Aglet

Triage CSV Delimiter Detection Failures

A preview showing one giant column or many unexpected columns can point to delimiter detection, but quoted values and malformed records can produce similar symptoms. Begin with the original file and a small set of representative records before changing field mappings.

Establish what is happening

  1. Compare records with preview columns

    Choose a header, a simple data record, and a record containing punctuation inside a value. Count their intended fields and compare those counts with the preview. Keep a sanitized copy of the original bytes; a spreadsheet resave may change the evidence.

  2. Record the detected dialect

    Capture the chosen delimiter, quote character, escape convention, and whether detection sampled the entire file. Check whether the preview and eventual importer use the same settings. A correct first row does not establish that later records follow the same format.

  3. Find the first structural divergence

    Locate the earliest record whose fields differ from the intended structure. Compare an explicit delimiter with automatic detection while leaving quoting unchanged. If both produce the same bad split, narrow the next investigation to escaping or source-file consistency.

What to carry forward

Finish with the expected field count, detected settings, first divergent record, and a reproducible preview comparison. Keep the cause provisional if only a screenshot is available. The next owner should know whether to inspect dialect detection, quoting, or the file producer.

Technical background: DuckDB documentation.

Keep the decision with the work.

Use a Work Item in Aglet to record the problem, the evidence you have, and the next decision. Add an owner and priority, then keep updates in the discussion so the next person can follow the reasoning.

Create an account See the product workflow