Aglet

Investigate Quoted Newlines in CSV Imports

A multiline record can fail because quoting is parsed incorrectly, a preprocessing step splits lines too early, or a chunk reader loses state. Follow one known record from file bytes to parsed values before changing validation rules. The same error message can arise from different boundaries.

Build a useful investigation brief

  1. Build contrasting multiline fixtures

    Create balanced quoted fields containing one newline, several newlines, and a quote escaped according to the chosen dialect. Add an intentionally unbalanced example. Specify the logical record count and complete field text independently for each case, rather than deriving expectations from the parser.

  2. Trace preprocessing and chunk reads

    Inspect any line-by-line preprocessing before the CSV parser. Compare a small file with one where the multiline field crosses a read boundary. Record whether quote state survives that boundary and whether newline normalization changes the bytes the parser actually receives.

  3. Follow error locations to records

    Compare parser errors with the user-facing row report. Determine whether reported numbers refer to physical lines or logical records. A correct parser can still leave users unable to repair an import if the interface points to the wrong record after a multiline field.

What to carry forward

Finish with a reproducible fixture, the first changed record boundary, and a clear explanation of line-number semantics. Separate parser, preprocessing, and error-reporting defects in the brief. A proposed fix should preserve valid multiline content while continuing to reject an actually unfinished quoted field.

Technical background: DuckDB documentation.

Keep the decision with the work.

Use a Work Item in Aglet to record the problem, the evidence you have, and the next decision. Add an owner and priority, then keep updates in the discussion so the next person can follow the reasoning.

Create an account See the product workflow