Project: CSV Import Validator with Row Errors
Validate an entire CSV import and return accepted records alongside deterministic row-specific errors.
Example check: Worked and guided code samples are checked against CPython 3.13. Practice uses Pyodide 0.29.5 (Python 3.13) in your browser. How we check lessons
Video lesson: Project: CSV Import Validator with Row Errors
The preview is stored on this site. YouTube loads only when you press Play. Watch on YouTube
Jump to a chapter
Read the video transcript
Select a timestamp to watch that moment on YouTube.
- 00:00
CSV imports often fail in two different ways. Sometimes one invalid row should abort the whole operation. In other workflows, a person needs a complete report of bad rows so they can repair a file in one pass. This project uses the second contract. We will read an in-memory CSV string with sku and qty columns, return accepted records, and return line-numbered reasons for rejected records. The function will not write to a database or pretend that partially accepted rows were committed anywhere. First we will run a simple fail-fast conversion and see how a bad quantity prevents a later valid row from being inspected.
- 00:42
Then we will use Python's csv.DictReader to preserve quoted fields and learn where a record appeared in the source. Finally we will implement the report, verify mixed input, a header-only file, reordered headers, and malformed row shapes. The matching PythonLessonLab project task checks these decisions automatically. The goal is a transparent parsing boundary: every accepted row has a normalized SKU and positive integer quantity, and every rejected row has a specific physical line and one reason. The first record is accepted. The second has a quantity that int cannot parse, so ValueError leaves the loop.
- 01:24
The later C-3 record is valid but never inspected. This behavior is not universally wrong; a fail-fast importer can be appropriate when no partial report is wanted. It is wrong for the contract we chose: collect all ordinary row errors and continue. We must be careful about what we catch. A malformed quantity is an expected data problem, so the importer can record it and move to the next row. A programming bug in our own validation code should not be swallowed by `except Exception` and mislabeled as an invalid quantity. We also should not silently drop B-2; a user needs its location and reason.
- 02:05
The result will have two lists: accepted tuples in source order and errors as `(physical_line, reason)` tuples. That means we need a reliable source line number. Counting accepted rows would be wrong because rejected rows still occupy lines, and a quoted CSV field can contain a newline. DictReader's line_num helps us report the physical line reached by a parsed record. The header declares two required fields. On line two, qty is missing. On line three, an extra cell appears; DictReader stores overflow values under the key None by default. Both are malformed row shapes under our format.
- 02:46
Line four is well shaped and must remain eligible even though earlier rows were bad. The line numbers shown are physical source lines reached by the reader, not the count of accepted records. If a quoted field spans multiple source lines, line_num reflects the end of that CSV record. Before examining rows, we should check that the header contains both required names. A file with only sku cannot produce a meaningful qty value, so we return a header error at line one and no accepted rows. A file with the right header and no records is different: it is valid and returns two empty lists.
- 03:26
After row shape, we validate content in a consistent order. Strip the SKU and report missing sku if it becomes empty. Then parse quantity as an integer and require it to be positive. Append exactly one reason for each rejected record and continue to the next row. The function validates the schema first. If sku or qty is absent from the header, there is no point attempting row conversion, so it returns a single header error. For each record, it checks shape, SKU content, and quantity in that order. A rejected row contributes one error and then continues. The first record is accepted as the tuple A-1 and integer two.
- 04:10
The blank SKU on physical line three is reported. The bad quantity on line four is reported. C-3 on line five is still processed and accepted. We chose exact reason labels to make the result stable for a user interface and for tests. Notice the function returns a report, not a half-written database. A real importer would need to decide whether accepted records may be committed when errors remain, whether duplicate SKUs are allowed, and whether a person must approve the preview. Those are separate policies. Here the code solves the narrower parsing problem and makes partial success visible rather than hiding it.
- 04:52
A valid header-only file returns empty lists, not a fake record or an error. A missing qty header returns a single line-one schema error. Reordered columns still work because DictReader uses header names, and a quoted comma inside a SKU stays within one field. The final sample checks three different invalid rows: missing qty, nonpositive qty, and an extra cell. The valid D-4 row after them still appears in accepted, proving the importer continued. These assertions cover the contract rather than merely repeating the prepared example. In the browser project, complete the validator from the starter code and run
- 05:33
Check task. If the first bad row prevents later accepted records, your code is still fail- fast. If line numbers shift after an error, check whether you are counting accepted rows instead of reading the CSV parser's physical line number. If malformed shapes become missing-SKU errors, check the validation order. Once the exercise passes, try a quoted field containing a newline and observe what line_num reports. This is a small importer, but the same design habit applies to larger data pipelines: decide whether partial success is allowed, report every rejected record
- 06:13
clearly, and never make missing data look like a silently successful import.
Understand the concept
A fail-fast CSV reader is useful when one bad record should stop an operation. Another common workflow must show all problems at once so the source file can be repaired efficiently. This project builds that second kind of importer. It returns a dictionary with accepted SKU and quantity tuples plus a list of errors. A malformed row is never silently dropped: every rejected nonblank record contributes a line-numbered reason.
csv.DictReader handles quoted fields and header positions. reader.line_num gives the physical line reached after a record, which is useful when a quoted field spans multiple lines. For this small format, sku and qty are required headers. If the header is missing or incomplete, the function returns no accepted rows and a header error at line one. A header-only file with the correct columns is valid and returns two empty lists.
Each row is checked in order. Extra cells, missing cells, an empty SKU after stripping, a non-integer quantity, and a quantity below one all get explicit reasons. The importer keeps processing later rows instead of raising an exception for an ordinary bad cell. For a valid row it appends a normalized SKU and integer quantity. The result preserves the source order of both accepted records and errors.
The exercise differs from the earlier DictReader lesson: it teaches a report with partial success, not another fail-fast parser. Tests verify a mixed file, header order, quoted SKUs, a malformed row, empty data, and exact line numbers. A production importer might add duplicate-SKU policy, transactions, maximum error counts, and a preview before saving. This tutorial deliberately stops at parsing and reporting; it does not write to a database.
- Partial success requires an explicit accepted/error result
- Physical line numbers help locate CSV mistakes
- Schema failure differs from row failure
- Preserve source order while continuing past bad rows
See it step by step
Read the code, predict the output, then compare it with the result.
01. Observe physical record lines
import csv, io
reader = csv.DictReader(io.StringIO('sku,qty\nA-1,2\nB-2,3\n'))
for row in reader:
print(reader.line_num, row['sku'])2 A-1
3 B-2The reader reports the physical line reached after each parsed record.
02. Continue after a bad quantity
rows = [('A-1','2'),('B-2','bad'),('C-3','1')]
accepted, errors = [], []
for line,(sku,qty) in enumerate(rows,2):
try: accepted.append((sku,int(qty)))
except ValueError: errors.append((line,'invalid qty'))
print(accepted)
print(errors)[('A-1', 2), ('C-3', 1)]
[(3, 'invalid qty')]A row error is recorded while later valid rows are still processed.
03. Distinguish malformed row shape
import csv, io
reader = csv.DictReader(io.StringIO('sku,qty\nA-1\nB-2,3,extra\n'))
for row in reader:
print(reader.line_num, None in row or row.get('qty') is None)2 True
3 TrueMissing and overflow cells both fail the project's row-shape contract.
A closer look
Follow the reasoning, inspect each result, then try the suggested changes in the console below.
See what fail-fast parsing loses
A simple import that calls int without catching its error stops at the first bad quantity. In the demonstration, B-2 contains invalid text, so the valid C-3 row is never reached. Fail-fast can be a deliberate design, but this project's user needs a report of all ordinary row errors and all records that were accepted.
Run the broken loop and observe which SKU names it saw. The importer should not silently discard B-2 either. It needs two explicit outputs: accepted records and line-specific errors. That contract lets someone repair a source file without guessing where a row disappeared.
rows=[('A-1','2'),('B-2','not-an-int'),('C-3','1')]
accepted=[]
try:
for sku,raw_qty in rows:
accepted.append((sku,int(raw_qty)))
except ValueError:
print('stopped at bad quantity')
print(accepted)stopped at bad quantity
[('A-1', 2)]- A-1 is accepted.
- The B-2 conversion stops the whole loop.
- C-3 is never inspected under fail-fast behavior.
Locate damaged CSV records precisely
DictReader provides a useful physical line counter after each parsed record. With simple one-line rows it matches the number a person sees in a text editor. A quoted field can span several physical lines, so line_num may identify the record's final line rather than an abstract accepted-row number. Use it when reporting a row failure.
The two records below have a correct header but damaged shape. The first lacks a quantity cell; the second has an extra cell. DictReader represents these as None in different places. Check shape before stripping or calling int, and append exactly one reason for each rejected record.
import csv, io
reader=csv.DictReader(io.StringIO('sku,qty\nA-1\nB-2,3,extra\n'))
for row in reader:
print(reader.line_num, None in row or row.get('qty') is None)2 True
3 True- Physical line numbers remain attached to records.
- Short and long rows both violate the schema.
- A validator can report each and continue.
Return accepted records and row-specific errors
The completed importer checks required headers before reading data. It returns a single header error when they are missing. Otherwise it walks all rows, deciding one reason per rejected record and continuing to later records. A valid SKU is stripped, and a valid quantity becomes a positive integer. The output lists preserve source order.
The sample has two good rows and two bad rows. Their errors point to physical lines three and four, while the later C-3 record still appears in accepted. The task checker also uses a header-only file, reordered headers, a quoted SKU, and malformed row shapes. This makes partial success measurable rather than accidental.
import csv, io
def validate_orders(csv_text):
reader=csv.DictReader(io.StringIO(csv_text))
accepted,errors=[],[]
if not {'sku','qty'}<=set(reader.fieldnames or []):
return {'accepted':accepted,'errors':[(1,'missing required headers')]}
for row in reader:
line=reader.line_num
if None in row or not isinstance(row.get('sku'),str) or not isinstance(row.get('qty'),str):
errors.append((line,'malformed row')); continue
sku=row['sku'].strip()
if not sku:
errors.append((line,'missing sku')); continue
try: qty=int(row['qty'].strip())
except ValueError:
errors.append((line,'invalid qty')); continue
if qty<=0:
errors.append((line,'invalid qty')); continue
accepted.append((sku,qty))
return {'accepted':accepted,'errors':errors}
text='sku,qty\nA-1,2\n,4\nB-2,not-an-int\nC-3,1\n'
print(validate_orders(text)){'accepted': [('A-1', 2), ('C-3', 1)], 'errors': [(3, 'missing sku'), (4, 'invalid qty')]}- Check schema before data rows.
- Record one reason and continue after each bad row.
- Keep later valid rows in accepted.
Try it in Python
Edit the example and run it. Python starts in your browser the first time you click Run.
Python console
Ready to runNeed input()? Add one value per line
Your output appears here.
Need a hint?
After checking reader.fieldnames, loop through records and use reader.line_num. Append one error then continue for each invalid row. Parse positive qty only after confirming the row shape and nonblank SKU.
Complete the task and select Check task to verify your code.
Quick quiz
Three questions. You can change your answers and try again.
Typical mistakes
Everyone meets these errors. See what causes them and how to fix them.
Returning only valid rows
if invalid: continueerrors.append((reader.line_num, reason))
continueWhat happens: A user cannot tell which source rows were rejected or why.
Every rejected row needs a traceable report entry.
Failing the whole import at the first bad cell
qty = int(row['qty']) # uncaughttry: qty = int(raw_qty)
except ValueError:
errors.append((reader.line_num, 'invalid qty'))
continueWhat happens: Later valid rows are never inspected.
Catch ordinary row-level validation errors under the chosen partial-success contract.
Explore the runnable code
The GitHub project includes setup instructions and source code you can run on your computer. Use this lesson for the guided examples, browser task, and quiz.