Download the scripts and tests (ZIP)
Python scripts, tests, and ready-to-run merge, join, and date practice files with expected results. No signup required.
An import can fail because one line is malformed even when the rest of a file looks correct. This local validator checks a JSONL file one line at a time and returns line numbers for problems. Its contract is explicit: every physical line must contain one JSON object, with no blank lines, duplicate keys, or non-standard constants.
Tested: Python 3.10 on Windows, September 7, 2026. The test covers a valid nested object, a blank line, an array, a repeated key, NaN, and malformed JSON. It checks that the valid count and all five failing line numbers match the sample.
Reproduce an import failure
Save the following six lines as UTF-8 in sample.jsonl. Keep the empty second line:
{"id":"001","nested":{"ok":true}}
[]
{"a":1,"a":2}
{"a":NaN}
{"a":}
This is intentionally an invalid input for our object-per-line importer. JSON Lines can represent JSON values other than objects, but many record-oriented workflows require objects. Rejecting the array is a requirement of this particular validator, not a claim that arrays are always invalid JSONL.
The complete script
Save the following as check_jsonl.py:
"""Validate one JSON object per nonblank line. No uploads or API calls."""
import json
from pathlib import Path
def reject_constant(value):
raise ValueError(f'Non-standard JSON constant: {value}')
def unique_object(pairs):
result = {}
for key, value in pairs:
if key in result:
raise ValueError(f'Duplicate key: {key}')
result[key] = value
return result
def check_jsonl(path):
valid, errors = 0, []
with Path(path).open(encoding='utf-8') as handle:
for number, line in enumerate(handle, 1):
if not line.strip():
errors.append({'line': number, 'error': 'Blank line'})
continue
try:
value = json.loads(line, parse_constant=reject_constant, object_pairs_hook=unique_object)
if not isinstance(value, dict):
raise ValueError('Expected a JSON object')
valid += 1
except ValueError as exc:
errors.append({'line': number, 'error': str(exc)})
return {'valid_objects': valid, 'errors': errors}
if __name__ == '__main__':
import argparse
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument('path')
args = parser.parse_args()
result = check_jsonl(args.path)
print(json.dumps(result, indent=2))
raise SystemExit(1 if result['errors'] else 0)
Run:
python check_jsonl.py sample.jsonl
The report contains "valid_objects": 1 and errors for lines 2, 3, 4, 5, 6. Error messages include Blank line, Expected a JSON object, Duplicate key: a, and Non-standard JSON constant: NaN. The final malformed line produces a JSON decoder message. Its wording can vary across Python versions; the physical line number in the outer report identifies the record.
The program exits with status 1 when the report contains errors, so a calling job can stop before import. When there are no errors it exits with status 0. File access failures and invalid UTF-8 also cause the program to fail, but appear as exceptions rather than per-record JSON errors.
Why plain json.loads is not the whole check
Successful parsing does not guarantee that your intended record rules were enforced. We supply a hook that rejects repeated object keys rather than silently letting a later field replace an earlier field. That hook also processes nested objects. A second hook rejects NaN and positive or negative infinity tokens. These hooks are part of Python’s documented JSON decoder interface.
After parsing, the script checks that the top-level value is a dictionary. It does not require an id key, check email formats, validate a schema, or look for repeated IDs across different lines. Add those checks only after defining the receiving system’s schema; a generic format validator cannot infer it.
Correct the sample and verify again
Keep the invalid sample.jsonl for comparison. Save these two ordinary object records in a separate UTF-8 file named corrected.jsonl:
{"id":"001","nested":{"ok":true}}
{"id":"002","nested":{"ok":false}}
Run python check_jsonl.py corrected.jsonl. The expected report is:
{
"valid_objects": 2,
"errors": []
}
A normal line terminator after the last object does not create an extra empty record. An additional blank line does. A pretty-printed object spanning several lines does not satisfy this reader’s format; serialize each record onto one line when producing the export.
Verify both runs without overwriting the input
Save this as verify_sample.py beside the validator and both sample files, then run python verify_sample.py. It captures each command’s output in memory, checks its exit status and report, and verifies that both input files remain unchanged. It does not import any records.
import json
from pathlib import Path
import subprocess
import sys
inputs = {name: Path(name).read_bytes()
for name in ("sample.jsonl", "corrected.jsonl")}
for name, expected_exit in (("sample.jsonl", 1), ("corrected.jsonl", 0)):
run = subprocess.run(
[sys.executable, "check_jsonl.py", name],
capture_output=True, text=True, encoding="utf-8",
)
if run.returncode != expected_exit or run.stderr:
raise SystemExit(f"Unexpected execution failure for {name}: {run.stderr}")
report = json.loads(run.stdout)
if name == "sample.jsonl":
matches = (report["valid_objects"] == 1
and [item["line"] for item in report["errors"]] == [2, 3, 4, 5, 6])
else:
matches = report == {"valid_objects": 2, "errors": []}
if not matches:
raise SystemExit(f"Unexpected report for {name}")
if Path(name).read_bytes() != inputs[name]:
raise SystemExit(f"Input changed: {name}")
print("PASS: five rejected lines, two corrected objects, inputs unchanged")
The final line should be exactly PASS: five rejected lines, two corrected objects, inputs unchanged. A missing file or invalid UTF-8 may also produce exit status 1, but it produces an exception on standard error instead of a completed JSON report. The verifier therefore checks both streams, not the exit code alone. Python’s subprocess documentation describes these captured streams and return codes.
This check is specific to the synthetic examples above. It does not establish that production data meets your import schema. Do not automatically delete every rejected line: recover malformed records from their source, resolve duplicate fields deliberately, and reconcile expected record counts before importing. The corrected two-record fixture is a separate demonstration, not a repair that preserves all six original records.
Limits for large or untrusted files
The script reads one line at a time, but each complete line and its parsed object must fit in memory. It also retains every error in a list. A file with huge records or millions of errors therefore needs explicit size limits, an error cap, and a streaming report. This example is for small trusted local exports, not a public upload endpoint.
An empty file returns zero objects and no errors. If your job requires at least one record, reject a zero count before import. UTF-8 with a BOM is deliberately not normalized here: the first line will fail until you remove the marker or adopt a documented BOM-handling policy. Catching an encoding issue is preferable to silently discarding bytes.
The script reads your local file and prints the report. It has no network calls and does not send data to Dadidan Lab. When sharing an error report, remember that a duplicate-key message includes the field name; use synthetic examples when requesting help with sensitive datasets.
For tabular exports, see merging CSV files with validation. For duplicate report downloads, see finding duplicate files without deleting them.
Code in this article is original Dadidan Lab example code. You may use and adapt it, including commercially. It is provided without warranty. Corrections: contact Dadidan Lab.