Download the scripts and tests (ZIP)
Python scripts, tests, and ready-to-run merge, join, and date practice files with expected results. No signup required.
A reporting tool needs customer_id,total,status, but your export includes a name, an internal note, and those fields in a different order. Manually deleting columns works once. A repeatable export needs an explicit list of allowed columns, a predictable order, and an error when a required column disappears.
This guide creates a new CSV containing only the named columns in the order you specify. It preserves every data record and the selected cell values, including leading-zero IDs, empty strings, and quoted newlines. It does not edit the source. The complete standard-library script was tested with Python 3.10 on Windows on September 13, 2026, using synthetic data.
Define the output contract first
List the exact column names the receiving tool expects. Their order in the source does not matter: the script finds positions from the header. Their order on the command line determines the output order.
| Input condition | Behavior |
|---|---|
| Extra named columns | Excluded unless selected |
| Required column missing | Error before creating the output folder |
| Source columns reordered | Still selected by name |
| Duplicate or blank header | Error, even for an unselected column |
| Same column selected twice | Error |
| Data row with too many or too few fields | Error; any partial output has no completion marker |
Names match exactly, including case and spaces. Customer ID is not customer_id. Header-only input creates a header-only output with zero data records. An empty file fails because there is no header to interpret.
An allowlist is useful when exports gain new fields: those fields are not automatically included. It is not a privacy guarantee. A selected free-text field can still contain personal information, and excluded columns remain in your original file. Inspect the selected data before sharing it.
Run a reproducible example
Save this invented export as UTF-8 orders.csv:
status,customer_id,name,total,internal_note
open,00127,"Park, Mina",12.50,review
closed,00128,Renée,0.00,
Save the full code below as select_csv_columns.py beside the input. Run:
python select_csv_columns.py orders.csv report-run --columns customer_id total status
Use a new directory name for each run. The script refuses an existing directory so a failed attempt cannot be mistaken for fresh output. The successful command prints:
rows=2 columns=3
Inside report-run, selected.csv contains:
customer_id,total,status
00127,12.50,open
00128,0.00,closed
The second file, COMPLETE.txt, records the row and column counts. Check this marker before accepting a result from an automated job. For a header containing spaces, quote the entire name as one argument, for example --columns "Customer ID" "Order Total".
Complete script
"""Select named CSV columns in an explicit order, into a new output folder."""
import argparse
import csv
from pathlib import Path
def select_columns(source, destination, columns):
if not columns or len(set(columns)) != len(columns):
raise ValueError('Choose at least one column; repeated selections are not allowed')
source, destination = Path(source), Path(destination)
with source.open(encoding='utf-8-sig', newline='') as handle:
reader = csv.reader(handle, strict=True)
header = next(reader, None)
if not header or any(not name.strip() for name in header):
raise ValueError('Missing or blank column header')
if len(set(header)) != len(header):
raise ValueError('Duplicate input column headers')
missing = [name for name in columns if name not in header]
if missing:
raise ValueError(f'Missing required columns: {missing!r}')
positions = [header.index(name) for name in columns]
destination.mkdir()
count = 0
with (destination/'selected.csv').open('x', encoding='utf-8', newline='') as output:
writer = csv.writer(output)
writer.writerow(columns)
for number, row in enumerate(reader, 2):
if len(row) != len(header):
raise ValueError(f'Record {number}: expected {len(header)} fields, got {len(row)}')
writer.writerow([row[index] for index in positions])
count += 1
with (destination/'COMPLETE.txt').open('x', encoding='utf-8') as marker:
marker.write(f'rows={count} columns={len(columns)}\n')
return count
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument('source', type=Path)
parser.add_argument('destination', type=Path)
parser.add_argument('--columns', nargs='+', required=True)
args = parser.parse_args()
try:
count = select_columns(args.source, args.destination, args.columns)
except (OSError, UnicodeError, ValueError, csv.Error) as exc:
parser.exit(1, f'Selection failed: {exc}\nOutput without COMPLETE.txt is incomplete.\n')
print(f'rows={count} columns={len(args.columns)}')
if __name__ == '__main__':
main()
The Python CSV reader documentation specifies that ordinary reader values are strings and recommends opening CSV files with newline=''. The script keeps that behavior. It never converts a value such as 00127 to an integer or uses a comma split that would break a quoted name. A UTF-8 byte-order mark is accepted on input; output is UTF-8 without that mark.
Verify the exported contract
Run this check from the directory containing report-run:
import csv
from pathlib import Path
folder = Path('report-run')
assert (folder / 'COMPLETE.txt').is_file()
with (folder / 'selected.csv').open(encoding='utf-8', newline='') as handle:
rows = list(csv.reader(handle))
assert rows == [
['customer_id', 'total', 'status'],
['00127', '12.50', 'open'],
['00128', '0.00', 'closed'],
]
print('Selected columns verified')
Try moving the total column to the beginning of the input, together with its data values, and run into a different new folder. The output contract should be identical. Next, rename that header to amount: the command should fail with a missing-column error. Do not silently insert an empty total column to satisfy an importer; that would hide a source schema change.
The downloadable tests cover the exact example, source-column reordering, missing columns, repeated selections, duplicate and blank headers, Unicode, quoted commas and newlines, empty selected cells, short and long records, header-only input, and refusal to reuse an output directory. Original input bytes are checked after conversion.
Why validate fields you are dropping?
Suppose the input header has five columns but a later row has only four. Even when your selected columns are near the beginning, the broken record may indicate an incorrectly quoted field or a shifted export. This script rejects the whole record rather than guessing which value is missing. A trailing empty cell is different: a final comma explicitly represents that empty field and is accepted when the field count matches.
Record numbers in errors count CSV records including the header, not physical lines. A quoted newline belongs to its cell and does not start another CSV record. A blank line outside quotes is a zero-field record and is rejected.
Failure handling and limits
Header and selection checks happen before the output directory is created. Data rows are processed one at a time, so a malformed later row can leave a header and some valid rows in selected.csv. The program exits with an error and does not create COMPLETE.txt. Keep that folder for diagnosis and use a fresh folder after correcting the input. Disk errors or interruptions can also leave incomplete output; the workflow does not promise an atomic multi-file transaction.
Memory use depends on the header and current record rather than the number of rows, but unusually large fields still consume memory. Python’s default CSV field-size limit applies. This guide provides functional tests, not throughput measurements. It accepts comma-delimited UTF-8 input only and does not guess encodings, separators, or data types.
Selecting columns does not filter rows, rename fields, deduplicate IDs, or sanitize spreadsheet formulas. For distinct tasks, see removing duplicate records, joining a lookup CSV, and converting CSV to JSON while preserving IDs. You can select the final columns after a join, but first inspect its unmatched-record report so narrowing the output does not conceal missing lookup data.