Download the scripts and tests (ZIP)
Python scripts, tests, and ready-to-run merge, join, and date practice files with expected results. No signup required.
Two folders can contain the same filenames and file sizes while holding different bytes. Before retiring an original export folder, make an inventory of its file paths, byte counts, and SHA-256 digests. Create another inventory after copying it, then compare the inventories to locate missing, added, or changed files.
Tested: Python 3.10 on Windows, September 7, 2026, using local synthetic files. Tests cover nested paths, empty files, Unicode filenames, equal-size content changes, added and removed paths, unchanged copies, and refusing to overwrite a report or save it inside the scanned tree. No third-party packages or uploads are required. This is an inspection workflow, not a backup service.
The decision this report supports
Use it to check whether a quiet local copy contains the same file paths and bytes as your source at the time each was scanned. Keep the original until you have also checked the copied data in its intended application. The script does not restore anything, delete anything, or prove that the original files were correct.
The comparison is keyed by relative path. A renamed file appears as one removal and one addition, even when its digest stays the same. Empty directories, timestamps, permissions, ownership, and filesystem metadata are outside this report. For finding repeated content within one folder, use the separate duplicate-file guide.
Complete script
Save this as file_manifest.py in a scratch folder. Put your test folders beneath that scratch folder so the script and reports stay outside the folders being inventoried.
"""Create and compare SHA-256 inventories of quiet local directory trees."""
import argparse
import hashlib
import json
import stat
from pathlib import Path
def inventory(folder):
root = Path(folder)
files = {}
def visit(path):
info = path.lstat()
if stat.S_ISLNK(info.st_mode) or getattr(info, 'st_file_attributes', 0) & 1024:
raise ValueError(f'Links and Windows reparse points are unsupported: {path}')
if stat.S_ISDIR(info.st_mode):
for child in sorted(path.iterdir()):
visit(child)
elif stat.S_ISREG(info.st_mode):
digest = hashlib.sha256()
total = 0
with path.open('rb') as handle:
for chunk in iter(lambda: handle.read(1024 * 1024), b''):
digest.update(chunk)
total += len(chunk)
after = path.stat()
if (info.st_size, info.st_mtime_ns) != (after.st_size, after.st_mtime_ns) or total != info.st_size:
raise ValueError(f'File changed during reading: {path}')
files[path.relative_to(root).as_posix()] = {
'bytes': total, 'sha256': digest.hexdigest()}
else:
raise ValueError(f'Unsupported file type: {path}')
if not root.is_dir():
raise NotADirectoryError(root)
visit(root)
return {'format': 'dadidan-file-manifest-v1', 'files': files}
def compare(before, after):
for item in (before, after):
if item.get('format') != 'dadidan-file-manifest-v1' or not isinstance(item.get('files'), dict):
raise ValueError('Unsupported manifest format')
left, right = before['files'], after['files']
shared = left.keys() & right.keys()
return {'added': sorted(right.keys() - left.keys()),
'removed': sorted(left.keys() - right.keys()),
'changed': sorted(name for name in shared if left[name] != right[name]),
'unchanged_count': sum(left[name] == right[name] for name in shared)}
def main():
parser = argparse.ArgumentParser(description=__doc__)
commands = parser.add_subparsers(dest='command', required=True)
create = commands.add_parser('snapshot')
create.add_argument('folder', type=Path)
create.add_argument('output', type=Path)
diff = commands.add_parser('compare')
diff.add_argument('before', type=Path)
diff.add_argument('after', type=Path)
args = parser.parse_args()
try:
if args.command == 'snapshot':
if args.output.resolve().is_relative_to(args.folder.resolve()):
raise ValueError('Save the manifest outside the inventoried folder')
result = inventory(args.folder)
with args.output.open('x', encoding='utf-8') as output:
json.dump(result, output, indent=2, ensure_ascii=True)
print(f"Recorded {len(result['files'])} files")
else:
before = json.loads(args.before.read_text(encoding='utf-8'))
after = json.loads(args.after.read_text(encoding='utf-8'))
print(json.dumps(compare(before, after), indent=2, ensure_ascii=True))
except (OSError, ValueError, TypeError, AttributeError) as exc:
parser.exit(1, f'Error: {exc}\n')
if __name__ == '__main__':
main()
The code uses Python’s hashlib digest interface to process file bytes in blocks, and pathlib filesystem operations to list paths and write reports. Sorting paths makes comparisons easier to inspect. File content is read in 1 MiB blocks, although the path inventory itself remains in memory.
Reproduce an unchanged copy and a broken copy
Save this as make_manifest_sample.py beside the main script and run python make_manifest_sample.py. It creates fresh folders; run it only once in a new scratch directory.
from pathlib import Path
import shutil
source = Path('source-sample')
source.mkdir()
(source / 'nested').mkdir()
(source / 'a.txt').write_bytes(b'abc')
(source / 'empty.txt').write_bytes(b'')
(source / 'nested' / 'note.txt').write_bytes(b'keep')
shutil.copytree(source, 'copied-sample')
Create the two inventories. Report filenames are outside both scanned folders:
python file_manifest.py snapshot source-sample before.json
python file_manifest.py snapshot copied-sample copied.json
python file_manifest.py compare before.json copied.json
Each snapshot reports Recorded 3 files. The comparison has empty added, removed, and changed arrays, with unchanged_count equal to 3.
Now change the invented copy. Save this as change_manifest_sample.py and run it. These operations target only the scratch files created above:
from pathlib import Path
copy = Path('copied-sample')
(copy / 'a.txt').write_bytes(b'xyz')
(copy / 'empty.txt').unlink()
(copy / 'extra.txt').write_bytes(b'new')
The replacement xyz has the same byte count as abc, which illustrates why comparing only sizes misses a change. Generate a new report without overwriting the previous one:
python file_manifest.py snapshot copied-sample changed.json
python file_manifest.py compare before.json changed.json
Expected report:
{
"added": ["extra.txt"],
"removed": ["empty.txt"],
"changed": ["a.txt"],
"unchanged_count": 1
}
The comparison reads the two saved inventories. It does not rescan either folder, so an old inventory does not describe subsequent edits. Create fresh snapshots when checking a new transfer.
Check the report, not just the exit status
The compare command exits with status 0 when it successfully produces a report, including when that report contains differences. Status 0 does not mean the copy matches. Missing input, invalid JSON, and an unsupported manifest format instead produce an error and status 1. A typo in command arguments can produce argparse status 2.
After running the unchanged and changed examples above, save this as verify_manifest_sample.py beside the script. Run python verify_manifest_sample.py without -O. It checks the actual command results against independently written expectations:
import json
from pathlib import Path
import subprocess
import sys
reports = [Path(name) for name in ("before.json", "copied.json", "changed.json")]
saved = {p: p.read_bytes() for p in reports}
for after, expected in [
("copied.json", {"added": [], "removed": [], "changed": [], "unchanged_count": 3}),
("changed.json", {"added": ["extra.txt"], "removed": ["empty.txt"],
"changed": ["a.txt"], "unchanged_count": 1}),
]:
result = subprocess.run(
[sys.executable, "file_manifest.py", "compare", "before.json", after],
capture_output=True, text=True, check=False,
)
assert result.returncode == 0, result.stderr
assert result.stderr == "", result.stderr
assert json.loads(result.stdout) == expected
assert saved == {p: p.read_bytes() for p in reports}
print("PASS: both reports verified; saved manifests unchanged")
Expected output is PASS: both reports verified; saved manifests unchanged. The unchanged comparison still reports three matching files even though you have since edited copied-sample: it reads the old copied.json, not today’s folder. The changed comparison reads the fresh changed.json and finds the differences. Keep the scan time and source-folder identity with each report in your own records.
Recover from a refused report destination
With before.json already present, run python file_manifest.py snapshot source-sample before.json again. Expect status 1 and an error indicating that the file already exists; the OS-specific wording can vary. The existing report must remain unchanged. Exclusive creation uses Python’s open mode x.
Next run python file_manifest.py snapshot source-sample source-sample/report.json. Expect status 1 and Error: Save the manifest outside the inventoried folder. No report should appear inside the source tree.
Recover by choosing a fresh filename outside the tree:
python file_manifest.py snapshot source-sample before-recheck.json
python file_manifest.py compare before.json before-recheck.json
Expect Recorded 3 files, followed by empty difference arrays and unchanged_count: 3, assuming the source is still unchanged. Preserve the original report for comparison. These commands do not repair the altered copy; investigate and recopy affected files separately before taking another fresh snapshot.
This added workflow was verified on September 26, 2026 with Python 3.10 on Windows: both comparison outcomes, report overwrite refusal, in-tree report refusal, fresh-name recovery, and preservation of the original source bytes and saved reports.
Read a manifest entry
In before.json, a.txt has bytes: 3 and this SHA-256 value:
ba7816bf8f01cfea414140de5dae2223b00361a396177a9cb410ff61f20015ad
That value corresponds to the exact three bytes abc, without a newline. A text editor adding a line ending will produce a different digest. The script also inventories empty files; they are not discarded as irrelevant.
Unicode filenames are written using JSON escapes for terminal portability and decoded when the JSON is read. Paths use forward slashes inside the report. Keep reports private when filenames reveal client names or project information.
Errors that need attention
| Condition | Behavior |
|---|---|
| Existing report filename | Exclusive creation fails; the existing report is preserved. |
| Report placed inside the source tree | The command refuses it to avoid inventorying its own output on the next run. |
| Symbolic link or Windows reparse point | The scan stops instead of following it or silently omitting it. Use an ordinary local test folder. |
| Unreadable path or unsupported file type | The scan fails; do not treat a failed run as a complete inventory. |
| File size or modification time changes while reading | The scan fails with a changed-file error. Stop the writer and retry. |
| Disk error during report writing | A partial JSON report may remain. Choose a fresh report name after resolving the problem. |
The script deliberately fails for links, including Windows junctions and some cloud placeholder files. It does not promise compatibility with online-only cloud folders, network mounts, or changing trees. These special cases are not included in the local sample verification.
What equal hashes do not establish
Matching SHA-256 values are strong evidence of matching file bytes in this workflow. They do not authenticate the author or establish that the source was safe: someone who can change both a file and its manifest can replace both. This guide does not implement signatures or a protected chain of custody.
The before/after size and modification-time checks catch some concurrent edits, but they are not an atomic filesystem snapshot. Files added after a directory was listed can be missed; same-size modifications with restored timestamps can evade those checks. Pause writers or use a filesystem snapshot when consistency across a live tree matters.
For many small files, traversal and storage latency may dominate. For large files, every byte must be read to compute the digest. No production throughput or multi-gigabyte benchmark is claimed here. The manifest stores one entry per file and deeply nested trees can encounter Python’s recursion limit.
If you need to compare business records inside exports rather than whole-file bytes, use CSV comparison by ID. Different row ordering changes a file digest even when its logical records are equivalent.