Archiving & Evidence Preservation

What this covers

Capturing a source so that it still exists after someone deletes it, and so that you can show the copy you hold is the copy you took. The capture itself is the easy half; the hash, the third-party timestamp and the written provenance are what make it worth anything to an editor, a lawyer or a court. Everything below is organised around one question: what does this step actually prove?

Method

1. Check what already exists before you capture. A Wayback snapshot from before the content became interesting is worth more than one you took after. Query the availability and CDX APIs first; if there is an older capture, that is your baseline and the diff against today is itself a finding.
2. Capture to something you do not control, and to something you do. A third-party archive supplies a timestamp nobody can accuse you of forging. A local copy survives the archive being rate-limited, robots-excluded or taken down. Neither substitutes for the other.
3. Pick the capture tool by what the page is. Static HTML and media: wget --warc-file. JavaScript-rendered, infinite-scroll or login-walled: a real browser driving the capture, which in practice means browsertrix-crawler producing WACZ. Platform video: yt-dlp, which gets you the metadata sidecar a screen recording never will.
4. Hash on acquisition, before you look at the file. A hash taken after you cropped, converted or re-saved proves something about your working copy and nothing about the original.
5. Timestamp the manifest, not every file. One hash over the list of hashes, submitted to OpenTimestamps or an RFC 3161 authority, binds the whole collection to a point in time at one cost. That is the step that converts “I have a hash” into “I had this hash on that date”.
6. Write the provenance down in the same commit as the files. URL, UTC time of capture, the tool and version, who handed it to you, and how you reached the page. The route to a finding is part of the finding, and it is the part you will forget first.
7. Get a second independent capture. Two archives of the same URL taken by different infrastructure, or the same video from a second uploader, is the difference between a claim and a corroborated claim. Two copies that both trace to one upload are one copy.

Judgement calls worth naming: submitting a URL to a public archive creates a public record that somebody is interested in it, so on a target that watches its referrers and its Wayback entries, capture locally first and submit later. And a capture of a page you reached while logged in contains your session — strip or redact before you share the WARC.

Key tools

wget (WARC capture)

The standard way to get a byte-level record of an HTTP exchange rather than a rendered picture of it. A WARC stores the request and response headers alongside the body for every resource fetched, with a SHA-1 digest per record, so the file is a transcript of the conversation and not just its outcome. Use it whenever the page is server-rendered; it is the most portable evidence format in this area and every replay tool reads it.

brew install wget                # or: apt install wget

# the baseline capture: WARC transcript plus a CDX index of what went into it
wget --warc-file=capture-001 --warc-cdx \
     --page-requisites --adjust-extension --no-parent \
     'https://example.com/page'

# stamp the operator and case into the warcinfo record, so the file self-documents
wget --warc-file=capture-002 --warc-cdx \
     --warc-header="operator: J. Investigator" \
     --warc-header="description: case-2026-014, post cited in filing" \
     --warc-header="robots: off" \
     --page-requisites --adjust-extension 'https://example.com/page'

# leave it uncompressed while you are inspecting it; grep works on a plain WARC
wget --warc-file=capture-003 --no-warc-compression 'https://example.com/page'

# pull in the CDN that actually serves the images, or the capture renders blank
wget --warc-file=capture-004 --warc-cdx \
     --page-requisites --convert-links --adjust-extension \
     --span-hosts --domains example.com,cdn.example.com \
     'https://example.com/page'

# a whole section, politely: the delay is what stops you being blocked mid-capture
wget --warc-file=capture-005 --warc-cdx --recursive --level=2 --no-parent \
     --wait=2 --random-wait --limit-rate=500k \
     -U 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/140.0 Safari/537.36' \
     'https://example.com/newsroom/'

# a list of URLs in one WARC, split at 1GB so the files stay movable
wget --warc-file=capture-006 --warc-max-size=1G --warc-cdx -i urls.txt

# second pass over the same site without re-storing what you already have
wget --warc-file=capture-007 --warc-dedup=capture-005.cdx --recursive --no-parent \
     'https://example.com/newsroom/'

# read back what the capture contains, without a replay tool
zcat capture-001.warc.gz | grep -a '^WARC-Target-URI:' | sort -u
awk 'NR>1 {print $5, $6, $1}' capture-001.cdx   # status, SHA-1 digest, URL

Read the CDX file: the digest column is a base32 SHA-1 of the response payload, which gives you a free per-resource integrity check, and the status column tells you which requisites 404’d. A capture whose CDX is full of 302 and 403 rows did not get the page.

What this does not prove: the WARC records what your client received from an IP at that moment. It does not prove the page looked that way to anyone else, does not survive geo-targeted or personalised content, and carries no third-party attestation of the time — the WARC-Date is your own clock. --convert-links rewrites the mirrored files on disk, not the WARC records, so the transcript stays pristine while the browsable copy is usable; keep both. Above all, wget executes no JavaScript, so on a modern single-page application it faithfully archives an empty shell. If the page needs a browser, use one.

browsertrix-crawler

A headless Chrome driven by Webrecorder’s crawler, writing WARC and WACZ. This is the live, maintained answer for anything JavaScript-rendered, where wget returns a loading spinner. It scrolls, autoplays, runs site-specific behaviours for the big platforms, and extracts page text for search.

The older Python crawlers in this niche have aged out: wpull carries a 2013–2016 copyright and no release since 2019, and grab-site still pins Python 3.7/3.8 and was dropped from nixpkgs after 23.05. Neither is a reasonable thing to stand behind in 2026. Treat them as read-only history and use the crawler below.

docker pull webrecorder/browsertrix-crawler

# single page, as WACZ, with text extracted for full-text search
docker run -v "$PWD/crawls:/crawls/" -it webrecorder/browsertrix-crawler crawl \
  --url 'https://example.com/post/123' --scopeType page \
  --generateWACZ --text to-pages --collection case-2026-014

# the page plus whatever it links to on the same host, two levels deep
docker run -v "$PWD/crawls:/crawls/" -it webrecorder/browsertrix-crawler crawl \
  --url 'https://example.com/newsroom/' --scopeType host --depth 2 \
  --generateWACZ --collection newsroom

# a feed that only loads on scroll: give the behaviours time to finish
docker run -v "$PWD/crawls:/crawls/" -it webrecorder/browsertrix-crawler crawl \
  --url 'https://example.com/feed' --scopeType page-spa \
  --behaviors autoscroll,autoplay,autofetch,siteSpecific \
  --behaviorTimeout 300 --generateWACZ --collection feed

# several URLs from a file, one collection, so the manifest covers the set
docker run -v "$PWD/crawls:/crawls/" -it webrecorder/browsertrix-crawler crawl \
  --urlFile /crawls/urls.txt --scopeType page --generateWACZ --collection batch-01

# the WACZ is a zip: look inside before you trust it
unzip -l crawls/collections/case-2026-014/case-2026-014.wacz
unzip -p crawls/collections/case-2026-014/case-2026-014.wacz datapackage-digest.json | jq .

# the crawl log is JSON Lines — this is where blocked and failed pages surface
jq -r 'select(.logLevel=="error") | [.timestamp,.message] | @tsv' \
  crawls/collections/case-2026-014/logs/*.log

A WACZ is a WARC plus an index, a page list and a signed digest manifest, and it opens in ReplayWeb.page with no server. That makes it the format to hand to someone who is not going to install anything. Check datapackage-digest.json after every crawl: if the crawler was blocked, you get a valid WACZ containing a block page, and nothing about the file itself says so.

What it does not tell you: a browser capture is a recording of one rendering session, which means it includes whatever was personalised, A/B-tested or geo-fenced for that session. Running the same crawl from a different exit and getting different content is a finding about the site, not an error. Logged-in crawls bake the session into the archive — handle accordingly.

yt-dlp

Platform video and audio with its metadata intact. The point is not the video file; a screen recording gets you that. The point is the .info.json sidecar, which carries uploader ID, upload date, duration, view and comment counts, available formats and often the original title and description before anybody edited them.

brew install yt-dlp              # or: pipx install yt-dlp

# the provenance capture: video, metadata, description, thumbnail, subtitles
yt-dlp --write-info-json --write-description --write-thumbnail \
       --write-subs --sub-langs 'all' --no-mtime \
       -o '%(upload_date)s_%(uploader_id)s_%(id)s.%(ext)s' URL

# what formats exist, before you commit to one
yt-dlp -F URL

# the best single pre-merged file, so you archive bytes the platform served
yt-dlp -f 'best[ext=mp4]/best' --no-mtime URL

# metadata only — enough to date and attribute a clip without downloading it
yt-dlp --skip-download --write-info-json --write-thumbnail URL

# comments too; they routinely date an event more precisely than the upload does
yt-dlp --write-info-json --write-comments --skip-download URL

# a whole channel, newest first, with an archive file so re-runs are incremental
yt-dlp --write-info-json --no-mtime --download-archive seen.txt \
       -o '%(upload_date)s_%(id)s.%(ext)s' 'https://example.com/@account/videos'

# the intermediate pages the extractor fetched — useful when an extractor misreads a page
yt-dlp --write-pages --skip-download URL

# the fields you will actually quote, straight out of the sidecar
jq -r '[.id,.uploader_id,.upload_date,.timestamp,.duration,.view_count,.title] | @tsv' *.info.json

--no-mtime matters: without it the file’s mtime is set from the upload date, which silently overwrites your own acquisition timeline with the platform’s claim. timestamp in the sidecar is Unix epoch seconds and is usually more precise than upload_date, which is date-only and rendered in the platform’s timezone of choice.

What the sidecar does not establish: every field in it is the platform’s assertion, repeated. upload_date is when it was uploaded, never when it was filmed, and a re-upload resets it. Extractors break when platforms change, so a field that is null may mean absent or may mean the extractor lost it — check against the page. Rate limits and age or region gates will silently truncate a channel pull; compare the count you got against the count the channel claims.

Wayback Machine, from the command line

Three separate endpoints, used at three different moments. Availability answers “is there already a capture”; CDX answers “what captures exist, and did the content change between them”; Save Page Now creates a new one.

# is there a capture at all, and what is the nearest one to a date
curl -s 'https://archive.org/wayback/available?url=example.com/page&timestamp=20240101' | jq .

# every capture of a URL, with the payload digest that reveals real edits
curl -s 'https://web.archive.org/cdx/search/cdx?url=example.com/page&output=json&fl=timestamp,original,statuscode,digest,length' | jq -r '.[] | @tsv'

# collapse consecutive identical captures: what is left is the change history
curl -s 'https://web.archive.org/cdx/search/cdx?url=example.com/page&output=json&fl=timestamp,digest&collapse=digest' | jq -r '.[] | @tsv'

# everything ever captured under a host, which is how you find deleted pages
curl -s 'https://web.archive.org/cdx/search/cdx?url=example.com&matchType=domain&output=json&fl=timestamp,original,statuscode&limit=500' | jq -r '.[] | @tsv'

# only the captures in a window, for a page that mattered on one specific day
curl -s 'https://web.archive.org/cdx/search/cdx?url=example.com/page&from=20260301&to=20260331&output=json&fl=timestamp,digest' | jq -r '.[] | @tsv'

# submit a new capture: SPN2, authenticated with Internet Archive S3-style keys
curl -s -X POST -H 'Accept: application/json' \
     -H "Authorization: LOW $IA_ACCESS_KEY:$IA_SECRET_KEY" \
     -d 'url=https://example.com/page' -d 'capture_all=1' -d 'capture_screenshot=1' \
     'https://web.archive.org/save' | jq .

# poll the job until it reports success, and keep the returned timestamp
curl -s -H "Authorization: LOW $IA_ACCESS_KEY:$IA_SECRET_KEY" \
     "https://web.archive.org/save/status/$JOB_ID" | jq '{status, timestamp, original_url}'

# fetch the archived copy itself, and hash what you got
curl -sL 'https://web.archive.org/web/20260301120000id_/https://example.com/page' | shasum -a 256

Get the key pair from archive.org/account/s3.php and export it as IA_ACCESS_KEY / IA_SECRET_KEY. The unauthenticated GET /save/<url> form is no longer usable for work — an anonymous request to it returns HTTP 429 rather than a capture. Scripts that still rely on it fail silently, because a 429 body looks like a page.

The id_ infix in that last URL asks for the original bytes without Wayback’s navigation banner injected, which is what you want if you intend to hash or diff the archived copy. The digest column in CDX output is the payload hash: two captures sharing a digest are byte-identical, so collapsing on it turns a thousand snapshots into the handful of moments the page actually changed. That is the single most useful thing the CDX API does.

What Wayback does not give you: permanence. A site can retroactively exclude itself, and captures do disappear. It also honours robots directives at crawl time, so absence of a capture is not evidence the page did not exist. And a Save Page Now capture is still just a capture — it attests that the Internet Archive saw that content at that time, which is a strong third-party claim about time and a weak one about authenticity.

archive.today

A separate archive with a separate failure mode, which is exactly why you use it alongside Wayback. It renders pages that Wayback flattens, ignores robots directives, and has a long record of not removing things. There is no API and the submission form sits behind bot protection, so this one is a browser job — do not script it.

1. Open https://archive.today  (archive.ph and archive.is are the same service;
   if one mirror is blocked where you are, try another)
2. Paste the URL into the lower box, "My url is alive and I want to archive its content"
3. Solve the challenge if you get one. Do not automate this step — it is the step
   that gets the service to block your address.
4. Wait for the capture. The result URL looks like https://archive.ph/AbC12
   and the page header shows the capture time in UTC.
5. Click "screenshot" in the header to get the full-page render as a separate
   artefact, and save it. The HTML capture and the screenshot fail differently.
6. Record the short URL AND the capture time. The short URL is your citation.
7. Before submitting, use the upper box to search for existing captures of the
   same URL — the same reason you query Wayback's CDX API first.

Two things to know. The service is deliberately opaque about its infrastructure and funding, which means you should not treat it as your only copy of anything; keep the local WARC. And it fetches the page itself, from its own address, so a submission does not leak your IP to the target — but it does create a public, searchable record that someone archived that URL at that minute.

Hashing and the manifest

Hashes are the whole evidentiary argument. A hash recorded at acquisition and published or timestamped separately lets you show, later, that the file you are producing is the file you took. The manifest pattern — one file listing every hash, then one hash of the manifest — scales that to a collection without timestamping a thousand files.

# macOS ships shasum; coreutils ships sha256sum. Same output format.
shasum -a 256 evidence.mp4
sha256sum evidence.mp4

# hash the whole capture tree in a stable order, so the manifest is reproducible
find ./capture -type f -print0 | sort -z | xargs -0 shasum -a 256 > MANIFEST.sha256

# the manifest's own hash: this single line is what you timestamp and publish
shasum -a 256 MANIFEST.sha256 | tee MANIFEST.sha256.sha256

# verify the collection, any time later
shasum -a 256 -c MANIFEST.sha256

# show only what failed, which is what you actually want on a large set
shasum -a 256 -c MANIFEST.sha256 2>/dev/null | grep -v ': OK$'

# a provenance record that travels with the files, in the same directory
cat > PROVENANCE.txt <<'EOF'
case:        2026-014
url:         https://example.com/post/123
captured_at: 2026-10-04T04:07:56Z   # UTC, from `date -u +%FT%TZ`
captured_by: J. Investigator
tooling:     wget 1.25.0 --warc-file; yt-dlp 2026.08.19
route:       linked from https://example.com/newsroom/ on 2026-10-03
notes:       page required no login; no personalisation observed
EOF

# fold the provenance into the manifest so the two cannot drift apart
shasum -a 256 PROVENANCE.txt >> MANIFEST.sha256

# find duplicates across two collections by hash, not by filename
sort MANIFEST.sha256 other/MANIFEST.sha256 | awk '{print $1}' | sort | uniq -d

Use SHA-256. MD5 and SHA-1 still appear in forensic tooling and are fine as identifiers, but a collision-prone hash invites an argument you do not need to have. Note that find | sort matters: an unsorted manifest changes order between runs and filesystems, which makes two manifests of the same files look different.

What a hash proves and does not: it proves the bytes have not changed since the hash was taken. It says nothing about when they were taken, nothing about where they came from, and nothing about whether the content is true. On its own a hash in your own notes is worth very little, because you could have written it at any time — which is what the next section fixes.

Timestamping: OpenTimestamps and RFC 3161

The step that turns a hash into evidence. A timestamp is a third party’s signed assertion that a given digest existed before a given moment. Without it, your hash only proves internal consistency; with it, you can show you held that exact file on that date and could not have produced it later.

pipx install opentimestamps-client     # the `ots` command
# openssl is already present for the RFC 3161 route

# OpenTimestamps: anchors the digest in the Bitcoin blockchain, free, no account
ots stamp MANIFEST.sha256

# what the proof currently contains: pending calendar attestations, then a block
ots info MANIFEST.sha256.ots

# calendars need an hour or so to get into a block; upgrade the proof afterwards
ots upgrade MANIFEST.sha256.ots

# verify. Full verification wants a local Bitcoin node; without one you are
# trusting the calendar, which is weaker but still a third party
ots verify MANIFEST.sha256.ots

# verify a digest you were given, without holding the file
ots verify -d 355c357708e4840d12b0d4284d9fe1a911c79c013953874f03068b81127f3363 MANIFEST.sha256.ots

# RFC 3161 route: build a timestamp query over the file's SHA-256
openssl ts -query -data MANIFEST.sha256 -sha256 -cert -out manifest.tsq

# send it to a Time Stamping Authority and keep the signed reply
curl -s -H 'Content-Type: application/timestamp-query' \
     --data-binary @manifest.tsq https://freetsa.org/tsr -o manifest.tsr

# read the token: the TSA's time, its identity and the digest it signed over
openssl ts -reply -in manifest.tsr -text

# verify the token against the TSA's chain — this is the check a reviewer repeats
curl -sO https://freetsa.org/files/cacert.pem
curl -sO https://freetsa.org/files/tsa.crt
openssl ts -verify -data MANIFEST.sha256 -in manifest.tsr \
           -CAfile cacert.pem -untrusted tsa.crt

A successful verification prints Verification: OK, and that line is the thing you cite. Keep the .tsr, the CA chain you verified against, and the exact file — the token is signed over the digest, so a reviewer needs the file to recompute it.

Which route to use: OpenTimestamps costs nothing, needs no account, and anchors to a public blockchain, so the proof outlives any single company — but it is coarse (block granularity, roughly an hour) and full verification wants a Bitcoin node. An RFC 3161 TSA gives you a precise, signed, PKI-rooted token that institutional reviewers already recognise, at the price of trusting that TSA and its certificate remaining validatable. Do both; they are cheap and they fail differently.

What a timestamp does not prove: that the content is authentic, that the capture was complete, or that you obtained it lawfully. It proves the file existed no later than that moment. That is a narrow claim, and stating it narrowly is what makes it credible.

Bellingcat Auto Archiver

The pipeline to set up when you are collecting continuously rather than case by case. It reads URLs from a feeder, runs a platform-specific extractor, then runs enrichers that hash, thumbnail, timestamp and push to Wayback, and writes the results to a database you can read as a case index. It is the only tool here that performs the hash-and-timestamp discipline on every item without you remembering to.

pipx install auto-archiver
auto-archiver --help           # the flag list is generated from the installed modules

Configuration is a single YAML file; the default path is secrets/orchestration.yaml. Only modules named under steps are loaded, so the file is both configuration and a declaration of the pipeline.

# orchestration.yaml
steps:
  feeders: [cli_feeder]
  extractors: [generic_extractor]
  enrichers: [hash_enricher, meta_enricher, thumbnail_enricher,
              timestamping_enricher, opentimestamps_enricher]
  databases: [csv_db, console_db]
  storages: [local_storage]
  formatters: [html_formatter]

hash_enricher:
  algorithm: SHA-256          # the only alternative is SHA3-512

timestamping_enricher:
  tsa_urls:
    - http://timestamp.identrust.com
    - http://zeitstempel.dfn.de
  allow_selfsigned: false     # leaving this false is the whole point of the module

local_storage:
  save_to: ./archived
  path_generator: url         # flat | url | random
  filename_generator: static  # names the file after its hash

csv_db:
  csv_file: ./archived/results.csv

logging:
  level: INFO
  file: ./archived/archiver.log

# one URL through the whole pipeline
auto-archiver --config orchestration.yaml 'https://example.com/post/123'

# several at once; the CSV gets one row per URL
auto-archiver --config orchestration.yaml 'https://example.com/a' 'https://example.com/b'

# no config file at all, for a one-off capture with sane defaults
auto-archiver --mode simple 'https://example.com/post/123'

# override the enricher list without editing the file
auto-archiver --config orchestration.yaml \
  --enrichers hash_enricher timestamping_enricher opentimestamps_enricher \
  'https://example.com/post/123'

# push to Wayback as part of the run, with the same Internet Archive key pair
auto-archiver --config orchestration.yaml \
  --enrichers hash_enricher wayback_extractor_enricher \
  --wayback_extractor_enricher.key "$IA_ACCESS_KEY" \
  --wayback_extractor_enricher.secret "$IA_SECRET_KEY" \
  'https://example.com/post/123'

# feed a column of URLs from a spreadsheet export
auto-archiver --config orchestration.yaml \
  --feeders csv_feeder --csv_feeder.files urls.csv --csv_feeder.column link

# turn up logging when an extractor silently returns nothing
auto-archiver --config orchestration.yaml --logging.level DEBUG 'https://example.com/post/123'

# the results CSV is the case index: one row per URL, with hashes and archive links
column -s, -t < ./archived/results.csv | less -S

Two enrichers do the evidentiary work and they are not interchangeable. timestamping_enricher aggregates the item’s file hashes into one text file and gets an RFC 3161 token over it from each TSA in the list; opentimestamps_enricher anchors the same digest via the OpenTimestamps calendars. Note that freetsa.org is deliberately absent from the shipped TSA defaults because its certificate does not validate against system authorities — you can add it, but only together with allow_selfsigned: true, which weakens exactly the property you wanted.

The gsheet_feeder_db module is what makes this a team tool: analysts paste URLs into a Google Sheet, the archiver writes the hash, the stored path and the archive links back into adjacent columns. The wacz_extractor_enricher shells out to Docker for browser-based capture, and wayback_extractor_enricher needs the same key pair as Save Page Now.

Watch the version. The client checks on startup and says so when it is behind, and per-platform extractors break often enough that an old release produces empty captures that look like successes. After every run, scan the results CSV for rows that carry a hash but no media.

ExifTool, for capture-time metadata

Covered in depth on Image & Video Forensics; the archiving use is narrower. You are not asking whether the file is manipulated, you are recording what the file asserted about itself at the moment it entered your custody, so that a later disagreement is about the metadata rather than about your handling.

# the full tag dump, archived next to the hash and never edited again
exiftool -j -g1 -a -u capture/evidence.mp4 > capture/evidence.mp4.exif.json

# the fields that date and attribute a capture, in one line
exiftool -CreateDate -ModifyDate -FileModifyDate -GPSPosition -Make -Model -Software \
         capture/evidence.mp4

# every timestamp in the file, grouped by where it came from
exiftool -time:all -a -G1 -s capture/evidence.mp4

# the whole capture tree as one CSV, which doubles as a collection inventory
exiftool -r -csv -FileName -FileSize -MIMEType -CreateDate -GPSPosition ./capture/ > inventory.csv

# flag files whose structure does not match their declared format
exiftool -r -validate -warning -a ./capture/

# strip metadata from a working copy before you publish, to protect a source
cp capture/evidence.jpg publish/evidence.jpg
exiftool -all= -overwrite_original publish/evidence.jpg
shasum -a 256 publish/evidence.jpg    # a different file, so a different hash: say so

Run this on the copy, after hashing the original, and never with -overwrite_original on anything in the capture tree. Note the last block: a published, stripped file is a different artefact with a different hash, and conflating the two is a straightforward way to be accused of altering evidence. Record both hashes and the relationship between them.

What it does not tell you: metadata is written as easily as it is read, so a helpful CreateDate is consistent-with and never proof. Most files pulled from platforms have had their metadata stripped on upload, and that absence is normal rather than suspicious.

Case-management and pipeline tools

Tool reference

Pitfalls

• Screenshots alone are weak. No metadata, no hash, trivially edited. Capture the file.
• Archives honour robots.txt and takedowns. A site can retroactively remove itself from Wayback. Your local copy is what survives.
• Dynamic content does not archive well. Infinite scroll, lazy loading and interactive maps frequently fail; verify the capture actually shows what you saw.
• Hash after acquisition, not after editing. A hash of a file you already cropped proves nothing about the original.
• Archiving can notify. Submitting a URL to a public archive creates a public record that someone is interested in it.
• A capture that succeeded is not a capture that worked. A WARC full of 403s, a WACZ containing a block page, and a yt-dlp channel pull truncated by rate limiting all exit zero. Read the CDX status column, the crawl log and the row count before you file anything.
• Old scripts hitting GET /save/<url> now get HTTP 429. Anonymous Save Page Now is no longer usable; a 429 body is still a body, so a script that does not check the status silently records a failure as a success. Use the authenticated SPN2 POST.
• A hash in your own notes is nearly worthless. You could have written it at any time. The third-party timestamp is what makes it an assertion about the past rather than about your memory.

Worked example

One datum: a URL to a post carrying a video clip, https://example.com/@account/post/123, cited in a filing you have been asked to check. The URL, the hashes and the timestamps below are invented; the order of the steps and what each one proves are not.

# 1. what already exists, before you touch the page
curl -s 'https://web.archive.org/cdx/search/cdx?url=example.com/@account/post/123&output=json&fl=timestamp,digest,statuscode&collapse=digest' | jq -r '.[] | @tsv'
# timestamp        digest                            statuscode
# 20260228183044   H4XQ...                           200
# 20260302094112   ZK7B...                           200

Two distinct payload digests, two days apart. Collapsing on digest is what revealed that: the page was edited between 28 February and 2 March, which is a finding in itself and the reason to pull both old captures before capturing the live page.

# 2. local capture of the live page, byte-level, with the case in the warcinfo record
mkdir -p case-2026-014 && cd case-2026-014
wget --warc-file=live --warc-cdx --no-warc-compression \
     --warc-header="operator: J. Investigator" \
     --warc-header="description: case-2026-014, post cited at para 17" \
     --page-requisites --adjust-extension --no-parent \
     'https://example.com/@account/post/123'
awk 'NR>1 {print $5, $1}' live.cdx | sort | uniq -c
#  14 200 https://example.com/...
#   3 403 https://cdn.example.com/media/...

Three requisites returned 403, and they are the media files. The wget capture has the page text and not the clip, which is the common outcome and the reason the next two steps exist rather than being optional extras.

# 3. the rendered page, in a browser, because the post body loads via JavaScript
docker run -v "$PWD/crawls:/crawls/" -it webrecorder/browsertrix-crawler crawl \
  --url 'https://example.com/@account/post/123' --scopeType page \
  --generateWACZ --text to-pages --collection case-2026-014
unzip -p crawls/collections/case-2026-014/case-2026-014.wacz datapackage-digest.json | jq .
# { "path": "datapackage.json", "hash": "sha256:9f2c...", "signedData": null }

signedData: null means the WACZ is unsigned — fine, because you are about to timestamp it yourself. The digest over datapackage.json is the crawler’s own integrity check on the archive; record it, then stop relying on it and use your own hash.

# 4. the media, with its provenance sidecar, which the WARC never got
yt-dlp --write-info-json --write-description --write-thumbnail --no-mtime \
       -o '%(upload_date)s_%(uploader_id)s_%(id)s.%(ext)s' \
       'https://example.com/@account/post/123'
jq -r '[.id,.uploader_id,.upload_date,.timestamp,.duration,.view_count] | @tsv' *.info.json
# 123   account   20260226   1772120000   47   18422

The sidecar says the clip was uploaded on 26 February — two days before the earliest Wayback capture of the post, and before the edit the CDX diff exposed. That is the pivot: the video predates the post text it now sits under, so the claim to check is whether the caption was changed, not whether the footage is real.

# 5. hash everything, in a stable order, before any further handling
cd .. && find ./case-2026-014 -type f -print0 | sort -z | xargs -0 shasum -a 256 > MANIFEST.sha256
wc -l MANIFEST.sha256 && shasum -a 256 MANIFEST.sha256
# 31 MANIFEST.sha256
# 4b1c8e... MANIFEST.sha256

# 6. the step that makes the hash mean something: two independent timestamps
ots stamp MANIFEST.sha256
openssl ts -query -data MANIFEST.sha256 -sha256 -cert -out manifest.tsq
curl -s -H 'Content-Type: application/timestamp-query' \
     --data-binary @manifest.tsq https://freetsa.org/tsr -o manifest.tsr
openssl ts -reply -in manifest.tsr -text | grep -E 'Status:|Time stamp:'
# Status: Granted.
# Time stamp: Oct  4 04:06:07 2026 GMT

Now the claim is narrow and defensible: this set of 31 files, with these hashes, existed no later than 04:06 UTC on 4 October 2026, attested by a TSA and by the Bitcoin blockchain, neither of which you control.

# 7. a third-party capture, submitted last so the local copy exists first
curl -s -X POST -H 'Accept: application/json' \
     -H "Authorization: LOW $IA_ACCESS_KEY:$IA_SECRET_KEY" \
     -d 'url=https://example.com/@account/post/123' -d 'capture_all=1' \
     'https://web.archive.org/save' | jq -r '.job_id'
# spn2-9f3c...

Then the same URL by hand into archive.today, and its short URL and UTC capture time into PROVENANCE.txt. Submitting last is deliberate: both submissions are public, so if the account notices and deletes the post, you already hold the capture.

What you can assert: the page as it stood at a timestamped moment, the clip with the platform’s own metadata, and a change history from Wayback showing the post was edited after the video was uploaded. What it does not establish: that the footage shows what the caption claims, or where or when it was filmed. Those are Image & Video Forensics and Geolocation questions, and no amount of archiving substitutes for them.

What would falsify it: any manifest entry that fails verification, a timestamp token whose digest does not match the manifest, or a WARC/WACZ log showing that the claimed resource failed and only an error page was captured. The public archive copy is corroboration; the local capture, hashes and independently verifiable timestamps carry the claim.

Broader catalogues

• Archiving OSINT
• Data Extraction OSINT

More tools

Further tools for this area from the OSINT Newsletter Tools Library (Archiving OSINT), excluding those already listed above.

Sources

Both catalogues below are maintained by other people and are considerably larger than this page. Use them as the canonical index; this sheet is a working route through them.

• Bellingcat’s Online Investigation Toolkit — ~340 tools, each with its own review page covering cost, difficulty, requirements and limitations.
• OSINT Newsletter Tools Library — ~280 tools, organised by investigative goal.

Neither publishes a licence, so nothing here is copied from them: tool names, one-line descriptions, cost flags and links are catalogue facts, and the method and commentary are this site’s own. See credits.