Websites, Domains & Hosting Infrastructure

What this covers

Working out who runs a website and what else they run. Passive attribution — historical records, certificate logs, archives — gets you most of the way without ever touching the target. This overlaps with Enumeration, but the goal here is attribution rather than attack surface.

Method

1. Historical WHOIS first. Current records are almost always privacy-shielded; records from before the shield often are not.
2. Passive DNS shows which IPs the domain used over time, and which other domains used those IPs.
3. Certificate transparency logs enumerate subdomains for free, without touching the target, and often expose staging and internal hostnames.
4. Archives show what the site used to say, including contact details and staff pages since removed.
5. Fingerprint the page. Analytics IDs, ad IDs, favicon hashes and CMS quirks link sites that share an operator.
6. Check what shares the host. Shared hosting is meaningless; a dedicated IP hosting five related domains is not.

Page fingerprinting

Linking two sites to one operator comes down to finding a value that is identical in both and that nobody would share by accident. The signals are not equal:

Searching an ID string in PublicWWW or Grep.app finds the other pages that embed it; urlscan.io finds the sites whose scans requested it, which catches IDs injected at runtime rather than written into the HTML.

Key tools

RDAP and whois

RDAP is the structured replacement for whois, and the one to reach for first: JSON instead of free text, consistent field names across registries, and no scraping. whois is still worth running because some registries put things in the free-text blob that never make it into RDAP.

# RDAP, following the bootstrap redirect to whichever registry is authoritative
curl -s -L 'https://rdap.org/domain/example.com' | jq '{ldhName,status,events,entities}'

# the registration and expiry dates on their own
curl -s -L 'https://rdap.org/domain/example.com' \
  | jq -r '.events[] | "\(.eventAction)  \(.eventDate)"'

# nameserver history is not here, but the current delegation is
curl -s -L 'https://rdap.org/domain/example.com' | jq -r '.nameservers[].ldhName'

# straight to the registry when you know it, which is faster and never rate-limited by a proxy
curl -s 'https://rdap.verisign.com/com/v1/domain/example.com' | jq '.events'

# an IP or a netblock: who holds the allocation, and when it was assigned
curl -s -L 'https://rdap.org/ip/93.184.216.34' | jq '{handle,name,country,events}'

# an ASN, for working out whose network a host actually sits on
curl -s -L 'https://rdap.org/autnum/15169' | jq '{handle,name,entities}'

# the old tool, for the fields RDAP drops
whois example.com | grep -iE 'registrar|created|updated|expir|name server|status'

Current records are privacy-shielded almost everywhere, so RDAP mostly gives you dates, status codes and the registrar — all useful, none of them a name. The creation date is the one field worth trusting: it is set once and does not change, which makes it the backbone of any timeline. Registrar-level and ccTLD RDAP coverage is patchy; a 404 from rdap.org means no server is registered for that zone, not that the domain is free.

dig and passive DNS

Live DNS tells you the present; passive DNS — a third party’s historical record of what resolved to what — tells you the past, which is where attribution lives. Run both: the live records name the vendors in use, and the history names the host before the CDN went up.

# the basics, worth doing as a set rather than one at a time
for r in A AAAA MX NS TXT SOA CNAME; do echo "== $r"; dig +short example.com "$r"; done

# mail and verification records enumerate the SaaS vendors the operator uses
dig +short TXT example.com
dig +short TXT _dmarc.example.com
dig +short TXT _domainkey.example.com

# ask the authoritative server directly, bypassing your resolver's cache
dig @"$(dig +short NS example.com | head -1)" example.com ANY

# reverse lookup on the IP, which occasionally names the hosting account
dig +short -x 93.184.216.34

# the full delegation chain, for spotting a subdomain handed to a third party
dig +trace staging.example.com | tail -20

# a zone transfer, which should fail — when it does not, you have the whole zone
dig @ns1.example.com example.com AXFR

dig +short NS plus AXFR is the one active step here; everything else above only queries resolvers. For history, DNS History and Whoxy are the usual free-tier sources, and SecurityTrails has the deepest free passive-DNS allowance, metered per month. A shared IP in the history is worthless; a dedicated IP that three domains shared for two years is a finding.

crt.sh

Certificate transparency logs are public, complete and free, which makes them the best subdomain enumerator in existence — and it never touches the target. crt.sh is the searchable front end, and its JSON output is the part to automate.

# every name ever certificated under the domain, deduplicated
curl -s 'https://crt.sh/?q=%25.example.com&output=json' \
  | jq -r '.[].name_value' | tr ' ' '\n' | sed 's/^\*\.//' | sort -u

# with the issuance timeline, which dates when each host appeared
curl -s 'https://crt.sh/?q=example.com&output=json' \
  | jq -r '.[] | "\(.not_before)  \(.name_value)"' | sort -u | head -40

# who issues their certificates — a change of CA often marks a change of operator
curl -s 'https://crt.sh/?q=%25.example.com&output=json' \
  | jq -r '.[].issuer_name' | sort | uniq -c | sort -rn

# search by organisation name instead of domain, to find their other estates
curl -s 'https://crt.sh/?q=Example+Organisation+Ltd&output=json' | jq -r '.[].common_name' | sort -u

# one certificate's full detail by crt.sh id
curl -s 'https://crt.sh/?id=29510272303&output=json' | jq '{issuer_name,not_before,not_after,serial_number}'

# only names that still resolve, so you can tell live infrastructure from history
curl -s 'https://crt.sh/?q=%25.example.com&output=json' | jq -r '.[].name_value' \
  | tr ' ' '\n' | sed 's/^\*\.//' | sort -u \
  | while read -r h; do [ -n "$(dig +short "$h")" ] && echo "$h"; done

crt.sh is run by Sectigo as a public service and is frequently overloaded — a 502 on the web UI is normal and usually clears, and the JSON endpoint stays up more often than the HTML one does. When it is down, subfinder pulls the same logs through other indexes. A certificate proves a name was requested, not that a host ever answered on it: staging and internal hostnames show up here precisely because nobody expected them to be published.

subfinder

Passive subdomain enumeration across several dozen indexes at once — certificate logs, passive DNS providers, search engines, threat-intel feeds. Faster and broader than querying crt.sh alone, and it needs no keys to be useful, though adding free ones roughly doubles what it finds.

# install: a single Go binary
go install -v github.com/projectdiscovery/subfinder/v2/cmd/subfinder@latest

# the default run: passive only, every keyless source
subfinder -d example.com -silent

# which sources exist, and which ones your keys unlock
subfinder -ls

# all sources including the slow ones, written to a file
subfinder -d example.com -all -o subs.txt

# JSON lines with the source recorded per host, so a finding is attributable
subfinder -d example.com -oJ -cs -o subs.jsonl

# several root domains in one run
subfinder -dL roots.txt -oD ./out -silent

# only hosts that actually resolve, with their IPs
subfinder -d example.com -nW -oI -silent

# throttle a chatty provider rather than getting your free key banned
subfinder -d example.com -rls 'crtsh=1/s' -rl 10 -silent

Free API keys go in the provider config under your platform’s config directory; subfinder -ls shows which sources are active. Passive results are historical, so a long list includes hosts retired years ago — pass -nW when you need the live set. Sources disagree and some return wildcard noise, so a single-source hit deserves -cs output and a second look before you build on it.

urlscan.io

A hosted browser that visits a page for you, records every request it made, and keeps the result in a searchable archive. That gives you two separate capabilities: look at a hostile page without touching it yourself, and search what other people’s scans loaded to find sites sharing an unusual third-party request.

# search the archive without visiting anything — no key needed for modest volume
curl -s 'https://urlscan.io/api/v1/search/?q=domain%3Aexample.com&size=20' \
  | jq -r '.results[] | "\(.task.time)  \(.page.url)"'

# every scan whose requested URLs carried a specific analytics ID
curl -s -G 'https://urlscan.io/api/v1/search/' \
  --data-urlencode 'q=filename:"UA-12345678"' --data-urlencode 'size=100' | jq '.total'

# scans by the IP a page resolved to, for finding co-hosted estates
curl -s 'https://urlscan.io/api/v1/search/?q=page.ip%3A93.184.216.34&size=50' \
  | jq -r '.results[].page.domain' | sort -u

# ASN-wide search, which is how you sweep a bulletproof host
curl -s 'https://urlscan.io/api/v1/search/?q=page.asn%3AAS15169&size=100' | jq '.total'

# submit a scan — the header name is API-Key, and nothing else works
curl -s -X POST 'https://urlscan.io/api/v1/scan/' \
  -H "API-Key: $URLSCAN_KEY" -H 'Content-Type: application/json' \
  -d '{"url":"https://example.com","visibility":"unlisted"}' | jq -r '.uuid'

# then read the result a minute later: requests, certificates, redirect chain, screenshot
curl -s 'https://urlscan.io/api/v1/result/UUID/' \
  | jq '{page:.page, redirects:[.data.requests[].request.redirectResponse.url]}'

# your remaining quota, which is the only reliable statement of your limits
curl -s 'https://urlscan.io/user/quotas/' -H "API-Key: $URLSCAN_KEY"

visibility is the decision that matters. A public scan is visible to everyone immediately, including your target, who may well be watching the archive for their own domains; unlisted keeps it out of search but still reachable by URL. Unauthenticated search quotas are small and reset on the full minute and hour, so batch your queries. A scan is one browser, from a datacentre IP, in one country — a page that cloaks on geography or user agent will show you something the target’s visitors never see.

Shodan

The index of what is listening on the internet, and the only tool here that can answer “what else serves this exact favicon”. Its value in attribution work is the pivot: a hash, a certificate serial or an HTTP header becomes a list of hosts.

# install and register the key once
pipx install shodan
shodan init YOUR_API_KEY

# free-tier reality check: how many results a query has, before spending credits on it
shodan count 'http.favicon.hash:-247388890'

# the pivot that matters — every host serving the same favicon
shodan search --fields ip_str,port,hostnames,org 'http.favicon.hash:-247388890'

# everything known about one address, including historical ports
shodan host 93.184.216.34

# hosts sharing a TLS certificate serial, i.e. the same operator
shodan search --fields ip_str,hostnames 'ssl.cert.serial:"73d94ae1858e6f4613fd389a8efe5267"'

# a distinctive response header or body string, which copy-pasted templates carry
shodan search --fields ip_str,http.title 'http.html:"Example Org internal portal"'

# facet counts, for describing an estate rather than listing it
shodan stats --facets country:10,org:10 'ssl.cert.subject.CN:*.example.com'

# the subdomains and DNS records Shodan has seen for a domain
shodan domain example.com

# bulk download then parse offline, so you pay for the query once
shodan download results.json.gz 'http.favicon.hash:-247388890'
shodan parse --fields ip_str,port,org --separator , results.json.gz

Search filters need a paid or academic key; a free account can look up hosts but not run filtered queries, so budget for that before planning a workflow around it. Installed under Python 3.12 the CLI can fail with ModuleNotFoundError: pkg_resources — install setuptools into the same environment. Shodan’s data is a crawl with a lag of days to weeks, so a result describes what was listening at scan time, and hostnames are reverse-DNS, not proof the site is served there.

httpx

Probes a list of hosts and reports what answered, with the fingerprints you need for linking sites in one pass: favicon hash, JARM, title, technology, body hash. The natural next step after subfinder, and the way you generate the favicon hash that Shodan then searches.

# install: ProjectDiscovery's httpx, NOT the Python HTTP library of the same name
go install -v github.com/projectdiscovery/httpx/cmd/httpx@latest

# what is actually live, with status and title
subfinder -d example.com -silent | httpx -silent -sc -title

# the fingerprint set, as JSON you can diff between estates
httpx -l subs.txt -json -favicon -jarm -tech-detect -title -web-server -o fingerprints.jsonl

# the favicon hash on its own — this is the number you paste into Shodan
httpx -u https://example.com -favicon -silent

# group hosts by response-body hash to find the ones serving identical pages
httpx -l subs.txt -hash mmh3 -silent | sort -k2 | uniq -c -f1 | sort -rn | head

# whether a CDN is hiding the origin, and whose network it is
httpx -l subs.txt -silent -cdn -ip -asn

# screenshots, for the record rather than the analysis
httpx -l subs.txt -screenshot -silent -o shots.txt

# slow it down: this is the one tool on this page that touches the target
httpx -l subs.txt -rate-limit 5 -silent -sc

This is active: every probe is a request from your address to their server, logged at their end. Everything above it on this page is passive, so do the passive work first and know what you are looking for before you knock. A favicon hash is only a link when the favicon is distinctive — thousands of hosts share the default WordPress and cPanel icons, so check the hash’s global count in Shodan before treating a match as meaningful.

Wayback Machine CDX API

The archive’s index, and far more useful than the calendar view: it lists every capture of a URL pattern with timestamp, status code and content digest, which turns “what did this site used to say” into a diffable dataset.

CDX='https://web.archive.org/cdx/search/cdx'

# every distinct capture of a path, collapsed so identical content appears once
curl -s -G "$CDX" --data-urlencode 'url=example.com/contact*' \
  --data-urlencode 'output=json' --data-urlencode 'collapse=digest'

# the first and last capture, i.e. when the site appeared and when it stopped
curl -s -G "$CDX" --data-urlencode 'url=example.com' --data-urlencode 'output=json' \
  --data-urlencode 'limit=1'
curl -s -G "$CDX" --data-urlencode 'url=example.com' --data-urlencode 'output=json' \
  --data-urlencode 'limit=-1'

# every URL ever archived under the domain — the archive as a site map
curl -s -G "$CDX" --data-urlencode 'url=example.com/*' --data-urlencode 'output=json' \
  --data-urlencode 'fl=original' --data-urlencode 'collapse=urlkey' | jq -r '.[1:][] | .[0]'

# only captures that returned 200, within a date window
curl -s -G "$CDX" --data-urlencode 'url=example.com/*' --data-urlencode 'output=json' \
  --data-urlencode 'filter=statuscode:200' --data-urlencode 'from=2018' --data-urlencode 'to=2020'

# pull one archived page as it was served, without the archive's own toolbar
curl -s 'https://web.archive.org/web/20190101120000id_/http://example.com/contact'

# diff two captures of the staff page, which is where removed names live
for ts in 20180301000000 20210301000000; do
  curl -s "https://web.archive.org/web/${ts}id_/http://example.com/team" > "team-$ts.html"
done
diff team-20180301000000.html team-20210301000000.html | head -40

Use https — the http form of the CDX endpoint returns nothing rather than redirecting. The id_ suffix on a snapshot URL serves the original bytes, which is what you want for forensics and for hashing. The archive honours removal requests and robots.txt retroactively in places, so absence is not evidence of absence; archive.today catches things Wayback missed and is harder to have taken down. See archiving and evidence for making your own captures stand up.

The Information Laundromat

Web only, free, and purpose-built for the question the rest of this page answers one signal at a time: are these sites run by the same people. Paste in a URL or a block of content and it compares both the text and the technical indicators across its index.

Use it at informationlaundromat.com — note the hyphenated spelling of the domain no longer resolves. Two modes matter:

Content similarity   paste an article; it finds the other sites republishing the same text,
                     which is how a syndication network becomes visible
Metadata similarity  give it a domain; it compares analytics IDs, ad IDs, CDN and registrar
                     fingerprints against its index and scores the overlap
Read the result as:  which indicator matched, not the headline score — a shared Google
                     Analytics ID is near-conclusive, a shared Cloudflare nameserver is noise
Capture:             the matched indicator, its value, and the date you ran it

The scoring blends strong and weak indicators, so a high score built entirely on shared hosting means nothing. Open the per-indicator breakdown every time, and confirm the strong ones by hand — a claimed analytics ID should be visible in the page source or in a PublicWWW or Grep.app search before you rely on it.

Tools that have moved since this area settled

• Censys is retiring Legacy Search (search.censys.io) and its v1/v2 APIs through 2026, in favour of the Platform API at https://api.platform.censys.io/v3/global/. Scripts written against /api/v2/hosts/... need rewriting; certificate observation endpoints have already dropped from realtime to a six-hourly refresh.
• Amass is at v5 and no longer works the way most cheatsheets describe. Passive is now the default, so -passive is a deprecated no-op and -active is the flag that adds zone transfers and certificate grabs. Results land in its own asset database, read back with amass subs and amass viz rather than printed by amass enum. For plain passive subdomain lists, subfinder is the simpler tool.
• DNSDumpster now requires an account for anything beyond a handful of lookups a day. The keyless equivalents are crt.sh and subfinder.

Infrastructure tooling at a glance

Tool reference

Pitfalls

• Privacy shields hide almost everything current. History is where the answer is.
• Shared hosting proves nothing. Thousands of unrelated domains share a CDN IP. Only a dedicated or small shared host is meaningful.
• CDNs hide the origin. Cloudflare in front of a site means the IP you see is Cloudflare’s.
• Active scanning is not passive. nmap against the target leaves logs. Urlscan and CT logs do not.
• Archives have gaps and honour exclusions. Absence from Wayback is not absence from the web.

Worked example

One datum: the domain example-news-daily.com, which published a story you are checking. The question is who runs it and what else they run.

1. Dates first. curl -s -L 'https://rdap.org/domain/example-news-daily.com' gives a registration event in March 2024 and a privacy-shielded registrant. A site presenting itself as a long-running local paper, registered eighteen months ago, is already a finding.
2. Names it has had. crt.sh for %.example-news-daily.com returns six hostnames, including old. and wp. subdomains and — more usefully — a certificate from 2024 that also covers example-city-times.com. One certificate across two brands is a strong link.
3. Broaden the host list. subfinder -d example-news-daily.com -all -oJ -cs adds four names crt.sh missed and records which source found each, so you can cite them individually.
4. See what is live without being seen. A urlscan archive search for domain:example-news-daily.com returns existing public scans, including the third-party requests the page makes. One is a Google Analytics beacon carrying a property ID.
5. Pivot on the ID. That ID searched in PublicWWW returns eleven other domains embedding it, and the urlscan query filename:"UA-…" returns nine of the same. Overlapping lists from two independent indexes is what makes the network claim defensible.
6. Confirm the shared template. httpx -l hosts.txt -json -favicon -jarm -title shows the same favicon hash across the whole set. That hash in shodan count returns 14 hosts globally — small enough that the match means something, which a default-WordPress icon would not.
7. Read the history. The Wayback CDX index for example-news-daily.com/* shows the earliest capture in April 2024 and an /about page archived in May 2024 naming two editors; the live /about names neither. diff of the two captures is the record.
8. Check the network, not just the pair. The Information Laundromat’s metadata comparison on the domain scores the same cluster and surfaces two more domains that share the ad ID but not the analytics ID — a wider ring, confirmed on a second indicator.

What you can assert: a domain registered in March 2024, sharing a certificate, an analytics property, an ad publisher ID and a distinctive favicon with eleven other sites, with two editor names removed from its own about page in 2024 and preserved in the archive. What you cannot: who those people are. Nothing on this page crosses from infrastructure to identity — that pivot runs through people search and email and phone work.

What would falsify it: a fresh page capture showing that the analytics or ad identifier was injected by a shared third-party template rather than by the operator; certificate history showing that the cross-domain certificate was issued to a hosting platform for unrelated customers; or a global favicon count large enough to make the hash commonplace. The load-bearing measurements are the account-linked analytics and ad identifiers. The shared IP and favicon are corroboration, not identity evidence, and must be discarded if either is common outside the eleven-site cluster.

Broader catalogues

• Domain Name OSINT
• Network Infrastructure OSINT
• Cyberthreat Intelligence OSINT

More tools

Further tools for this area from the OSINT Newsletter Tools Library (Domain Name OSINT, Network Infrastructure OSINT, Cyberthreat Intelligence OSINT), excluding those already listed above.

Sources

Both catalogues below are maintained by other people and are considerably larger than this page. Use them as the canonical index; this sheet is a working route through them.

• Bellingcat’s Online Investigation Toolkit — ~340 tools, each with its own review page covering cost, difficulty, requirements and limitations.
• OSINT Newsletter Tools Library — ~280 tools, organised by investigative goal.

Neither publishes a licence, so nothing here is copied from them: tool names, one-line descriptions, cost flags and links are catalogue facts, and the method and commentary are this site’s own. See credits.