Open Infrastructure for the Independent Web
A decentralised, agent-generated, privacy-preserving directory of every meaningful domain on the independent web — scoped sensibly, resistant to SEO by construction, and runnable by anyone.
The dominant search engines are centralised, surveillance-dependent, and progressively poisoned by SEO optimisation. The independent web — the tools, blogs, niche references, and personal projects that make the web worth having — has become effectively invisible.
Alternatives exist, but they are architecturally identical: still centralised, still operated by a single entity, still dependent on observed queries as a data product.
The missing piece is a discovery layer — one that is decentralised, privacy-preserving by architecture rather than policy, and scoped to the problem it actually needs to solve.
TLD registry zone files provide the complete seed list. No crawl required for initial domain inventory — the namespace is already documented. New registrations appear in zone file diffs automatically.
An agent makes a minimal pass on each live domain: resolve, fetch root, extract title, description, and headings. The output is a structured card describing what the domain is. It cannot be gamed — the site owner didn't write it.
With the index held locally or synced peer-to-peer, queries are entirely local computation. A query never leaves your device. No intermediary sees what you searched for. Surveillance disappears as an architectural consequence.
Maintenance is event-driven, not a global re-crawl. Zone file diffs queue new registrations. Query-time 404s flag domains for recheck. Dead domains drop out naturally. The index stabilises over time.
The combinatorial explosion of web content comes from recursion within domains. Meaningful distinct domains number in the tens of millions — a bounded, tractable problem. The discovery layer and retrieval layer are separate concerns.
ScopeSurveillance-based search depends on the query being observed by a centralised intermediary. Remove the intermediary and the query disappears as a data product entirely — not by policy, but by construction.
PrivacyThe metadata being ranked against was generated by an agent reading the site, not written by the site owner to influence ranking. Keyword stuffing, backlink schemes, and SEO content are irrelevant — none of it is in the index.
IntegrityBecause the indexing methodology is a replicable process rather than a proprietary database, the index cannot be captured or killed. No foundation to defund, no repository to take down. Anyone can reproduce a functionally identical index.
ResilienceRegest's usefulness depends on knowing what it isn't for. Platforms already have their own discovery systems. Regest covers the long tail that falls outside them.
"It is not a search engine. It is the directory layer that should have existed alongside DNS from the beginning — a phone book for the web that is local, accurate, and nobody's product."— Regest Project Document
Regest is currently a protocol specification, not yet a running system. The design phase is complete. Implementation tooling is underway.
The goal is that anyone should be able to run a full indexing pass independently, produce a compatible index, and share or sync it with others without coordination.
Human curation adds a quality tier on top of the automated baseline — but is not required. The automated agent alone is sufficient to produce a useful and trustworthy index.