You Don’t Need a Ministry of Truth to Build a Memory Hole

What happens when a thousand independent sources turn out to have one parent?

A while back I went looking for a specific piece of television. A 2012 late-night interview with a sitting president, forty-five minutes long, broadcast on a major network to several million people.

Finding out about it was trivial. An episode database has the record: season, episode number, air date, runtime, and a summary of what was discussed. Wire coverage exists. Clips exist. A national newspaper posted the complete video the following morning, and the URL for that page is still indexed. Contemporary articles quote it. Later articles quote those articles.

Finding the thing itself was considerably harder.

A version of this essay treats that as sinister. This is not that essay. Broadcast rights change hands. Video platforms get retired. Formats go obsolete. Nobody has to intend anything for a forty-five-minute artifact to become difficult to inspect while everything written about it remains a search away.

What interests me is the shape that leaves behind, because I think it is becoming the normal shape of our information environment:

What happens when the source disappears but everything derived from it remains?

The memory hole doesn’t have to be empty

We tend to imagine information loss as absence. A document disappears, a database is deleted, a recording is destroyed. Something that existed no longer does.

Modern information systems produce a stranger failure mode. The original can vanish while its descendants multiply.

Picture a primary artifact that generates ten contemporary news stories. Another hundred articles cite those stories. Wikipedia summarizes several of them. Blog posts cite Wikipedia. Podcasts discuss the blog posts. Social posts quote the podcasts. Years later, AI systems ingest some combination of it all.

The ecosystem now contains thousands of references to an artifact almost nobody can examine. Retrieval works. Search works. There may be enormous agreement about what the original contained. But something has quietly changed underneath all that agreement.

The system hasn’t forgotten the story. It has forgotten how to prove the story.

That leads to the claim this whole essay rests on, so I will state it once, plainly, before dressing it up in examples:

Document count is a terrible proxy for evidentiary independence.

417 sources can’t all be wrong, right?

Suppose an AI system is answering a question about a disputed event and finds 458 relevant documents. Of those, 417 support one interpretation and 41 support another. The tempting conclusion writes itself.

417 > 41

But documents aren’t votes.

Suppose 290 of those 417 ultimately trace back to the same wire-service report. Another 92 descend from the same organizational statement. The remaining 35 cite one another. Meanwhile, the 41 documents supporting the competing interpretation include several independent primary sources.

The interesting question was never how many documents agree. It is how many independent provenance chains support the claim.

The web is exceptionally good at copying information, which is precisely why counting copies tells you so little. Generative AI sharpens the problem, because the final answer collapses hundreds of derivative sources into one confident paragraph. The reader sees consensus without seeing the genealogy that produced it.

This is not a thought experiment, and you can check it yourself in about a minute. Ask an answer engine a general knowledge question and look at what it cites. A crowd-edited encyclopedia will turn up more often than you might expect. That encyclopedia is a tertiary source: a summary of secondary reporting about primary artifacts. When it appears in a citation list, nothing in the interface mentions that the chain already runs three deep before it reaches anything anyone actually witnessed.

Enter the Golden Country Tire Company

Real disputes carry emotional freight, so let’s use tires.

Imagine the Golden Country Tire Company is the world’s largest tire manufacturer. Golden Country has just released its flagship product, the Super-Duper Road Tire. It’s fine—perfectly adequate tire. Golden Country would nevertheless very much like the world’s humans, search engines, and AI systems to regard it as one of the finest achievements in the history of vulcanized rubber. The company and every site named below are invented. The structure is not exotic.

So Golden Country does what any competent marketing organization does. It creates genuinely good content: technical documentation, comparison pages, FAQs, buying guides, structured data, product specifications, expert commentary, and articles answering every question a person might plausibly ask about road tires.

From a GEO and AEO standpoint, Golden Country is doing its job well.

Then Golden Country goes further and funds or controls a collection of apparently independent sites:

RoadTireExperts.example
UltimateDrivingGuide.example
TirePerformanceLab.example
BestRoadTiresToday.example
DefinitelyNotGoldenCountry.example

Each publishes high-quality, well-structured, machine-readable content. Each concludes that the Super-Duper Road Tire is fantastic.

Now ask an answer engine which tires are best for highway driving.

Retrieval surfaces dozens of sources praising the Super-Duper Road Tire. The model isn’t hallucinating. The documents exist. The recommendations exist. The citations exist.

Five sources are not five independent sources if Golden Country is standing behind all five of them.

Golden Country may not have fabricated a single claim. Every individual statement might be technically defensible. What Golden Country manufactured is not a falsehood. It is the appearance of consensus.

An answer engine that understands URLs sees five sources. An answer engine that understands provenance sees one organization speaking through five domain names. Those are very different information environments, and nothing in the retrieval layer distinguishes them.

When the copies start citing one another

The problem gets more interesting once Golden Country’s ecosystem develops internal links.

RoadTireExperts.example publishes a review calling the tire exceptional, citing a braking-distance comparison from TirePerformanceLab.example. The lab article points to a roundup at UltimateDrivingGuide.example. That roundup cites customer-satisfaction figures summarized by BestRoadTiresToday.example, which links back to the original Road Tire Experts review.

From the outside, the provenance graph looks rich:

Four boxes labeled Road Tire Experts, Tire Performance Lab, Ultimate Driving Guide, and Best Road Tires Today. Arrows labeled "cites" run from each to the next, and a final arrow runs from Best Road Tires Today back to Road Tire Experts, closing the chain into a loop. No other elements appear.

Multiple domains. Multiple articles. Multiple authors. Multiple citations. Apparent corroboration throughout.

The graph is a circle. No independent evidence ever entered the system. The sources don’t corroborate one another. They are recursively laundering the same claim.

Here is the same graph with one more fact restored:

The same four sites and the same loop of "cites" arrows as the previous diagram. A fifth box, Golden Country Tire Company, now sits apart from the loop with dashed arrows labeled "controls" running from it to each of the four sites. The four sites still cite only one another; every ownership arrow points inward from the single outside node.

The dashed edges are the only thing that changed, and they are the only thing that matters. They are also the only part of this picture that no retrieval system draws, because nothing in a URL, a byline, a schema block, or a citation announces who funded the page.

This is where counting citations becomes as misleading as counting documents. A densely connected graph can look authoritative while having almost no independent roots. If every path eventually terminates at Golden Country, the graph contains repetition, not corroboration.

None of this is new. Human information ecosystems have always contained circular citation, press-release recycling, unattributed copying, and claims that gain acceptance through sheer repetition.

What changes with generative AI is the economics. Another plausible article is cheap. Another plausible site is nearly as cheap. Rephrasing a claim so it reads as linguistically independent is cheap. Producing structured, answer-friendly content at volume is cheap.

Apparent consensus can now grow much faster than independent evidence.

I could give you a number here. A widely circulated figure estimates how much of the newly published web is now AI-generated, and I have seen it quoted in a dozen places this month. I went looking for where it came from. The first article cited a second article. The second cited a marketing blog. The marketing blog cited a crawl study, described but not linked. I gave up at the fourth hop, which is either a failure of diligence on my part or the entire thesis of this essay demonstrating itself at my expense. Possibly both.

So take the number as read, and notice instead that I cannot show you its parents.

And synthetic content no longer needs to copy the original wording. Fifty pages can express the same unsupported claim fifty different ways. Textual similarity becomes a weaker signal of shared ancestry, even though the underlying provenance hasn’t changed at all.

The result is an information environment optimized beautifully for retrieval and architecturally terrible for verification.

But doesn’t somebody catch this?

The reasonable objection is that platforms already police this. They do, and they do it reasonably well. Search engines have spent years developing policies against scaled content abuse and coordinated networks built to manipulate rankings, and enforcement actions have removed entire sites from indexes.

Notice what those policies target: low-quality content produced at volume, and thin content built to game a ranking. That is the crude version of Golden Country, and the crude version does get caught.

Our Golden Country doesn’t do that. Its technical documentation is accurate. Its comparison pages are useful. Its specifications are correct. Its structured data is well-formed. Every site in the network would survive a quality review on its own merits, because every site deserves to.

Golden Country is not violating the spam policy. It is following the content marketing playbook competently, five times, from five domains it happens to own. The enforcement regime was built to detect garbage, and Golden Country isn’t producing garbage. It is producing a well-made monoculture.

That deserves a name, because it will keep happening. Call it a provenance monoculture: an information environment that is diverse in sources, formats, and domains, and uniform in origin. Nothing in it is false. Nothing in it is thin. Everything in it grew from the same root.

That is the gap. Quality enforcement and independence verification are different problems, and we currently have infrastructure for one.

Nobody needs a pneumatic tube to the furnace

In 1984, controlling history requires destroying evidence. Winston Smith rewrites the record and the original goes down the memory hole.

Our systems don’t require anything that dramatic. A primary source becomes gradually inaccessible. Links rot. Licensing changes. Platforms retire. Archives migrate. Formats go obsolete. Meanwhile the derivative material stays exactly where it is, and new material keeps accumulating around one interpretation of the missing source.

Nothing has to be deleted on purpose. Nothing has to be centrally coordinated. The environment simply becomes asymmetric, and the systems grounded on that environment inherit the asymmetry.

A modern memory hole is surrounded by more information than ever.

The economics of the hole

GEO and AEO are usually discussed as marketing disciplines: make your organization, product, expertise, or terminology retrievable and comprehensible to answer engines. That is legitimate work, and good technical content should be understandable by humans, search engines, and answer engines alike.

The problem starts when information availability gets confused with independent corroboration.

When a primary source is missing, something determines which secondary representation becomes its machine-readable substitute. An organization with sufficient resources can produce a large body of coherent, optimized material around its preferred representation of reality. It doesn’t need to falsify anything. It only needs to become disproportionately represented in the environment from which answers get assembled.

The Super-Duper Road Tire doesn’t become better. It becomes better represented.

If answer engines treat frequency as confidence, domain count as independence, citation density as authority, or repetition as corroboration, then the organizations best equipped to populate the environment gain an advantage with no relationship whatsoever to the quality of their evidence.

There is a further wrinkle, and it’s also easy to test. Put the same question to two different answer engines and compare the lists of sources underneath. The answers will often agree. The evidence behind them frequently does not overlap much at all. Whatever consensus a reader perceives is partly an artifact of which pipe they happened to ask.

When the source is missing, say so

This is where provenance stops being an archival concern and becomes part of memory architecture.

A trustworthy system should distinguish among primary evidence, independent corroboration, derivative reporting, organizational claims, unknown provenance, and unavailable primary sources. Those are not equivalent categories of knowledge, and collapsing them is a design decision, not a technical necessity.

If 417 documents descend from three sources, the system should know that. If five apparently independent tire sites belong to Golden Country, the system should know that too. If a citation graph contains no independent evidentiary root, the number of edges in the graph should not manufacture authority.

And if a source once existed but can no longer be examined, the system should preserve that fact rather than silently filling the gap with the statistical weight of everything surrounding it. Absence is itself a provenance category. It is a thing worth recording, not a hole to be smoothed over.

A provenance-aware system might answer like this:

Multiple secondary sources report this claim, but the primary artifact they reference is unavailable. Several of those sources also derive from the same upstream reporting, so they should not be treated as independent corroboration.

That is not a weaker answer. It is a more honest one, and honest answers are the only kind worth building infrastructure for.

Information without provenance is just gossip

Memory is not simply the ability to preserve information. Trustworthy memory preserves the relationship between information and its origins.

Who created this? What evidence supported it? Was the source primary or derivative? Was it independent? Can that authority still be verified? Does this source depend on another that no longer exists? Are apparently independent sources controlled by the same organization? Does the citation graph lead outward to evidence, or eventually curl back onto itself?

Without those relationships, a system can accumulate an extraordinary volume of knowledge while gradually losing the ability to explain why any of it should be believed. That isn’t memory. That’s a very well-indexed rumor mill.

Orwell imagined that controlling history required destroying the evidence. Our problem is subtler and considerably cheaper. We can preserve enormous quantities of information while losing the provenance required to evaluate it, and we can surround a missing source with so many summaries, restatements, and synthetic corroborations that the absence itself becomes invisible.

Increasingly, machines stand between that environment and the person asking the question. So the question is no longer whether a system can find an answer. It is whether the system can tell a thousand independent witnesses from one witness repeated a thousand times.

Because when a source falls into a memory hole, something always fills the space around it. Information without provenance is just gossip, and gossip scales beautifully.


An open question, and I mean it as one.

I have described a problem and stopped short of a fix, because I am not sure what the first move is. Disclosure obligations for funded networks? An independence signal carried alongside citations? Answer engines surfacing shared upstream sources when they detect them? Something else entirely, or nothing, because the incentives point the other way?

If you build retrieval systems, work in trust and safety, or just have a view: what would a first step actually look like, and who is positioned to take it?

Facebooktwitterredditlinkedinmail

The New Information Borders

Recently I came across a discussion about AI crawlers and robots.txt files. The conversation centered on a simple question:

Should website owners allow AI systems to access their content?

One proposed configuration looked something like this:

User-agent: ClaudeBot
Allow: /

User-agent: GPTBot
Disallow: /

User-agent: ChatGPT-User
Disallow: /

User-agent: PerplexityBot
Disallow: /

At first glance this is a reasonable policy decision.

Perhaps a company has a commercial relationship with one AI vendor and not another. Perhaps it trusts one organization more than another. Perhaps it simply dislikes a particular company and would rather that company not benefit from its content.

These are all rational decisions. And worth remembering: robots.txt is a request, not a wall. It governs the crawlers that choose to honor it. The borders we are about to talk about form through compliance norms and licensing agreements, not through technical enforcement.

The interesting part is what happens when thousands of organizations make similar decisions at once.

The Web We Assumed

For most of the modern Internet era, there was an implicit assumption that people were operating from a broadly shared information environment.

Search engines differed in quality. Ranking algorithms differed. Some sources were easier to discover than others. But in general, if two people searched for information on a topic, there was a good chance they were drawing from many of the same underlying sources.

The web functioned as a largely shared corpus of knowledge.

That assumption may not hold forever.

Fragmentation Without Malice

When people discuss information fragmentation, they often jump straight to government censorship, national firewalls, or deliberate propaganda systems.

Those are real examples. But fragmentation does not require any malicious intent.

Imagine the following:

  • Company A blocks OpenAI but allows Anthropic.
  • Company B licenses content exclusively to OpenAI.
  • Company C blocks all AI crawlers.
  • Company D optimizes specifically for one AI platform.
  • Company E maintains a private agreement with a commercial search provider.

None of these organizations is trying to create information silos. Each is making what looks like a reasonable local decision.

Collectively, those decisions begin to produce different information environments. The divergence does not emerge from AI reasoning. It emerges from AI access.

None of these organizations is trying to create information silos. They are simply trying to protect their intellectual property or negotiate a survival-level licensing deal in an ecosystem that no longer sends them traffic. Each is making what looks like a reasonable local decision.

Two Kinds of Access

It helps to separate two things that fragment differently.

The first is what a model was trained on. The second is what a model can reach at the moment you ask it a question.

Today these overlap heavily. Most large models are built from many of the same underlying sources: the same crawled archives, the same bulk licensing deals, the same public web that has been scraped for years. At the training layer, the corpus is still mostly shared.

Retrieval is where the divergence is already happening.

When a model answers using live access to the web, the robots.txt rules, the licensing agreements, and the private deals all decide what it is permitted to pull in right then. One system can cite a source. Another is told it may not look. Same question, different evidence, and the difference has nothing to do with how either model reasons.

So the honest version of the claim is not that Claude and ChatGPT already see two different webs. It is narrower and more defensible:

Retrieval access is fragmenting now. Training access could follow.

That second part is the one worth watching. If exclusive licensing becomes the norm rather than the exception, the divergence stops being a retrieval-time quirk and starts being baked into what each model knows at all. The shared corpus we have taken for granted would quietly stop being shared.

The Difference Between Thinking and Seeing

When two AI systems produce different answers, we tend to assume the difference lies in how the models reason.

Sometimes that is true. Increasingly, though, the more important question may be a different one: what information was the model allowed to see?

An answer generated from complete evidence and an answer generated from partial evidence can both arrive with equal confidence. Only one of them may reflect the full record.

The distinction matters.

A model cannot mourn the data it was never allowed to read. It simply synthesizes a flawless, highly confident answer out of the fragment it has, leaving the user entirely unaware of the missing horizon.

Museums Learned This Long Ago

One reason I spend so much time thinking about provenance is that museums, archives, and historians have wrestled with these questions for decades.

Researchers care not only about what artifacts exist. They care about what artifacts are missing. Absence affects interpretation. A collection missing half of its records tells a different story than a complete one, and a careful researcher never mistakes the surviving fragment for the whole.

AI systems face the same challenge. A model can only reason from the evidence available to it. If the evidence becomes fragmented, the resulting interpretations may diverge even when the underlying reasoning processes remain sound.

The Sovereign Systems Perspective

The Sovereign Systems Specification is built around a simple observation:

Information without provenance is just gossip.

Most discussions of provenance focus on where information came from. The harder and more neglected question is what was left out.

Not only:

Where did this information originate?

But also:

What information was unavailable?

What information was excluded?

What information was never allowed into the system at all?

Absence is itself a provenance category. A record of what a system could not see is as much a part of its lineage as a record of what it could. Those questions become more important, not less, as AI systems become primary interfaces to knowledge.

While commercial cloud models hide their data deficits behind a smooth conversational curtain, a Sovereign system must explicitly map its own borders—declaring exactly what lies within its registry, and where the boundary of its knowledge ends.

The New Information Borders

I do not believe AI is creating separate realities. We are.

Not through any coordinated effort. We are simply making thousands of local decisions about access, licensing, trust, governance, and control.

The cumulative effect may be the emergence of informational borders that are far less visible than national borders but no less consequential.

So here is the thing to watch for. The next time two AI systems hand you different answers, do not stop at asking which one reasoned better. Ask what each one was allowed to see. The gap between them may have nothing to do with intelligence and everything to do with access.

The web once assumed a largely shared corpus of knowledge. The next generation of knowledge systems may not.

When two AI systems disagree, are we observing different reasoning? Or are we observing different worlds?

Facebooktwitterredditlinkedinmail