How far does a retraction reach? Tracing the science built on a retracted paper with a graph

In 2006, Nature published a paper on Alzheimer's disease. It said that one specific assembly of amyloid-β in the brain impairs memory. Eighteen years later, in 2024, it was retracted.

Today, 2,768 papers cite it.

Some of them just mention it in passing. Others built their own work on top of it: they used its method, its data or its result. And some papers built on those papers. When a paper is retracted, which of the papers downstream are standing on shaky ground?

Retraction alerts don't answer that. Zotero warns you about a retracted paper in your own library, and scite's Reference Check flags retracted references in a manuscript, but they all stop one step away, and they treat a passing mention the same as a foundation. That's the part that matters: a 2025 study of citation networks found that retracted papers more prominently affect the papers that cite them indirectly. I built Shaky Ground during Graph Hacks (WeMakeDevs × FalkorDB) to answer it properly: follow the chain of citations, read the sentence behind every link, and only call a paper dependent when every link really is.

• Try it live (read only): shaky-ground-production.up.railway.app

• Code (MIT): github.com/aryangorde6/shaky-ground

• Demo video (4 min): youtu.be/64SxLfHHHHY

It's a question about paths

"Does paper C depend on retracted paper A?" isn't a property of C. It's a property of the chains between them. C might cite B, which used a method from A. Or C might cite B for something B never took from A. One step isn't enough, and counting citations isn't enough. You need the paths, and what sits on every edge of them.

That's a graph. So the whole thing lives in FalkorDB:

(:Paper {id, doi, title, year, retracted, risk, risk_proven, ...})
(:Paper)-[:CITES {cls, reliance, evidence, judged_by, carries, carry_votes, ...}]->(:Paper)
(:Retraction {record_id, date, reason})-[:RETRACTS]->(:Paper)
(:Investigation {id, verdict, ...})-[:EXPLORED {round, reason, decided_by}]->(:Paper)

The papers come from OpenAlex, the retraction notices from the Retraction Watch database (published by Crossref), and the citing sentences from Semantic Scholar. Every load is a MERGE keyed by an ID, so loading twice changes nothing. For the paper in this post, the graph holds 10,852 papers and 147,434 citations.

No author names, institutions or countries are stored. It scores papers, never people.

Reading the sentence behind every citation

A citation on its own says nothing about dependence. The sentence around it does. Compare "we used the protocol described in [12]" with "amyloid has been widely studied [3–12]".

So for every citation, Shaky Ground fetches the sentences where the citing paper cites the cited one, and a language model reads them. The model never picks a number. It answers yes/no questions: does the citing paper dispute this work? Does it use its method or data? Does it rely on its specific finding? Fixed rules in code turn the answers into a class, and a table turns the class into a weight:

Class	The citing paper…	Weight
FOUNDATIONAL	uses its method, data or result	1.0
SUPPORTING	relies on its finding as evidence	0.6
BACKGROUND	mentions it in passing	0.15
CRITICAL	disputes it, or reports the retraction	0
UNKNOWN	no sentence to read	0.3 (estimate only)

The deciding sentence is stored on the edge, word for word, along with the model that judged it. A judged edge is never sent to the model again.

A chain's risk is the product of its weights, and a paper's risk is its riskiest chain of up to three citations. The weights were picked so that the result reads cleanly. Three SUPPORTING links in a row score 0.6³ = 0.216, while any chain containing a passing mention scores at most 0.15. So with a bar of 0.2, "builds on" means exactly this: some chain where every link is a foundation or real support. One passing mention anywhere breaks the chain.

Scoring every chain in Cypher

My first idea was to treat it as a shortest-path problem: with cost = −ln(weight), the shortest path is the riskiest chain, and algo.SPpaths finds it. That worked until the rules got more interesting (more on that below). A path algorithm wants one fixed number per edge, and my weights started to depend on which retracted paper you're asking about.

So instead, risk flows backwards from the retracted paper, one hop per query. Simplified (the real query adds the "carry" rule from the next section):

MATCH (a:Paper)-[c:CITES]->(b:Paper)
WHERE b.risk_proven > 0 AND c.reliance > 0 AND c.cls <> 'UNKNOWN'
WITH a, max(b.risk_proven * c.reliance) AS r
WHERE r > coalesce(a.risk_proven, 0.0)
SET a.risk_proven = r

The retracted paper starts at 1.0, and the query runs three times, once per hop. Each run is one pass over the edges, so the cost grows with the number of edges, not the number of chains.

To show one paper's chain with its evidence, a variable-length path with reduce() does the multiplication (simplified again):

MATCH p = (a:Paper {id: $id})-[:CITES*1..3]->(:Paper {id: $root})
WHERE all(c IN relationships(p) WHERE c.reliance > 0 AND c.cls <> 'UNKNOWN')
WITH p, reduce(r = 1.0, c IN relationships(p) | r * c.reliance) AS risk
RETURN risk, nodes(p), relationships(p)
ORDER BY risk DESC LIMIT 1

I didn't want to trust the fast version on faith, so python -m shaky.risk also scores every chain one at a time and compares. On the full graph: 0 mismatches. The propagation took 0.45 seconds; the one-chain-at-a-time check took 287.8 seconds. That gap is the whole argument for doing this in a graph database.

Does the dependence actually carry?

Here's the case that changed the design. A 2014 mouse study extracted proteins using a lysis buffer adapted from the retracted paper. Fair enough: that study builds on it. But papers that cite the mouse study for its brain-wave recordings rely on the mouse study, not on anything it took from the retracted paper. Multiplying weights along the chain would wrongly drag them in.

So for every strong link past the first, a model gets two things: what the middle paper took from upstream, and how the next paper uses the middle one. It writes down both, then answers whether they're the same part.

That turned out to be the hardest judgment in the project. Across four prompt variants, one model's count of "yes" answers on the same 31 links ranged from 0 to 30. So now two model families vote (Groq's gpt-oss-120b and Google's Gemini). A link carries only if both say yes, and is cleared only if both say no. Until it's shown to carry, it counts as a passing mention at most. Both votes are stored on the edge.

Saying "not shown" instead of "clean"

Most citations come with no sentence to read. Semantic Scholar has citing sentences for 754 of the 2,767 loaded citations of this paper, about 27%. The easy move would be to treat the rest as harmless. Shaky Ground doesn't. Without a sentence, a paper is not shown either way: not blamed, and not cleared.

The same rule applies in the other direction. The foundation check asks whether a paper rests on retracted work, by walking its references two levels up. It only says CLEAN when every one of those references was checked. Check Deep Residual Learning (ResNet) and you get "no retracted work found", not clean, because 23 of its 79 references point at works that OpenAlex can no longer resolve.

Results for one retracted paper

For the 2006 Nature paper:

• 45 papers build on it. For 10 of them, the chain runs through another paper, and both models agreed that each later link relies on the part that came from the retracted work.

• 2 of those 45 were later retracted themselves.

• 868 cite it only in passing, and 11 dispute it.

• 3,373 are not shown, because no citing sentence was available.

An agent that decides where to dig

Loading every citer of every citing paper would burn through the free API budgets fast: OpenAlex allows 10,000 credits a day, and Semantic Scholar one request a second. So an agent chooses. Each round it sees the 12 riskiest papers whose citers aren't loaded yet, with each one's risk, strongest link and sentence. It picks which to expand, and it must say why. The reason is stored in the graph as an EXPLORED edge, next to the paper it chose.

The code doesn't take the agent's word for anything. It keeps only picks that were on the list and fit the budget. On one run, a paper the agent picked with the reason "Highest remaining citers with proven foundational link" turned out to be retracted itself.

Check your own reference list, or let your AI assistant check it

The same checks work before you publish. Paste a reference list or a BibTeX file into the web UI, and each reference gets a verdict. The list is drawn as a graph: your list, its references, and the retracted papers they lead to. In the example on the live site, one of five references is retracted itself, and two more rest on retracted work.

Shaky Ground also runs as an MCP server, so an AI writing assistant can call check_foundations before it cites a source. In the example agent, the model checks four sources for a claim, keeps two (one with a warning) and drops two. The rules are enforced in code, not left to the model: if it tries to keep a retracted source, the answer is sent back.

Running it on a free tier

The live site runs on Railway and reads a FalkorDB Cloud instance on the free tier, which has 100 MB. A snapshot without the raw reference lists is 29 MB instead of 113 MB, so it fits. The FalkorDB team pointed out on Discord that query memory counts against the same 100 MB, so a burst of visitors mustn't turn into a burst of queries. Every read now has a timeout, at most four API requests touch the graph at once, and visitors who arrive together share one view build. The public site is read only: it shows stored results and never spends API credits.

What I learned

• Think in hops, not in paths. I started this week new to Cypher, and my first instinct was to list every chain and score it. Into this one paper there are 221,189 chains of up to three citations. Letting risk flow one hop at a time, and keeping only the strongest value at each step, gives the same answers as checking every chain, in under half a second.

• Check the counts, every time. My first snapshot export returned 10,000 of 109,774 citations, with no error, because of a default limit on result size. Only a count check caught it. Now a restore fails loudly if its counts don't match the snapshot's, and the live site is checked against the same counts before a demo.

• One model's answer is not a measurement. On the same 31 links, one model's count of "yes" answers moved from 0 to 30 depending on how I worded the prompt. Asking two model families and storing both votes on the edge doesn't make the question easy, but it makes the disagreement visible instead of hiding it inside one label.

• "Not shown" is an answer. For about three quarters of the citations there was no sentence to read. Treating that silence as safe would have made the numbers look better and the tool worse.

What I'd do next

• Get sentences from open-access full text, to shrink the "not shown" pile.

• Compare what was taken with what the retraction was about. Using a buffer recipe from a paper is not the same as relying on its manipulated images. Today Shaky Ground says "depends on", never "is wrong", and that's deliberate.

• Hold risk for many retracted papers at once. Today it's one at a time.

• Measure model agreement against human labels on a sample of links. So far it's model against model: a third model, from another family, re-labelled 30 judged links blind and gave the same class on 27, and none of the 3 differences changes the 45 papers that build on the retracted one.

The thing I'll take away: the graph didn't just store the data. Once each citation carried its sentence, its class and the model that judged it, the hard question ("does this paper really depend on that one, three steps back?") became one Cypher query, and every answer came with its evidence attached.

---

Built for Graph Hacks: Context for AI Agents (WeMakeDevs × FalkorDB). Data: Retraction Watch (via Crossref), OpenAlex, Semantic Scholar. Built with Claude Code as my coding assistant; every decision in the project is mine, and I can explain every part of it.