Pick 3,900 published social-science papers at random. Hand 164 of them, and the 274 specific claims buried inside them, to independent teams with a budget to run the experiment again. See what survives. That’s Investigating the Replicability of the Social and Behavioural Sciences, published in Nature on Oct. 29, 2026 by Andrew H. Tyner, Timothy M. Errington, Brian A. Nosek, and the SCORE project team, and covered the same day by PsyPost, Yahoo News, The College Fix, and Karolinska Institutet, a partner institution on the project. Of the 274 claims individually assessed, 55.1% replicated. 35.8% came back with no effect at all. 8.8% reversed — the opposite finding from what got published the first time. And the effects that did replicate were, on average, less than half the size of the originals. Sextus Empiricus didn’t need a DARPA grant to arrive at the conclusion this study keeps landing on. In the Outlines of Pyrrhonism, he built an entire method around one move: when a claim shows up, don’t assent yet. Wait for the counter-argument. See if it’s just as strong. It usually is.

The Quick Verdict

Question	What we found
What was studied?	Whether published social-science findings hold up when independent teams try to reproduce them
Who ran it?	Andrew H. Tyner, Timothy M. Errington, Brian A. Nosek, and the SCORE (Systematizing Confidence in Open Research and Evidence) project, with Karolinska Institutet as a partner institution
Where and when?	Nature, published Oct. 29, 2026; covered the same day by PsyPost, Yahoo News, and The College Fix
Scope	3,900 quantitative papers (2009–2018) across 62 journals in business, economics, education, political science, psychology, and sociology; 164 papers and 274 individual claims actually re-tested
Headline number	55.1% of the 274 claims replicated; 35.8% showed a null effect; 8.8% reversed direction entirely
Effect size	Replicated effects averaged less than half the magnitude of the original published effects — a pattern DARPA’s own team ties to publication bias toward novel, positive results
What Sextus Empiricus already argued	Outlines of Pyrrhonism: suspend judgment (epoché) on any claim until you’ve weighed it against an opposing argument of equal force (isostheneia) — a single confident result was never grounds for assent

The Quick Version

A seven-year project tried to reproduce 274 specific claims pulled from 3,900 published social-science papers. A little over half held up. More than a third vanished on retest. Nearly one in eleven flipped to the opposite conclusion. Sextus Empiricus spent his life arguing that no single claim — however confidently stated, however published, however widely believed — deserves your agreement until it’s survived a real counter-argument. This study is what happens when you run his method on a filing cabinet of modern social science.

What the Study Actually Found

SCORE — Systematizing Confidence in Open Research and Evidence — started with a deceptively simple question: if you picked social-science findings at random instead of letting researchers submit their favorite ones for replication, what fraction would survive? Most prior replication efforts (the famous 2015 psychology project, for instance) worked from papers researchers volunteered or reviewers flagged as important. SCORE went the other way. Random sample first, then test what came up.

From the initial pool of 3,900 papers spanning business, economics, education, political science, psychology, and sociology, the team narrowed to 164 papers across 54 journals that were actually feasible to re-run, yielding 274 distinct scientific claims. For 98 of those papers, independent teams collected brand-new data from scratch. For 66, they re-analyzed existing datasets. Both approaches found almost the same failure rate, which matters — it means the problem isn’t confined to one kind of methodology.

The topline: 55.1% of the 274 claims replicated by the study’s main measure. 35.8% produced a null result — no effect where one had been reported. 8.8% reversed outright, meaning the replication found evidence pointing the opposite direction from the original paper. And even among the claims that did replicate, the magnitude shrank. The median effect size dropped from 0.25 in the original papers to 0.10 on retest — less than half.

The rates weren’t even across fields. Education came in at 63.1%. Economics sat at 42.5%. Psychology landed almost exactly in the middle at 49.0%. No discipline came out clean, and none came out so broken that the whole enterprise should be thrown out — which is, itself, a very Pyrrhonist thing for the data to do: refuse to let you form a tidy verdict in either direction.

What Did Sextus Empiricus Mean By This?

In Outlines of Pyrrhonism, Sextus defines his method’s central term directly: epoché is “a standing still of the intellect, owing to which we neither deny nor affirm anything.” He explains what produces it — isostheneia, “equality with regard to credibility and incredibility,” meaning no opposing argument strikes us as more convincing than its rival.

Sextus wasn’t arguing that nothing is true, which is the version of skepticism people usually assume and then dismiss. Read closely, his target is narrower and harder to wave off: he’s arguing that assent — actually committing to a claim as settled — is the thing worth withholding, and withholding it specifically until a claim has been tested against its strongest opposite. Not until you feel confident. Not until it’s published somewhere reputable. Until the counter-case has had its turn and the scale still doesn’t tip.

He built the Ten Modes of Aenesidemus and later arguments around demonstrating, case after case, that the same object produces contradictory appearances depending on who’s observing it, how, and under what conditions — a healthy eye sees one color, a jaundiced one sees another, and neither observer has independent access to which is “real.” His conclusion wasn’t despair. It was a discipline: live by appearances, act in the world, but stop handing out the full confidence of certain knowledge to claims that haven’t earned it by surviving the opposing argument.

A published finding is an appearance. A persuasive one, dressed in a p-value and a peer-reviewed journal name, but an appearance all the same — one side of a case that hasn’t yet met its isosthenic opposite. SCORE’s replication teams were, mechanically, that opposite. And more than a third of the time, once the counter-argument actually ran, the scale didn’t hold where the original claim had left it.

Why Do Published Findings Fail to Replicate?

1. Novel, positive results get published; null results mostly don’t. The study’s own authors point to this directly — the shrinking effect sizes on replication are consistent with a filtering process upstream, where papers showing a dramatic effect clear the publication bar more easily than papers showing nothing.
2. Small original samples inflate the effects that do get through. A modest true effect, measured noisily in a small sample, sometimes gets measured as large purely by chance — and chance doesn’t repeat itself on command.
3. Researchers (not maliciously) have many ways to analyze the same data, and some of those ways produce a significant result. Choosing, consciously or not, among plausible analysis paths after seeing the data tilts the odds toward finding something, whether or not something real is there.
4. A field’s standard methods can be systematically weaker than assumed. The spread between education’s 63.1% and economics’ 42.5% in this study isn’t about one field trying harder — it suggests some combination of smaller typical samples, noisier measurement, or weaker theory varies a lot by discipline, and readers rarely adjust their trust accordingly.

Why the Study and the Philosophy Rhyme

The honest version of this comparison isn’t “an ancient skeptic predicted a 2026 Nature paper.” It’s narrower, and it holds up better for being narrower: Sextus’s specific claim was that a single appearance of a claim — however confident, however authoritative its source — isn’t grounds for assent until it’s been weighed against its strongest opposite. SCORE’s researchers weren’t testing Pyrrhonism. They were testing whether published findings, selected at random rather than cherry-picked, survive a real second look. But the structural result is the same shape Sextus described in the second century: a claim that looked settled on first appearance lost roughly half the time once a genuine counter-case got its turn.

It connects to the piece on Socrates’ maieutic method, which makes an adjacent point about confident answers arriving fast and understanding arriving slow — Socrates wanted you to interrogate the answer, Sextus wanted you to withhold agreement from it, and this study gives both of them a body of evidence to point to. It sits close to the piece on Bacon’s “false mirror”, since Bacon’s Idols of the Tribe describe exactly the kind of built-in distortion — toward novel, flattering, publishable results — that SCORE’s authors finger as the likely source of the gap between original and replicated effect sizes. It rhymes with the piece on Seneca and the ultimatum-game algorithm study and the piece on Aristotle and the duty-sex study, both of which took one confident paper at face value for the length of an article — a pattern this post would ask you to hold a little more loosely, this one included.

The Gap Sextus Didn’t See Coming

Here’s where the comparison is more interesting than a clean match.

Sextus’s method assumes a patient, available observer: someone with the time, standing, and inclination to actually seek out the counter-argument before forming a view. That’s a reasonable assumption for a philosopher in Alexandria with no deadline. It’s a bad description of how a published finding reaches most people now. A study drops, a headline compresses it to one sentence, the sentence circulates for months or years, and the correction — if a correction ever gets run at all — arrives quietly, in a journal almost nobody outside the field reads, long after the original claim has settled into common knowledge. Sextus had no model for a claim that spreads faster than its own counter-argument could ever be produced, let alone noticed.

SCORE is the correction machine working as designed — seven years, hundreds of researchers, a random sample instead of a curated one. It’s also proof of how rare that machine is. Most published findings never get this treatment. The ones that circulate as “studies show” in conversation, in articles, in this very genre of post, mostly never face an isosthenic opposite at all. They just get believed, by default, because nothing pushed back yet.

What This Study Doesn’t Show

Be precise about the size of this. The framing invites more confidence than the data earns in either direction.

A Failed Replication Isn’t Proof the Original Was Fraudulent

Null or reversed results can come from a genuinely weaker original effect, from real differences between the original and replication samples, or from chance in either direction. SCORE’s authors aren’t alleging misconduct across a third of social science — they’re documenting a measurement problem, not an ethics scandal.

55.1% Is a Headline Number, Not the Only One

Different ways of scoring “success” moved the overall estimate across a wide range in related SCORE analyses. The 55.1% figure for individual claims is real and reported, but treating it as the single precise truth about “how much of social science is right” overstates what one metric can tell you.

This Covers Six Fields, Not All of Science

Business, economics, education, political science, psychology, and sociology. Biology, physics, and medicine have their own, separately documented replication issues, and this study doesn’t speak to any of them directly.

The Sample Was Papers From 2009–2018

Open-science reforms — preregistration, larger standard sample sizes, data-sharing requirements — have become more common since then in several of these fields. Whether a similar random sample of 2024–2026 papers would replicate at a higher rate is a reasonable guess, not something this study measured.

What Actually Follows From This

1. Treat “a new study found” as the opening move in an argument, not the verdict. Sextus’s discipline wasn’t cynicism — it was patience. Wait for the second look before you update your whole worldview off a single headline.
2. When you do hear about a replication or a failed one, ask what actually failed to replicate — the direction of the effect, or just its size. This study found plenty of claims that shrank on retest without reversing. A weaker-than-advertised effect and a nonexistent one are different findings, and they call for different amounts of doubt.
3. Hold your own strongly-held beliefs about psychology and society the same way, not just the ones in the news. If roughly half of randomly sampled, peer-reviewed social-science claims didn’t survive a well-funded second look, the belief you picked up from one compelling article or one striking statistic deserves at least the same epoché before it gets to run your decisions.

Sextus Empiricus never imagined a DARPA-funded consortium re-running 274 claims across six disciplines, and the SCORE team wasn’t out to vindicate a Greco-Roman skeptic. But a 2026 project built to measure whether published findings survive a second look arrived at the same structural lesson Sextus spent the Outlines of Pyrrhonism arguing for: a claim’s first appearance, however confident, was never the end of the inquiry. It was the opening bid.

---

This post discusses a 2026 study published in Nature (Tyner, Errington, Nosek, and the SCORE project), covered by PsyPost, Yahoo News, The College Fix, and Karolinska Institutet on Oct. 29, 2026, and Sextus Empiricus’s Outlines of Pyrrhonism. Replication rates varied by field and by scoring method, and a failed replication does not by itself prove the original finding was wrong.