How the Fiction Slop Index is measured

The headline number is the Human Prose Score, on a 0-100 scale where 100 reads like a person wrote it. It combines five components:

Until the Arena ships, the leaderboard reports a Mechanical Prose Score: the four live axes combined at their relative weights, which renormalise to 0.308 for Tells and 0.231 for each of the other three.

MechanicalProseScore =
    0.308 · Tells
  + 0.231 · Templating
  + 0.231 · Rhythm
  + 0.231 · ShowTell

Scoring runs in three stages, all reproducible from the raw passages:

1. Per passage. Each detector reads one passage and returns a slop signal. The signal is inverted to a 0-100 human-likeness score, so higher is always better.
2. Per model. A model's passages are pooled and each axis is averaged, giving four axis scores per model.
3. Blend and rank. The axes are combined by the weights above into one score, and models are ranked by it.

Scores are absolute, not graded on a curve, so a model is measured against the human baseline rather than against the other models in the run. That keeps a snapshot comparable across months even as the roster changes.

Every axis is a pure function of the text with no model call, so the same passage always yields the same score. Each is described below with the signal it reads and the formula that turns it into a 0-100 human-likeness number.

3.1 Tells

Density of known AI writing tells from the 758-entry AI Cliché Corpus, including fiction-specific clichés like body-language tics and said-bookisms. Each match counts by its strength, how often it repeats, and whether the tell is still current, then the total saturates against a density midpoint so one stray word cannot dominate a long passage.

d = Σ (strength · repeat(count) · lifecycle) / words · 1000
Tells = 100 - ( 100 · d / (d + 100) )

3.2 Templating

Structural repetition a word list cannot catch: runs of sentences that open the same way, the negation pivot ("she did not run, she walked"), repeated phrase frames, a symbolic final sentence that echoes the opening, and detachable epigrams. Each structural tell adds points, which saturate to a slop score.

pts = 2·anaphoraRuns + 2·negationPivot
    + 2·framingFrames + 3·symbolicEnding + 1·epigrams
Templating = 100 - ( 100 · pts / (pts + 4) )

3.3 Rhythm

Human sentences vary in length; AI prose tends toward an even cadence, or, after a heavy edit, an over-corrected staccato chop. Five sub-signals each become a 0-1 penalty against a calibrated threshold and are averaged. Passages under six sentences are too short to read a cadence and score a neutral 75.

penalty = mean( variation, comfortZone, subordination,
                lowSimpleRatio, staccato )   // each 0..1
Rhythm = 100 - ( 100 · penalty )

3.4 Show / Tell

The fiction axis no functional-prose leaderboard has: how often the writing tells a feeling instead of showing it. It reads filter verbs ("she saw", "he felt"), emotion-naming ("a wave of dread"), and flat telling labels ("was terrified"), weights emotion-naming highest, and saturates against a lower midpoint tuned to fiction.

d = Σ (hit · weight) / words · 1000
ShowTell = 100 - ( 100 · d / (d + 22) )

Each model writes to a fixed grid of six craft tasks by three genres, 18 scenarios in all, at three samples each. Prompts are persona-rich on purpose: bare instructions produce bare, unrepresentative prose, so each cell fixes point of view, tense, and a real constraint and asks for roughly 400 words. Models run at default settings with no cherry-picking.

Model roster

The current snapshot tests 9 models from 5 labs, all reached through a single routing layer at their default settings. Reasoning models get a large output budget so they finish thinking and still emit prose.

The Arena adds blind human judgment. A visitor sees two passages written to the same prompt, with the model names hidden, and flags the one that reads more like AI. Votes update a Bradley-Terry rating (K = 32): the passage left alone gains, the flagged one loses.

Safeguards keep it honest: a minimum dwell time before a vote counts, a daily cap per visitor, and human-written control passages mixed into the pairs. A voter who consistently rates real human prose as the sloppier option is down-weighted.

The mechanical axes are tied to human writing from before generative AI, so the ranking cannot be shaped by any model. The rhythm thresholds and the corpus of tells are the reference for those axes directly. For length and cadence distributions, and for the Arena reference pairs, the baseline is drawn from prompt-shaped human fiction written before October 2022, with classic out-of-copyright fiction as a secondary source. The baseline is frozen and versioned so a dated snapshot stays reproducible.

• No AI judge in the headline score. Only human baselines and crowd votes inform the Human Prose Score. No model grades itself.
• Default settings, no cherry-picking. Every model runs with default parameters, and every generated passage is scored.
• Slop-free is not the same as good. A separate Craft Quality board keeps blind judge scores for voice, originality, and resonance. A model can read perfectly human and still be dull, so the two scores stay apart.
• Open by default. The corpus is public and downloadable, and the detectors are deterministic, so anyone can reproduce a score.

• Rhythm has little spread at 400 words. Leading models rarely hit the cadence extremes at this length, so the axis reads near the top for most of them today. It will earn its weight across a wider model set and longer passages.
• This is a preview snapshot. The Arena is not yet live, and the sample size is small. Scores will move as the test set and roster grow.
• Style, not authorship. The score describes how templated the prose reads. It never claims who or what wrote a given passage; edited AI drafts can score clean and humans write in AI-cliché-heavy styles constantly.