The SlopIndex has emerged as a new reference benchmark designed to quantify exactly how much AI models write like "AI slop" rather than genuine humans. Unlike many existing evaluations that rely on other LLMs to judge output, this index uses a hybrid approach: mechanical analysis against historical human writing and a live blind human vote. The platform currently tracks 58 frontier models across 112 scenarios, generating over 64,000 data points to calculate a slop score where higher numbers indicate worse, more robotic writing.

The Measurement Methodology

The core innovation here is the avoidance of circular logic. The benchmark measures "slop" as the distance from genuine human writing collected before ChatGPT existed, ensuring the yardstick wasn't shaped by the models themselves. The corpus includes real workplace emails from 2000โ€“2002, tweets from before 2022, and essays and chat logs that predate generative AI entirely. The final score is a blend of five components: 40% from the human arena vote, and 15% each from conciseness, templating, rhythm, and specific linguistic tells.

Identifying the Tells

The index explicitly calls out the linguistic patterns that betray AI authorship, such as the ubiquitous "I hope this finds you well," three tidy bullet points, and closing phrases inviting the reader to not hesitate. These phrases are measured against how often real people actually used them before 2022. The platform contrasts these AI habits with natural human communication, such as a direct request to push a deadline because of waiting on design files, noting that real people rarely use the formal, hedging language typical of LLMs.

The Human Arena

Beyond the mechanical scoring, the SlopIndex integrates a live human element through its "Arena." Here, users participate in blind comparisons, choosing between two samples generated for the same task where one is objectively sloppier. Every vote contributes to the live human axis of the score, effectively crowdsourcing the definition of what feels like AI. This dual approach of hard data and subjective human perception aims to create a more robust definition of quality writing in the era of synthetic text.

Key Takeaways

  • The benchmark covers 58 frontier models, scoring them on a 0-100 scale where higher scores mean more AI-like writing.
  • Scoring is based 40% on human blind votes and 15% each on mechanical analysis of conciseness, templating, rhythm, and specific phrase usage.
  • The reference corpus consists exclusively of human writing data from before 2022 to prevent contamination by LLM training data.
  • Specific tells like "I hope this email finds you well" are penalized based on their rarity in pre-AI human correspondence.

The Bottom Line

SlopIndex proves that evaluating AI writing quality doesn't require an AI judge, and its reliance on pre-ChatGPT human data offers a necessary, unbiased baseline for distinguishing synthetic filler from authentic communication.