New AI models are almost always announced with a chart of benchmark results, and the model with the tallest bars is usually described as the best. This article explains what those tests measure, where they break down according to Stanford researchers and the US standards agency NIST, and how to read a score without over-trusting it.
What is an AI benchmark, exactly?
A benchmark is a fixed set of tasks with a scoring rule. A model is given questions, such as math problems, multiple-choice science questions or software bugs to fix, and its answers are marked automatically. The result is a single number, often a percentage, that can be compared across models and over time.
The appeal is obvious. Automated tests are cheap to repeat, they do not depend on a reviewer’s mood, and they let outsiders compare systems built by different companies. NIST’s Center for AI Standards and Innovation (CAISI) says in its January 2026 draft guidance that benchmarks are a common measurement instrument that can be especially useful when organizations are short of time, expertise or resources. The same page is equally clear that benchmarks cannot meet every evaluation objective.
A benchmark is a sample, not the whole job. A model that scores well on a coding test has shown it can pass that test. Whether it will behave the same way inside a company’s own software is a separate question.
Why do scores climb so quickly?
Part of the answer is real progress, and part is the way tests age. The Stanford Institute for Human-Centered AI, in its 2026 AI Index report, says that evaluations intended to be challenging for years are now saturated in months. Saturation means models score so close to the maximum that the test can no longer tell them apart.
The report gives examples. Frontier models gained 30 percentage points in a single year on Humanity’s Last Exam, a benchmark built to be hard for AI and favorable to human experts. On OSWorld, which tests whether AI agents can complete tasks on a computer, accuracy rose from about 12% to 66.3%, against a human baseline of 72.35%. That is a gap of roughly six points (72.35 minus 66.3 is 6.05).
Saturation matters because it shortens the window in which a benchmark is useful for tracking progress. When many models cluster near the top, small differences in a score may reflect noise or tuning rather than meaningful capability.
Can the test itself be wrong?
Yes, and this is one of the less publicized findings. The Stanford report cites a review of nine widely used benchmarks that looked for invalid questions, meaning items with errors in the question or the answer key. The share varied from about 2% on MMLU Math to 42% on GSM8K, according to the report. Stanford summarizes the section as growing reliability and gaming concerns, with error rates of up to 42% on widely used evaluations.
The review measured how well a detection method flagged invalid questions, so the percentages describe what the review found in each test, not a direct count of wrong answers by models. Even so, the direction is plain: a score from a flawed test carries less information than its precision suggests.
A simple worked example shows the effect. Suppose a test has 1,000 questions and 10% of them have faulty answer keys. A perfect model could score at most 90%, and a model that is right on every valid question but is marked wrong on the faulty ones would look 10 points worse than it is. Two models separated by a few points in that situation may be separated mainly by how each handles the broken items.
What does gaming a benchmark look like?
When a number becomes the goal, effort flows toward the number. Developers can tune models, prompts and evaluation settings to do well on a particular test. In some cases test material leaks into training data, which is usually called contamination. A model that has effectively seen the answers can look more capable than it is.
The Stanford report addresses a related problem on public leaderboards. It says that standing on the Arena leaderboard, a ranking built from human votes on model responses, may partly reflect adaptation to the platform rather than general capability, citing separate research. The report gives no figure for the size of the effect. It also notes that, as of March 2026, the top four providers on that leaderboard sat within 25 Elo points of one another (Elo is a rating scheme borrowed from chess): Anthropic at 1,503, xAI at 1,495, Google at 1,494 and OpenAI at 1,481. The spread from first to fourth is 22 points, which suggests the leaderboard separates the leading models less sharply than it once did.
The Stanford page examined for this article did not discuss contamination directly, so no figures on its prevalence are offered here. NIST’s work on the subject is aimed at the same underlying concern: whether a score reflects the property it claims to.
What are NIST and its partners proposing?
In January 2026, CAISI released an initial public draft titled NIST AI 800-2, Practices for Automated Benchmark Evaluations of Language Models. NIST describes it as preliminary, voluntary guidance for evaluating language models and AI agent systems. It is organized into three parts:
| Stage | What it covers |
|---|---|
| Defining objectives | Deciding what you want to measure and selecting benchmarks that fit |
| Running evaluations | Implementing the tests and running them consistently |
| Analyzing and reporting | Interpreting results and describing them honestly |
NIST says the draft is meant to support the validity, transparency and reproducibility of AI evaluations. Its intended readers include technical staff at developers, deployers and third-party evaluators, and NIST adds that people who read evaluation reports, such as procurement specialists and business decision-makers, can benefit too. Public comment closed on March 31, 2026. A final version had not been located at the time of writing.
The agency has also pointed to international work. On February 13, 2026, NIST reported that the International Network for Advanced AI Measurement, Evaluation, and Science published a list of key practices and open questions on automated evaluations. CAISI founded the network in November 2024, and NIST says it includes government bodies from ten jurisdictions: Australia, Canada, the European Union, France, Japan, Kenya, the Republic of Korea, Singapore, the United Kingdom and the United States. The NIST page does not itself list the practices, which are hosted by the UK AI Security Institute.
A separate CAISI research post from December 2025 argues for stronger measurement science and sets out open questions in three areas: measurement validity, interpreting results, and measurement in the field. NIST describes those questions as an initial, non-exhaustive selection.
Do high scores predict real-world performance?
Often less than people assume. The Stanford report uses the phrase “jagged intelligence” for the uneven pattern in which AI systems excel at some hard tasks and stumble on seemingly simple ones. It notes that Gemini Deep Think earned a gold-medal-level score of 35 points at the 2025 International Mathematical Olympiad, yet the best model read analog clocks correctly only about half the time on ClockBench (the report’s text gives 50.6%).
The gap between controlled tests and physical or messy settings can be large. The report says robots reached 89.4% on RLBench, a simulated benchmark, but succeeded in only 12% of real household tasks. A simulation is a tidy approximation of the world, and the figures show how much can be lost when systems leave it.
This is also why a hard-to-fool result matters for products that act on a user’s behalf. Our explainer on AI agents describes the systems that benchmarks like OSWorld try to measure, and the limits there are not only about accuracy but about reliability across many steps.
How should a reader interpret a vendor’s score table?
Company announcements are the main place readers meet benchmark numbers, and they are produced by the party with the most to gain. A few questions help separate information from marketing:
- Who ran the test? Results reported by the developer are claims. Results reproduced by an independent evaluator are stronger evidence.
- Which settings were used? Prompts, number of attempts and the use of tools can all move a score. A comparison is only fair if the conditions match.
- How big is the difference? With scores clustered near the top, a one- or two-point gap may sit within the test’s own noise, and a test with faulty questions has an even lower ceiling.
- Is the test still informative? A benchmark that most models nearly max out says little about the differences that matter.
- Does it resemble your task? A score on competition math says little about summarizing legal documents or drafting support replies.
None of this means benchmarks are worthless. It means a single number is a starting point. For a business choosing a tool, a small trial on its own data, checked by the people who will use the output, tells it more than any leaderboard.
Benchmarks also interact with other concerns. Readers wondering why strong-scoring chatbots still invent facts can read our piece on AI hallucinations, and those weighing how regulation treats evaluation can look at the EU AI Act high-risk delay, since regulators increasingly rely on technical testing standards to define what compliance means.
What is still unknown about measuring AI?
Several things remain unsettled. NIST’s draft is not final, and it describes some practices as emerging rather than established, which means the field has not agreed on a full rulebook. The Stanford report gives no measure of how much leaderboard adaptation distorts rankings. And as models improve, tests must be rewritten faster, a pattern the report describes in terms of saturation within months.
There is also an unresolved tension between public and private tests. Public benchmarks are transparent but can be learned from; private ones resist that but are harder for outsiders to scrutinize. Neither source reviewed here says which approach will win out.
A better way to read the next score table
The next time a model launch comes with a chart, treat it as a claim to be examined. Look for who ran the test, how large the gap is, whether the test is saturated or flawed, and whether it looks like the work you care about. The research from Stanford and the guidance from NIST point the same way: benchmarks are useful instruments, but they are measurements with error bars, and the honest question is not “which model has the biggest number” but “what does this number actually tell me.”
This article is general information about AI evaluation, not a substitute for professional advice on selecting or procuring AI systems.
Sources: Stanford Institute for Human-Centered AI; National Institute of Standards and Technology
