Tuesday, November 17, 2026 Independent Review Analysis Methodology · Archive

Reviewing the Reviewers

Seven Out of Ten

Methodology

How Seven Out of Ten works

Two questions, answered from public data: does an outlet rank games higher or lower than the players who bought them, and higher or lower than its peers? And does it move away from its own habit on particular kinds of games?

The panel and the data

16 outlets. Scores, bylines and dates come from each review page's own structured data (schema.org Review markup), found through the outlet's review feed and its public sitemap or review listing. Scores are rescaled to 0–100 (an 8/10 and a 4/5 are both 80). Unscored outlets (Kotaku, Rock Paper Shotgun, The Verge) count for coverage and the disclosure audit only. We do not use OpenCritic data (its API terms bar building an aggregator) and we do not scrape Metacritic (its terms bar automated access); Metacritic appears only as a link.

OutletScaleArchive sourceEthics policy
IGN0–10sitemappolicy
PC Gamer0–100sitemappolicy
GamesRadar+0–5 starssitemappolicy
Eurogamer1–5 starssitemappolicy
Game Informer0–10listingpolicy
Game Rant0–10sitemappolicy
TheGamer0–10sitemappolicy
Screen Rant0–10sitemappolicy
VGC1–5 starssitemap_indexpolicy
Push Square0–10listingpolicy
Nintendo Life0–10listingpolicy
Pure Xbox0–10listingpolicy
Rock Paper Shotgununscoredsitemappolicy
Kotakuunscoredsitemappolicy
The Vergeunscoredsitemappolicy
GameSpot0–10 (not readable: bot wall)feed onlynone found

Window: reviews published 2026-07-20 to 2026-11-17 (120 days). 525 panel reviews, 479 of them scored, covering 198 games. We backfilled 120 days rather than 90 because the main measure needs a game's first 30 days on Steam to have passed.

Players: Steam

Steam's public review totals (no key), saved as a raw snapshot every day because players edit and delete reviews later. Each game is matched to a Steam app by exact (normalized) title through Steam's store search; console exclusives and unmatched games get no player score and are marked. Apps released long before the window (re-releases, long-running games) are excluded from Steam measures.

Windows are cumulative from release: day 7, day 30 and day 90. Variants: filtered (Steam's default, which drops reviews it flags as off-topic review bombs) and unfiltered; all languages and English only; playtime of at least two hours. Small samples are adjusted with an empirical-Bayes Beta prior fitted to the panel's games (day 30, filtered: prior mean 81.67%, fitted on 82 games with 50+ reviews), so a game with 30 reviews is pulled toward the typical game more than one with 30,000. A game needs 20+ reviews in a window to be ranked.

The measures

  1. Steam gap. For each scored review, the score's percentile among all panel scores of same-year games, minus the game's Steam day-30 percentile among same-year games. An outlet's Steam gap is the mean over its games, partially pooled toward the panel (normal–normal empirical Bayes) so small samples cannot produce extreme values. Shown only with 15+ games, with a 90% interval.
  2. Consensus gap. The same, against the mean of the other panel outlets' scores for that game (our own consensus, at least 2 other scores), ranked among same-year games' panel means. If the Steam gap is high and the consensus gap is near zero, the press as a whole is more generous than players; a high consensus gap means this outlet stands out.
  3. Conditional gaps. On a category of games (major publisher, indie, likely console exclusive, rocky launch, a single publisher with 8+ games, or credit-verified narrative consultancy), how far the outlet's gap moves from its own average gap, shrunk toward zero. Shown only with 8+ games and a 90% interval. We call it a "difference", not bias, unless the interval excludes zero against both players and critics.
  4. Robustness strip. Every Steam gap is recomputed with unfiltered Steam, English only, playtime ≥ 2h, day 7, day 90, and excluding games with a documented non-game review bomb. A finding counts only if its sign survives every variant with enough games.
  5. Patch drift. Steam's adjusted percent positive at day 90 (or to date past day 30) minus day 7.
  6. Disclosure audit. Whether the outlet publishes an ethics or standards policy, and the share of its reviews whose page text states where the review copy came from or discloses affiliate links.

What a gap is not

Steam is not ground truth. It is a second panel with its own biases: buyers who chose the game, a thumbs-up scale, launch-week bugs, prices and account requirements, regional and language splits. Critics and Steam players correlate only moderately in published research (about r = 0.59), and they diverge most on big-budget games. A critic who diverges from players may be right. Correlation is not motive: a gap toward a publisher is a pattern on the published record, not evidence of a deal.

Guardrails

Known limits

Corrections

If we matched a review to the wrong game, misread a score or byline, or a policy page has moved, we will fix it and note the change.