At AllSides, we’ve developed multiple media bias review methods to discover how people across the political spectrum perceive news. Our methodology balances the perception of experts trained to see bias with the opinions of everyday people.
In addition to our Editorial Reviews and Blind Bias Surveys, we’ve developed a new method — the Editorial Review Survey.
This methodology is housed under our traditional Editorial Review method, preserving insights from a multipartisan team trained to spot bias. But rather than relying on a generally synchronous, discussion-based panel review, the Editorial Review Survey allows reviewers to independently evaluate a structured content sample from a news outlet more quickly, allowing AllSides to conduct more bias reviews more frequently.
Below are some more details about this methodology and a case study of our first outlet rated using this method.
How the Editorial Review Survey Works
Each Editorial Review Survey is conducted by a multipartisan, expert AllSides Editorial Review Panel, with six total reviewers representing left, center, and right perspectives.
The process follows a standardized structure:
- Content Collection: Headlines and articles are drawn from a media outlet’s homepage over a multi-day period.
- Sampling: 15 headlines and 7 full articles are randomly selected from the collected content. Full articles are presented to reviewers in their original format.
- Evaluation: Reviewers assess each piece of content and indicate what they perceive the political bias of each to be. This is done using AllSides’ pre-established bias scale. Reviewers may also provide open-ended responses explaining where they perceived bias in the piece.
- Scoring: Assessments of individual pieces of content are converted into numerical bias values and averaged across reviewers to ensure balance across left, center, and right reviewers.
- Aggregation: In the final rating, full articles are weighted more heavily than headlines to reflect the depth of analysis, and then aggregated to produce an overall rating reflective of all reviewers’ analyses of the content.
- Reporting: The overall rating along with findings from the analysis, including qualitative reviewer feedback, are published to justify confirmation or changes in bias ratings.
Editorial Review Surveys carry the same weight as other AllSides Editorial Reviews and are reflected on media bias source pages accordingly.
RELATED: How AllSides Rates Media Bias
Case Study: Editorial Review Survey of The Bulwark
In December 2025, after internal testing in early 2025, AllSides administered its first Editorial Review Survey. Upon review, AllSides moved The Bulwark’s bias rating from Lean Right to Left (-3.57).
_0.png)
Across the review, team members evaluated 15 headlines and 7 full articles, assigning bias ratings to each piece. When results were aggregated and weighted, the content overall exhibited a Left bias, with a majority of full articles perceived as Left.
There were two reviewers with a bias on the left, two in the center, and two on the right. Two reviewers — one on the left and one in the center — perceived The Bulwark as Lean Left, on average, while all others perceived the outlet as Left.
Examples of articles cited as Left by reviewers included:
- New Trump Security Strategy Turns Against the Free World – flagged for framing the Trump administration policies as aligned with far-right ideology and carrying a “strong whiff of white nationalism.” A Lean Left reviewer noted the article “arguably downplays the extent to which mass migration is affecting European culture” while a Right reviewer noted no right-leaning perspectives present. A Center reviewer noted left-leaning slant when the article described Trump’s 2020 election denials and the January 6th, 2021 capitol riot.
- “Heritage Americans” and the Nazi Horse – cited for drawing comparisons between immigration policy in the modern United States and Nazism, which reviewers across the spectrum noted as having left framing. The piece argues that Trump's border and immigration policies are "about ethnic purity and creating an illiberal order," which was noted by a reviewer on the right as being opinion presented as fact.
- Netflix and Chill Out—Trump Hasn’t Decided Yet – the article argues Trump has been “dozing off at cabinet meetings,” which a Lean Left reviewer regarded as presenting opinion as fact. A Lean Right reviewer noted mindreading in the article for saying, "Trump loves being sucked up to, and he loves distributing head-pat rewards to the suck-ups."
Reviewers noted that bias varied by article, but the dominant takeaway was that the general direction of the outlet was Left.
As with all AllSides Media Bias Ratings, results reflect the content as it was reviewed in December 2025; ratings may evolve as an outlet’s coverage changes.
Interested in helping AllSides rate media bias? Sign up to participate in our media bias research surveys here.
