Fox News is one of the most successful media companies in America, beating out both CNN and MSNBC in terms of viewers for the past several years. Americans at large agree that Fox has solidified its place as one of America’s mainstream media networks. With its success, however, came criticisms of its journalistic integrity, with some suggesting factual reporting has taken a back-burner to push a political agenda. So do these claims have any merit? Is Fox News truly a reliable media source or is it bogged down by subjective and inaccurate reporting?
Source: Pew Research
Does Reliability Matter?
Reliability, in general, refers to how trustworthy or accurate information, or in this case, a news source is. If we consider this definition, it quickly becomes clear why reliability is important in media sources. If we can’t trust the things we read then there isn’t much of a point in continuing to consume content from that source, after all. So how exactly can we gauge the reliability of a news source anyways?
There are several potential measures of reliability to look out for when trying to determine whether a media source is reliable or not. Red flags for an unreliable article can include the presence of wild unsubstantiated claims, facts dependent on other unreliable sources, heavy use of opinionated language, and more. Some indicators of a reliable news source, on the other hand, include things like:
- Absence of subjective/opinionated language in articles
- Credible sources cited (e.g., neutral sources, .gov, .edu websites)
- Facts and statistics backed by multiple relevant outside sources
- Use of primary sources when possible (e.g., interviews, quotes)
- Information that remains consistent across news sources
So How Does Fox Fair?
The political reliability index developed by Biasly objectively assesses news organizations’ accuracy and trustworthiness. Fox News’s overall Reliability Score has been rated as ‘Good’ by Biasly. This rating is a weighted average of two distinct scores: the Fact Analysis Score and the Source Analysis Score, each evaluating separate components of Fox News’s Reliability. When computing the Average Reliability of the article the Fact Analysis score is more heavily weighted. These ratings are as follows in the next two paragraphs:
Fox News’s Fact Analysis Score is ‘Good,’ which suggests readers can trust most of Fox News’s content online. The Fact Analysis score focuses more on the accuracy of claims, facts, and sources presented in the article and any hints of selection and omission bias, which we will discuss further in the article.
Fox News’s Source Analysis Score is ‘Fair,’ which suggests readers can trust some of the sources, links, and quotes provided by the news source. This score, which is based on A.I., focuses on assessing the quality of sources and quotes used including their number, lengths, uniqueness, and diversity.
However, since these scores are based on percentages and averages, individual articles could be more or less trustworthy depending on the context, author, and other factors. Our findings show that Fox News’s reliability is mostly but not all factual because they have retracted several stories in the past or had pieces that were not factual.
Let us analyze the supporting data for Fox News’s rankings and discuss what to watch out for while searching for trustworthy news sources.
Fox News Accuracy and Reliability
Many have questioned the accuracy and reliability of the articles put out by Fox News, with many claiming their reporting places greater importance on pushing an ideology than reporting the facts, but is this really the case? Where exactly does Fox News reside on the ideological scale, and what does it have to do with the reliability and accuracy of the articles they produce?
When trying to assess an article’s level of bias, one important measure we consider is an article’s accuracy, or whether an article is able to report factually on the claims they make. Two important things to keep in mind when trying to determine how factual and accurate an article is is by screening for both selection and omission bias.
Selection bias is when stories and facts are selected or deselected, often on ideological grounds, to create a narrative in support of the new sources’ ideology. Omission bias, on the other hand, is when different opinions and political views regarding a situation are left out, so that the reader is only exposed to the ideological perspective supported by the author. It’s important to keep in mind these two types of biases when trying to assess an article’s level of accuracy.
Biasly rates accuracy on a percentage scale of 1-100, with 1 being the least accurate, and 100 the most accurate. Ratings are determined by measuring claims with factually-backed reasoning, credible sources, and the number of credible external sources used within an article. Fox News was given a score of “Good” reliability based on an average of 17 articles rated. This means that, on average, news articles published by Fox News tend to be fairly reliable.
If you check out Fox News’ page on the Biasly website, you can see the reliability and accuracy ratings given on some of their most recent articles. You’ll see that while there is some variance in reliability from article to article, it remains fairly close to our average. You’ll also notice that changes in reliability are often associated with changes in bias, often due to the presence of selection and omission bias, meaning articles rated as being more politically biased tend to score lower in terms of reliability, and less biased articles score higher.
These differences are present due to the way bias can influence some of the indicators of reliability listed above. A more biased article tends to include more subjective and emotional language and tends to place greater importance on proving an opinion. One article titled “Vermont Senate OKs noncitizens voting in capital city’s elections,” was given an analyst bias score of somewhat conservative and an author bias score of somewhat conservative. This article received a lower-than-average reliability score of “Fair.”
A more reliable article, on the other hand, focuses on informing the reader first, before pushing any sort of agenda. Reliable articles use more neutral language and back their facts with credible sources. An article titled “Biden operatives begin opposition research on potential 2024 GOP presidential candidate,” received an analyst bias score of somewhat conservative and an author bias score of somewhat conservative. This article received a higher-than-average reliability score of “Excellent.”
Analysis of Fox News Opinion Pieces
While some of our examples above may have given the impression that bias in Fox News can be absent or present at varying intervals, there is one type of journalism where bias is universal. The opinion piece. Opinion pieces tend to be the complete opposite of objective reporting. These pieces often are meant to convey a persuasive argument, leading to lower rates of accuracy and reliability overall. Let’s look at this opinion piece, for example, titled “What Biden’s woke military has wrought.
“Unfortunately, recent Democrat administrations – and Joe Biden’s is no exception – have made race, gender, and controversial political teachings an integral part of military training and values, from the Pentagon to our service academies.”
The author makes his position very clear in the thesis of his article by displaying his disagreement with what he perceives to be the Biden administration’s priorities. We can expect the rest of the article to follow suit and defend his argument. Opinion articles also do not require strict reporting. Take the following for example:
“A recent Reagan National Defense Survey found that the number of Americans with a “great deal” of confidence in the military has plummeted from 70 percent in 2018 to 45 percent now. The largest decline in trust comes among Republicans, whose strong confidence fell from 87 percent to 53 percent – a disturbing 34 percent drop.”
The article uses two pieces of evidence to explain this drop, which we’ll review later. It is important to note that causality here is completely interpreted by the author, and no direct evidence is shown to link the prior quote to the following quotes. Here we can see several instances of selection and omission bias in the picking and choosing of facts to frame the Biden administration in a bad light:
“Soon after Joe Biden became president, the Pentagon ordered an unprecedented military-wide ‘stand down’ of service members to root out right-wing domestic extremists, wasting 5.8 million man-hours. The Pentagon then hired Diversity, Equity, and Inclusion Officers at salaries as high as $200,000. For the past two years, the Department of Defense rigidly enforced vaccine requirements, cutting off tens of thousands of service members from benefits, even when many have legitimate objections and are young, healthy adults.”
“In recent years, the academies have also taught radical doctrines like Critical Race Theory under the label of ‘diversity and inclusion.’ The Air Force Academy established a special ‘Diversity and Inclusion reading room’ that it described as a “safe space” for America’s young warriors. It also created an organization of hand-picked cadets – identified by a purple braid on their dress uniform—to spread these controversial ideals in the ranks. At West Point, cadets can now minor in “Diversity and Inclusion Studies” alongside cadets studying serious subjects like Grand Strategy, Aeronautical Engineering, and Nuclear Science.”
Starting from the top, we can see selection bias in the initial choice to include only facts supporting the idea that confidence in the military has plummeted due to the actions taken by a democratic administration. We can see omission bias in the exclusion of other potential causes of this decline in confidence, leaving the reader to believe that the Biden administration must be the sole cause. The article then selects to focus on spending associated with military diversity initiatives, and vaccine initiatives, choosing to omit any statistics regarding spending in other areas. Finally, the article focuses on diversity education programs, criticizing the programs while omitting any contrasting opinions, or facts illustrating the benefits of such programs.
The presence of this degree of omission and selection bias highlights the issues of accuracy and reliability that arise within the opinion article. Because this is an opinion article, only one side is articulated, and only facts supporting the opinion being made are given. While the facts and studies supporting the opinions being made might be reliable and credible, by choosing to exclude facts that go against the main opinion of the article, the author is reducing the overall reliability of the article as a whole by preventing the reader from forming a full picture of the issue being discussed. For the sake of balance, let’s look into why these investigations, and diversity initiatives, are happening.
Multiple defense agencies came together to agree on pursuing investigations into extremism within the military. According to this CSIS blog post, these investigations “came amid growing concern over extremist network efforts to tailor recruitment toward military personnel and an increase in criminal cases involving extremism in the ranks.” The author argues that these investigations are important because “military personnel can add larger-than-average value to extremist networks due to their specialized knowledge and abilities—including communications, logistics, and tactical skills.” The post also discusses that this isn’t the first time the DOD has investigated extremism, with the last time being in 1995 due to a bombing in Oklahoma and the Fort Bragg murders.
If we consider the facts provided by the blog post above and contrast them with the narrative presented in the Fox article, we can see two different perspectives on the issue of diversity and anti-extremism in the military. In one, these measures are framed as a waste of time by an incompetent democratic administration, and in another, an important part of ongoing counter-terrorism efforts.
By examining how the same issue can be framed in two very different ways through the selection and omission of different facts, we can see how opinion pieces are particularly susceptible to issues of reliability. Furthermore, as a more conservative news source, Fox news, in an effort to appeal to its viewers, is incentivized to engage in further selection bias by choosing only to publish opinion pieces that support more conservative ideals, even at the cost of their overall reliability.
So is Fox Reliable?
The answer to that, like for many things, is it depends. While Fox News on average has a fairly high degree of reliability and credibility as a news source, things tend to differ from article to article, which highlights the importance of having an indicator like Biasly’s News Check chrome extension to analyze the bias and reliability scores of any given article.
While some articles do a great job managing their objectivity and neutrality by providing credible and factual reporting, others fall short of the bar. You’ll find that as you continue to see how different articles are rated, you can gain a clearer understanding of how to spot markers of reliability and accuracy issues within an article, allowing you to improve your media literacy.
Disclaimer: Pew Research Center bears no responsibility for the analyses or interpretations of the data presented here. The opinions expressed herein, including any implications for policy, are those of the author and not of Pew Research Center.