ANN ARBOR—As the crucial mid-term election approaches, the University of Michigan Center for Social Media Responsibility offers media and the public a tool to help monitor the prevalence of fake news on social media through a Platform Health Metric called the Iffy Quotient.

A web-based dashboard that shows the Iffy Quotient for Facebook and Twitter, dating back to 2016, will be updated regularly.

The Center for Social Media Responsibility, supported with funding from the Office of the Provost and Executive Vice President for Academic Affairs, is housed in the U-M School of Information. The first report issued by the interdisciplinary center confirms from a deep analysis what was suspected to be true about the 2016 U.S. presidential election: the Iffy Quotient increased dramatically on both social media sites during the election.

It also shows that the two social media platform companies have made progress since early 2017 on their promises to crack down on misinformation, but one has succeeded more than the other.

“We at the School of Information are committed to meeting the intellectual and social challenges of this new era of unregulated public communication via social media. At the Center for Social Media Responsibility, we are working directly with social media companies to produce the designs, systems and metrics to steer social media toward more beneficial discourse,” said Thomas Finholt, dean of the School of Information.

“Contributing to the accountability of social media platforms is one reason why the CSMR was created. This is an important area of focus for our faculty research. Our aim is to serve as a valued resource in the battle against misinformation.”

The School of Information announced the center in March to bring together existing campus research expertise to partner with others to address the explosive, unbridled growth of social media and its resulting challenges and opportunities. The center helps media platforms meet their public responsibilities by identifying the challenges, investigating potential solutions and measuring progress.

The Iffy Quotient is the center’s first public measurement tool. It draws data from two external entities: NewsWhip and Media Bias/Fact Checker.

NewsWhip, a social media engagement tracking firm, collects URLs on hundreds of thousands of sites every day and then gathers information on which of those sites have engagements on Facebook and Twitter.

Iffy Quotient queries NewsWhip for the top 5,000 most popular URLs on the two social media platforms. Then the U-M tool checks to see if those domain names have been flagged by Media Bias/Fact Check, an independent site that classifies various sources based on their reliability and bias.

The U-M tool divides the URLs into three categories based on the Media Bias/Fact Check lists: “Iffy,” if the site is on the Questionable Sources or Conspiracy lists; “OK,” if the site is on any other list, such as Left-Bias, Right-Bias or Satire; “Unknown,” if not on any list.

Here’s what else the analysis found:

• On both Facebook and Twitter, the Iffy Quotient approximately doubled from January to November 2016.
• The Iffy Quotient was higher at Facebook than Twitter in 2016 and into 2017.
• The Facebook Iffy Quotient has declined steadily since early 2017 and has now returned to its early 2016 levels.
• The Twitter Iffy Quotient has not declined much and is still nearly twice its level in early 2016.

The Facebook and Twitter Iffy coefficients were roughly comparable through most of 2018 but Facebook’s is now somewhat lower. The contrast between Facebook and Twitter is even more pronounced in an engagement-weighted version of the Iffy Quotient, which we can think of as a rough proxy for the fraction of total user attention.

In 2016 the Iffy sites’ share of attention was about twice as high on Facebook as Twitter; now it is 50 percent higher on Twitter.

“By contrast with the current environment of accountability by ‘gotcha’ examples of bad outcomes, the Iffy Quotient tells us something about the overall performance of the platforms,” said Paul Resnick, founder and acting director of the center, and associate dean for research at the School of Information. “The platforms can track metrics internally with their own data, but hesitate to report them externally. By publishing continuously, we can provide accountability when things get worse and credibility for claims of progress.”

Resnick said reporters and members of the public who would like to monitor social media for the upcoming election and beyond can sign up to receive alerts when the Iffy Quotient changes.

The team plans to add other sources and additional social platforms to the tool. When that happens registered users will also receive an alert.

CSMR leaders said there are some limitations to the tool and study. Iffy sites sometimes publish reliable information but the metric treats all their URLs as Iffy. NewsWhip also may miss some popular URLs, especially from newer fly-by-night sites. Media Bias/Fact Check may not have gotten around to listing newer Iffy sites. And NewsWhip’s URLs include some that are not related to news or public affairs; the team plans to automatically filter those out in a future version of the tool.

Because of these limitations, the report authors caution against over-interpreting the Iffy Quotient.

“Trends in the Iffy Quotient over time are meaningful,” Resnick said. “The absolute number…not so much.”

CSMR’s Aviv Ovadya and Garlin Gilchrist contributed to the report.

More information:

• Report (PDF): Iffy Quotient: A Platform Health Metric for Misinformation
• Dashboard
• Center for Social Media Responsibility
• NewsWhip
• Media Bias Fact Check