It Ain't East Knowing Who You Can Trust: News Organizations? Bloggers? Citizen Journalists? What's a news consumer to do? by Steve Outing

It Ain't Easy Knowing Who You Can You Trust
News organizations? Bloggers? Citizen journalists? What's a news consumer to do?

By Steve Outing

NEW YORK (July 30, 2007) -- Ah, life was so easy when I was younger. The daily newspaper was my major source of news, supplemented by the occasional TV news program, a bit of radio news, maybe Time magazine for some weekly analysis. There was no doubt about the veracity of any of those sources.

But today, holy moly! I still read my daily newspaper (perhaps only because I'm middle age and it's a habit I can't break), but it's a tiny slice of my daily news and information diet. There are all the blogs, of course -- from guys and gals typing in their PJs from home, to "placebloggers" (covering their home communities independently), to niche bloggers trying to make a living from their expertise, to bloggers being paid by news or other organizations, to soldiers in Iraq blogging and spouses of soldiers, to politicians, to college students.

There are all the grassroots media (a.k.a., citizen journalism) sites, with content submitted by people who I have no idea of their competence or political bias. Community micro-news, once the purview of community weeklies, is now often provided by citizen journalists. Uber grassroots media sites like Associated Content publish content from a wide variety of amateur writers (as well as pros).


There are news and information websites published by non-profits and all sorts of organizations that I can't easily categorize or assess. And there are the many online-only media outlets that have sprung up over the last decade or so.

For consumers of news and searchers of information, these are heady times. Yet there's a huge downside to this abundance: How as consumers do we know if we can trust what we read? How do we know if it's balanced, or serving someone's narrow agenda? ... OK, you might figure that out after reading a source for a while, but it's not easy at a glance to know.

Most of us are adding new news sources to our information diet all the time. We discover a new blog that we think is great, and add it to our regular reading list. That blogger refers us to other bloggers and unusual news sources. ... We frequent news aggregators like Google News, Yahoo! News and Topix, often discovering new sources that we haven't visited before. We search Google and hit upon a new source covering a topic we care about.

The problem isn't just that of the consumer -- who does need to be a lot more savvy when filling him/herself from the fattened flow of information. It's also a problem that newspapers need to deal with, because increasingly they are adding alternative sources to their websites (and to a lesser extent, but still, to their print editions).

Many a newspaper site these days publishes local bloggers, citizen journalism, eyewitness reports, etc. Set within the context of the overall news online presentation, there's typically no or little guidance offered about the likely veracity of these people.

It's time, I'd like to suggest, for news publishers to get serious about rating and ranking the alternative sources that they now publish -- to give their readers a bit of help in assessing what they're reading.

Enter Newstrust

Help, it appears, is on the way. A non-profit initiative called Newstrust is developing a system for rating news from a very broad array of sources. And unlike previous efforts that employed teams of paid reviewers (a model that proved economically unviable), this one is a social network model which uses the intellect of the masses to rate all manner of news content and news sources.

In beta now and due out in early 2008, Newstrust will not only be a stand-alone site where consumers can come to find the best journalism as ranked by an army of volunteer media reviewers, but more importantly it will (we can hope) be deployed over all manner of online news sources so that readers will on any news-related website see an objective rating of that site's quality and of specific news content.

I've been thinking for some time that there needs to be some way devised to differentiate the various flavors of content on a typical news site. As a reader, I should be made aware when an article is authored by a professional journalist, vs. a story posted by a volunteer citizen journalist. Something I've advocated is making that clear in the byline. And a link off the author's name should go to a bio page, where we'll learn that the writer is either a staff pro journalist with a degree from Medill or a school teacher who enjoys gardening and writing.

But Newstrust, I think, goes further than that, and even in another direction. The promise that the Newstrust model presents is that wherever our news search and browsing takes us, we'll see aids that will help us determine the quality of an online news source and ratings of individual stories.

After all, what we as consumers most care about in our news is accuracy, quality and balance. If we can be informed about those, then perhaps it doesn't really matter whether you're reading a piece by a New York Times reporter or a blogger you've never heard of who writes just for fun but is really good.

The Nuts and Bolts

Newstrust is not a big-money endeavor at this point. According to executive director Fabrice Florin, the project has raised only about $150,000 to date, from foundations and interested parties, including Craigslist founder Craig Newmark. The project is further along than $150,000 will typically get you because of considerable volunteer time from others involved, and Florin himself.

I used the phrase "army of media reviewers" earlier, and that's accurate. Anyone can sign up to be a Newstrust reviewer, with nothing more than an e-mail address. That gets you the right to start reviewing news sources (websites, blogs, etc.) and individual articles.

Reviewers are asked to rate news sources using a simple 5-star selection. Each source gets an aggregate score. Ratings for individual stories are more interesting and deeper; story reviewers are asked a series of questions about the article (rate with 1-5 stars):

Is it a good story?
Do you trust this publication?
Is the story informative?
Is the story fair?
Is the story well sourced?
Does the story show the "big picture"?

The reviewer also can leave a text review, which will show up on Newstrust along with the story.

What's exciting about this is the thought of Newstrust ratings showing up on stories around the web -- on all manner of news sites, from NYTimes.com to your local blogger's. Any website can sign up to run a rating button on all its content; online users rate a story and the aggregated results show up on the site as well as within Newstrust.

This thing is going to have to take off in a big way for the ratings to be really useful to consumers, of course. What I want is to find ratings of obscure news sources -- ones that I don't have a sense yet of how much I can trust them. I already have an opinion of how much I can trust (or not) the Washingtonpost.com and Fox News, but not so for many of the little guys publishing online.

Of course, I don't know what to think of alternative sources within mainstream news websites -- like community bloggers published within a newspaper site. I hope Newstrust can help with that.

With the success of news recommendation social networks like Digg, there's reason for hope that Newstrust can become popular and widely used enough to get there.

The Bias Factor

If you look at Newstrust during its beta period, take the ratings with a grain of salt. Florin reports that the site's volunteer reviewers currently have a left tilt. It started out with far more self-described liberals, but as more people sign up to be reviewers, that's lessening and headed more toward balance. Ergo, Fox News is not going to do well on the ratings during the Newstrust beta. (Fox has an aggregate rating of 2.3 out of 5 currently -- the lowest rating of any national TV news program or network.)

Florin says that will even out in time, and the project is very much a non-partisan effort. Indeed, reviewers are instructed to rate sources and articles in an objective manner, and leave personal politics and bias out of it.

More importantly, Newstrust's developers are building bias filters into the algorithm that drives the site. Every member reviewer of Newstrust gets a "member rating," which is an assessment of their performance as a reviewer. When administrators notice a reviewer that consistently reviews sources and articles with a particular political bias, for example, that reviewer's member rating gets knocked down. While the biased member can continue rating things, his impact on the overall scores is neglible, and his written reviews are either placed where few will see them or not shown at all.

Administrators actively look for reviewers who are trying to game the system and influence ratings toward a particular point of view (e.g., a conservative reviewer who regularly gives 5 stars to opinion pieces on Fox News while ranking New York Times articles with 1 star). Such people are then sent an e-mail with a request to comply with the non-partisan spirit of Newstrust and start reviewing content and sources objectively.

The site also has tiers of reviewers. Everyone starts out as a Reviewer, but if you really get into this and perform in an objective manner, you can be promoted to a Host or even an Editor.

A future enhancement to the site will be bias-adjusted reviews -- that is, optional ratings based exclusively on reviews by Hosts and Editors. (That may be a paid premium service.)

Hope for theFuture

It's way too early to tell if Newstrust will make it, and if its usage will become widespread enough to be truly useful. I hope this initiative succeeds, because it addresses a sore cultural need.

The wealth of news and information now available freely on the Internet is both blessing and curse. That we have this much at our fingertips -- and so inexpensively -- is an amazing development of the last decade. But we're still in the information dark ages, in a way, because we don't yet have tools for consumers to assess what they're reading.

Let's hope that Newstrust and its inevitable coming competitors figure out how to solve this problem.
Steve Outing (letters@editorandpublisher.com) has been writing a column for E&P for most of the past decade.
http://www.editorandpublisher.com/eandp/columns/stopthepresses_display.jsp?vnu_content_id=1003618132