April 12, 2008

Pollution as revenge

One of my students alerted me to a recent dramatic episode. Author and psychologist Cooper Lawrence appeared on a Fox News segment and made some apparently false statements about the Xbox game "Mass Effect", which she admitted she had never seen or played. Irate gamers shortly thereafter started posting (to Amazon) one-star (lowest possible score) reviews of her recent book that she was plugging on Fox News. Within a day or so, there were about 400 one-star reviews, and only a handful any better.

Some of the reviewers acknowledged they had not read or even looked at the book (arguing they shouldn't have to since she reviewed a game without looking at it). Many explicitly criticized her for what she said about the game, without actually saying anything about her book.

When alerted, Amazon apparently deleted most of the reviews. Its strategy apparently was to delete reviews that mentioned the name of the game, or video games at all (the book has nothing to do with video games). With this somewhat conservative strategy, the reviews remaining (68 at the moment) are still lopsidedly negative (57 one-star, 8 two-star, 3 five-star), more than I've ever noticed for any somewhat serious book, though there's no obvious way to rule these out as legitimate reviews. (I read several and they do seem to address the content of the book, at least superficially.)

Aside from being a striking, and different example of book review pollution (past examples I've noted have been about favorable reviews written by friends and authors themselves), I think this story highlights troubling issues. The gamers have, quite possibly, intentionally damaged Lawrence's business prospects: her sales likely will be lower (I know that I pay attention to review scores when I'm choosing books to buy). Of course, she arguably damaged the sales of "Mass Effect", too. Arguably, her harm was unintentional and careless (negligent rather than malicious). But she presumably is earning money by promoting herself and her writing by appearing on TV shows: is a reasonable social response to discipline her in her for negligence? (And the reviewers who have more or less written "she speaks about things she doesn't know; don't trust her as an author" may have a reasonable point: so-called "public intellectuals" probably should be guarding their credibility in every public venue if they want people to pay them for their ideas.)

I also find it disturbing, as a consumer of book reviews, but not video games, that reviews might be revenge-polluted. Though this may discipline authors in a way that benefits gamers, is it right for them to disadvantage book readers?

I wonder how long it will be (if it hasn't already happened) before an author or publisher sues Amazon for providing a nearly-open access platform for detractors to attack a book (or CD, etc.). I don't know the law in this area well enough to judge whether Amazon is liable (after all, arguably she could sue the individual reviewers for some sort of tortious interference with her business prospects), but given the frequency of contributory negligence or similar malfeasances in other domains (such as Napster and Grokster facilitating the downloading of copyrighted materials), it seems like some lawyer will try to make the case one of these days. After all, Amazon provides the opportunity for readers to post reviews in order to advance its own business interests.

Some significant risk of contributory liability could be hugely important for the problem of screening pollution in user-contributed content. If you read some of the reviews still on Amazon's site in this example, you'll see that it would not be easy to decide which of them were "illegitimate" and delete all of those. And what kind of credibility would the review service have if publishers made a habit of deciding (behind closed doors) which too-negative reviews to delete, particularly en masse. I think Amazon has done a great job of making it clear that they permit both positive and negative reviews and don't over-select the positive ones to display, which was certainly a concern I had when they first started posting reviews. But it authors and publishers can hold it liable if they let "revenge" reviews appear, I suspect it (and similar sites) will have to shut down reviewing altogether.

(Thanks to Sarvagya Kochak.)

Posted by jmm at 01:42 PM | Comments (0) | Permalink »

November 30, 2006

Yelp: Local reviews via social networking site: why contribute?

So, reviews of local businesses written by local patrons are popular. Why not? Newspapers have always done well running "Best of ___" or "Reader's Choice" contests. Now we have Yelp.com, Judy's Book, Intuit's Zipingo, Insider Pages, and offerings from Yahoo!, Microsoft Live and other players. Even our small city (Ann Arbor, MI) has about 250 businesses reviewed by the newest entrant, Yelp:

And the venture capitalists are giving the new players some dough.

But, why? These sites will make revenues if they sell ads, which should work if there are eyeballs since the eyeballs will be looking specifically for businesses in the local area so advertising on the page should have a good return. But to get eyeballs, these sites have to get volunteer labor to enter ratings and write reviews. And those volunteers come from a diffuse group of local business patrons, many of whom don't know from Web 2.0, and even fewer know about Yelp.com. And even if they know, what's in it for the volunteers?

It's possible that these Web 2.0 companies are simply using Incentives 1.0: They could hire paid reviewers who at least seed the site with reviews on a number of popular businesses in each city. Yelp and the others claim that they don't do this: "real reviews from real people" (I guess we're supposed to assume that paid employees are not real people). But how would users know if they did? What forfeitable bond is Yelp posting to convince us they are trustworthy? Or if they bribed "real people" to do reviews by sending a salesperson to the establishments and handing out bling in exchange for promises to enter a review?

There's another old-school way to get review content generated, too: tell the business owners about your site, and they'll take the initiative to write their own reviews (the "Amazon" problem). And so that they look popular -- not just loved by one critic -- they ask their mothers and cousins to submit reviews too. Again, how could we tell?

Posted by jmm at 01:46 AM | Permalink »

October 11, 2006

Market for Diggs

Something valuable happening on the Net? Then a market will emerge for it. The Blog Herald reports that there is now a service that sells "Diggs" (gets users to "digg" stories posted on digg.com so that they get ranked higher),

Publishers get to pay $20 and an additional $1 per dig, and digg users can get paid $0.50 for every 5 stories they digg.

Posted by jmm at 04:16 PM | Permalink »

April 08, 2006

Polluting user-contributed reviews

A recent First Monday article by David and Pinch (2006) documents an interesting case of book review pollution on Amazon. A user review of one book critically compared it to another. Immediately following a "user" entered another review blatantly plagiarizing a favorable review of the first book, and further user reviews did additional plagiarizing.

When the author of the first book discovered the plagiarism, he notified Amazon which at the time had a completely hands-off policy on user reviews, so it refused to intervene even for blatant plagiarism. (The policy since has changed.) Another example of the problem of keeping bad quality contributions out.

David and Pinch remind us that when an Amazon Canada programming glitch revealed reviewer identities,

a large number of authors had "gotten glowing testimonials from friends, husbands, wives, colleagues or paid professionals." A few had even 'reviewed' their own books, and, unsurprisingly, some had unfairly slurred the competition.

David and Pinch address the issue of review pollution at some length. First, the catalogue six discrete layers of reputation in the Amazon system, including user ratings of reviews by others, and a mechanism to report abuse. Then they conducted an analysis of 50,000 reviews of 10,000 books and CDs. Categories of review pollution they identified automatically (using software algorithms):

They also make an interesting point about the arms-race limitations of technical pollution screens:

The sorts of practices we have documented in this paper could have been documented by Amazon.com themselves (and for all we know may have indeed been documented). Furthermore if we can write an algorithm to detect copying then it is possible for Amazon.com to go further and use such algorithms to alert users to copying and if necessary remove material. If Amazon.com were to write such an algorithm and, say, remove copied material, this will not be the end of the story. Users will adapt to the new feature and will no doubt try and find new ways to game the system.

Posted by jmm at 02:35 PM | Comments (0) | Permalink »

February 07, 2006

Incentives to review

IgoUgo: Travel Reviews, Vacation Pictures, Travel Deals

IgoUgo is (apparently) a popular travel review site (I found it mentioned in the NYT article linked in my preceeding entry).

They offer to pay incentives to people who post reviews: "Go Points" they will redeem for gift cards, frequent flier miles (natch!), etc, from Amazon, iTunes, and others. Here's a screen shot (in case they change it or take it away):

Posted by jmm at 08:21 AM | Comments (0) | Permalink »

Incentives to misrepresent

Hotel Reviews Online: In Bed With Hope, Half-Truths and Hype - New York Times

The NYT discusses an increasing problem with informal review and recommendation sites: insincere or misleading postings. Here, they talk about hotels that either post fake (positive) reviews about themselves, or that offer inducements (discounts, etc.) to customers to post positive reviews, or that bribe web sites and blogs to remove negative reviews.

The Times suggests this is not a problem for professional reviewers in the traditional media, but of course that's horse manure: restaurant critics who are spotted get special treatment, book and movie reviewers are offered inducements. The "payola" scandals in broadcast radio are famous (and ongoing).

Chris Dellarocas has written a paper about this (motivated in part by revelations about a year ago that Amazon's book review system was being scammed by authors and publishers).

Where is the ICD in this? Well, for starters, those with an interest in the outcome are offering incentives to induce (misleading) behavior. Which drives home the countervailing incentives problem: how do we discourage low-quality reviews and recommendations, and encourage high quality?

(It may be obvious, but this is another example of the growing world of "spam": unsolicited communications that are undesired or misleading. They are not pushed into users' mailboxes, but other than that, how different is it from adds touting the miraculous powers of herbal "viagra"?)

Posted by jmm at 08:06 AM | Comments (0) | Permalink »