Yelp is courting bad press, again. Earlier this month, the company released an update to its iPhone App that will enable users with the highest number of check-ins to become “Duke” or “Duchess” of a venue and reap special benefits. Fair enough Yelp, you’ve gotta keep up with the Jones’ of tech, but at least come up with your own name for a badge. It’ll be interesting to see how Foursquare, which has already dubbed hundreds of thousands of users “Mayor”, will respond to such a blatant rip-off (are they still nursing a grudge from the check-in copycat?).
Don’t get me wrong. I’ve been a Yelp fan and “registered user” for years and I visit the site nearly every day for user ratings, directions, pictures and even to post the occasional review. But the controversy surrounding the company just never seems to end, and it’s starting to feel like the signal to noise ratio of its reviews may be getting weaker and weaker…
The allegations began in early 2009 when dozens of business owners claimed that Yelp was manipulating user reviews (deleting positive reviews and highlighting negative ones) in order to extort advertising revenue from them. The MO of Yelp sales reps, as reported by several newspapers, is to call a business owner with the standard sales pitch, persist in calling even after he/she has turned down the service and then threaten him/her with negative reviews. Is this true? Or are these just bad businesses searching for a scapegoat after an inevitable demise?
Yelp CEO Jeremy Stoppelman vehemently denies the claims (and I want to believe him). He explains that Yelp uses a computer algorithm to rank the user reviews, displaying them in order of “usefulness” rather than chronology (unlike typical user comments). This “usefulness” is determined based on a number of criteria including the ranking / popularity of the reviewer and the frequency with which he/she posts reviews. Sound familiar? Digg ranks its stories and comments with, what many users consider to be, the ultimate black box algorithm but allows users to vote up or bury comments (unlike Yelp).
Stoppelman says the Yelp algo protects against restauranteurs and retailers having their friends and family post glowing reviews and bumping up the venue’s ratings. It does this by strategically burying and highlighting certain reviews. So it isn’t an unbiased system, clearly. It is just unclear whether the bias here is “fair” or extortionary.
As the protests gained traction, a civil action lawsuit was filed against Yelp by 10 business owners alleging unfair business practices. On April 5, Yelp acknowledged the issue. It added links to deleted comments and took away paying members’ ability to place positive evaluations atop their profiles. The outcome is TBD.
But there’s another issue with Yelp’s model that continues to bother me, and that is the user motivation behind writing a review. The typical user will be inclined to post a review in one of two cases: either he had a stellar experience and felt compelled to share his joy with the world or he had the worst experience ever and wanted to tell everyone how much it sucked. This means the majority of reviews are going to fall into an extreme bucket and the average restaurant/retailer will either get a much better, or much worse, rating than it actually deserves.
I’m hoping (guessing) that Yelp’s algo accounts for this somehow (give greater weighting to the frequent and equitable reviewers?) but since moving to San Francisco reviews, I’m losing faith. So many places that me and my friends absolutely adore have mediocre Yelp ratings and so many of the reviewers cite the strangest rationales for their poor ratings (see below) but still receive prime real estate (“above the fold”) on the review page.
Am I grossly out of whack with the San Francisco foodie? Or…could it be that more reviews on Yelp (in SF) = less accuracy?
Its unclear, but Yelp’s move to go social, by partnering with Facebook, may be one solution to the above. By pulling your social graph into the site, Yelp enables you to see exactly where your friends have been going (thanks for checking in), how often they go there (number of check ins) and what they think of it (reviews). Stalking aside, the value to this seems tremendous. People are much more comfortable trusting their friends’ opinions, recommendations and biases vs. those of Yelp’s unknown millions.
Whether socializing the site will be the silver bullet to all of Yelp’s woes remains to be seen, but it still feels like there’s a market gap and a recommendation problem here that needs solving. It just may not be in Yelp’s destiny to solve it.
A Wandering Feast 