Score Inflation in Restaurant Reviews

Ever since the demise of The Egon Ronay guide we’ve used a number of guides to help us choose restaurants: Michelin, Gault Millau (with its delightful French turn of phrase), Time Out – depending on where we are geographically.

As we used to travel abroad a fair amount for work we used Zagat quite a lot. Zagat depends wholly on the reviews of the punters. So obviously you have to believe in crowdsourcing etc at least a little bit. We used to find it reasonably reliable albeit not perfect. But we did notice that restaurants in New York, San Francisco, etc seemed to get higher ratings for ‘Food’ than the equivalent quality in London. It seemed easier for a New York restaurant to get a high score, of, say 28 out of 30. OK – that’s no big deal.

Recently, however, I’ve noticed two interesting discrepancies in London. There is a great little Tapas place in Camden Town, inner North London. It’s little more than a ‘caif’, really. But the food is honest and tasty, rustic and not ‘refined’. It’s well-priced. It gets a Food rating of 25 out of 30 in Zagat, which seems perfectly reasonable to me. However, we recently discovered a really nice Tapas place in Westbourne Grove, near Notting Hill, inner(ish) West London. There, the food is both tasty and refined and adventurous. Both places seem equally very busy. And we shall continue to go to both places although Westbourne Grove is further from home. But here’s the thing. The somewhat posher and also better Westbourne Grove place gets a measly 22. I have no idea how to begin calculating whether there is any statistically significant difference between any scores in Zagat. For a start you have no idea how many people are rating the restaurants. So we have to take them at face value or not take them at all. And Zagat has, in the past at least, definitely been useful to us. But it struck us as an odd difference. It should, to our way of thinking, have had a better score than the Camden Town place, not a poorer score. I kind of hypothesised that the possibly better-heeled Notting Hill crowd had higher expectations than the Camden Town crowd. And maybe foodie Londoners have higher expectations than foodie Washingtonians (DC). So Zagat scores are geographically sensitive.

So far so good. I can live with that – within limits.

But the limit was breached a couple of nights ago when we explored a restaurant in Highbury/Islington area of slightly grungy inner North London which had been given a 28 Food score. We thought – that should be interesting.  The important thing to note is that the highest score for Food in the London Zagat appears to be 29. That’s for The Waterside Inn – (way out of London). Some really, really posh and also truly excellent and well known places get 28 in Central London. So, what about this place? Folks, it was… err… OK. But is was certainly nowhere near the quality of a Central London 28.  In fact, by comparison I would only have given it a 20 or 21 – perhaps because of possible food sourcing policies – and, err, not sure –  a bit higher than the estimable Cote chain – restaurants within which tend to be given a Food score of 18, and which I’m very happy to eat in, thank you. To be fair, this place was priced quite reasonably – ‘decent’ local eatery prices.

But there you have it. Punters’ scores are as heavily influenced by who they are and where they live as the actual quality and sophistication of the food itself. I should not be in the least bit surprised by this. It’s in all the psychology textbooks in one context (aha! Context! Framing! etc, etc) or another. It’s just that a food guide punter ‘score’, especially that which is an average of those given by a large (?) number of people, has an aura of objectivity to it – which is totally unwarranted in reality.

Or maybe we were just there on a bad night? Perhaps Chef was on holiday? Dunno.

Bring back Egon Ronay?

My lady wife points out to me that we should expect to see many more discrepancies in Zagat – since it was taken over by Google and has become free. Before, you needed to be some kind of committed foodie to write a review for Zagat – as you needed to pay for the guide. Now, however, anyone, committed foodie or not, can and, at the behest of the owner (or, indeed, an aggrieved diner), may write a review, whether or not they know or care anything about restaurant food quality. Expect many more reviews of poorer quality…


About DMO

Market Research Consultant View all posts by DMO

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: