Wine Spectator reveals Top 100 Wines of 2009, but... are all wine rating systems flawed?
Given the high price of wine and the enormous number of choices, a system in which industry experts comb through the forest of wines, judge them, and offer consumers the meaningful shortcut of medals and ratings makes sense.
But what if the successive judgments of the same wine, by the same wine expert, vary so widely that the ratings and medals on which wines base their reputations are merely a powerful illusion? That is the conclusion reached in two recent papers in the Journal of Wine Economics.
He's referring to findings published by Robert Hodgson, a retired statistics professor and the proprietor of Fieldbrook Winery. A few years ago, Hodgson joined the California State Fair wine competition advisory board, which allowed him to run a controlled scientific study of its tastings.
The results, published in the Journal of Wine Economics, showed that the judges' ratings varied by ±4 points on a standard 100-point rating scale. And "only about one in 10 [judges] regularly rated the same wine within a range of ±2 points."