Wednesday, April 21, 2010

2010 Chocolate Tasting

The results are in from this years chocolate tasting and the winner is... Switzerland. Yes, the whole country. This year's Swiss chocolates resoundingly topped the rankings in every category. In fact, only 2 of the top 10 chocolates are not Swiss.

Winning Swiss Chocolates
In the milk category, Lindt retains the number one position for the third year running with familiarly delicious caramel flavors. In the darks category, the small family run chocolateir Gilgen was a stand out winner. Gilgen's creamy "home-made chocolate" was dramatically popular, beating Callier, Lindt, Teuscher and Bachman, all swiss makers themselves which placed 2nd through 5th respectively. Callier, the oldest Chocolate maker in switzerland, won the super dark category in a decisive victory with their Sublime bar, which was the least contravertial (lowest standard deviation) chocolate on the table- everyone loved it.

Other Tasty Chocolate Makers
While Swiss chocolate dominates the winnings, several other chocolates are worth noting. Haigh's Chocolates of Australia performed very well, taking 2nd place in super dark and 6th place in dark making them the best non-Swiss chocolate in each category. Eco+, a disount milk chocolate sold in French grocery stores also performed very well, taking 3rd place in that category. Divine retained it's position as 5th in the milk chocolate category but was stripped of it's winning rank in super dark- a bevy of imported Chocolates knocked Divine down to 16th place overall. It's all realative though- Divine still got very high ratings, and compared only against chocolates bought in the US it takes 2nd place. The most delicious chocolate aquired in the USA was also made in the USA: Amano Chocolates' Montanya bar from Orem, UT. We'll have to see how they perform against Guittard in next year's tasting, as Guittard has historically taken the top marks amoung the American chocolates.

Several new chocolates in the milk category are noteworthy- the Lake Champlain Milk chocolate pretty popular- mostly amoung boys. Neuhaus, the famous Belgian maker also received good scores for their milk chocolate- that one was more popular amoung the ladies. Valhrona also deserves a special note this year, (not for their regular bars, which had only average performance)- Valhrona is the first maker to produce an Organic chocolate which most people really liked.

We had three chocolates in the milk category that were marketed as dark chocolates, but since Milk was a main ingredient they were officially classified as milk chocolates. Of these, Alter Eco Fairtraide's "Dark Chocolate Velvet Touch of Milk" bar and Goya's "Dark chocolate" bar both did very well, placing 2nd and 4th places respectively. The third dark milk, Sarotti, didn't do very well over all, but all three chocolates received top marks from super-dark chocolate fans. In fact, these three chocolates were the most popular "milks" amoung the milk haters- which calls their overall ranking into question. If we remove these dark milks, Neuhaus and Valrhona (Organic) move up to 4th and 5th places, respectively.

Some Interesting Correlations
The chocolate with the most sugar per gram and also the most cholesterol was Ikea's Milk bar, which did surprisingly well, ranking 10th in milks. This was also the cheapest chocolate on the table at 99 cents per 100g bar. This year there was a slight correlation between chocolate and price- a negative one ;). This was probably skewed by the high price and horrible quality of the Hawaiian Chocolates. After three years of data on price correlations, we can safely say price is no indication of chocolate quality.

Whittaker's Dark Chocolate from New Zealand was the saltiest chocolate on the table. People with a sweet tooth really loved it, wheras people whose scores had a negative correlation to sugar content absolutely hated it. Other "sugar dimorphic" chocolates included Spruengli's Trinitario de Santo Domingo (sweet-toothers love it) and Nestle Club Extra-Fine 70% (low-sugar fans loved it). There was a high correlation between sugar content and chocolate score- allmost all the high-sugar chocolates were ranked between 2.5 and 3.5, suggesting that you if you add enough sugar to your chocolate it will always come out "good enough," but you can never make a really delicious chocolate by adding more sugar. This was likewise the case with Chocolates low in saturated fat- they are always "good but not great."

Chocolates with no cholesterol and no additives performed better, in general. Chocolates performed by companies over 50yrs old performed the best. No chocoloate over 80% received good ratings, even when only looking at the reviews of super dark chocolate fans. As we keep seeing, it's just hard to make chocolate that dark with a good taste and melt.

Overall, Europe beat out America once again, but that contest could be summed up as "Switzerland vs Anyone Else"- there's just no competition. One very striking result of this tasting is correlation with bean source. The two largest growing regions for Cocoa beans are Africa and South/Central America. With very high confidence we can say that chocolates with South/Central American beans outperform chocolates made from African beans. Though, while the beans made in South/Central America produce delicious chocolate, we've seen that the chocolate bars made there are universally unpleasant.

Under the heading of interesting but not delicious we have a few chocolates of note. Amadei's Toscano Black Bar from Italy was the most contravertial chocolate, and also the the most sexually dimorphic chocolate- girls thought it was OK, but boys really hated it. The highest calorie chocolate was the Amano's Milk Chocolate bar, which performed poorly this year as it did last year. This lead me to investigate if, as we've seen with Sharffen Berger, artisan chocolate makers either good at dark or milk but not both. The verdict is still out but the correlation looks more like "American milk isn't very good" then anything else. The worst chocolates on the table were unmistakably the Theo Organic bars, which were hated by almost everyone and described as "acid bitter death."

What to Buy
We can conclude from this tasting a straightforward approach to choosing tasty chocolate. Buy swiss chocolate, preferably from a company at least 50 years old, that sources their beans in Central or South America, with low sugar, high saturated fat, no artificial additives and a cacao content of 25-30% for milk chocolate, 45-50% for dark, or 70% for super dark.

A note on data integrity
We threw a little surprise in this year's tasting- 4 chocolates appeared twice on the table. Out of ~1700 tastings, 120 were duplicates. We noticed something interesting... only 47% of the time did the same chocolate receive the same score by the same person. Almost all of these (95%) were one-off's (a 2 and a 3 was most common). Which means that, in general, your score is only good to within +/-1. What about the overall data quality then? Well, it actually seems to mostly average out across all the tasters. For example, the overlapping chocolates received ranks of 1st/6th and 5th/7th in the milks category, with a very small difference in normalized score. What I take away from that is not to read the absolute rankings as gold truth on "which is the best chocolate?" but instead to use the rankings as a more general guide for which chocolates are likely to be tasty. We've already seen over the years that, with the exception of Lindt, chocolate quality seems to fluctuate from year to year anyway.

Thank You
A big thanks to Madleina for the Swiss chocolates, Jan for the German chocolates, and Cecile for the French chocolates. This International tasting was only possible because they purchased and carried home almost 10lbs of Chocolate. Thanks also to all of our 42 testers, who not only braved a torrential thunderstorm to get here, but also faced the far more intimidating challenge of 85 heaping plates of chocolate. Thanks also to Amano Chocolate for donating chocolate for this tasting- we enjoyed it.

Further Information
Do you care about the math? Are you curious how some of your favorite chocolates performed? Want to see how your scores measured up against everyone elses? Here's the complete anonymized* dataset.

* If you tasted chocolates and you'd like to know which anonymous taster you were (e.g. to see your sugar correlation or other information from the person-profiling analysis), email me.