I started brewing in 1997 and man has the beer landscape changed in 17 years. Craft beer was around but you had to seek it out – you’d be very lucky to even find a Sam Adams at your local chain restaurant. Finding respectable samples of many styles was tough – Sierra Nevada Pale Ale was around but the IPA craze had not hit. Belgians were a rare breed. If a home brewer chose to make one of these rare styles, then finding a good quality sample of a classic style was not easy. That made me seek out information about beer judging and due to the internet the Beer Judge Certification Program was a quick find. It intrigued me. So much to learn about styles and brewing methods. I knew I wanted to try and become a judge.
It took awhile, but eventually I found my was to an exam and four years ago this January I took the BJCP exam, and even better I passed it. It was a tough test, 3 solid hours of writing and some beer tasting. And today I am a Certified Beer Judge.
But when I look back on why I originally sought out judging information, it was self-doubt. I made these beers – but were they flawed? I read about all of these off flavors – diacetyl, oxidation, acetone. Were they in my beer? Was I missing something? Well today I worry much less about such things. I have a little experience and know what I like – and I make beer to drink what I like. But judging others’ beers is a different thing. Not only are you charged with finding flaws, you are charged with picking out the best beers.
The reason I mention this is Robert Hodgson. He happens to be a winemaker and a statistician. And he noticed something about the wines he entered into competitions. Sometimes a particular wine would do very well, other times it would be judged poorly. So he proposed an experiment to the California State Fair competition in 2005, and he repeated for eight years. And what he discovered is that wine judges are pretty inconsistent at what they do. His bottom line: if a wine wins an award it most probably by chance, not merit.
Hodgson was interviewed on the Science for the People podcast. Check it out. The same episode includes a great interview with Charlie Bamforth, the professor of brewing at Cal-Davis. If you have never heard Charlie talk beer you are missing a treat.
So science says that my ability to consistently discern the flavor characteristics of a particular beer is quite low. I can live with that. I just hope all the brewers whose beers I have judged can live with it too.