Testing and cheating

Has LG Electronics been naughty? Apparently it had installed a ‘circumvention device’ in a couple of its refrigerators. This, it is said, turns down the fridges when it senses they are in a 22C environment. The aim was to reduce their energy consumption in what would be expected to be standard test environments, thereby allowing them to achieve better scores and, consequently, more stars on their energy rating labels.

When this was pointed out to me today, I was quite prepared to take this at face value. LG naughty. But, as I wrote in an email this afternoon:

I am a bit surprised that LG would think it worth the extra sales/kudos from having an extra star on the fridge to take the chance of cheating.

Especially, now that I think about it, 22C is not an uncommon temperature in a home, so to have the unit function at a reduced level at that temperature could be positively dangerous, what with food spoiling and the like.

Now in one of those weird coincidences, later in the day I listened to a Quick Hitts podcast by Dave Hitt. I’d recommend these to anyone. He’s somewhat of a mad libertarian like me, but as a speaker on the radio he sounds so amiable and reasonable that it must take an effort of will to be offended. At the end of the podcast he asked for links to help boost the Google rating of his blog, and I thought I’d try to oblige. He suggested Googling his site for possible subjects of interest, so I Googled it for ‘audio’.

That led to this post on how the United State’s version of Choice magazine, Consumer Reports, is ‘clueless’ on audio, on cars and on … refrigerators.

This Blog entry was going to be about audio, but with the forgoing on LG’s fridge woes in mind, the most interesting thing was the closing section of Dave Hitt’s post.

Hitt is American and the post was written in September last year, so he had no knowledge of the current LG matter. But in it, he relates how a decade ago he overheard the engineers in a refrigerator company trying to deal with the fact that their fridges scored poorly in Consumer Reports‘ tests. The problem was that their fridges were optimised for normal room temperatures, but CR’s test procedure used a much higher temperature environment. Somehow other manufacturers’ fridges were able to perform well there. Read the details here, including how optimising for CR would actually reduce overall efficiency.

Now let’s put that story together with LG’s current troubles. Could LG’s ‘circumvention device’ have been developed initially to deal with CR’s test procedure? Might there have been a whole lot of shonky practices emanating from this, and not just by LG alone?

This entry was posted in Testing. Bookmark the permalink.

1 Response to Testing and cheating

  1. Cindi Knox says:

    Building to the test:

    In the USA, we have “No Child Left Behind”. It’s a program that requires standardized testing of schoolchildren, and schools that do not perform well can lose funding or even be closed. Teachers whose students do not perform well can be fired.

    The response is that teachers are encouraged to “teach to the test”. Given students a well-rounded understanding of the subject is secondary to performing well on the standardized test.

    I can see the same thing happening with appliances. Companies (schools) can suffer loss of revenue or even close if their appliances do not perform well. Engineers (teachers) can lose their jobs. And so there is great incentive to build (teach) appliances (students) to do well on the tests, at the expense of overall performance.

    Institutional measurements are usually simple compared to real-world use. Whether you’re testing a child on dates and names instead of historical significance, a refrigerator on efficiency at 22C instead of overall performance, or an audio component on THD at 80% of rated power instead of overall sound, you are taking a snapshot instead of a three dimensional movie.

    These benchmarks are useful for comparison because they are based on easily comparable standards. But when we narrow our information stream to a single source using a snapshot standard, we lose the opportunity to judge overall performance. Worse, we encourage suppliers to focus on one or a few aspects of performance at the expense of overall usability.

    It is up to consumers to recognize that benchmarks are only on source of information and to stop ceding their decisions to others. To do so is lazy and will only result in useless products that do well on tests.

Leave a Reply

Your email address will not be published. Required fields are marked *