May 08, 2011

Benchmarks. Can we trust them?

Benchmarks can be useful, whether we're talking mobile or any other computer hardware. There is a problem though -- you have to depend on everyone being honest. Let me explain. Anytime you see a video or a screenshot of a benchmark score, what you're really seeing is a measure of how the software interacts with the hardware. There are two big problems with that: No two Android phones run the same software, and there's always some joker who wants to cheat. Recently our pal John from Phone Arena noted some Quadrant scores of a new handset that are just too low compared to the other phones listed. And good on John for pointing it out.