Benchmarks aren’t useful anyway, says cofounder
The Middle Kingdom manufacturer sent out review units to a few publications before the launch (El Reg isn’t on their lovey list) and coder blog XDA Developers analyzed a unit. According to its research, the new OnePlus 5 handset has been set up to game popular benchmarking software.
“It is almost certain that every single review of the OnePlus 5 that contains a benchmark is using misleading results, as OnePlus provided reviewers a device that cheats on benchmarks,” self-described Android obsessive Mario Tomás Serrafero wrote.
“This is an inexcusable move, because it is ultimately an attempt to mislead not just customers, but taint the work of reviewers and journalists with misleading data that most are not able to vet or verify.”
According to the research, the OxygenOS Android skin used by OnePlus has software that detects if the phone is running one of seven popular performance-testing applications. If so it ramps up performance to a maximum of 1.9Ghz. The affected suites are:
- Geekbench 4
- Nenamark 2
XDA managed to disable the cheat codes and found that this dropped performance levels by an average of 5 per cent. That’s not a lot, but enough to make the OnePlus handset look considerably more capable than it is.
This isn’t the first time this has happened. Back in January, XDA noted that the OnePlus 3 – the manufacturer’s previous phone (the number 4 is considered unlucky in China and was skipped) – also included benchmark cheating code, albeit in a slightly different form.
OnePlus has not responded to requests for comment from The Reg, but cofounder of the company Carl Pei went onto Reddit to defend his handset. He listed the ways the software doesn’t cheat, by not overclocking the chip or lowering screen resolution, and said that what OnePlus had done was perfectly acceptable practice.
“We have made it so that when running benchmark apps, the phone performs the same as when running resource-intensive apps such as 3D games. We also fully activate our chipset in other parts of OxygenOS, for instance when launching apps to make the launch experience faster and smoother,” he said.
“When users run benchmark apps – which I agree aren’t a useful proxy for real-life performance – we believe that they want to see the full potential of their device without interference from tampering. That’s what we’ve unlocked.”
Interestingly, one performance-testing app – 3D Mark – wasn’t affected by the coding, so if you do test your phone that looks like the one to select. ®