Galaxy Note 3 Benchmark Boosts Miss The Point That No One Cares About Benchmarks

Galxy Note3_031_set2

Samsung is being called out by the highly respected and thorough Ars Technica for apparently “artificially” boosting the Galaxy Note 3′s performance specifically when it comes to benchmark testing.  The blog found that while under normal testing the Note 3 vastly outperformed the LG G2, which has the same processor, after stripping away some fancy benchmark-specific code, the phone scored about the same as its LG competitor.

Ars has a very good, very long explanation of how they arrived at their findings, and the end result is an artificial benchmark bump of between 20 and 50 percent in all areas depending on which benchmarking tools you use, including industry standards like Antutu and Geekbench. It’s a good read if you’re interested in that sort of thing, but the upshot is, you probably aren’t. Which is why Samsung has even more egg on its face.

Artificially enhancing performance benchmarks for a smartphone these days is like artificially enhancing the smoothness of your elbows via plastic surgery: it may mean that overall, you technically present a more attractive package on the surface, but no one’s really going to know or care that you’ve had any work done.

Apple’s iPhone 5s reportedly benchmarks up in the same ranks as some fairly recent Mac computers, for instance, but that’s not something your average iPhone 5s buyer is likely to know. Also, it doesn’t mean anything; benchmark scores doesn’t mean one device will be able to handle the same tasks as the other, like running a professional video editing software suite for example.

Long ago, Apple realized that a specs race wasn’t the same as the race for market dominance. Actual buyers cared about the phone experience, not abstract numbers which may or may not be borne out by really using software and apps. It’s true that Apple still talks about performance when it touts new devices – but it does so relatively, explaining only how much faster or more efficient something is compared to previous generations. That frames the discussion in terms that everyday users can understand, making it genuinely useful information.

The end result is that Samsung looks like it’s grasping when it takes an abstract (essentially meaningless, for all intents and purposes) number and artificially builds that up to win praise from some whitecoats who test these things for a living. It seems to be doing this as a matter of course now, as Ars says it’s seeing similar behaviour in testing the new Galaxy Note 10 Android tablet from Samsung as well. And, in the end, its unadjusted numbers were actually faster than competitors like the G2 anyway; if for some reason as an OEM you’re still concerned with winning a specs race on paper at this point (which you shouldn’t be), you don’t need to win by a wide margin, especially at the risk of looking foolish.

Unadjusted numbers would’ve won faint praise from the crowd that likes them, and gone unnoticed by most. Artificially altered ones attract a whole lot of negative attention and result in a net bad look for Samsung. The Note 3, like most high-end Samsung hardware, is probably a great phone, but now it’s embroiled in a doping scandal, over a number nobody really cares about.

MobileBench group aims to improve mobile benchmarking, recruits Samsung but lacks Qualcomm, NVIDIA

Industry group established to simplify and improve mobile device benchmarking, both Qualcomm and NVIDIA absent
It’s called MobileBench: an industry consortium planning to offer “more effective” performance assessments on mobile devices — most likely centered on, but not limited to, Android. Unsurprisingly after recent developments, Samsung joins as a founding member, alongside Broadcom, Huawei, Oppo, and Spreadtrum. While that’s who’s in, who isn’t? Well, both NVIDIA (responsible for the Tegra series of mobile chips) and the increasingly ubiquitous Qualcomm, which makes the Snapdragon mobile processor range. Between them, they power the likes of Microsoft’s Surface series, Amazon’s new Kindle Fire range, not to mention numerous flagship devices from LG, Samsung, Sony and Motorola.

The group gathered for the first time yesterday in Shenzhen, China and outlined how it aims to offer more useful tools for mobile platform designers and “more reliable indices” for assessing user experience. MobileBench plans to establish impartial guidelines and a more sophisticated evaluation methodology for both its first benchmark tool, MobileBench and MobileBench-UX, for testing system-level applications. The benchmarking tool will assess hardware performance, including high-level processes like video and image viewing, camera use and other real-life use cases, with one of the primary aims being result consistency and less deviation between repeated tests. Another app is planned for consumer use in the future, likely similar to the benchmarking apps Engadget uses in its reviews. The bigger question is how much the consortium can achieve without wider adoption inside the industry — it’s apparently “actively seeking” more members.

Filed under: ,

Comments

Source: MobileBench consortium (PDF)

iPhone A7 Chip Benchmarks: Forget the Specs, It Blows Everything Away

iPhone A7 Chip Benchmarks: Forget the Specs, It Blows Everything Away

We just ran benchmarks on Apple’s new iPhone 5S, revealing that, yup, this is the dopest smartphone silicon ever made. This thing freaking churns, crushing every other smartphone out there on both computational power and graphics. But if you look at common specs like core-count and clock speed for the hardware, you’d never know it.

Read more…


    



iPhone 5S benchmarks imminent as 3DMark arrives on iOS

There’s little doubt that Futuremark – folks responsible for creating the benchmarking app 3DMark – decided on syncing up the release of their gamer-aimed test with the release of Apple’s new iPhone (or iPhones). What you’ve got here is a set of two tests with lots and lots of variables, all of it already released […]

Galaxy Note 3 Benchmarks Allegedly Leaked

Galaxy Note 3 Benchmarks Allegedly Leaked

We’re less than a day away from Samsung’s event where it will unveil the Galaxy Note 3 as well as its Galaxy Gear smartwatch. Rumors about this new phablet have consistently tricked in over the past few weeks and they continue to do so. The ever reliable @evleaks has posted screenshots of alleged AnTuTu benchmark test results of the Galaxy Note 3, and boy does it pack a mean punch.

The alleged benchmark results reveal that Galaxy Note 3 scores an impressive 24,497 points. Back in August an early prototype of Galaxy Note 3 was put through the same test and it scored 22,361 points. These alleged benchmark results show a significant improvement over Galaxy S4′s score which runs slightly north of 21,000 points. Samsung’s upcoming flagship is expected to be powered by a Qualcomm Snapdragon 800 2.3GHz quadcore processor aided by 2.5GB or 3GB of RAM. One variant might also have Samsung’s own eight core Exynos 5 Octa processor. It is expected to come with Android 4.3 Jelly Bean pre-installed. Despite these results, we can only be sure about the phablet’s performance once it is really in our hands. Benchmarks aren’t really set in stone, a fact we talked about in detail after that Samsung benchmark rigging fiasco. Not to worry though, the veil over Galaxy Note 3 will finally be lifted tomorrow.

  • Follow: CellPhones, , , ,
  • Galaxy Note 3 Benchmarks Allegedly Leaked original content from Ubergizmo.

        



    LG G2 Benchmarks: The Raw Numbers

    LG G2 Benchmarks: The Raw Numbers

    Since we have an LG G2 unit in the office and are working on the complete review, we have run some synthetic benchmark numbers to see where it stand in the overall mobile hardware landscape. Note that this is not using the final firmware, so the numbers may change by the time the phone is released on American networks. Ready? Here they are: (more…)

  • Follow: CellPhones, Featured, , ,
  • LG G2 Benchmarks: The Raw Numbers original content from Ubergizmo.

        



    Windows 8 found to skew benchmark results on overclocked hardware

    Windows 8 found to skew benchmark results on overclocked hardware

    Overclocking may yield impressive benchmark results, but it turns out scores from Windows 8 PCs may not be reliable. The management at overclocking community HWBOT has discovered that tests provide inaccurate stats when then CPU base clock frequency is fiddled with from within the OS. Hardware-based real-time clocks (RTCs) help keep accurate track of time, but the operating system’s timekeeping somehow slows down or ramps up when processing speeds are tweaked. When underclocked by six percent, the outfit’s Haswell-infused system lagged 18 seconds behind actual time, fooling the benchmark into a higher score since it seemingly finished in a shorter period of time. Conversely, a boost to CPU speeds results in a lower mark as the internal timepiece ticks away faster than usual. However, modifying processor speeds at boot time avoids these issues.

    As a result of the revelation, HWBOT is no longer accepting benchmarks from computers running the eighth iteration of Ballmer and Co.’s software, and will invalidate those already in its database. “Simply no benchmark – not even 3DMark – is unaffected by Microsoft’s RTC design decisions,” the outlet adds. The timing issues are said to stem from Windows 8’s support of disparate hardware setups, including embedded and budget PCs that don’t have a fixed RTC. If you’d like to see the inconsistencies for yourself, head past the break for video proof.

    Filed under: ,

    Comments

    Via: ExtremeTech

    Source: HWBOT

    Benchmarks hint at Snapdragon-based Kindle Fire HD with 2,560 x 1,600 display

    Benchmarks hint at Snapdragon-based Kindle Fire HD with 2,560 x 1,600 display

    While benchmark leaks can be wobbly info-stones to tread upon, the arrival of some purported Kindle Fire HD scores could be the first steps toward a timely refresh. The results in question refer to a “Amazon KFAPWA” device with a reported Adreno 330 GPU, which by implication would likely mean a Snapdragon 800 SoC. The field that really leaps out, however, is that 2,560 x 1,600 resolution. If true, it could mean the 8.9-inch version is getting a bump from the previous iteration’s 1,920 x 1,200 display. This certainly chimes with earlier reports, and, if nothing else, would certainly make that new browser UI all the more pretty to look at.

    Filed under: ,

    Comments

    Via: Android Community

    Source: GFXBench

    Galaxy S4 Benchmark “Optimization” Drama Hits Samsung

    Galaxy S4 Benchmark “Optimization” Drama Hits Samsung

    There’s a new wave of drama hitting the Samsung Galaxy S4 today. After running some tests tech site Anandtech has noticed that Samsung was allowing at least a version of the S4 to run its graphics processor (GPU) at higher frequency (533MHz) in some benchmarks, while games yielded a lower frequency of 480MHz. In other words, Samsung would be inflating its graphics horsepower by 11.04%. The reviewers were able to get accurate frequency numbers thanks to a built-in command available to anyone who can log as an administrator (root) in Android. (more…)

    Like It , +1 , Tweet It , Pin It | Galaxy S4 Benchmark “Optimization” Drama Hits Samsung original content from Ubergizmo.

        

    Samsung Denies Trying to Artificially Inflate Its Benchmark Scores

    Samsung Denies Trying to Artificially Inflate Its Benchmark Scores

    Yesterday, it was reported that Samsung had rigged the Galaxy S4 to perform better in benchmarks. Today comes Samsung’s terse response, which answers some questions, but leaves others flapping in the wind.

    Read more…