Google Pixel XL Performance Stress Test, Throttling and Thermals Analysis — A Remarkable, Consistent Performer

Google's new Pixel phones assume the mantle of yesteryear's Nexus devices, which were consistently some of the fastest devices around. But while the user experience rarely suffered on a Nexus, there were some caveats near the end of the line's life.

The Nexus 6P and Nexus 5X both packed a particularly flawed set of processors – the infamous Snapdragon 810 and its brother the 808 – that were ultimately tamed for the everyday user experience in the real world (by the Nexus line, in particular), but nevertheless demonstrated severe throttling and temperatures under heavy stress. A year later, the Nexus line is no more and the Pixel devices pack Qualcomm's newest processor, albeit the Snapdragon 821 "Pro-AB" variant which comes with the same clockspeeds as the Snapdragon 820 across the board. This is not necessarily a bad thing, and I don't believe consumers should feel cheated –the new chipset is 5% more power-efficient (according to what Qualcomm told us, this gain is achieved when running at the same clockspeeds as the 820) and it also means slightly more breathing room for overclocking.

But overclocking is only truly viable when other variables allow for it, namely thermal efficiency and throttling. The Nexus 6P, which we tested for one of our performance-over-time articles, displayed pretty poor endurance with severe throttling and high temperatures. That wasn't exclusive to the 6P as other 810 devices suffered the same fate, but with the Snapdragon 820 line, we found much better results despite the surprising variance across devices. The OnePlus 3 is the best example (that we've tested, anyway), as it rarely ever throttled in any of our tests, with the surprising ability of sustaining performance through multiple 30-minute sets of GFXBench Manhattan sessions (even if it had to reach 45° C | 113° F in the process). The Pixel phone theoretically has a slightly better (yet not faster) processor, so can it achieve the same level of stamina?

We set out to find if the Pixel XL displayed any kind of throttling, either small or significant, and to figure out what its relative standing is among other 2016 devices. A few notes to keep in mind: these tests are only representative of the Pixel XL which has a different body than the regular Pixel, and thus different thermal properties. Most importantly, these tests are not representative of real-world usage (other than "worst case scenarios"), but of the processor's strengths and the phone's behavior under stress. To minimize extraneous variables, we disabled nearly every app that could interfere with the testing. Other variables we tried to control are temperature (same starting temperature for all tests and devices), room temperature and surface heat absorption by running all tests for all devices in nearly-identical starting conditions. We measured device surface temperatures using SEEK and FLIR thermal cameras backed by IR thermometers, so take into account temperatures could have a ±1°C margin of error.

---

The body of the Pixel XL has both glass and metal, which actually complicates measuring the temperature of the device under stress as we had to adjust our measurement tools to account for both the glossy and matte textures. The first question is, where does one measure the temperature? Both materials have different heat transfer coefficients, with aluminum being able to both heat up and cool down several times faster than the glass window. Glass is not a good thermal conductor like metal, and it does act as an insulator, a result clearly reflected in our thermal imaging. It's worth noting that the glass lays on top of the chassis and is not in direct contact with the internals, as shown by iFixit's teardown.

Finally, the fingerprint scanner seems to lay at the heart of some shielding, and it ultimately gathers the most heat from the device, even if you won't often feel it. Given it always shows up at the hottest point of the device, this is where we decided to take our measurements. Because of this, though, I must point out that the temperatures listed  in these tests are actually higher than what you'll feel in-hand throughout the rest of the body, making the results even more favorable for the Pixel XL. This behavior is similar to that of other devices with fingerprint scanners on the back, too, so it is nothing new.

Due to Geekbench 4's long runtime plus several pauses to defer throttling, that benchmark is not suited for endurance and performance over time tests, as it gives breathing room to the processor. We'll be using Geekbench 3 as usual to look at the performance drop over time — keep in mind we'll be focusing on the differentials, and not the magnitudes of the peak scores for the comparisons, to assess the impact of throttling on the device's performance. Geekbench 3 has a shorter runtime with no real pauses, making it better to stress the processor. Below you can find the 3 data sets from 3 different sets of 10 consecutive Geekbench 3 runs on the Pixel XL.

These tests show extremely good results for the Pixel XL, as there is no significant drop in performance over the 10-run tests, and the differences in scores are easily attributed to inherent variance in the test. Temperature, too, barely goes up over time, hitting a peak of 33.4°C | 92.1°F  on the fingerprint scanner — this isn't only lower than what we found on other devices, but it's also at the hottest point in the device. This marks a tremendous year-on-year improvement for the 821 over the 810, and for the Pixel XL over the Nexus 6P as shown in graphs below.

Moreover, the trends across the different data sets are similar, which indicates there is consistency to the lack of throttling found in this round of benchmarks. It's important to note that while this device does show lower peak scores in Geekbench 3 than what we found on our OnePlus 3 test, the drop in scores (or lack thereof, rather) is similar in proportion yet the Pixel stayed slightly cooler at its hottest point in our testing.

Other Snapdragon 820 devices like the HTC 10 showed more significant drops in performance after the 10-test mark, and the Snapdragon 820 Galaxy Note 7 also showed similar behavior with a maximum drop in score of 6.1% in single core and 3.5% in multi core, whereas the Pixel doesn't quite reach a 4% difference between its highest and lowest scores in multi core and it's highest drop in single core is ~5.2% . To sum up, we found no significant throttling in our set of Geekbench 3 tests for the Pixel, although the variance between sets of 10-run tests seems to increase slightly into each run. Overall and from what we can infer from these results, the Pixel XL shows remarkable CPU performance over time with no thermal constraints, and its hottest point is still neither hot enough to be detrimental to the user experience, nor large enough for you to really notice.

---

Moving on to GPU and graphics performance over time, we ran a set of graphics-intensive benchmarks to see the proportional drop in scores after 7 tests for 3DMark Sling Shot using ES 3.1, and 30 (consecutive) iterations of GFXBench's Manhattan tests (within the battery benchmark). Beginning with the set of 3DMark tests below, you'll see that the Pixel XL actually does a surprising job at keeping its performance, and the throttling we observed is actually one of the least severe ones we've found: it doesn't quite reach a 10% drop in scores. For reference, the OnePlus 3 saw a score loss of 8% in its fifth test, while the Snapdragon Note 7 has a loss of 18% in its final 3DMark score, and the Exynos Note 7 lost up to 27%.

While the Pixel XL throttled significantly less than those Galaxy devices, its final temperature was around the same, ranging from 43.1°C | 109.6°F  to 43.6°C | 110.5°F. This isn't unconventionally hot for these kinds of tests, but once more we must stress the fact that this is the temperature on the fingerprint scanner, and the rest of the body feels cooler to the touch. The sustained score is not as big of an improvement over last year's Nexus 6P's 3DMark performance, which actually did surprisingly well despite the Snapdragon 810 inside it. That being said, not all Snapdragon 810 devices were created equal, and the OnePlus 2 saw a drop in score of ~21%.

Looking at the trends throughout the different tests of sets, we can see remarkable consistency in the temperature curves, and also similar proportional drops in score once the throttling kicks in. The only discrepancy is that in two of the sets the score drop kicks in on the 5th test, while the odd one out has the throttle show up on the 6th test. If you look at the graph you'll also find that its temperature is slightly lower throughout and up until that point, suggesting it maybe hadn't reach a breakpoint for the throttling to kick in.

But regardless of that slight difference, these test ultimately show very unsubstantial throttling on the Pixel when it comes to graphics performance. We also put the Pixel XL through GFXBench tests, however, to see whether it'd perform as well during that intensive 30-minute benchmark, where many devices fail such as the HTC 10, but others like the OnePlus 3 do a more-than-remarkable job. While 3DMark renders the Sling Shot ES 3.1 benchmark at 1440p regardless of resolution (and then scales it), GFXBench's Manhattan does take the phone's native resolution into account, so we tried the test on both 1440p and 1080p to see how it stacks up to all kinds of 820 devices. Below you can find the results we obtained from various sets as well as a short timelapse of the heat distribution on the body throughout the test.

When running GFXBench at 1440p, the Pixel XL sheds the consistency of previous tests and gives slightly-varying results without clearly-identifiable throttling patterns. Those who read our Snapdragon 820 vs Exynos 8890 Note 7 comparison might recall that the throttling pattern for those devices was extremely replicable throughout various tests, but even when controlling the starting conditions, the Pixel XL shows wildly-different results — I made sure to run this 30-minute test many, many times. Even if I couldn't get a clear and satisfying pattern down, all of my results were above the average. Indeed, the Pixel XL actually beats the Snapdragon 820 Note 7 and the HTC 10, the former shedding up to half its score and the latter losing close to a third. The Pixel XL, by comparison, saw drops between 5% and 20%, with most results sitting around a 10% drop in performance at most. Temperatures never rose past 44°C | 111.2°F very much like in 3DMark.

In order to compare the device to 1080p Snapdragon 820 devices, we had to downgrade the device's resolution using simple a simple adb shell command (wm size 1080x1920) and then re-adjusting the DPI. The OnePlus 3 has been the absolute champion in GFXBench from what we've tested, as it simply didn't throttle nor did it see drops above 5% in a controlled environment. While the OnePlus 3 was running at 1080p, it's worth pointing out that devices like the Note 7 and HTC 10 still saw similar performance loss when tested on 1080p, as the processor is still pushed to its limits and it still hits the same physical thermal constraints. That being said, the Pixel XL actually offered us more-consistent results when running at 1080p, with an ~11% drop at most in our set of tests. As predicted, maximum temperature remained nearly the same with the fingerprint scanner reaching 43°C | 109.4°F to 44°C | 111.2°F (for comparison, the OnePlus 3 could hit 45°C | 113°F throughout a larger area of its body). As always, seeing close to double the framerate when lowering the resolution is a nice reminder of the level of performance we trade for 1440p screens.

What about gaming? Unfortunately, with the Pixel running Android 7.1, all of the framerate measurement tools we've grown used to for our tests seemingly need updating. The same goes for much of our other methodology, and up until yesterday, we did not have root to get in-depth results for many areas of our review. We've begun updating some of our tools for the Pixel and the new Android version, however. One curious example is our in-house battery logger tool, which keeps track of battery voltage/current and temperature so that we can get better data, nice charging curves and see what's going on behind the chassis. The changes to Android 7.1 made it so that root is seemingly required to fetch these stats, but now that Chainfire has released his clever root method for the Pixel and Pixel XL, we've been able to update it. As a side note, this device stayed rather cool on the outside while charging, so that's nice (more will be explained in our full battery life article or review section). We'll work on updating our methodology (and even expanding it!) to work around changes like this for the full in-depth review coming soon.

---

What do these tests tell us? Once more I want to empathize that we haven't used these benchmarks to compare theoretical maximums and/or the practical peak performance of a device, but how it behaves over time. While there are many devices that share the same processor, they are never created truly equal, and we've seen great variability in the endurance and behavior of the Snapdragon 820 devices we've tested to date. The Snapdragon 821 in the Pixel XL is not very different at all from these devices, so if we were to compare it to that category, it clearly sits at the top with the OnePlus 3. The Pixel XL has done a remarkable job even when pushed to its limits, and even at its hottest points it remains relatively cool for the level of performance and consistency it outputs.

Measuring performance over time and the heat a device generates is, to us, an aspect as important as the peak speed of the processor. Last year was a reminder to all of us that a cool and consistent processor ultimately wins the race, and we are glad to see Qualcomm has offered OEMs a solution that in great part redeems last year's failure. Google has done an excellent job with the Snapdragon 821 in every aspect we've admired so far: the phone is one of the snappiest around, it's cool to the touch and it's extremely consistent in delivering a smooth UI experience. The software is just as important as the hardware here, as Google's optimizations to Android Nougat and 7.1 in particular result in a faster and more pleasant UI experience — our favorite improvement being the changes relating to touch latency, which we'll cover in a separate article and in our review with the appropriate data and explanations.

What else is there to say? Not much, because the Pixel is one of those devices where we begin our tests  half-expecting the kind of results we'll get — it's something you can't escape inferring when you actually get to use the device day-to-day and experience its fluidity and consistency. And as we half-expected, the device is not half-bad: it's one of the more consistent and consistently-good performers we've put through these tests, and I wouldn't expect any issues with overheating or throttling during day-to-day usage with Google's phone.

Stay tuned for more Google Pixel and Pixel XL testing and our review!

Check out XDA's Pixel XL Forum! >>>

Special thanks to Aamir and Mishaal for their contributions to this article!

from xda-developers http://ift.tt/2ftTlyp
via IFTTT