Puget Systems print logo
Read this article at https://www.pugetsystems.com/guides/1000
William George (Product Development)

Stop Directly Comparing CPU Specs

Written on August 8, 2017 by William George

Every time a new generation of CPUs is announced, I see a number of people writing about how they think it will be faster (or slower) than current technology because of the advertised specifications. As an example, here are some quotes from an article about the Intel Core i9 7900-series processors and how they stack up against AMD’s Threadripper chips:

Intel released the final details of its Core i9 microprocessors Monday morning, and the data is in: AMD’s high-end Threadripper processors will be faster—except when you really need it.

When Intel inadvertently revealed that its 12-core Core i9-7920X was 2.9-GHz—slower than the comparable AMD Threadripper—a subset of the Internet had a small freakout.

Essentially, both Intel and AMD can claim the title of fastest processor.

These are not the most egregious cases of this that I have seen, but they were top of my mind when sitting down to write this. Here is the problem with these statements: none of them are based on any empirical testing. AMD sent out Threadripper samples, but this article was published several days before the embargo on performance results was lifted. The Core i9 processor models in question were announced at this point, but still several weeks away from availability. The entire article was in reaction to the published specs - core count, clock speed, etc - from AMD and Intel.

Are there truths that can be gleaned from such information? Yes! You can safely make some comparisons between CPUs within the same processor family, for example. If Intel puts out two processors which are the same architecture, generation, and core count - and the only difference is clock speed - we can safely say which is faster. Or if they are the same clock speed, and the only difference is core count, then we can safely assume which will be faster in programs that can use a lot of cores (multithreaded workloads). However, you cannot make such assumptions across brands of processors... or even across different architectures and generations. Even features as small as the amount of cache memory on a processor, or details like the way cores communicate with each other inside the chip, can have profound impacts on real-world performance. Modern technologies like Turbo Boost that adapt clock speeds depending on the number of active cores throw yet another wrench into all of this.

That is why we have a team here at Puget Systems, which I am honored to be part of, that focuses on understanding how popular applications work and then testing how processors and other computer hardware perform. We run realistic benchmarks which involve software acting as it does in normal usage, rather than artificial tests which look at raw capabilities in abstract ways that don’t reflect real applications. When such benchmarks do not exist, we create our own testing methodologies to emulate the workflow of professional users, so that we can give our customers accurate information about what hardware will best meet their needs.

And lest you think that I would end this article without backing up these claims, here is an example of performance in several Adobe Creative Cloud applications, looking at the Intel Core i7 6900X and AMD Ryzen 1800X. Both are 8-core processors with their respective manufacturers SMT technologies - Hyperthreading is what Intel calls that - so they can work on 16 threads at a time. The Intel chip clocks in at 3.2GHz base speed with a max all-core Turbo Boost speed of 3.5GHz and single-core of 4GHz. The competing AMD processor has a base clock of 3.6GHz and can turbo as high as 4GHz. In all fairness, I should also note that Intel’s processor is almost a year older but AMD’s entry costs half as much (MSRP). On the surface, though, AMD’s chip has higher clock speeds - right? So if that was all that mattered, it should be faster - right? And in single-threaded applications they should be tied, both reaching 4GHz. Let’s see what real-world testing says...

[Results gathered from our various articles on Ryzen performance]

As you can see, these two processors do not perform the same despite having very similar specs. In some tests they come very close, but in others either AMD or Intel wins by a pretty wide margin. These are just a handful of specific procedures within a few applications, but it is enough to tell the true story: specs alone cannot tell you how two different CPUs will perform. Real-world testing is needed in order to truly determine which is faster, and in most cases there will be strong and weak points to each product.

Beware those who write about the performance of unreleased hardware without having test data to back it up.

Tags: CPU, processor, core, clock, speed, comparison, Intel, AMD, i7, i9, Ryzen, Threadripper, performance, benchmark

Thank you for showing real world numbers. This is what I need to see. I only want to upgrade from what I have now, if I can "half" my work load times.

To that end, it would be nice if you could offer a system that your audience could remote into and allow us to install our own software and test our own workloads before purchase. Automatically reconfigure the system to a clean state after each usage.

An example workload to test:
* Compile a C++ project in VS2015
* Compile a C++ project in C++ Builder 2007
* Render a video to an AVI file in Sony Vegas
* Convert an AVI video file to MP4 using Handbrake
* Output Canon Raw files using Canon DPP
* Resize JPG files using FastStone Photo Resizer
* Test searching the NVMe drive using FileSearchEX
* Test installing an OS in VMWare Workstation
* Do all the above at the same time :-)

Posted on 2017-08-11 02:05:50

As a professional photographer (who also happens to live in Puget Sound), you have become my go-to source for CPU testing.
Thanks for all of your hard work.

Posted on 2017-08-11 17:56:48

You are very welcome! I'm glad that our testing is helpful to you :)

Posted on 2017-08-14 22:24:52

What was the RAM used on Ryzen? As Samsung B-die is easy to get running at 3200mhz CL14 on it and can help performance in some cases, but overall I think Ryzen is still the go to since in reality you'd likely get a cheaper R7 1700, or go Threadripper now for more cores at a lower cost.

Posted on 2017-08-14 21:50:41

We use what the manufacturers (Intel and AMD) officially support in terms of memory speeds. Yes, both Intel and AMD platforms can handle higher memory speeds on some motherboards and in certain combinations, but we find that cranking memory speeds higher leads to increased failure rates and more stability issues. For our customers, performance and stability are both critical - so we prefer to stay on the safe side. In this specific instance, both CPUs were tested with 2400MHz memory (which is what the manufacturers rate these CPUs for).

Posted on 2017-08-14 22:24:23
Earl Watkins

Are any number available to show the following information provided by AMD? Like Ryzen, Threadripper scales well with faster memory, with increased DDR4 memory speeds speeding up thread-to-thread communication in Ryzen by speeding up the CPU's Infinity Fabric. Infinity Fabric speed is tied to memory speeds, so the faster your memory the faster your thread-to-thread communication, even across multiple dies.

Posted on 2017-08-14 23:44:53