Every time a new generation of CPUs is announced, I see a number of people writing about how they think it will be faster (or slower) than current technology because of the advertised specifications. As an example, here are some quotes from an article about the Intel Core i9 7900-series processors and how they stack up against AMD’s Threadripper chips:
Intel released the final details of its Core i9 microprocessors Monday morning, and the data is in: AMD’s high-end Threadripper processors will be faster—except when you really need it.
When Intel inadvertently revealed that its 12-core Core i9-7920X was 2.9-GHz—slower than the comparable AMD Threadripper—a subset of the Internet had a small freakout.
Essentially, both Intel and AMD can claim the title of fastest processor.
These are not the most egregious cases of this that I have seen, but they were top of my mind when sitting down to write this. Here is the problem with these statements: none of them are based on any empirical testing. AMD sent out Threadripper samples, but this article was published several days before the embargo on performance results was lifted. The Core i9 processor models in question were announced at this point, but still several weeks away from availability. The entire article was in reaction to the published specs – core count, clock speed, etc – from AMD and Intel.
Are there truths that can be gleaned from such information? Yes! You can safely make some comparisons between CPUs within the same processor family, for example. If Intel puts out two processors which are the same architecture, generation, and core count – and the only difference is clock speed – we can safely say which is faster. Or if they are the same clock speed, and the only difference is core count, then we can safely assume which will be faster in programs that can use a lot of cores (multithreaded workloads). However, you cannot make such assumptions across brands of processors… or even across different architectures and generations. Even features as small as the amount of cache memory on a processor, or details like the way cores communicate with each other inside the chip, can have profound impacts on real-world performance. Modern technologies like Turbo Boost that adapt clock speeds depending on the number of active cores throw yet another wrench into all of this.
That is why we have a team here at Puget Systems, which I am honored to be part of, that focuses on understanding how popular applications work and then testing how processors and other computer hardware perform. We run realistic benchmarks which involve software acting as it does in normal usage, rather than artificial tests which look at raw capabilities in abstract ways that don’t reflect real applications. When such benchmarks do not exist, we create our own testing methodologies to emulate the workflow of professional users, so that we can give our customers accurate information about what hardware will best meet their needs.
And lest you think that I would end this article without backing up these claims, here is an example of performance in several Adobe Creative Cloud applications, looking at the Intel Core i7 6900X and AMD Ryzen 1800X. Both are 8-core processors with their respective manufacturers SMT technologies – Hyperthreading is what Intel calls that – so they can work on 16 threads at a time. The Intel chip clocks in at 3.2GHz base speed with a max all-core Turbo Boost speed of 3.5GHz and single-core of 4GHz. The competing AMD processor has a base clock of 3.6GHz and can turbo as high as 4GHz. In all fairness, I should also note that Intel’s processor is almost a year older but AMD’s entry costs half as much (MSRP). On the surface, though, AMD’s chip has higher clock speeds – right? So if that was all that mattered, it should be faster – right? And in single-threaded applications they should be tied, both reaching 4GHz. Let’s see what real-world testing says…
[Results gathered from our various articles on Ryzen performance]
As you can see, these two processors do not perform the same despite having very similar specs. In some tests they come very close, but in others either AMD or Intel wins by a pretty wide margin. These are just a handful of specific procedures within a few applications, but it is enough to tell the true story: specs alone cannot tell you how two different CPUs will perform. Real-world testing is needed in order to truly determine which is faster, and in most cases there will be strong and weak points to each product.
Beware those who write about the performance of unreleased hardware without having test data to back it up.