We test a lot of software here at Puget Systems, and in most cases what we are looking for is what hardware lets a given program run the fastest – or in some cases, what is the most cost effective. If you can get 95% of the best possible performance for half the price that it would cost to get a full 100%, for example, that is often a compelling way to go.
However, ANSYS Mechanical (and FLUENT) present a different challenge: how can you get the best performance within the limitations of the ANSYS licensing model? This is a tricky question for several reasons.
First, the license structure itself is a bit different: you get to use two cores with a base ANSYS license, and then to add on to that you need “HPC Packs”. These increase the amount of cores you can run ANSYS on at a given time, but it is not linear. Adding one HPC Pack brings the core total from 2 to 10 (previously 8, in versions before 18.0). Adding a second HPC Pack brings the total to 32, though, rather than 18 or 20. A third HPC Pack takes that all the way to 128, and it continues to go exponentially higher. That is good news for those wanting to run on a cluster of systems, but for someone aiming at a single workstation it really limits the field of logical hardware choices. It would be ridiculous, for example, to run a 12-core system that needs two HPC Packs but isn’t even using half of the licensing potential those packs offer. Considering the cost per HPC Pack, it is ideal to max-out the hardware you are allowed to use depending on what licenses you have available.
Second, what counts as a “core” to ANSYS is either a single logical CPU core (not including Hyperthreading, if a CPU supports it – in fact, using HT causes problems in ANSYS) *or* a whole GPU (video card). That’s right: you can have a single core of a modern multi-core processor or a whole $5000+ GPU, and ANSYS licensing treats them the same. However, the list of officially supported / tested GPUs is very limited… and in our testing, we found that multi-GPU configurations were not reliable in ANSYS.
Finally, we found in testing that having a lot of CPU cores and a single GPU can actually hurt performance more than it helps. Some people refer to this situation as being “oversubscribed”: each instance of ANSYS, running on a different CPU core, is trying to get GPU time to help with computations… and as you have more and more of those threads the GPU cannot keep up with it all and that leads to a traffic jam of sorts. In the standard ANSYS 18.X benchmarks, this caused a 28-core system with a single GPU to perform more slowly in the majority of tests than the same 28-core system without the GPU being utilized at all.
Because of all that, we have found that there are really just two ideal hardware configurations before you start to get into cluster territory:
1) For users with a single HPC pack, a 10-core processor at the highest possible clock speed and a Quadro GP100 video card. This gives you the option to use either 10 CPU cores or 9 CPU cores plus the GPU, depending on whether the specific calculations you are running will utilize the GPU well. Either way you are maxing-out the base ANSYS license plus the HPC Pack.
2) For users with two HPC packs, a 32-core dual Xeon workstation – with or without a GPU, depending on your specific workload. We found that most of the example tests in the ANSYS benchmark run better without a GPU when so many CPU cores are being utilized, but there were two tests (out of ten) that ran a little faster with a GPU included.
Before we started testing, there was hope that something like a 28-core workstation with four GPUs might be a solid option as well. Unfortunately, the problems we have had getting ANSYS to run through all of the benchmark samples with multiple GPUs makes that a bad idea at this time.
Moreover, as odd as it may seem, the two designs above cost about the same amount up-front. A 10-core CPU with a powerful GPU is in the same price range as a dual 16-core (32 cores total) system without a strong GPU (just a basic video card for graphics output). The licensing costs, though, are nearly doubled if you want to use the higher core count system. The question really becomes: how much do you want to pay for ANSYS licensing? The answer to that will determine which hardware configuration is best for you, more so that your budget for the actual computer itself.