Puget Systems print logo

https://www.pugetsystems.com

Read this article at https://www.pugetsystems.com/guides/1258
Article Thumbnail

GeForce RTX 2080 Multi-GPU Scaling in OctaneRender and Redshift

Written on October 12, 2018 by William George
Share:

Introduction

We found previously that stacking multiple RTX 2080 video cards next to each other for multi-GPU rendering led to overheating and significant performance throttling, due to the dual-fan cooler NVIDIA has adopted as the standard on this generation of Founders Edition cards. Now that manufacturers like ASUS are putting out single-fan, blower-style cards we can repeat our testing to see if the throttling issues are resolved and find out how well these video cards scale when using 1, 2, 3, or even 4 of them for rendering in OctaneRender and Redshift.

Four Asus Turbo RTX 2080 Video Cards During OctaneRender and Redshift Benchmarking

Test Setup

When benchmarking we often run each test three times, make sure the results are pretty close, and then take the best one (highest score or lowest time, depending on the situation) as the final result. This helps ensure that something going on in the background doesn't throw things off and that each hardware configuration we test gets a fair shake. Because of our previous experience with the NVIDIA Founders Edition RTX 2080s throttling, though, we wanted to conduct longer tests - so we ran each combination of video cards and software for 30+ minutes straight, collecting far more data points and ensuring that we would see throttling if it was going to happen.

To keep the testing fair, and to be able to compare performance directly to our previous test, we used the same motherboard, CPU, and power supply. If you would like full details on the hardware configuration we ran these tests on, just .

Benchmark Results & Analysis

The first question we wanted to answer is whether these single-fan, blower-style cards would throttle like the NVIDIA Founders Edition models did - so the first thing we tried was a full set of four GPUs. Running both OctaneRender and Redshift for over 30 minutes each, we looked at the difference between the highest and lowest results - and found there was only 1.4% variance in OctaneRender and 1.6% variance in Redshift. That is well within the margin of error when looking at this many test runs, and since we saw huge performance dips after just the second run with these programs before it is clear that the blower-style cooling solution fixes the overheating and throttling issues we had with the dual-fan Founders Edition cards.

With that question out of the way, we continued to run sets of three, two, and finally just one of these cards to see how well they scale. Here are charts showing those results for OctaneRender - both absolute scores as well as percent increase compared to a single card:

OctaneBench 3.08 Multi-GPU Scaling with 1 to 4 Asus Turbo RTX 2080 GPUs

OctaneBench 3.08 Multi-GPU Relative Performance Scaling with 1 to 4 Asus Turbo RTX 2080 GPUs

That looks great - near perfect scaling, with just some minor variance... exactly what we have come to expect from OctaneRender. Moving on, we have the same sort of result charts for Redshift as well:

Redshift 2.6.22 Demo Multi-GPU Scaling with 1 to 4 Asus Turbo RTX 2080 GPUs

Redshift 2.6.22 Demo Multi-GPU Relative Performance Scaling with 1 to 4 Asus Turbo RTX 2080 GPUs

Here, again, we see excellent results. Redshift has always shown a little more drop-off in performance when we've tested it with multiple GPUs, compared to Octane, but it still benefits greatly from having more than one video card to utilize. Because these blower-style cards are not throttling significantly under heavy load, there is no artificial penalty from using a full set of four cards to maximize performance.

ASUS Turbo vs NVIDIA Founders Edition Cards

There are some additional things to consider when looking at different video cards built with the same GPU. To illustrate some of those issues with these two particular GeForce RTX 2080 models...

Single Card Performance vs Multi Card Performance - If you look at just a single card of each type, the NVIDIA Founders Edition models are clocked slightly higher than the reference-speed ASUS Turbo cards. The specs NVIDIA lists on their website show a 5% clock speed difference (1800 vs 1710 MHz), and that matches almost exactly what we found when comparing single GPU performance:

Single GPU Performance in OctaneRender on ASUS Turbo RTX 2080 versus NVIDIA Founders Edition GeForce RTX 2080

Single GPU Performance in Redshift on ASUS Turbo RTX 2080 versus NVIDIA Founders Edition GeForce RTX 2080

However, when we ramp up to a full set of four graphics cards, the multi-GPU friendly cooler on the ASUS Turbo model maintains better performance than the dual-fan NVIDIA Founders Edition card - even though the NVIDIA version starts off with higher clock speeds:

Multi-GPU Performance in OctaneRender on ASUS Turbo RTX 2080 versus NVIDIA Founders Edition GeForce RTX 2080 Cards After 30 Minutes of Load

Multi-GPU Performance in Redshift on ASUS Turbo RTX 2080 versus NVIDIA Founders Edition GeForce RTX 2080 Cards After 30 Minutes of Load

Noise Levels - As individual cards, dual-fan models like the NVIDIA Founders Edition RTX 2080 will usually run quieter under load than a single-fan blower card. However, when the cards are stacked next to each other in a three or four GPU configuration, those dual fan designs have to ramp fan speeds higher to try and keep cool. I didn't have the equipment to measure noise levels, but the set of four Founders Edition cards sounded much louder to my ears under load than the ASUS Turbo models. Both sets of cards did ramp their fans up higher when crowded together than when used as single cards, though, which makes sense since airflow is more restricted and there is more heat surrounding the cards in that situation.

Which Type of GeForce RTX Card Should You Get for Multi-GPU Rendering?

If you just want a single video card, the NVIDIA Founders Edition GeForce RTX 2080 and 2080 Ti are great choices - and there are other brands that make similar dual-fan models. If you want to use multiple GPUs, however, single-fan cards that exhaust heat out of the system like the ASUS Turbo RTX 2080 are much better.

NVIDIA Founders Edition GeForce RTX 2080 versus ASUS Turbo RTX 2080 Pros and Cons

How Well Do GeForce RTX 2080 Cards Scale in OctaneRender and Redshift?

Assuming you use video cards with coolers that can keep them running well in multi-GPU configurations, the GeForce RTX video cards have excellent performance scaling in both OctaneRender and Redshift. GPU-based rendering has always benefited from multiple cards, in our experience, and the RTX series is no exception... but make sure your system's cooling is up to the task.

OctaneBench 3.08 and Redshift 2.6.22 Demo Multi-GPU Relative Performance Scaling with 1 to 4 Asus Turbo RTX 2080 GPUs

Recommended Systems for OctaneRender

Recommended Systems for Redshift

Tags: Multi, GPU, Scaling, Rendering, Octane, Render, Redshift, Benchmark, NVIDIA, GeForce, RTX, 2080, Performance, Heat, Cooling, Fans, Video, Card, Asus, Blower
Håkon Broder Lund

So you are saying there is a 25-27% performance difference between blower style card and dual fan card?! That is huge! And no one is really talking about this except you guys!

With the cards so close the airflow is really restricted, obviously. I'm wondering. On some motherboards there is three slots between the first and second x16 slots. This way you have room for a tripple cooler card, or more breathing room for a FE or blower card. Would you expect to see this large of a performance difference on a dual GPU config as with your 4x config?

Posted on 2018-10-12 21:22:21

Yes, if you are using these cards in a configuration where they are right next to each other the blower style are much, much better. As a single card, or in pairs where there is added space between them, the dual-fan cards are okay... though they still pump a lot of hot air back into the computer, which may raise temps on other components in the system. Back when I did the previous article, showing how badly four of the Founders Edition RTX 2080 cards throttled, I did a sanity check with just two of them spaced a couple slots apart - and they did not throttle, at least in an open-air testbed.

We haven't really done any work with triple-wide cards, mostly because you can't fit as many of them in a system. They are also very large and heavy, making them less than ideal for being shipped out to customers in fully assembled computers. I would expect that two of them in a system would be okay, provided there was some space between the cards for airflow / intake for the fans. With GPU based rendering, though, it is really nice to be able to cram as many video cards into a single system as possible (from a performance standpoint).

Posted on 2018-10-12 21:27:29
Håkon Broder Lund

Makes sense. Really goes to show how important airflow in the case is, like you guys are experts on. True, The main negative with the triple slot cards that I see is that it makes it difficult to add other card like 10Gb/s/SFP+ network card or video output cards for NLEs.

Posted on 2018-10-12 21:33:38
R3D

https://www.redshift3d.com/...

Why the latest 2.6.23 version not used for benchmarks?

Posted on 2018-10-12 23:11:16

We often start these benchmarks days or weeks before publishing, and sometimes software versions get changed in the middle of our testing. In this case, and in another article that will be coming out Monday, we started with 2.6.22 a few weeks ago as it was the most recent version available when the new GeForce RTX 2080 and 2080 Ti cards got to our lab. Rather than go back and re-test a bunch of stuff when 2.6.23 came out, we stuck with 2.6.22... plus, I don't think there were any major changes between those versions (as far as I can tell from the 2.6.23 release notes: https://www.redshift3d.com/... ).

Posted on 2018-10-12 23:46:05
Turing

https://www.nvidia.com/en-u...

https://www.nvidia.com/en-u...

Nvidia Quadro RTX cards being blower fan designs makes a lot of sense now, since they'll be used in multi-GPU configurations.

Will Puget be testing Quadro RTX 6000 & RTX 5000 when they are available?

Posted on 2018-10-13 13:13:03
HellPedre .

having an rtx VS quadro benchmark on red shift and octane would be EYE OPENING...

Posted on 2018-10-13 15:36:00

Certain companies get a little bit sensitive when mainstream and "professional" video cards are compared directly. However, if we have two articles with performance data on two separate lines of video cards but using the same underlying platform and same version of software it can make for an easy comparison for those who want that information ;)

Posted on 2018-10-13 15:45:55

Yes, I fully intend to test the new Quadro cards when we get samples. Probably just as individual cards, since we already have a good idea of how multi GPU scaling goes and they use blower fans which means we don't have to really worry about throttling.

Posted on 2018-10-13 15:44:53
José León Molfino

Thanks! what about temps in full load? would 3 or 4 of these cards perform without temp issues during several hours in full load?... recomended PSU? thanks

Posted on 2018-10-22 22:58:21

Our open testbeds wouldn't be a great way to look at temperatures, since things will be a little warmer in the confines of an enclosed chassis, but we didn't see any significant throttling with these cards - so as long as they are in a chassis with sufficient airflow (intake, in particular - the cards themselves will exhaust most of their heat) you shouldn't have any trouble with heat. We ran them under constant load for about 1 hour per render engine / GPU combination, which is usually enough time to reach a steady temperature state, so I would expect these to do fine for even longer rendering sessions... but I haven't personally run them constantly overnight or anything like that.

As for the PSU, for the RTX 2080 cards you need an 8-pin and a 6-pin for each GPU. I would advise something rated for at least 1200-1300W for four of them (225W per GPU = 900W plus some for the rest of the system). Moving up to the 2080 Ti cards requires a pair of 8-pin connections for each card. That may be a little hard to find, and you would want a 1500-1600W power supply for four of those.

Posted on 2018-10-22 23:10:46
howdygoop

If I have two blowers and two open air 2080 GPUs, what's the best arrangement? I ask because https://www.pugetsystems.co... suggested that the middle GPUs suffer the most. If the middle GPUs are blowers and the top and bottom are open air, then is this less of an issue?

Posted on 2018-10-25 05:07:20

The bottom slot, where there is nothing else below it to block the fan intake, is definitely the best spot for a dual-fan card in a multi-GPU setup. This is assuming, I suppose, that your chassis has room below that card for it to breathe - if the card bottoms out on something that blocks the fans, it won't be any better.

A second spot, though? That is tough. Probably the top slot, where the fans will be blocked but at least there isn't another card above it to prevent heat from radiating out. It will still throttle up there, but not as badly as if it were in the middle.

Posted on 2018-10-25 18:34:49
howdygoop

> It will still throttle up there, but not as badly as if it were in the middle.
By your measure, is the throttling mostly from the accumulated heat in the system or its inability to get air? If there are blowers, then the accumulated heat should be less. Yet if the open air cards need their fans to have access to much air, then the performance for an OBBO setup should not be much better than an OOOO setup (O=open, B=blower).

Posted on 2018-10-25 18:53:57

I'm honestly not sure - I haven't tested a mixed blower + dual fan setup before, or tried to artificially create poor cooling conditions in order to measure their effect. I suspect, though, that it comes down to how hot the air being pulled in to "cool" the card is. If it is already very hot, it won't actually help pull more heat off the card's heatsink. So having a couple blower cards should help in that regard... hopefully. Good luck :)

Posted on 2018-10-25 19:09:01