Puget Systems print logo

https://www.pugetsystems.com

Read this article at https://www.pugetsystems.com/guides/1330
Article Thumbnail

NVIDIA NVLink Bridge Compatibility Chart

Written on January 15, 2019 by William George
Share:

Introduction

NVLink is a technology from NVIDIA for creating a high bandwidth link between two compatible video cards. It can be used for many things, from basic SLI for faster gaming to potentially pooling GPU memory for rendering large and complex scenes. What NVLink can be used for depends on how software developers write their applications, and there is a lot of exciting potential for this technology.

However, NVLink requires a physical bridge between the cards in order to enable these capabilities - and NVIDIA isn't transparent as to which models of the NVLink bridge will work with which video cards. Several Quadro video cards support NVLink, along with some of the new GeForce RTX 20-series cards and the Titan RTX. NVLink bridges are available with Quadro or GeForce branding in various sizes.

If you have already set up a system with NVLink, you can find instructions for enabling and testing it on another article.

Compatibility Matrix

Here is a chart of all the NVLink compatible video cards, as of publishing time, along with the various NVLink bridges that are available:

NVLink Bridge →
↓ Video Card
Quadro GP100
2-Slot Pair
Quadro GV100
2-Slot Pair
Quadro RTX
5000 2/3-Slot
Quadro RTX
6000 2/3-Slot
GeForce RTX
3/4-Slot
Titan RTX
3/4-Slot
Quadro GP100 Works (Tested) Should Work Does Not Work Does Not Work Does Not Work Does Not Work
Quadro GV100 Does Not Work Works (Design) Does Not Work Does Not Work Does Not Work Does Not Work
Quadro RTX 5000 Does Not Work Does Not Work Works (Design) Does Not Work Does Not Work Does Not Work
Quadro RTX 6000 Does Not Work Should Work Does Not Work Works (Tested) Works (Tested) Should Work
Quadro RTX 8000 Does Not Work Should Work Does Not Work Works (Design) Should Work Should Work
Titan RTX Does Not Work Should Work Does Not Work Works (Tested) Works (Tested) Works (Design)
GeForce RTX 2080 Does Not Work Should Work Does Not Work Works (Tested) Works (Tested) Should Work
GeForce RTX 2080 Ti Does Not Work Should Work Does Not Work Works (Tested) Works (Tested) Should Work

For the combinations indicated as "Works", we specified whether we have actually tested it or it is just assumed to work because NVIDIA designed it to. For example, the Quadro 6000 RTX NVLink Bridge is clearly designed to work on the Quadro RTX 6000, as well as the upcoming 8000 - but we only have RTX 6000s in-house to test currently. Likewise, the RTX 5000 has its own bridge designed for its smaller NVLink connector - so it should work, by design. "Should Work" indications are used where we believe the combination will work based on other testing we have done, but we don't have samples on hand to verify for sure. "Does Not Work" should be self explanatory.

Insights

The main takeaways from our testing are as follows:

  • The older Quadro GP100 cards were equipped with the first generation of NVLink, and had less bandwidth per link than newer models. As such, the Quadro GP100 bridges do not work with newer cards.
  • The Quadro GV100 cards use the second generation of NVLink, and like the GP100 have two NVLink connectors per card. The newer RTX series bridges can fit on one set of connectors, but their plastic housings are physically larger and cannot be paired up to fully connect both sets of NVLink connectors. That means newer RTX bridges won't fully work on GP100 or GV100 cards.
  • Quadro RTX 5000 cards have a smaller NVLink connector than all the other cards, requiring a completely separate set of NVLink bridges. Because of this they do not interchange with any of the other existing video cards, whether RTX generation or older. This seems to be a design decision by NVIDIA, but is strange since the GeForce RTX 2080 has the same lower NVLink bandwidth (50 GB/s total, versus 100 GB/s on the RTX 6000, 2080 Ti, and Titan RTX) but uses the same full-length connector.
  • Quadro RTX 6000 and GeForce RTX NVLink bridges may look different on the outside, but they appear to be functionally identical. We've tested both types and found they work across most Quadro, Titan, and GeForce cards in this generation. We cannot yet verify that on the Quadro RTX 8000, because they are not yet shipping, but we have tested on a pair of RTX 6000s and all four bridge variants work. Likewise, we have tested them on GeForce RTX 2080, 2080 Ti, and Titan RTX successfully.
  • It is worth noting, however, that the LED logo on GeForce RTX bridges does not light up when used on Quadro RTX cards. The Titan RTX bridges also appear to have a NVIDIA logo, but as we don't have them in hand I am not sure if it lights up.

Conclusion

The good news here is that, if you are using hardware from the RTX generation - except for the Quadro RTX 5000 - it looks like you can use a GeForce, Titan, or Quadro branded bridge without needing to worry about compatibility. That is particularly helpful since they come in different sizes: 2- and 3-slot for Quadro and 3- and 4-slot for GeForce / Titan. I'm not sure why NVIDIA didn't just make 2-, 3-, and 4-slot solutions that were brand agnostic... but at least there don't seem to be any weird limitations on which bridges work with which cards. They are also all the same price, as of publishing time, if purchased directly from NVIDIA. The older GP100 and GV100 are a different story, and if you are using those you should just stick with the paired bridges built for each model, respectively.

Additional Resources

If you want more info about NVLink in Windows, check out another article we published on the topic.

If you need a walk through on enabling NVLink in Windows, or want to make sure it is work, we have an article covering those topics too.

If you are interested in NVLink, Linux, and machine learning, check out Dr Kinghorn's HPC Blog.

For a list of all articles where we've talked about NVLink, use our website's search function.

Tags: NVIDIA, NVLink, Bridge, SLI, Windows 10, GeForce, RTX, 2080, 2080 Ti, Titan, Quadro, 5000, 6000, 8000, GP100, GV100

I just found out that NVIDIA has gone and made Titan RTX NVLink bridges as well - which look just like the GeForce ones, but gold instead of silver (to match the color of the Titan RTX's cooler). I thought about updating the chart above, but if I add any more columns it will not fit in the same width... so I'm just adding a note here that I fully expect it to work in all the same places the GeForce RTX NVLink bridge does - which is to say, across all current RTX series cards.

Posted on 2019-01-25 22:05:55

I also just found out that NVIDIA pulled a stupid and made the Quadro RTX 5000's NVLink connector a smaller size. We knew it was already going to be a 1-link (50 GB/s) connection, rather than 2-link (100 GB/s) like the RTX 6000 and 8000. However, that same split exists on the GeForce side: RTX 2080 is 50 GB/s and 2080 Ti has the full 100 GB/s... but they have the same connector and share the same bridges. I don't know what led NVIDIA to change that for the Quadro RTX 5000, especially since it means yet another set of NVLink bridges (with the smaller connector). I am going to go ahead and update the chart above with this, since it is a pretty major issue.

Posted on 2019-01-25 23:49:39
mathieu soleil

I think it would be worth it to update the paper with those comments as well as the chart (and if needed to resize the width of the columns), because a good part of the readers won't read comments, only the article, an quickly. Also it would be very interesting to verify that a GV100 Nvlink bridge works with 2x2080ti, in particular because of 1) the absence of 2 slot size bridge among the consumer nvlink bridges, and 2) the 100G speed and 3) the cost difference.

Posted on 2019-01-26 15:49:44

I did update the chart, as well as some of the descriptive text in the article, based on the new info about the smaller Quadro RTX 5000 connectors and the Titan RTX bridges. I would be happy to test with the GV100's bridges, if we had any... but we don't, and they are so expensive (~$500!) that it isn't worth ordering them in. If you just need a 2-slot bridge, just the one designed for the Quadro RTX 6000 and it works (we have one and tested it here) with the GeForce RTX 2080 & 2080 Ti.

Posted on 2019-01-28 17:09:05