TensorFlow Performance with 1-4 GPUs -- RTX Titan, 2080Ti, 2080, 2070, GTX 1660Ti, 1070, 1080Ti, and Titan VWritten on March 14, 2019 by Dr Donald Kinghorn
I have updated my TensorFlow performance testing. This post contains up-to-date versions of all of my testing software and includes results for 1 to 4 RTX and GTX GPU's. It gives a good comparative overview of most of the GPU's that are useful in a workstation intended for machine learning and AI development work.
There are 2 recent Intel processors that are really strange, the Xeon W-3175X 28-core, and the Core i9 9990XE overclocked 14-core. I was able to get a little time in on the these processors. I ran a couple of numerical compute performance tests with the Intel MKL Linpack benchmark and NAMD. I used the same system image that I had used recently to look at 3 Intel 8-core processors so I will include those results here as well. **There will be results for W-3175, 9990XE, 9800X, W-2145, and 9900K**.
RTX Titan TensorFlow performance with 1-2 GPUs (Comparison with GTX 1080Ti, RTX 2070, 2080, 2080Ti, and Titan V)Written on January 30, 2019 by Dr Donald Kinghorn
I've done some testing with 2 NVIDIA RTX Titan GPU's running machine learning jobs with TensorFlow. The RTX Titan is a great card but there is good news and bad news.
In this post I'll take a brief look at the numerical computing performance of three very capable 8-core processors -- i9 9900K, i9 9800X and Xeon 2145W All three are great CPU's but there are some significant differences that can cause confusion. I'll discuss these differences and see how the processors stack up when running Linpack and NAMD molecular dynamics simulations.
There has been some concern about Peer-to-Peer (P2P) on the NVIDIA RTX Turing GPU's. P2P is not available over PCIe as it has been in past cards. It is available with very good performance when using NVLINK with 2 cards. I did some testing to see how the performance compared between the GTX 1080Ti and RTX 2080Ti. There were some interesting results!
In my recent testing with the AMD Threadripper 2990WX is was impressed by the CPU based performance with the molecular dynamics program NAMD. NAMD makes a good benchmark for looking at CPU/GPU performance since it requires a balance and is usually limited by CPU. After some discussions I decided it would be good to look at multi-GPU performance with NAMD on Threadripper.
I recently wrote a post about building and running AMD Threadripper 2990WX with HPL Linpack - a "How-To". Most of the time I had with the processor went into getting that to work. However, I did run a few other test jobs that I thought the 2990WX would do well with. I compared that against my personal workstation with a Xeon-W 2175. In this post I share those test runs with you. It's not thorough testing by any means but it was interesting and I was surprised a couple of times with the results.
How to Run an Optimized HPL Linpack Benchmark on AMD Ryzen Threadripper -- 2990WX 32-core PerformanceWritten on November 30, 2018 by Dr Donald Kinghorn
The AMD Ryzen Threadripper 2990WX with 32 cores is an intriguing processor. I've been asked about performance for numerical computing and decided to find out how well it would do with my favorite benchmark the "High Performance Linpack" benchmark. This is used to rank Supercomputers on the Top500 list. It is not always simple to run this test since it can require building a few libraries from source. This includes the all important BLAS library which AMD has optimized in their BLIS package. I give you a complete How-To guide for getting this running to see what the 2990WX is capable of.
RTX 2080Ti with NVLINK - TensorFlow Performance (Includes Comparison with GTX 1080Ti, RTX 2070, 2080, 2080Ti and Titan V)Written on October 26, 2018 by Dr Donald Kinghorn
More Machine Learning testing with TensorFlow on the NVIDIA RTX GPU's. This post adds dual RTX 2080 Ti with NVLINK and the RTX 2070 along with the other testing I've recently done. Performance in TensorFlow with 2 RTX 2080 Ti's is very good! Also, the NVLINK bridge with 2 RTX 2080 Ti's gives a bidirectional bandwidth of nearly 100 GB/sec!
NVLINK is one of the more interesting features of NVIDIA's new RTX GPU's. In this post I'll take a look at the performance of NVLINK between 2 RTX 2080 GPU's along with a comparison against single GPU I've recently done. The testing will be a simple look at the raw peer-to-peer data transfer performance and a couple of TensorFlow job runs with and without NVLINK.
Are the NVIDIA RTX 2080 and 2080Ti good for machine learning? Yes, they are great! The RTX 2080 Ti rivals the Titan V for performance with TensorFlow. The RTX 2080 seems to perform as well as the GTX 1080 Ti (although the RTX 2080 only has 8GB of memory). I've done some testing using **TensorFlow 1.10** built against **CUDA 10.0** running on **Ubuntu 18.04** with the **NVIDIA 410.48 driver**.
NVIDIA recently released version 10.0 of CUDA. This is an upgrade from the 9.x series and has support for the new Turing GPU architecture. This CUDA version has full support for Ubuntu 18.4 as well as 16.04 and 14.04. The CUDA 10.0 release is bundled with the new 410.x display driver for Linux which will be needed for the 20xx Turing GPU's. If you are doing development work with CUDA or running packages that require you to have the CUDA toolkit installed then you will probably want to upgrade to this. I'll go though how to do the install of CUDA 10.0 either by itself or along with an existing CUDA 9.2 install.
PyTorch for Scientific Computing - Quantum Mechanics Example Part 4) Full Code Optimizations -- 16000 times faster on a Titan V GPUWritten on September 14, 2018 by Dr Donald Kinghorn
This is the 16000 times speedup code optimizations for the scientific computing with PyTorch Quantum Mechanics example. The following quote says a lot, "The big magic is that on the Titan V GPU, with batched tensor algorithms, those million terms are all computed in the same time it would take to compute 1!!!"
PyTorch for Scientific Computing - Quantum Mechanics Example Part 3) Code Optimizations - Batched Matrix Operations, Cholesky Decomposition and InverseWritten on August 31, 2018 by Dr Donald Kinghorn
An amazing result in this testing is that "batched" code ran in constant time on the GPU. That means that doing the Cholesky decomposition on 1 million matrices took the same amount of time as it did with 10 matrices! In this post we start looking at performance optimization for the Quantum Mechanics problem/code presented in the first 2 posts. This is the start of the promise to make the code over 15,000 times faster! I still find the speedup hard to believe but it turns out little things can make a big difference.
PyTorch for Scientific Computing - Quantum Mechanics Example Part 2) Program Before Code OptimizationsWritten on August 16, 2018 by Dr Donald Kinghorn
This is the second post on using Pytorch for Scientific computing. I'm doing an example from Quantum Mechanics. In this post we go through the formulas that need to coded and write them up in PyTorch and give everything a test.
Doing Quantum Mechanics with a Machine Learning Framework: PyTorch and Correlated Gaussian Wavefunctions: Part 1) IntroductionWritten on July 31, 2018 by Dr Donald Kinghorn
A Quantum Mechanics problem coded up in PyTorch?! Sure! Why not? I'll explain just enough of the Quantum Mechanics and Mathematics to make the problem and solution (kind of) understandable. The focus is on how easy it is to implement in PyTorch. This first post will give some explanation of the problem and do some testing of a couple of the formulas that will need to be coded up.
NAMD Custom Build for Better Performance on your Modern GPU Accelerated Workstation -- Ubuntu 16.04, 18.04, CentOS 7Written on July 20, 2018 by Dr Donald Kinghorn
In this post I will be compiling NAMD from source for good performance on modern GPU accelerated Workstation hardware. Doing a custom NAMD build from source code gives a moderate but significant boost in performance. This can be important considering that large simulations over many time-steps can run for days or weeks. I wanted to do some custom NAMD builds to ensure that that modern Workstation hardware was being well utilized. I include some results for the STMV benchmark showing the custom build performance boost. I've included some results using NVIDIA 1080Ti and Titan V GPU's as well as an "experimental" build using an Ubuntu 18.04 base.
PyTorch is a relatively new ML/AI framework. It combines some great features of other packages and has a very "Pythonic" feel. It has excellent and easy to use CUDA GPU acceleration. It is fun to use and easy to learn. read on for some reasons you might want to consider trying it. I've got some unique example code you might find interesting too.
In this post I go through a simple modification to the VGG Image Annotator that adds easy to use buttons for adding labels to image object bounding-boxes. It is very fast way to do what could be a tedious machine learning data preparation task.
In this post I'll walk you through the best way I have found so far to get a good TensorFlow work environment on Windows 10 including GPU acceleration. YOU WILL NOT HAVE TO INSTALL CUDA! I'll also go through setting up Anaconda Python and create an environment for TensorFlow and how to make that available for use with Jupyter notebook. As a "non-trivial" example of using this setup we'll go through training LeNet-5 with Keras using TensorFlow with GPU acceleration. We'll get a setup that is 18 times faster than using the CPU alone.
In this post I'll be going over details of Installing Ubuntu 18.04 including the NVIDIA display driver and, any one of the available desktop environments. I'll do this starting from a base server install. I'll go over a few possible pitfalls and end with a short discussion on the new netplan configuration tool for Ubuntu networking.
In this post I'll walk you through the best way I have found so far to get a good TensorFlow work environment on Windows 10 including GPU acceleration. I'll go through how to install just the needed libraries (DLL's) from CUDA 9.0 and cuDNN 7.0 to support TensorFlow 1.8. I'll also go through setting up Anaconda Python and create an environment for TensorFlow and how to make that available for use with Jupyter notebook. As a "non-trivial" example of using this setup we'll go through training LeNet-5 with Keras using TensorFlow with GPU acceleration. We'll get a setup that is 18 times faster than using the CPU alone.
TensorFlow is a very important Machine/Deep Learning framework and Ubuntu Linux is a great workstation platform for this type of work. If you are wanting to setup a workstation using Ubuntu 18.04 with CUDA GPU acceleration support for TensorFlow then this guide will hopefully help you get your machine learning environment up and running without a lot of trouble. And, you don't have to do a CUDA install!
One of the questions I get asked frequently is "how much difference does PCIe X16 vs PCIe X8 really make?" Well, I got some testing done using 4 Titan V GPU's in a machine that will do 4 X16 cards. I ran several jobs with TensorFlow with the GPU's at both X16 and X8. Read on to see how it went.