I got my hands on a GTX 1080 today and decided to see if I could get it working on Linux and do some quick CUDA testing. I succeeded! These are preliminary results and will no doubt change when official drivers and CUDA 8 is released. Also, this is a new architecture so there are bound to be problems with some existing CUDA accelerated programs. Take these results for what they are, just a quick hack of a setup to see if things work. Later when official drivers and CUDA are are released we will get a much better picture of how compute is on the new GeForce cards.
I'll get right to the first CUDA results since they are stunning.
CUDA 7.5 samples nbody simulation (benchmark single precision)
GTX 1080 –> 5816 GFLOP/s
GTX 980 –> 2572 GFLOP/s
I would expect your jaw to drop right now, mine did! I also, ran a "real" program, NAMD molecular dynamics simulation.
NAMD 2.11 500 time steps of stmv (satellite tobacco mosaic virus – million atom)
GTX 1080 –> 54 sec .81 day/ns
GTX 980 –> 71 sec 1.21 day/ns
33% speedup with 1080 which is very good for this program!
Testing setup
- Intel Xeon E5-1650 v3 6-core 3.5 GHz (3.6 GHz all-core-turbo)
- 64GB DDR 4 2133 REG ECC
- GTX 980
- GTX 1080
- 2 x Intel Xeon E5-2687W v4 12-core (Broadwell) 3.0 GHz (3.2 GHz all-core-turbo)
- 256 GB DDR4 2133 REG ECC
- GTX 980
- GTX 1080
Software/OS
- Ubuntu MATE 16.04
- NVIDIA display driver 367.18 installed from ppa:graphics-drivers/ppa The GTX 1080 is detected as "Graphics Device" with this driver 🙂
-
CUDA 7.5-18 I used my usual hacks to get CUDA working on Ubuntu 16.04.
- NAMD 2.11 Note: I used the Peak Dual for NAMD since the Single was CPU bound i.e. NAMD uses both the CPU and GPU and I needed more CPU to balance the performance of the GTX 1080.
Please note that getting all this working on Ubuntu 16.04 requires some sysadmin skill and it's not supported by anyone! It may be possible to get all of this working in a more sensible setup with Ubuntu 14.04. I will try that and do another post with details. I may do another detailed post on getting everything working on Ubuntu 16.04 after CUDA 8 is out and a "release" driver for the 1080 is released. (both of those I expect to see soon)
Here's some raw output so you can see a few more details about the card.
kinghorn@u16-1080:~/testing/samples/bin/x86_64/linux/release$ nvidia-smi Thu May 26 14:04:51 2016
+------------------------------------------------------+
| NVIDIA-SMI 367.18 Driver Version: 367.18 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
|===============================+======================+======================|
| 0 GeForce GTX 980 Off | 0000:05:00.0 On | N/A |
| 26% 37C P8 15W / 180W | 80MiB / 4037MiB | 0% Default |
+-------------------------------+----------------------+----------------------+
| 1 Graphics Device Off | 0000:09:00.0 Off | N/A |
| 27% 30C P8 6W / 180W | 1MiB / 8113MiB | 0% Default |
+-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+
| Processes: GPU Memory |
| GPU PID Type Process name Usage |
|=============================================================================|
| 0 1113 G /usr/lib/xorg/Xorg 79MiB |
+-----------------------------------------------------------------------------+
kinghorn@u16-1080:~/testing/samples/bin/x86_64/linux/release$ ./nbody -benchmark -numbodies=256000 -device=0 > Windowed mode > Simulation data stored in video memory > Single precision floating point simulation > 1 Devices used for simulation gpuDeviceInit() CUDA Device [0]: "Graphics Device > Compute 6.1 CUDA device: [Graphics Device] number of bodies = 256000 256000 bodies, total time for 10 iterations: 2259.061 ms = 290.103 billion interactions per second = 5802.057 single-precision GFLOP/s at 20 flops per interaction kinghorn@u16-1080:~/testing/samples/bin/x86_64/linux/release$
kinghorn@u16-1080:~/testing/samples/bin/x86_64/linux/release$ ./bandwidthTest --device=0 [CUDA Bandwidth Test] - Starting... Running on... Device 0: Graphics Device Quick Mode Host to Device Bandwidth, 1 Device(s) PINNED Memory Transfers Transfer Size (Bytes) Bandwidth(MB/s) 33554432 12147.9 Device to Host Bandwidth, 1 Device(s) PINNED Memory Transfers Transfer Size (Bytes) Bandwidth(MB/s) 33554432 12866.1 Device to Device Bandwidth, 1 Device(s) PINNED Memory Transfers Transfer Size (Bytes) Bandwidth(MB/s) 33554432 278829.1 Result = PASS NOTE: The CUDA Samples are not meant for performance measurements. Results may vary when GPU Boost is enabled.
kinghorn@u16-1080:~/testing/samples/bin/x86_64/linux/release$ ./deviceQuery
./deviceQuery Starting...
CUDA Device Query (Runtime API) version (CUDART static linking)
Detected 2 CUDA Capable device(s)
Device 0: "Graphics Device"
CUDA Driver Version / Runtime Version 8.0 / 7.5
CUDA Capability Major/Minor version number: 6.1
Total amount of global memory: 8113 MBytes (8507555840 bytes)
MapSMtoCores for SM 6.1 is undefined. Default to use 128 Cores/SM
MapSMtoCores for SM 6.1 is undefined. Default to use 128 Cores/SM
(20) Multiprocessors, (128) CUDA Cores/MP: 2560 CUDA Cores
GPU Max Clock rate: 1734 MHz (1.73 GHz)
Memory Clock rate: 5005 Mhz
Memory Bus Width: 256-bit
L2 Cache Size: 2097152 bytes
Maximum Texture Dimension Size (x,y,z) 1D=(131072), 2D=(131072, 65536), 3D=(16384, 16384, 16384)
Maximum Layered 1D Texture Size, (num) layers 1D=(32768), 2048 layers
Maximum Layered 2D Texture Size, (num) layers 2D=(32768, 32768), 2048 layers
Total amount of constant memory: 65536 bytes
Total amount of shared memory per block: 49152 bytes
Total number of registers available per block: 65536
Warp size: 32
Maximum number of threads per multiprocessor: 2048
Maximum number of threads per block: 1024
Max dimension size of a thread block (x,y,z): (1024, 1024, 64)
Max dimension size of a grid size (x,y,z): (2147483647, 65535, 65535)
Maximum memory pitch: 2147483647 bytes
Texture alignment: 512 bytes
Concurrent copy and kernel execution: Yes with 2 copy engine(s)
Run time limit on kernels: No
Integrated GPU sharing Host Memory: No
Support host page-locked memory mapping: Yes
Alignment requirement for Surfaces: Yes
Device has ECC support: Disabled
Device supports Unified Addressing (UVA): Yes
Device PCI Domain ID / Bus ID / location ID: 0 / 9 / 0
Compute Mode:
Default (multiple host threads can use ::cudaSetDevice() with device simultaneously)
Device 1: "GeForce GTX 980"
CUDA Driver Version / Runtime Version 8.0 / 7.5
CUDA Capability Major/Minor version number: 5.2
Total amount of global memory: 4037 MBytes (4233297920 bytes)
(16) Multiprocessors, (128) CUDA Cores/MP: 2048 CUDA Cores
GPU Max Clock rate: 1216 MHz (1.22 GHz)
Memory Clock rate: 3505 Mhz
Memory Bus Width: 256-bit
L2 Cache Size: 2097152 bytes
Maximum Texture Dimension Size (x,y,z) 1D=(65536), 2D=(65536, 65536), 3D=(4096, 4096, 4096)
Maximum Layered 1D Texture Size, (num) layers 1D=(16384), 2048 layers
Maximum Layered 2D Texture Size, (num) layers 2D=(16384, 16384), 2048 layers
Total amount of constant memory: 65536 bytes
Total amount of shared memory per block: 49152 bytes
Total number of registers available per block: 65536
Warp size: 32
Maximum number of threads per multiprocessor: 2048
Maximum number of threads per block: 1024
Max dimension size of a thread block (x,y,z): (1024, 1024, 64)
Max dimension size of a grid size (x,y,z): (2147483647, 65535, 65535)
Maximum memory pitch: 2147483647 bytes
Texture alignment: 512 bytes
Concurrent copy and kernel execution: Yes with 2 copy engine(s)
Run time limit on kernels: Yes
Integrated GPU sharing Host Memory: No
Support host page-locked memory mapping: Yes
Alignment requirement for Surfaces: Yes
Device has ECC support: Disabled
Device supports Unified Addressing (UVA): Yes
Device PCI Domain ID / Bus ID / location ID: 0 / 5 / 0
Compute Mode:
Default (multiple host threads can use ::cudaSetDevice() with device simultaneously)
Peer access from Graphics Device (GPU0) -> GeForce GTX 980 (GPU1) : No
Peer access from GeForce GTX 980 (GPU1) -> Graphics Device (GPU0) : No
deviceQuery, CUDA Driver = CUDART, CUDA Driver Version = 8.0, CUDA Runtime Version = 7.5, NumDevs = 2, Device0 = Graphics Device, Device1 = GeForce GTX 980
Result = PASS
Happy computing! –dbk