Puget Systems print logo

https://www.pugetsystems.com

Read this article at https://www.pugetsystems.com/guides/775
Dr Donald Kinghorn (Scientific Computing Advisor )

NVIDIA CUDA with Ubuntu 16.04 beta on a laptop (if you just cannot wait)

Written on March 8, 2016 by Dr Donald Kinghorn
Share:

I was configuring a laptop to run GPU accelerated molecular dynamics simulations as a demo at the Spring 2016 ACS meeting. This laptop is to be used for running simulations directly and as a visualization front end for some larger machines. The laptop is "bleeding edge", a monster, with an Intel Skylake i7 6700K, 32GB mem, m.2 storage, and a 4K display! That hardware can be a challenge for Linux so I decided to go "bleeding edge" with the OS install too. I was not optimistic but miraculously it all worked including the needed NVIDIA CUDA setup!

Disclaimer!

First things first. No one endorses, recommends, suggests or condones doing this setup. Including myself, Puget Systems, NVIDIA and all other sane persons.  If that is completely understood then read on to see what I had to do to get this to work.  Note: also, that this is not a guide! If you don't understand what I did then don't try it!

Seeing is believing ...That's CUDA nbody running on Ubuntu 16.04

Hardware used

I've done this "Ubuntu 16.04 beta plus NVIDIA CUDA" setup on two systems so far, a laptop and a Peak Mini.

Puget Systems Traverse Pro 

  • 15 inch Notebook w/ TPM and 4K Matte IPS Screen
  • Intel Core i7 6700 (3.4GHz) 
  • 2 x 16GB DDR4 SODIMM
  • NVIDIA GTX 980M 8GB
  • Samsung 500GB 950 Pro M.2 SSD

Puget Systems Peak Mini

  • Intel Xeon E5-1650 (3.5GHz) 6-core
  • 64GB DDR4 Reg ECC
  • 2 x NVIDIA GTX 980
  • Samsung 500GB 850 Pro SSD

Software used

Ubuntu MATE 16.04 beta 1

Note: I used the MATE desktop spin because Unity and Gnome3 failed to work on the laptop (as usual, the 3D compositors are very problematic with the NVIDIA drivers). Also, MATE works very well with remote desktops like X2go. ... and MATE is my favorite Linux desktop at the moment! 

NVIDIA CUDA 7.5.18

Note: The most recent Ubuntu supported for CUDA is 15.04 so installing on 16.04 beta is stretching things a bit! The main limitation is that version 5 gcc compilers are not supported yet. 

NVIDIA Display Driver 361.28

NVIDIA driver from the Ubuntu "Software & Updates" --> "Additional Drivers" utility.

gcc version 5.3.1

You have to modify a cuda header file to enable this gcc version to work.

VMD and NAMD

These are wonderful molecular dynamics tools from the great folks at the Theoretical  and Computational Biophysics Group at UIUC.

OS Install

The basic OS install was trouble free on both systems. I used the live install image ubuntu-mate-16.04-beta1-desktop-amd64.iso. You could also try using the ISO image from  Ubuntu MATE 16.04 (Xenial Xerus) Daily Build. The Lubuntu and Xubuntu spins may also work but I would recommend to stay away from Unity and Gnome 3 unless you are feeling adventurous.  

After the basic install do an "apt-get update" and "upgrade", reboot, and then go to "Software & Updates" --> "Additional Drivers"  and install the latest tested NVIDIA driver. I didn't have any trouble with this using the MATE spin. After a reboot open the "nvidia-settings" app and check that everything is as expected. Configure you system to taste as you would for any install.

I tried to use the normal NVIDIA CUDA Ubuntu 15.04 repo deb files to do the CUDA install but failed. We will be doing the CUDA install manually from the ".run" file. Dependencies that would normally get pulled in from doing a CUDA install from the NVIDIA CUDA repo will have to be added.    

There are lots of dependencies for a CUDA install. To make life easier I generated a dependency list from another system that was running 15.04.

The "-s" flag for apt-get will show the details of what would be done for an install including dumping the dependency list.

 kinghorn@i7:~$ apt-get -s install cuda

In the output below I have removed all of the deb files that would be in the CUDA repo itself and just left dependencies that would get pulled in from the standard Ubuntu repos.  

ca-certificates-java default-jre default-jre-headless fonts-dejavu-extra freeglut3 freeglut3-
dev java-common libatk-wrapper-java libatk-wrapper-java-jni  libdrm-dev libgl1-mesa-dev libglu
1-mesa-dev libgnomevfs2-0 libgnomevfs2-common libice-dev libpthread-stubs0-dev libsctp1 libsm-
dev libx11-dev libx11-doc libx11-xcb-dev libxau-dev libxcb-dri2-0-dev libxcb-dri3-dev libxcb-g
lx0-dev libxcb-present-dev libxcb-randr0-dev libxcb-render0-dev libxcb-shape0-dev libxcb-sync-
dev libxcb-xfixes0-dev libxcb1-dev libxdamage-dev libxdmcp-dev libxext-dev libxfixes-dev libxi
-dev libxmu-dev libxmu-headers libxshmfence-dev libxt-dev libxxf86vm-dev lksctp-tools mesa-com
mon-dev openjdk-7-jre openjdk-7-jre-headless tzdata-java x11proto-core-dev x11proto-damage-dev
 x11proto-dri2-dev x11proto-fixes-dev x11proto-gl-dev x11proto-input-dev x11proto-kb-dev x11pr
oto-xext-dev x11proto-xf86vidmode-dev xorg-sgml-doctools xtrans-dev libgles2-mesa-dev 
nvidia-modprobe build-essential

You can "sudo apt-get install" the above list and that should get most or all of the dependencies. There is a possibility that there will still be missing debs. For example I added the last three entries to the list when I discovered that they were missing on my new 16.04 install after the others were installed. That's is probably because they were already installed on the 15.04 system it ran "apt-get -s install cuda" on i.e. they didn't come up as needed dependencies because they were already installed.

CUDA Install

Download the CUDA .run file from the NVIDIA download site.  I used this, http://developer.download.nvidia.com/compute/cuda/7.5/Prod/local_installers/cuda_7.5.18_linux.run

kinghorn@skylap:~/Downloads$ chmod 755 cuda_7.5.18_linux.run

kinghorn@skylap:~/Downloads$ sudo ./cuda_7.5.18_linux.run --override

The "--override" is needed so you don't get the error, "Toolkit:  Installation Failed. Using unsupported Compiler."

Be sure to not install the NVIDIA driver that is in the .run file since you already have a more up to date version installed. Here's the command history with the long EULA text cut out.

kinghorn@skylap:~/Downloads$ sudo ./cuda_7.5.18_linux.run --override

-------------------------------------------------------------
Do you accept the previously read EULA? (accept/decline/quit): accept
You are attempting to install on an unsupported configuration. Do you wish to continue? ((y)es/(n)o) [ default is no ]: y
Install NVIDIA Accelerated Graphics Driver for Linux-x86_64 352.39? ((y)es/(n)o/(q)uit): n
Install the CUDA 7.5 Toolkit? ((y)es/(n)o/(q)uit): y
Enter Toolkit Location [ default is /usr/local/cuda-7.5 ]:
Do you want to install a symbolic link at /usr/local/cuda? ((y)es/(n)o/(q)uit): y
Install the CUDA 7.5 Samples? ((y)es/(n)o/(q)uit): y
Enter CUDA Samples Location [ default is /home/kinghorn ]: /usr/local/cuda-7.5
Installing the CUDA Toolkit in /usr/local/cuda-7.5 ...
Finished copying samples.

===========
= Summary =
===========

Driver:   Not Selected
Toolkit:  Installed in /usr/local/cuda-7.5
Samples:  Installed in /usr/local/cuda-7.5

 

CUDA Environment

Set up the  environment and libs for the install

kinghorn@skylap:~/projects$ sudo emacs -nw /etc/profile.d/cuda.sh

export PATH=$PATH:/usr/local/cuda/bin

kinghorn@skylap:~/projects$ sudo emacs -nw /etc/ld.so.conf.d/cuda.conf

/usr/local/cuda/lib64

kinghorn@skylap:~/projects$ source /etc/profile.d/cuda.conf

​kinghorn@skylap:~/projects$ sudo ldconfig 

Force it to work with gcc 5

Fix/break the header file that doesn't want to let us use gcc > 4.8. All we are going to do is comment out (//) the error line that drops you out of a build.

kinghorn@skylap:~/projects$ sudo emacs -nw /usr/local/cuda/include/host_config.h

line: 115 comment out error
//#error -- unsupported GNU version! gcc versions later than 4.9 are not supported! 


Compiling the CUDA Samples

You can now copy the CUDA sample to your testing directory and build some code.

kinghorn@skylap:~/projects$ rsync -av /usr/local/cuda/samples .

The "Makefile" in the samples directory does not pick up all of the paths for this install and I haven't fix all of the problems, just hacked around a few of them. One change I made was in the file "common/findgllib.mk

kinghorn@skylap:~/projects/samples/common$ emacs -nw findgllib.mk
line : 61  changed nvidia-352 to nvidia-361
    UBUNTU_PKG_NAME = "nvidia-361"

 

Another common Ubuntu problem is needing to specify  GLPATH=/usr/lib on the make line. 

GLPATH=/usr/lib make 

That will build a lot of the sample code. I only got one error in the output,

/usr/bin/ld: cannot find -lnvcuvid
collect2: error: ld returned 1 exit status
Makefile:346: recipe for target 'cudaDecodeGL' failed

That one was strange since the lib is there and other libs around it link OK?? ... still some bugs.

The "Makefile" did not follow all of the directories for some reason so a bunch of sample were missed. I was able to cd into the individual directories and rum make to get them to build. In total I had 145 of the samples build without trouble. That is nearly all of them!

Here's the output from the nbody benchmark run on the laptop (the openGL rendering works fine too),

kinghorn@skylap:~/projects/samples/bin/x86_64/linux/release$ ./nbody -benchmark -numbodies=256000

> Windowed mode
> Simulation data stored in video memory
> Single precision floating point simulation
> 1 Devices used for simulation
GPU Device 0: "GeForce GTX 980M" with compute capability 5.2

> Compute 5.2 CUDA device: [GeForce GTX 980M]
number of bodies = 256000
256000 bodies, total time for 10 iterations: 7207.025 ms
= 90.934 billion interactions per second
= 1818.670 single-precision GFLOP/s at 20 flops per interaction

 To finish off here's a snapshot of the laptop running a molecular dynamics simulation on the "Satellite Tobacco Mosaic Virus" with NAMD and displaying the running simulation to a 4K monitor in real time using VMD.  Awesome!

Happy computing! --dbk

Tags: Ubuntu 16.04 beta, NVIDIA, CUDA
Lasse Pedersen

Excellent article. Helped me figure out the last details I needed to make it work.
I'm actually on Unity with no issues. Also laptop.

I found several more instances of nvidia-352 being the "package library" rather than nvidia-361 and thats actually the cause of the "cannot find lnvcuvid" issue as well as the "makefile doesnt make all samples" issue.
By running: grep -r nvidia-352 -l --null . | xargs -0 sed -i 's#nvidia-352#nvidia-361#g'
In the root of the samples directory I fixed all compile issues. The Makefile now goes through ALL samples.

I didnt need to set GLPATH.
I think you want the second last line in the CUDA Environment to read: ~/projects$ source /etc/profile.d/cuda.sh (instead of cuda.conf)

My poor computer only managed about 400 single-precision GFLOP/s at 20 flops per interaction...

Posted on 2016-03-13 12:23:10
Sarmad Shams

i am getting error of premission denied when run grep -r nvidia-352.... command

Posted on 2016-06-24 12:09:38
C Vasco

did you try using sudo? EDIT: sudo needs to be on the xargs/sed command
grep -r nvidia-352 -l --null . | sudo xargs -0 sed -i 's#nvidia-352#nvidia-361#g'

Posted on 2016-07-02 00:27:35
pavlos kairis

tried to install on a Desktop, mate 15.10, I have a GT 640. I get the error, X server is running. Do I need to run this in single user mode?

Posted on 2016-03-27 16:48:07
Alice Rose

I used the below website for NVIDIA-Linux-x86_64-361.45.11 driver

http://www.nvidia.com/objec...

Login from any Ubuntu tty terminal:

$ sudo service lightdm stop

$ cd ~/Downloads/

$ wget us.download.nvidia.com/XFre...

$ sudo chmod +x NVIDIA-Linux-x86_64-361.45....

$ sudo ./NVIDIA-Linux-x86_64-361.45....

$ sudo service lightdm restart

$ reboot

Posted on 2016-06-09 01:50:52
Rob Reid

Seems CUDA 7.5.18 it is available in the Xenial multiverse repositories now: `apt-get install nvidia-cuda-toolkit`

Posted on 2016-04-21 06:02:04
vintermann

Thanks for the warning! It wasn't listed on Nvidia's site, so I was about to try to follow these instructions...

Posted on 2016-05-31 17:08:07
Thusi

Hi, thanks for the comment. I did install it using apt-get, but I dont think it installed the cuda driver. Is there a way to install cuda driver using the repo?

Posted on 2016-06-05 15:55:19
Han Qiu

up!
up!

Posted on 2016-06-29 15:31:53
C Vasco

I did the apt-get install at first, but the libs and includes end up all over the place, so other programs have a hard time linking to CUDA. This approach seems more useful for linking with OpenCV or caffe...

Posted on 2016-07-02 00:23:47
zhy

That's true. With apt-get installation, cuda files locate in `/usr/bin`, `/usr/lib`, and I can only specify CUDA_PATH as `/usr` and this cause problems when compiling CUDA samples from the run file.

Posted on 2016-07-22 07:54:01
C Vasco

As Lasse Pedersen said, there's an error in the instructions (still there today).

~/projects$ source /etc/profile.d/cuda.sh (instead of cuda.conf)

Posted on 2016-07-02 00:32:30
Eric J. Humphrey

+100 for this!

btw, anyone who got here looking to get Theano up and running, here's the last piece: https://github.com/Theano/T...

Posted on 2016-07-14 14:39:33
Harvitronix

If you get the error:
WARNING (theano.sandbox.cuda): CUDA is installed, but device gpu0 is not available (error: cuda unavailable)

You need to add the following to your ~/.theanorc file:

[nvcc]
flags=-D_FORCE_INLINES

Per: https://github.com/Theano/T...

Posted on 2016-07-21 04:14:06
Gaurav Waghmare

Thanks to you I'm successfully using TensorFlow 0.10rc on my notebook.

Posted on 2016-08-01 16:18:13
Kaustuv Datta

Facing a problem in the CUDA installation stage:

===========
= Summary =
===========

Driver: Not Selected
Toolkit: Installed in /usr/local/cuda-7.5
Samples: Installed in /usr/local/cuda-7.5

Please make sure that
- PATH includes /usr/local/cuda-7.5/bin
- LD_LIBRARY_PATH includes /usr/local/cuda-7.5/lib64, or, add /usr/local/cuda-7.5/lib64 to /etc/ld.so.conf and run ldconfig as root

To uninstall the CUDA Toolkit, run the uninstall script in /usr/local/cuda-7.5/bin
To uninstall the NVIDIA Driver, run nvidia-uninstall

Please see CUDA_Installation_Guide_Linux.pdf in /usr/local/cuda-7.5/doc/pdf for detailed information on setting up CUDA.

***WARNING: Incomplete installation! This installation did not install the CUDA Driver. A driver of version at least 352.00 is required for CUDA 7.5 functionality to work.
To install the driver using this installer, run the following command, replacing <cudainstaller> with the name of this run file:
sudo <cudainstaller>.run -silent -driver

Logfile is /tmp/cuda_install_12175.log

I was using the 367.35 driver at the time.

Would appreciate some help since I'm quite new to all this!

Posted on 2016-08-16 04:21:54
Tirta Setiawan

Actually, There are same problem coming to me when try to install CUDA-8.0. later, I successfully install CUDA-7.5 on my ubuntu 14.04. Some simulation using Gromacs have been completely done in that configuration, but now i completely change my OS from Ubuntu 14.04 to 16.04 within CUDA-8.0 which unfortunatelly make my dependeny broken. So, any one could give us a solution.

Posted on 2016-08-18 15:26:36
Piotr Trochim

Great article, thank you very much for sharing.

Posted on 2016-08-28 20:00:40
Arjun

Almost same as Lasse Pedersen's answer, I was getting the same error at /usr/bin/ld: cannot find -lnvcuvid because it still searches in /usr/lib/"nvidia-352" of the corresponding folder and findgllib.mk was modified in the common folder. Found that, findglllib.mk is present in subfolders too. What I did was to modify UBUNTU_PKG_NAME from nvidia-352 to nvidia-361 in all the files inside samples folder.

grep -rl 'nvidia-352' ./ | xargs sed -i 's/nvidia-352/nvidia-361/g' recursively finds the nvidia-352 string and replaces it with the nvidia-361. So with this we don't get any errors while compiling. :)

Posted on 2016-09-13 06:27:36
tom eric

Thanks A lot. Brilliant Article.
I am running CUDA on Ubuntu 16.04.1 LTS 64-bit with GeForce GTX 750 Ti/PCIe.

Posted on 2016-09-20 11:14:19
Rohan Raj

I did the same thing many times. It installed successfully from http://randomized.me/index.... !
I ran the run file in ALT+CTRL+F1 and stopped my lightdm ! It installed sucessfully ! But after the reboot there is porblem with my ubuntu login ! It goes to low graphics mode ! Now I can easily do
sudo apt-get install nvidia-current-updates
But this will defeat all the installation I did. Have any solutions to this

Posted on 2016-10-15 15:11:58
Jesus Pico

i Got a Question When i Do the override still fails and update on these matetr

Posted on 2016-11-30 20:57:27
Maxime

Hello!

GLPATH=/usr/lib make

I don't find this line... There are severals closed so I don't know what I can modify... Any help?

Posted on 2016-12-20 13:28:37