Dr. Donald Kinghorn (HPC and Scientific Computing)

NVIDIA CUDA GPU computing on a (modern) laptop

Written on March 13, 2015 by Dr. Donald Kinghorn
Share:

***NOTE*** I added a small update at the end of the post for the CUDA 7 finial release which is out now.

Modern high-end laptops can be treated as desktop system replacements so it's expected that people will want to try to do some serious computing on them. Doing GPU accelerated computing on a laptop is possible and performance can be surprisingly good with a high-end NVIDIA GPU. [I'm looking at GTX 980m and 970m ]. However, first you have to get it to work! Optimus technology can present serious problems to someone who wants to run a Linux based CUDA laptop computing platform.

I just spent considerable time exploring options and failing many times trying to get a laptop setup for GPU computing with Linux (including Optimus GPU switching). I finally succeeded with a straight forward install without including Optimus support. Even though I did succeed a couple of times with full optimus GPU switching support it proved to be very unstable and would break at the slightest provocation. I have to concede defeat on Optimus under Linux, it was just too fragile. However, we really didn’t care about it anyway! Our intent was to have a nice NVIDIA GPU accelerated laptop for CUDA development and modest load job runs. For that, the setup I describe here should work great. It’s essentially what I would do on a desktop machine.

I have a couple of CUDA compute performance numbers for you using the GTX970m and GTX980m at the end of the post ...

The setups with Optimus that I tried suffered badly from side effects. When I made a change and then reverted the change, I didn’t recover the previous system state. It was also the case that doing something like a system shutdown while the Intel graphics were running and then rebooting and changing to NVIDA and then shutting down could leave the system in a different state and possibly a black screen or lockup. I would have a system setup that seemed stable and working properly and then some sequence of events would break it. Reinstall repeating the same process would not guarantee a system in the same state that I had done a few days ago. I’m not going to try to list the number of things I tried to get a stable setup because there were just too many and it would likely be pointless for anyone to try to repeat them since they were all effectively non-repeatable after a few days with any OS update involving the kernel, drivers, or X.

My impression was that a lot of the trouble I had was related to the gnome desktop core since both Ubuntu Unity and Gnome3 desktops were very susceptible to black-screen states and/or corrupted workspaces. KDE didn’t seem to suffer from these problems.

I have confidence that the following procedure will give you a working, NVIDIA driven, CUDA development laptop. I believe this mainly because it’s simple!

Laptops

The two laptops I’ve been working with are;

[They are similar and based around Clevo P150SM-A and P170SM-A]

Installation and Setup

The Linux version for these build was Ubuntu 14.04.1 server with kubuntu-desktop.

I start with a simple Ubuntu 14.04 server install. I do this routinely for Ubuntu installs since it’s easier to automate than using a “Live-Image” install and the added packages will come from the update repos rather than the (stale) install media.

… basic server text based install …

Then

sudo apt-get update
$ sudo apt-get dist-upgrade
$ sudo shutdown -r now

That gives you a fresh install with the latest kernel updates.

Then add the desktop and other needed packages. These will pull down LOTS of dependencies!

sudo apt-get install kubuntu-desktop build-essential emacs synaptic dkms
sudo apt-get upgrade
sudo shutdown -r now

At this point you will have a Ubuntu with a KDE desktop login screen and the systems will be using the Intel video driver. [ Yes, this is essentially Kubuntu ] I personally prefer synaptic for a graphical package manager on Ubuntu and, of course, emacs is essential! :-) build-essential will install the needed compilers, build tools and kernel headers and dkms (Dynamic Kernel Module Support) is useful for automatically (hopefully) rebuilding kernel driver modules on boot whenever there is a kernel version change.

That is a working laptop! And if you didn’t have the nice NVIDIA graphics card in the system that you desperately want to use for computing you could go on your merry way and be happy with your new Linux laptop.

NVIDIA graphics driver and CUDA setup.

DO NOT install NVIDIA drivers from the Ubuntu repos or any of the independent ppa’s (repos)!

We will install the NVIDIA drivers as part of the CUDA setup. This is simple and gives a stable setup ready to go with CUDA.

I used the Ubuntu 14.04 CUDA repo version 7rc. That is the “release candidate” for version 7. I’ve been using it for all of my CUDA testing for several weeks now without problems and it may be at full release by the time you read this.

You will need to be a registered developer to get the rc version! If you do any CUDA work then you should go ahead and register. NVIDIA is very good about the email that they send out to registered developers. It’s infrequent and they are not “spamy” -- I always find the information interesting.

The full file I downloaded was;

cuda-repo-ubuntu1404-7-rc_7.0-18_amd64.deb

Install that with,

sudo dpkg -i   cuda-repo-ubuntu1404-7-rc_7.0-18_amd64.deb

Then do

sudo apt-get update
sudo apt-get install cuda
sudo shutdown -r now

The cuda install pulls down a lot dependencies and runs some very good configure scripts to properly setup the NVIDIA graphics kernel modules and initializes the dkms settings for you. This is, in my opinion, the easiest way to install the NVIDIA graphics drivers on Linux.

After the reboot you will be running on your nifty NVIDIA graphics card and won't be pestered by that Intel video i.e. you wont have a simple way to switch over to it. You may consider that good or bad. I consider it good!

Run nvidia-settings and see that everything is as expected. You will not have the NVIDIA “optimus prime” panel to switch drivers. Count your blessings!

To finish up the CUDA setup I like to add a couple of system files;

You will need to be root to create the files so use your favorite sudo friendly editor (sudo -s to a root shell) and create;

/etc/profile.d/cuda.sh

containing the line

export PATH=$PATH:/usr/local/cuda/bin

This will execute when you login and put the cuda tools on your path.

You can also add the cuda libraries to the system by creating

/etc/ld.so.conf.d/cuda.conf

containing the line

/usr/local/cuda/lib64

 

Then run

 

sudo ldconfig 

Now on to some testing.

Copy the CUDA samples directory to your home directory or wherever you want to put it.

 

I do (be careful with rsync if you are not used to it -- you could just use cp -a if you like)

 

rsync -av /usr/local/cuda/samples  . 

Then

cd samples/
make -j4

and watch as all of the sample codes compile.

My favorite simple test code is the “nbody” sample. It has a “benchmark" mode with switches to to make it useful.

cd bin/x86_64/linux/release/
./nbody -benchmark -numbodies=256000

and marvel at the incredible single precision floating point performance of the GPU. It’s pretty impressive!

Here’s what I got;

GTX 970m    1415    single-precision GFLOP/s
GTX 980m    1835    single-precision GFLOP/s

On the order of 1.5 Tera flops of compute performance from a laptop!!

for reference the desktop card gives

GTX 980    2496    single-precision GFLOP/s
2 x 980    4561    single-precision GFLOP/s

Here’s another testing result (as a teaser for a post I’ll do about running GROMACS with GPU acceleration) “lysozyme in water” molecular dynamics simulation with GROMACS on the GTX970m powered laptop gave

58 ns/day

That may not mean that much to you if you are not familiar with gromacs and that particular test job but I’ll assure you it’s impressive (about the same as a simple desktop machine with a Core i7 4770 4-core @ 3.4 GHz CPU with a Tesla K40 for acceleration! Note: that the GPU compute part of the calculation is single-precision so the GTX GeForce card is at it’s best for this.)

ADDENDUM FOR CUDA 7 FINIAL

A few days after I wrote this (while I was at GTC) NVIDIA finalized cuda 7. I did a fresh install and it worked fine. If you are going to follow what I’ve outlined above then just use the release version. I used the network install repo, https://developer.nvidia.com/cuda-downloads

If you have already done an install using the 7rc then you can update to the release version by grabbing either the network repo or the full repo (you may as well use the network version) then,



sudo dpkg -i  cuda-repo-ubuntu1404_7.0-28_amd64.deb
sudo apt-get update
sudo apt-get upgrade  (it will see the updated repo and offer to upgrade cuda)
sudo shutdown -r now

On reboot you should have the latest CUDA and NVIDIA graphics drivers. Easy!

Happy computing! --dbk

 

Tags: NVIDIA, CUDA, HPC, laptop, 980m
maverick

Thanks for your post. Unfortunately, this did not work for me. On executing the final reboot command, I was greeted by a black screen on reboot. The GPU fan started spinning really fast for ~5 seconds and then stopped. I can still access the console but there is no Unity or Kubuntu desktop (Kubuntu needs to be chosen, BTW, on a reboot; Unity is default for me).

I started from a clean Ubuntu 14.04 install. The only difference I can find is that you have used Ubuntu Server. I am not sure if this makes a difference or not.

Posted on 2015-03-18 04:35:18
olena

Hi Donald, I recently got an MSI laptop for sole purposes of using the power of its GPUs for deep learning. I tried everything. Installing ubuntu (at least 3 times), opensuse (twice), and even a failure in getting ubuntu-server set up.
After one successful install of cuda on ubuntu 14.04, the graphics went shimmering/glittering (not sure what's the right term to describe this phenomenon).
So I tried openSuse. After a successful install of cuda on openSuse 13.2 the screen resolution went to 600x480, and nothing I found online helped changing it back.
Did you experience graphics issues after cuda installation? If so, how did you resolve those?

Posted on 2015-04-13 00:14:47
Donald Kinghorn

Sorry to hear about problems with SuSE. I hadn't tried it but it's is often a well behaved distro. On your SuSE setup you might try removing any xorg.conf file that got created during the install. You may also have luck by adding a nomodeset to your kernel startup line.

****************************************************************************
nomodeset can help with some of the crazy display problems, try it!
Just search for nomodeset and the ditro that you are using ...
****************************************************************************

In General the ...

The key points to get this to work are;
DO NOT install any gnome based desktop (ubuntu default unity, gnome3 etc...)
DO NOT install the NVIDIA drivers from anywhere other than the CUDA repo
DO INSTALL a KDE desktop (on Ubuntu that means install straight from the Kubuntu live media or install Ubuntu server (no GUI) and then add kubuntu-desktop )
DO INSTALL the NVIDIA CUDA repo (on Ubuntu that means the deb file on rpm based distro get the net install repo.

If you do the above then you at least have a good chance at getting it to work. However, it is not guaranteed! There are things that can go wrong and it is constantly changing. I expect the gnome issues to be resolved at some point. ... and the new KDE could break everything. SO WHY DO WE DO THIS? Because it's fun and the challenge of getting it working is half the enjoyment :-)

Hang in there and keep at it! Best wishes -Don

Posted on 2015-04-13 16:01:47
Scott Le Grand

In Olena's case, I'm pretty sure it's not Gnome and it's not even Ubuntu. I think it's something to do with the GT72-QE's internal display: either some sort of weird EDID/BIOS issue or some sort of bug in the display driver. I have the same problem and:

1) it goes away on external monitors

2) It's reproducible under Ubuntu 12.04 LTS, 14.04 LTS, and 14.10 LTS.

3) The real cincher is that one sees this the second one enables a minimum X server on Arch Linux (no Gnome installed). Updating the laptop BIOS to the 3/24/15 edition does not help either.

I'm going to file a bug with NVIDIA and hope for the best.

Posted on 2015-04-20 15:50:12
Vivek Saxena

I followed the steps you outlined on Kubuntu (and not Ubuntu). I ended up with a dual-screen configuration of 3840x1080 instead of 1920x1080. Also the power light on my laptop is still orange, signaling that the Intel driver is in use, in spite of the NVIDIA driver having been installed.

Posted on 2015-07-07 02:51:40
Leonid Volnitsky

I was able to install Cuda 7.0 on standard Ubundu desktop 15 (vivid). But I agree that we need to avoid any extra Gnome packages. I had to reinstall Ubuntu after bricking the system with some extra gnome packages.

Posted on 2015-07-24 20:51:12
Karesh Arunakirinathan

Thanks for the post. On my 14.04.2 ubuntu I have installed nvidia driver and cuda toolkit 7. After I installed I have re started then added the cuda path variables and then ran deviceQuery and it is passed. seems like everything working. but when I restart my notebook it freezes on splash screen. Do you know the reason for it? is anything problem with 14.04 version of ubuntu?

Posted on 2015-10-13 01:22:42
Donald Kinghorn

Hi Karesh, I didn't see your post earlier ... it's always a little
tricky figuring out nvidia + laptop problems but you could be seeing a
problem with gnome3/unity. First thing to try when you get a black
screen on boot is to try reboot and add a nomodeset to the end of the
kernel boot line from grub menu. Secondly, if you have a working
system and then it fails after a reboot it may be from a kernel update
(ubuntu is really bad about nagging about updates) it can happen that
the nvidia kernel modules fail to rebuild by dkms on boot. If you can
get to a terminal your can try to force rebuild dkms managed modules AND
if you get it working again search for "apt-mark hold" as a way to
freeze kernel updates ... Happy Holidays! --Don

Posted on 2015-11-27 18:05:44
EstherJ

I am having problems on my MSi laptop. It has a GTX860 and *should* work. But when I open nvidia-settings I get these errors and I can't change it from the Intel to the NVIDIA driver. I have an existing Ubuntu 14.04 install. Not too keen on starting with a fresh install, since it was a hassle to install it in the first place, with 3 partitions etc.

** (nvidia-settings:2730): WARNING **: Couldn't connect to accessibility bus: Failed to connect to socket /tmp/dbus-tJw9cLM6Wv: Connection refused
** Message: PRIME: Requires offloading
** Message: PRIME: is it supported? yes

ERROR: nvidia-settings could not find the registry key file. This file should
have been installed along with this driver at
/usr/share/nvidia/nvidia-application-profiles-key-documentation. The
application profiles will continue to work, but values cannot be
prepopulated or validated, and will not be listed in the help text.
Please see the README for possible values and descriptions.

Posted on 2015-11-27 08:46:47
Donald Kinghorn

Hi Ester, When you are dealing with the NVIDIA drivers you are always at risk of needing a reinstall to clean up a mess! There are times when you just can't cleanly back out of changes. I hope that is not needed for you now.

The message you are getting from nvidia-settings looks like what you would see if the drivers failed to load i.e. you are back on the nouveau driver or Intel. Look at the reply I made to Karesh above about freezing your kernel updates and also, maybe, look at dkms (if your nvidia modules are managed by it) You may just need to reinstall/recompile the nvidia kernel modules against a kernel update. I wish I had a better advise for you! Best wishes! --Don

Posted on 2015-11-27 18:18:45
EstherJ

I tried something today. I downloaded the cuda 7.5 run file. Logged out of my x server, stopped lightdm, installed cuda 7.5 without the opengl drivers, rebooted.

Now when I ran nvidia-settings I got an error saying: nvidia-settings can't find registry key files. But even so, cuda installed the nvcc program and I can run theano using the gpu so that is enough for me.

Posted on 2015-12-01 10:36:36
Donald Kinghorn

Yes, doing the cuda install will set things up even if you don't use the display drivers. Your X server is having trouble starting up with the nvidia drivers and is falling back to nouveau. But, as you see, it doesn't mean the kernel modules are not there. When you start up a cuda program it will usually try to load the kernel modules if they are not already loaded. You don't necessarily have to have them running for the display.

It is unfortunate that the nvidia Linux display drivers cause so much trouble! I've been doing some laptop setups and had a very difficult time with Fedora 23. Next I will have to deal with high DPI screens ( 4K ) I expect that to be a mess too :-)

Best of luck with your Machine Learning work! ( assuming that's what you are using theano for ) I'm doing some work with this myself. Take care --Don

Posted on 2015-12-01 19:49:06
EstherJ

Thanks! That is indeed what I am using it for. I am able to run my machine learning code but unfortunately when it starts to do the big stuff it says it runs out of memory. Maybe a laptop is not the best machine for trying to do this computational-intensive work.

Posted on 2015-12-02 10:49:15
Harriman

Donald - I am so grateful to you for this post and others. After days of pain with trying to get cuda and the nvidia drivers working with the standard Unity desktop on my laptop - the simple solution was to use Ubuntu Mate. Wish I had found this post earlier. I owe you a beer or your beverage of choice!

Posted on 2016-05-01 17:12:58