Puget Systems print logo


Read this article at https://www.pugetsystems.com/guides/903
Dr Donald Kinghorn (Scientific Computing Advisor )

Docker and NVIDIA-docker on your workstation: Installation

Written on February 24, 2017 by Dr Donald Kinghorn

Hopefully the last post on "Docker and NVIDIA-Docker on your Workstation" provided clarity on what is motivating my experiments with Docker.  Being able to run NVIDA GPU accelerated application in containers was a big part of that motivation. In this post I'll go through the basic install and setup for Docker and NVIDIA-Docker.

I'll be using our base DIGITS workstation hardware for the installation since it's sitting on my test-bench. :-) I'll use Ubuntu 16.04 for the workstation OS. I have been doing testing on other systems too; My "gaming" laptop with a GTX 1070 on Ubuntu 16.10, and my older home workstation with a GTX 980 GPU currently running Ubuntu 16.04 (will be experimenting with Ubuntu 17.04 soon).

What Workstation Host OS to use for Docker?

There are Linux distributions that are focused on providing a host environment for Docker like, CoreOSAtomic, RancherOS, etc.. These specialized "micro" OSs are interesting in their own right but they are not at all what we are interested in for a desktop workstation. For our experiments we want a "standard" Linux workstation desktop distribution for the base OS. Docker is something we are adding to supplement usage of a standard workstation setup. A reasonable choice would be Ubuntu 16.04.

Why Ubuntu 16.04?

  • It is common well known Linux distribution with an active user community
  • It is a LTS (long term support) version
  • There are several optional Desktop user interfaces to chose from
  • It is a base distribution for other popular distributions such as Linux Mint
  • It is up-to-date and maintained to provide recent versions of common desktop applications

It should be possible to use any Linux distribution that supports Docker but starting with Ubuntu 16.04 seems "reasonable".

I normally do Ubuntu installs by starting with a simple server install and then add a desktop interface on top of that. It's quick and can be scripted. Following is a simple, lightly commented script that would setup the desktop. [Note: I'm using MATE because I like it] Of course you will need to run this script (or the individual commands) using sudo.

# Do a Desktop GUI and NV driver setup on top of Ubuntu server
# DESKTOP -- Ubuntu desktop to install
# ubuntu-desktop
# kubuntu-desktop
# ubuntu-gnome-desktop
# ubuntu-mate-desktop
# xubuntu-desktop 

# NVDRIVER -- NVIDIA driver version from ppa:graphics-drivers/ppa

# EXTRAS -- Extra packages (your taste may vary)
EXTRAS="build-essential dkms synaptic emacs ssh gdebi"

# Do system Updates
apt-get update
apt-get -y dist-upgrade

# Desktop environment install
tasksel install $DESKTOP

# Extra programs
apt-get install -y $EXTRAS 

# NVIDIA driver
add-apt-repository -y ppa:graphics-drivers/ppa
apt-get update
apt-get install -y $NVDRIVER

# Move manual network config that happened from server install to NetworkManager control
# This sed line comments out the primary nic interface 
sed -i '/The primary network interface/,/^$/ s/^/#/' /etc/network/interfaces
# This line enables NetworkManager for everything
sed -i 's/managed=false/managed=true/' /etc/NetworkManager/NetworkManager.conf

#end of script -- return 0
exit 0

How to install Docker Engine

The next step is to get Docker engine on the system. Docker has installation instructions for several Linux distributions. There are currently instructions and deb packages for Ubuntu 14.04 (Trusty), 16.04 (Xenial) and 16.10 (Yakkety). [ There were incorrect instructions for proper Ubuntu repo configuration for several months but as of this writing it looks OK ].  The install and initial configuration follows as a series of commands to be run using sudo

  1. Add some dependencies for the install
    apt-get install -y apt-transport-https ca-certificates curl software-properties-common
  2. Get and check the Docker GPG key
    curl -fsSL https://apt.dockerproject.org/gpg | sudo apt-key add -
    apt-key fingerprint 58118E89F3A912897C070ADBF76221572C52609D
  3. Add the Docker repo for Ubuntu 16.04
    add-apt-repository "deb https://apt.dockerproject.org/repo/  ubuntu-xenial main"
  4. Install Docker
    apt-get update
    apt-get -y install docker-engine
  5. Add your login user name to the docker group so you can run docker commands without being root
    usermod -aG docker yourLoginUsername
  6. Enable docker to start on boot
    systemctl enable docker

Note: The commands above will give you a "normal" docker install and setup. In the next post in this series I will discuss using kernel user-namespaces for a more secure "single user workstation" setup.

How to Install NVIDIA-Docker 

One of the driving factors for investigating Docker on the desktop is that NVIDIA GPUs can be cleanly accesses from containers now. I wouldn't have considered it there had not been a good way to access the GPU hardware for compute. The reason that this is a problem in the first place is that the graphics driver is in kernel space and Docker is not able at this time to directly special drivers like this. NVIDIA has created a wrapper and daemon to make the GPU available to containers without a driver install in the container. This allows running GPU accelerated applications in containers. 

Fortunately  NVIDA has a deb file that will install and setup NVIDIA-Docker easily on Ubuntu 16.04. Ubuntu 14.04 and CentOS versions should also work well. 

I recommend that you go to the main "nvidia-docker" GitHub page and read what it there. You will find installation instructions however, I do recommend that you have a look at the release page to check for the latest build. As of this writing the build was 1.0.0 (yay! it's at version 1) You can download the deb file from your browser or you could use wget 

wget https://github.com/NVIDIA/nvidia-docker/releases/download/v1.0.0/nvidia-docker_1.0.0-1_amd64.deb

The just install the deb package,

sudo dpkg -i nvidia-docker*.deb

That should install the nvidia-docker service and the nvidia-docker-plugin and have them configured to start automatically on boot.

Note for Ubuntu 16.10: as of this writing, the install fails to configure the systemd service file for nvidia-docker-plugin on Ubuntu 16.10.  This daemon can be started manually from the command line with 

sudo -b nohup nvidia-docker-plugin > /tmp/nvidia-docker.log

We have made several configuration changes on the system with new services and adding our user to another group. Before testing the install it would be good to go ahead and reboot the system to be sure all of the services start up correctly. [ I didn't have any trouble with Ununtu 16.04 ]


For a quick test of your main Docker setup you can try the following.

docker run --rm hello-world

That will give you a "hello-world" with a bit of an explanation of what just happened. We will look at details of using the docker command and examples in later posts.

To test NVIDIA-Docker try the following,

nvidia-docker run --rm nvidia/cuda nvidia-smi

That nvidia-docker command will pull down a NVIDA CUDA image from the NVIDIA repository on  Docker Hub based on Ubuntu 14.04 libs and CUDA 8 and then run then system management interface nvidia-smi. That will show you some general information about your GPU from within the container. There is a lot going on the first time you do that command. If you run it again it will start a new container and run that command almost instantly. 

In the next post on this Docker Desktop setup we will make some configuration changes to get a more secure and sensible configuration using "user-namespaces". We'll also do some more interesting examples including connecting a container to the display!

Happy computing! --dbk


Tags: Docker, NVIDIA-docker, GPU
Gabriel Miretti

Great post! On Ubuntu 16.04.3 with drivers nvidia-384 and nvidia-387 you also need to install nvidia-modprobe package. Thanks for sharing!

Posted on 2017-10-28 17:01:11
Herodion Hartono

Hi, I was trying to use the nvidia-docker to play with nvidia-digits. However, I am a bit confused about how to get training images into digits since using the digits-root folder to import feature images or labels would result in an invalid path. Do you happen to know how to load them?

Posted on 2018-02-05 07:06:44
Donald Kinghorn

... First I should mention that I'm doing a new series of posts about setting up and using docker including using NVIDIA's NGC registry on a workstation ... that is with the new nvidia-docker version 2 ...

Here's basically how I started up DIGITS when I was testing the Titan V with a container using **version 2** and the NGC image registry along with some tuning ...
docker run --runtime=nvidia run --shm-size=1g --ulimit memlock=-1 --rm -it -p 8888:5000 -v /home/kinghorn/projects/data-mnt:/data -v /home/kinghorn/projects/data-mnt/jobs:/workspace/jobs nvcr.io/nvidia/digits:17.12

**version 1** would look something like
nvidia-docker run --rm -it -p 8888:5000 -v /home/kinghorn/projects/data-mnt:/data -v /home/kinghorn/projects/data-mnt/jobs:/workspace/jobs nvidia/digits:latest

The key for you here is to look at the volumes that I'm mounting. On my workstation I have a disk with image data and the DIGITS "jobs" directory mounted in my home directory under projects/data-mnt (I have directories there for data and jobs) I then bind those directories to the places that DIGITS expects to see them in the container i.e. /data and /workspace/jobs with that you should be able to use DIGITS to create you image database and start up jobs ... and when you finish your model and job info will be sitting there in you local system directories after the container is shut down.

You can use variations on that, just be sure to bind using -v the directories you want to use from your local system to the expected places in the container.

Best wishes --Don

Posted on 2018-02-05 17:44:23
Herodion Hartono

Thanks a lot! I was not expecting such a fast reply. That's very helpful. Does that mean that i put something like /home/myname/projects/data-mnt/data/training/images on the feature image folder or /data/training/images ?

Oh, and by the way, how much faster are the TITAN Vs compared to like a 1080 Ti for a typical fcn-alexnet segmentation workload? I saw the video Linus made but it didn't really show how are they better for AI purposes....

Again thanks!.

Posted on 2018-02-05 23:33:27
Donald Kinghorn

... you should be OK to just use /data in the container, that's where the container is expecting to find things. You should then be able to navigate to your image directory in the digits interface. I would probably use -v /home/myname/projects/data-mnt/data:/data you may have to experiment a little but you are on the right track.

I have some results for the Titan V vs Titan Xp is this post that will probably give you a good idea of performance. The 1080Ti probably within 20% of the performance of a Titan Xp

Posted on 2018-02-14 20:30:23

Hello Donald - great tutorials on how to get this stuff working - so many different approaches!!!
I am fairly new to this.

I managed to follow all of your instructions down to:
"nvidia-docker run --rm nvidia/cuda nvidia-smi"

Then I get the error:
"docker: Error response from daemon: oci runtime error: exec: "nvidia-smi": executable file not found in $PATH."

I have tried to edit the execstart with:
sudo mkdir -p /etc/systemd/system/docker.service.d
sudo tee /etc/systemd/system/docker.service.d/override.conf <<eof [service]="" execstart="ExecStart=/usr/bin/nvidia-docker-plugin" -s="" $sock_dir="" -d="" usr="" lib="" nvidia-418="" bin="" eof="" sudo="" systemctl="" daemon-reload="" sudo="" systemctl="" restart="" docker="" usr="" lib="" nvidia-418="" bin="" is="" where="" my="" nvidia-smi="" is="" but="" now="" i="" get="" a="" conflict="" with="" the="" override.conf="" file="" when="" docker="" tries="" to="" start="" up:="" an="" error="" occurred="" trying="" to="" connect:="" get="" http:="" %2fvar%2frun%2fdocker.sock="" v1.24="" images="" json:="" read="" unix="" @-="">/var/run/docker.sock: read: connection reset by peer

What am I doing wrong?

(BTW I started out trying to install in Ubuntu 18.04 with no luck, so I have completely reinstalled 16.04 hoping is would work following your instructions...)

Cheers, Ted:)

Posted on 2019-06-23 02:44:49
Donald Kinghorn

Hi Ted, There have been some changes since this post, mostly on the nvidia-docker side. They're onto version 2 now which is set as a docker "runtime" instead as a plug-in. I did think the plug-in still worked but it looks that may not be the case! (I think I'll ad a note to the top of this post)

The most recent post I've done on doing a docker setup is
I was using Ubuntu 19.04 in that mainly because it's not officially supported and I want to try it. It should work for Ubuntu 18.04 (or 16.04 too!) It will be a good post because it goes over (carefully) the most common issue of getting things working, namely version mismatch.

Try the stuff in the new post. Since you have gone through this older one it should be reasonably clear. If you hit any snags with the new post drop me a comment there! I want to be sure that one is reasonable to follow.

P.S. I'll be working on this stuff myself this week as a possible turn-key offering. So if you hit any problems you'll know I'll be up to my elbow's in it too :-) --Don

Posted on 2019-06-24 15:35:45
Donald Kinghorn

Adding a second reply ... I just noticed that this is the old, old, post :-) What I referred you to with the 19.04/18.04 setup is good but it's pretty concise. I would recommend that you at least read through the 5 parts series I did in 2018. It covers all the details and why's! Here is a link to the last part, it will have links to the other 4 earlier posts. That guide should work well as it is but the new one I did for 19.04 goes over the biggest gotcha that come up so it's good too. It covers the essence of the install without all the details or why's --Don

Posted on 2019-06-24 15:45:22

This is great news Donald! Actually I got a bit further with the version 2 nvidia-docker install on ubuntu 16.04 but kept hitting conflicts between the ExecStart calls (I tried to use to overcome the path issues for nvidia-smi inside and outside the container) and docker itself. Anyway, I would much prefer to use ubuntu 19.04 (a very nice version!) - so, I look forward to following your latest guide - and I will report back if I find any anomalies. After more than a week of no success it would be good to get it working and back onto the good stuff - actually using it to do something. Cheers, Ted. BTW you mentioned in your second post "Here is a link to the last part..." but no link?

Posted on 2019-06-24 22:22:42