Puget Systems print logo

https://www.pugetsystems.com

Read this article at https://www.pugetsystems.com/guides/903
Dr Donald Kinghorn (Scientific Computing Advisor )

Docker and NVIDIA-docker on your workstation: Installation

Written on February 24, 2017 by Dr Donald Kinghorn
Share:

Hopefully the last post on "Docker and NVIDIA-Docker on your Workstation" provided clarity on what is motivating my experiments with Docker.  Being able to run NVIDA GPU accelerated application in containers was a big part of that motivation. In this post I'll go through the basic install and setup for Docker and NVIDIA-Docker.

I'll be using our base DIGITS workstation hardware for the installation since it's sitting on my test-bench. :-) I'll use Ubuntu 16.04 for the workstation OS. I have been doing testing on other systems too; My "gaming" laptop with a GTX 1070 on Ubuntu 16.10, and my older home workstation with a GTX 980 GPU currently running Ubuntu 16.04 (will be experimenting with Ubuntu 17.04 soon).


What Workstation Host OS to use for Docker?

There are Linux distributions that are focused on providing a host environment for Docker like, CoreOSAtomic, RancherOS, etc.. These specialized "micro" OSs are interesting in their own right but they are not at all what we are interested in for a desktop workstation. For our experiments we want a "standard" Linux workstation desktop distribution for the base OS. Docker is something we are adding to supplement usage of a standard workstation setup. A reasonable choice would be Ubuntu 16.04.

Why Ubuntu 16.04?

  • It is common well known Linux distribution with an active user community
  • It is a LTS (long term support) version
  • There are several optional Desktop user interfaces to chose from
  • It is a base distribution for other popular distributions such as Linux Mint
  • It is up-to-date and maintained to provide recent versions of common desktop applications

It should be possible to use any Linux distribution that supports Docker but starting with Ubuntu 16.04 seems "reasonable".

I normally do Ubuntu installs by starting with a simple server install and then add a desktop interface on top of that. It's quick and can be scripted. Following is a simple, lightly commented script that would setup the desktop. [Note: I'm using MATE because I like it] Of course you will need to run this script (or the individual commands) using sudo.

#!/bin/bash
#
# Do a Desktop GUI and NV driver setup on top of Ubuntu server
#
##############
# VARIABLES: #
##############
# DESKTOP -- Ubuntu desktop to install
# ubuntu-desktop
# kubuntu-desktop
# ubuntu-gnome-desktop
# ubuntu-mate-desktop
# xubuntu-desktop 
DESKTOP='ubuntu-mate-desktop'

# NVDRIVER -- NVIDIA driver version from ppa:graphics-drivers/ppa
NVDRIVER='nvidia-375'

# EXTRAS -- Extra packages (your taste may vary)
EXTRAS="build-essential dkms synaptic emacs ssh gdebi"

# Do system Updates
apt-get update
apt-get -y dist-upgrade

# Desktop environment install
tasksel install $DESKTOP

# Extra programs
apt-get install -y $EXTRAS 

# NVIDIA driver
add-apt-repository -y ppa:graphics-drivers/ppa
apt-get update
apt-get install -y $NVDRIVER

# Move manual network config that happened from server install to NetworkManager control
# This sed line comments out the primary nic interface 
sed -i '/The primary network interface/,/^$/ s/^/#/' /etc/network/interfaces
# This line enables NetworkManager for everything
sed -i 's/managed=false/managed=true/' /etc/NetworkManager/NetworkManager.conf

#end of script -- return 0
exit 0


How to install Docker Engine

The next step is to get Docker engine on the system. Docker has installation instructions for several Linux distributions. There are currently instructions and deb packages for Ubuntu 14.04 (Trusty), 16.04 (Xenial) and 16.10 (Yakkety). [ There were incorrect instructions for proper Ubuntu repo configuration for several months but as of this writing it looks OK ].  The install and initial configuration follows as a series of commands to be run using sudo

  1. Add some dependencies for the install
    apt-get install -y apt-transport-https ca-certificates curl software-properties-common
    
  2. Get and check the Docker GPG key
    curl -fsSL https://apt.dockerproject.org/gpg | sudo apt-key add -
    
    apt-key fingerprint 58118E89F3A912897C070ADBF76221572C52609D
    
  3. Add the Docker repo for Ubuntu 16.04
    add-apt-repository "deb https://apt.dockerproject.org/repo/  ubuntu-xenial main"
    
  4. Install Docker
    apt-get update
    
    apt-get -y install docker-engine
    
  5. Add your login user name to the docker group so you can run docker commands without being root
    usermod -aG docker yourLoginUsername
    
  6. Enable docker to start on boot
    systemctl enable docker
    

Note: The commands above will give you a "normal" docker install and setup. In the next post in this series I will discuss using kernel user-namespaces for a more secure "single user workstation" setup.


How to Install NVIDIA-Docker 

One of the driving factors for investigating Docker on the desktop is that NVIDIA GPUs can be cleanly accesses from containers now. I wouldn't have considered it there had not been a good way to access the GPU hardware for compute. The reason that this is a problem in the first place is that the graphics driver is in kernel space and Docker is not able at this time to directly special drivers like this. NVIDIA has created a wrapper and daemon to make the GPU available to containers without a driver install in the container. This allows running GPU accelerated applications in containers. 

Fortunately  NVIDA has a deb file that will install and setup NVIDIA-Docker easily on Ubuntu 16.04. Ubuntu 14.04 and CentOS versions should also work well. 

I recommend that you go to the main "nvidia-docker" GitHub page and read what it there. You will find installation instructions however, I do recommend that you have a look at the release page to check for the latest build. As of this writing the build was 1.0.0 (yay! it's at version 1) You can download the deb file from your browser or you could use wget 

wget https://github.com/NVIDIA/nvidia-docker/releases/download/v1.0.0/nvidia-docker_1.0.0-1_amd64.deb

The just install the deb package,

sudo dpkg -i nvidia-docker*.deb

That should install the nvidia-docker service and the nvidia-docker-plugin and have them configured to start automatically on boot.

Note for Ubuntu 16.10: as of this writing, the install fails to configure the systemd service file for nvidia-docker-plugin on Ubuntu 16.10.  This daemon can be started manually from the command line with 

sudo -b nohup nvidia-docker-plugin > /tmp/nvidia-docker.log

We have made several configuration changes on the system with new services and adding our user to another group. Before testing the install it would be good to go ahead and reboot the system to be sure all of the services start up correctly. [ I didn't have any trouble with Ununtu 16.04 ]

...reboot


For a quick test of your main Docker setup you can try the following.

docker run --rm hello-world

That will give you a "hello-world" with a bit of an explanation of what just happened. We will look at details of using the docker command and examples in later posts.

To test NVIDIA-Docker try the following,

nvidia-docker run --rm nvidia/cuda nvidia-smi

That nvidia-docker command will pull down a NVIDA CUDA image from the NVIDIA repository on  Docker Hub based on Ubuntu 14.04 libs and CUDA 8 and then run then system management interface nvidia-smi. That will show you some general information about your GPU from within the container. There is a lot going on the first time you do that command. If you run it again it will start a new container and run that command almost instantly. 

In the next post on this Docker Desktop setup we will make some configuration changes to get a more secure and sensible configuration using "user-namespaces". We'll also do some more interesting examples including connecting a container to the display!

Happy computing! --dbk

 

Tags: Docker, NVIDIA-docker, GPU
Gabriel Miretti

Great post! On Ubuntu 16.04.3 with drivers nvidia-384 and nvidia-387 you also need to install nvidia-modprobe package. Thanks for sharing!

Posted on 2017-10-28 17:01:11
Herodion Hartono

Hi, I was trying to use the nvidia-docker to play with nvidia-digits. However, I am a bit confused about how to get training images into digits since using the digits-root folder to import feature images or labels would result in an invalid path. Do you happen to know how to load them?

Posted on 2018-02-05 07:06:44
Donald Kinghorn

... First I should mention that I'm doing a new series of posts about setting up and using docker including using NVIDIA's NGC registry on a workstation ... that is with the new nvidia-docker version 2 ...

Here's basically how I started up DIGITS when I was testing the Titan V with a container using **version 2** and the NGC image registry along with some tuning ...
docker run --runtime=nvidia run --shm-size=1g --ulimit memlock=-1 --rm -it -p 8888:5000 -v /home/kinghorn/projects/data-mnt:/data -v /home/kinghorn/projects/data-mnt/jobs:/workspace/jobs nvcr.io/nvidia/digits:17.12

**version 1** would look something like
nvidia-docker run --rm -it -p 8888:5000 -v /home/kinghorn/projects/data-mnt:/data -v /home/kinghorn/projects/data-mnt/jobs:/workspace/jobs nvidia/digits:latest

The key for you here is to look at the volumes that I'm mounting. On my workstation I have a disk with image data and the DIGITS "jobs" directory mounted in my home directory under projects/data-mnt (I have directories there for data and jobs) I then bind those directories to the places that DIGITS expects to see them in the container i.e. /data and /workspace/jobs with that you should be able to use DIGITS to create you image database and start up jobs ... and when you finish your model and job info will be sitting there in you local system directories after the container is shut down.

You can use variations on that, just be sure to bind using -v the directories you want to use from your local system to the expected places in the container.

Best wishes --Don

Posted on 2018-02-05 17:44:23
Herodion Hartono

Thanks a lot! I was not expecting such a fast reply. That's very helpful. Does that mean that i put something like /home/myname/projects/data-mnt/data/training/images on the feature image folder or /data/training/images ?

Oh, and by the way, how much faster are the TITAN Vs compared to like a 1080 Ti for a typical fcn-alexnet segmentation workload? I saw the video Linus made but it didn't really show how are they better for AI purposes....

Again thanks!.

Posted on 2018-02-05 23:33:27
Donald Kinghorn

... you should be OK to just use /data in the container, that's where the container is expecting to find things. You should then be able to navigate to your image directory in the digits interface. I would probably use -v /home/myname/projects/data-mnt/data:/data you may have to experiment a little but you are on the right track.

I have some results for the Titan V vs Titan Xp is this post that will probably give you a good idea of performance. The 1080Ti probably within 20% of the performance of a Titan Xp
https://www.pugetsystems.co...

Posted on 2018-02-14 20:30:23
twyeld

Hello Donald - great tutorials on how to get this stuff working - so many different approaches!!!
I am fairly new to this.

I managed to follow all of your instructions down to:
"nvidia-docker run --rm nvidia/cuda nvidia-smi"

Then I get the error:
"docker: Error response from daemon: oci runtime error: exec: "nvidia-smi": executable file not found in $PATH."

I have tried to edit the execstart with:
sudo mkdir -p /etc/systemd/system/docker.service.d
sudo tee /etc/systemd/system/docker.service.d/override.conf <<eof [service]="" execstart="ExecStart=/usr/bin/nvidia-docker-plugin" -s="" $sock_dir="" -d="" usr="" lib="" nvidia-418="" bin="" eof="" sudo="" systemctl="" daemon-reload="" sudo="" systemctl="" restart="" docker="" usr="" lib="" nvidia-418="" bin="" is="" where="" my="" nvidia-smi="" is="" but="" now="" i="" get="" a="" conflict="" with="" the="" override.conf="" file="" when="" docker="" tries="" to="" start="" up:="" an="" error="" occurred="" trying="" to="" connect:="" get="" http:="" %2fvar%2frun%2fdocker.sock="" v1.24="" images="" json:="" read="" unix="" @-="">/var/run/docker.sock: read: connection reset by peer

What am I doing wrong?

(BTW I started out trying to install in Ubuntu 18.04 with no luck, so I have completely reinstalled 16.04 hoping is would work following your instructions...)

Cheers, Ted:)

Posted on 2019-06-23 02:44:49
Donald Kinghorn

Hi Ted, There have been some changes since this post, mostly on the nvidia-docker side. They're onto version 2 now which is set as a docker "runtime" instead as a plug-in. I did think the plug-in still worked but it looks that may not be the case! (I think I'll ad a note to the top of this post)

The most recent post I've done on doing a docker setup is
https://www.pugetsystems.co...
I was using Ubuntu 19.04 in that mainly because it's not officially supported and I want to try it. It should work for Ubuntu 18.04 (or 16.04 too!) It will be a good post because it goes over (carefully) the most common issue of getting things working, namely version mismatch.

Try the stuff in the new post. Since you have gone through this older one it should be reasonably clear. If you hit any snags with the new post drop me a comment there! I want to be sure that one is reasonable to follow.

P.S. I'll be working on this stuff myself this week as a possible turn-key offering. So if you hit any problems you'll know I'll be up to my elbow's in it too :-) --Don

Posted on 2019-06-24 15:35:45
Donald Kinghorn

Adding a second reply ... I just noticed that this is the old, old, post :-) What I referred you to with the 19.04/18.04 setup is good but it's pretty concise. I would recommend that you at least read through the 5 parts series I did in 2018. It covers all the details and why's! Here is a link to the last part, it will have links to the other 4 earlier posts. That guide should work well as it is but the new one I did for 19.04 goes over the biggest gotcha that come up so it's good too. It covers the essence of the install without all the details or why's --Don

Posted on 2019-06-24 15:45:22
twyeld

This is great news Donald! Actually I got a bit further with the version 2 nvidia-docker install on ubuntu 16.04 but kept hitting conflicts between the ExecStart calls (I tried to use to overcome the path issues for nvidia-smi inside and outside the container) and docker itself. Anyway, I would much prefer to use ubuntu 19.04 (a very nice version!) - so, I look forward to following your latest guide - and I will report back if I find any anomalies. After more than a week of no success it would be good to get it working and back onto the good stuff - actually using it to do something. Cheers, Ted. BTW you mentioned in your second post "Here is a link to the last part..." but no link?

Posted on 2019-06-24 22:22:42
Francis

Hi Donald,

I am using Dell 7559 PC which is running i386 Ubuntu-mate 18.04 and can be installed nvidia graphical driver GTX 960M, but it can't be installed nvidia-docker, because nvidia-docker_1.0.0-1_amd64.deb need to be install on 64 bits Ubuntu OS. Could you advise whether or not there is
nvidia-docker for i386 Ubuntu-mate 18.04 please?

Thanks,

Francis

Posted on 2020-05-30 00:35:06
Donald Kinghorn

If there is no 32-bit libraries being built, then I'm afraid you are out of luck. In general Linux support for 32-bit is dropping almost everywhere ...except for ARM etc.. 32-bit x86 Code is even being removed from the Linux kernel.

That Dell is a pretty good laptop! ... and probably has at least 16GB mem in it ... I highly recommend that you reinstall with a 64-bit Ubuntu. You will have a lot better experience!

Posted on 2020-06-01 15:13:05
Francis

Hi Donald,
Thank you for your suggestion; however, I had install 64 bits Ubuntu 18.04 in my Dell 7559 before, unfortunately the installation of a 64-bit Ubuntu 18.04 system were usually hacked every time. I could not install it successfully every time beofre, but only Ubuntu-mate 18.04, even Ubuntu 20 that can be installed on my Dell 7559 successfully before. I had checked the hardware configuration of CPU in my Dell 7559, which is adopting Intel i7-6700HQ CPU and the memory RAM is 8G; NVIDIA display card is Geforce GTX is 960M. So what technical advise you can propose to me again please?
Best wish,
Francis

Posted on 2020-06-02 15:30:36
Donald Kinghorn

I had a very similar laptop and used it for a long time with Ubuntu 16.04 and 18.04 along with the MATE desktop.
If by being "hacked" you mean you suffered a security breach then that is indeed unfortunate but it should have little to do with whether you are using 32-bit or 64-bit version of the OS. If you want to remain on a 32-bit version of the OS then you will not be able to use any applications that do not provide 32-bit versions of their executables or libraries.

My recommendation is that you install Ubuntu-MATE 20.04 and study how to secure you system, be sure you keep the firewall enabled etx.. You could also try a different Linux distribution ...

Posted on 2020-06-02 16:26:28
Francis

Hi Donald,
Sorry for my typo mistake. I made mistake to type "hang" to become "hack". When I install Ubuntu 18.04 on my Dell 7559 computer, it was usually unsuccessful because of being hanged (or call it "crashed") during installation of Ubuntu 18.04, but it can be installed of Ubuntu-mate 18.04. If I install Ubuntu 16.04 or Ubuntu 14.04, it could be successful on my Dell 7559. However, I couldn't install Ubuntu 16.04 only, because I found that it was not unsuccessful to install nvidia driver of Geforce GTX 960M supported by Ubuntu 16.04 OS. I had tried to install nvidia driver of GTX 960M with the method from the display setting of desktop window and the method by typing command on the terminal console before. In contrary, it is workable to run nvidia driver of GTX 960M on Ubuntu-mate 18.04 due to the updated driver supported by Ubuntu-mate 18.04; however, I still can not install nvidia-docker by typing command on the terminal right now. So that I would like to enquire your advise.
Many thanks,
Francis

Posted on 2020-06-03 02:34:23
Donald Kinghorn

OH OK, I do understand now! I may have blog post to help you ...
https://www.pugetsystems.co...

The method I describe in that post is how I install Linux that gives the least trouble. I start with a simple server install, then install the NVIDIA driver and then add the graphical interface.

The most common problem encountered during an install on hardware that has an NVIDIA GPU is having the screen hang on a black screen or some other display trouble. There is a kernel command that can be passed during the install boot process "nomodeset" that will often take care of the problem. ... but, try the method I use in the post given above. I almost always succeed with it. Best wishes --Don

Posted on 2020-06-03 15:31:16
Francis

Dear Donald,
Many thanks for your help. I will follow the web post you provided for reference.
One more thing I need to clarify with you about your instruction on installing Ubuntu 18.04 Server. Is it possible to install Ubuntu 18.04 Desktop? It's because that I would like to install nvidia-docker and then run Nvidia Digit for deep learning.
Could you advise please?
Many thanks again.
Francis

Posted on 2020-06-05 15:57:31
Donald Kinghorn

If you do the server install and the nvidia driver install before you install a desktop things should work for you. The tasksel command is easy to use and you can install the full Ubuntu Desktop with it (on top of server) I personally use the MATE desktop and I do all of my Linux installs starting with a server install. Here's a screenshot of what tasksel looks like you run it with sudo, select what you want and hit return ...
https://uploads.disquscdn.c...

After you have the system setup then you should be all set to do the nvidia-docker install

Posted on 2020-06-08 15:19:04
Francis

Hi Donald,
Many thanks for your help again.

So you meant that installation of Ubuntu 18.04 server and then install Ubuntu desktop is better than installation of Ubuntu 18.04 desktop directly. What reason you suggest to install Ubuntu 18.04 server and then install Ubuntu desktop will be better? Is it the answer about the installation of nvidia gpu driver and nvidia-docker that will be succeed easily on Ubuntu 18.04 server? or is it the answer about "no hang" phenomena will be occurred on Dell 7559 if installing Ubuntu 18.04 server?
I had run across the installation problem when installing Ubuntu 18.04 desktop, the system of Ubuntu 18.04 was hanged up with black color screen during the installation of Ubuntu 18.04 desktop on Dell 7559 several times before.

I will follow your instruction to try the installation of Ubuntu 18.04 server and then to select Ubuntu desktop instead of installing Ubuntu 18.04 desktop directly. I hope this procedures can make the installation of Nvidia GPU drive workable and can install nvidia-docker successfully.

Best regards,

Francis

Posted on 2020-06-09 05:26:35
Donald Kinghorn

You are installing from server to avoid the "hang" during install.

I need to do an updated post about doing a lot of these setups. Here is a set of steps that should get you going.

1) Install Ubuntu server Do not use the "live" installer (it fails on many hardware configurations!) use the legacy installer. for 18.04 the ISO is here,
http://cdimage.ubuntu.com/r...

[For 20.04 the legacy server the ISO is here,
http://cdimage.ubuntu.com/u... ]

2) Reboot

3) Install the NVIDIA display driver in Ubuntu server (using the graphics drivers ppa)
sudo apt-get install build-essential dkms
sudo add-apt-repository ppa:graphics-drivers/ppa
sudo apt update
sudo apt-get install nvidia-driver-440

4) Reboot

5) sudo tasksel (pick the desktop environment you want and hit return

6) Reboot You should now have have a full desktop along with the NVIDIA driver

7) By installing from server, Ubuntu will have used "netplan" for the network setup. You may want to change that to NetworkManager (good for a laptop) then do step 7) in this post https://www.pugetsystems.co...

Now you should have a good Ubuntu desktop install with NVIDIA drivers ready for anything else you want to do.

Posted on 2020-06-09 15:07:15
Francis

Hi Donald,
Many thanks for your further explanation in details.
I will follow your instruction to re-install Ubbuntu 18.04 Server and Ubuntu-mate desktop again.
Best wish,
Francis

Posted on 2020-06-10 16:52:47
Francis

Hi Donald,

I have another issue abut how to use
nvidia to provide the improved source codes for Windows Linux Subsystem
(WSL) to run nvidia-docker and nvidia display card driver. Please see
his website on Github:
https://github.com/NVIDIA/l...

Could you share your opinions on how to run such source codes of nvidia-docker and nvidia display card driver on WLS please?

Many thanks,

Francis

Posted on 2020-06-17 21:19:47
Donald Kinghorn

This actually just became available yesterday! ... and YES, I have tried it :-) and it works!
I'll be writing up a blog post soon.

In order to do this you have to be in the Microsoft insider program Developer channel (fast ring) on build 20150 which was just released on June 17th. You also have to be registered as an NVIDIA CUDA developer in order to get the needed Windows display driver. (It might require a recent video card too?? it does work with both GeForce and Quadro GPU's) ...It's easy and safe to register with NVIDIA...

It is still very early to be considering this but I'm hopeful that it will be available in the normal Windows 10 Pro Fall 2020 release.

Keep an eye on my blog posts :-)

Posted on 2020-06-18 14:54:52
Francis

Dear Donald,
Many thanks for your introduction. I will try to run WLS for building up Ubuntu 18.04 and to try connecting to nvidia driver in WLS. Certainly, I will wait for your post about this part later.

Best wish,
Francis

Posted on 2020-06-19 05:02:41