Dr Donald Kinghorn (HPC and Scientific Computing)

Docker and Nvidia-Docker on your workstation: Common Docker Commands Tutorial

Written on April 7, 2017 by Dr Donald Kinghorn

Up to this point I have not said much about actually using Docker commands. In this post I'll go through a first iteration of learning common Docker commands. It will be in the form of a tutorial creating a new Docker image from the NVIDIA CUDA image.

I've written four earlier posts in this series that were intended to establish a base setup and configuration for a "single-user-workstation" including GPU usage. Here is a list of the first four posts in this series for reference,

Docker Command Tutorial

I've mentioned before that Docker is complex. How complex is it?

  • The Docker engine is used in this format; docker [COMMAND]
    There are 51 commands listed if you do docker --help

  • One of those commands is run It has a format
    docker run [OPTIONS] IMAGE [COMMAND] [ARG...]
    If you do docker run --help you will see that there are 94 OPTIONS to the run command!

So, yes, Docker is complex! However, you don't need to know everything. In fact how could you?! You can do a lot with Docker using just a few commands. There is great documentation for when you need to know more. https://docs.docker.com

In what follows I'll show some common docker commands, give usage and examples. We will end up building an image for NVIDIA CUDA that has OpenGL support and the CUDA Samples in it.

Managing Images and Containers

If you have been experimenting with docker you may be accumulating images and containers that you really don't want to save. Here we'll see how to list and remove these. These commands are used often. You can refer to images and containers by their ID hash. The first few characters of the image/container hash are all that are usually needed since you only need enough characters to uniquely specify the ID.

List images:
docker images
docker images
REPOSITORY                   TAG                       IMAGE ID            CREATED             SIZE
continuumio/anaconda3        latest                    2f12b7e5bd80        3 weeks ago         2.23 GB
busybox                      latest                    00f017a8c2a6        4 weeks ago         1.11 MB
alpine                       latest                    4a415e366388        5 weeks ago         3.99 MB
nvidia/cuda                  latest                    1cce8839a2c5        5 weeks ago         1.62 GB
Remove an image:
docker rmi imageID

This would remove the alpine image from above (as long as there were not any containers running from it, see containers below.)

docker rmi 4a4
Untagged: alpine:latest
Untagged: alpine@sha256:58e1a1bb75db1b5a24a462dd5e2915277ea06438c3f105138f97eb53149673c4
Deleted: sha256:4a415e3663882fbc554ee830889c68a33b3585503892cc718a4698e91ef2a526
Deleted: sha256:23b9c7b43573dd164619ad59e9d51eda4095926729f59d5f22803bcbe9ab24c2
Remove all of your images:

If you want to completely clean things up you can pass the images listing as an argument to docker rmi. The -a flag means "all" and the -q flag makes the output a list of imageID's.

docker rmi $(docker images -a -q)
List containers:

Containers are handled in much the same way as images. ps is docker processes i.e. containers and -a means "all". You may have Exited and running Up containers.

docker ps -a
Stop a running Conatiner

This useful to stop containers taht are running in the background. Note: you can be more forceful and use docker kill containerID.

docker stop containterID
Remove a container:
docker rm containterID
Remove all containers:

If you want to clean up and remove all of your running containers you can do the following, (-f will try to force a shutdown of the container if it is running. )

docker rm -f $(docker ps -a -q)

Running Containers

The run command is the most elaborate docker command with 94 options. However, there are just a few options that are commonly used.

docker run [OPTIONS] IMAGE [COMMAND] [ARG...]


  • --rm remove the container after it exits

  • -i -tor-it interactive, and connect a "tty" i.e. a terminal

  • -d --detach run in the background

  • --name give the container a name (it will always have a unique ID hash)

  • -p 8080:80 port map from host to container i.e. port 8080 on host is connected to port 80 on container

  • -v ~/projects:/projects map storage volume from host to container (bind mount) i.e. bind the ~projects directory in your home directory to /projects in the container (you can use this multiple times)

  • -v /tmp/.X11-unix:/tmp/.X11-unix -e DISPLAY=$DISPLAY this bind your X socket to the container and set the DISPLAY environment variable so you can use the host display from your container for graphical output. Please see my earlier post on this so that you do it in a more secure way!

This is the repository, name and tag of the container you want to run i.e. dbkdoc/whalefortune or nvidia/cuda:8.0-devel-ubuntu14.04 Names without repository are used for local named containers and "official" containers on Docker Hub like ubuntu:14.04. Leaving off the "tag" will default to ":latest".

The command is the command to run when starting the container. Such as /bin/bash. There is usually a default command that is defined for the container.

These are arguments to the command above.


Note: The command nvidia-docker is the NVIDIA tool for setting up your host to pass it's kernel display modules through to the container so you have access to the GPU. After it does this nvidia-docker passes the rest of the command line on to the docker command.

The following would start the NVIDIA CUDA :latest container with the director $HOME/docker bound to /projects in the container and the host X socket bound to the container.

nvidia-docker run -it -v $HOME/docker:/projects -v /tmp/.X11-unix:/tmp/.X11-unix -e DISPLAY=$DISPLAY nvidia/cuda

This would start Tensorflow running a web server for a Jupyter notbook on port 8888

nvidia-docker run -it -p 8888:8888 gcr.io/tensorflow/tensorflow:latest-gpu

This example with elaborate [COMMAND] and [ARG...] would start up Anaconda3 Python, install Jupyter with conda, setup some directories for notebooks and start a Jupyter server available on the host from any ip on the host. [ I don't necessarily recommend this. ]

docker run -it -p 8888:8888 continuumio/anaconda3 /bin/bash -c "/opt/conda/bin/conda install jupyter -y --quiet && mkdir /opt/notebooks && /opt/conda/bin/jupyter notebook --notebook-dir=/opt/notebooks --ip='*' --port=8888 --no-browser"

Building an image from a Dockerfile

You can create a custom container using a 'Dockerfile'. If you look for conatiners on Docker Hub you will often find a copy of the Dockerfile that was used to create the container. They may also be available on GitHub or GitLab.

Lets examine the Dockerfile for the NVIDIA CUDA image based on Ubuntu 14.04. We will modify this container to add the 'Samples' source directory to the install and install the package dependencies to add OpenGL development support to the container. This will allow us to compile and run some of the example code and run it on our display.

FROM nvidia/cuda:8.0-runtime-ubuntu14.04
LABEL maintainer "NVIDIA CORPORATION <cudatools@nvidia.com>"
RUN apt-get update && apt-get install -y --no-install-recommends \
         cuda-core-$CUDA_PKG_VERSION \
         cuda-misc-headers-$CUDA_PKG_VERSION \
         cuda-command-line-tools-$CUDA_PKG_VERSION \
         cuda-nvrtc-dev-$CUDA_PKG_VERSION \         
         cuda-nvml-dev-$CUDA_PKG_VERSION \         
         cuda-nvgraph-dev-$CUDA_PKG_VERSION \         
         cuda-cusolver-dev-$CUDA_PKG_VERSION \         
         cuda-cublas-dev-$CUDA_PKG_VERSION \         
         cuda-cufft-dev-$CUDA_PKG_VERSION \         
         cuda-curand-dev-$CUDA_PKG_VERSION \         
         cuda-cusparse-dev-$CUDA_PKG_VERSION \         
         cuda-npp-dev-$CUDA_PKG_VERSION \         
         cuda-cudart-dev-$CUDA_PKG_VERSION \         
         cuda-driver-dev-$CUDA_PKG_VERSION && \
     rm -rf /var/lib/apt/lists/*
ENV LIBRARY_PATH /usr/local/cuda/lib64/stubs:${LIBRARY_PATH}

I'll break this down so you understand what is going on is this file.

  • FROM nvidia/cuda:8.0-runtime-ubuntu14.04
    This is instructing the docker build command to use nvidia/cuda:8.0-runtime-ubuntu14.04 as the base for the container. That container itself uses the ubuntu:14.04 container for it's base. Containers can be built up hierarchically this way.

  • LABEL maintainer "NVIDIA CORPORATION <cudatools@nvidia.com>"
    This is the "maintainer" for the container. I will keep this in the modified file as a comment and add myself as an test-example "maintainer".

  • RUN Is where most of the modifications to the base container get defined. These are mostly install commands with a little clean-up at the end. Note: That is all one line, the \ is a line continuation character.

  • ENV LIBRARY_PATH is defining an environment variable in the container for the CUDA libs.

Modifying the Dockerfile

I am going to add two "features", the CUDA Samples code and deps for OpenGL. Here's the modified file with these changes.

FROM nvidia/cuda:8.0-runtime-ubuntu14.04
# This is a modification of the NVIDIA CUDA Dockerfile
# for cuda 8 devel on Ubuntu 14.04
LABEL maintainer "dbk for personal use"
#LABEL maintainer "NVIDIA CORPORATION <cudatools@nvidia.com>"
RUN apt-get update && apt-get install -y --no-install-recommends \
         cuda-core-$CUDA_PKG_VERSION \
         cuda-misc-headers-$CUDA_PKG_VERSION \
         cuda-command-line-tools-$CUDA_PKG_VERSION \
         cuda-nvrtc-dev-$CUDA_PKG_VERSION \         
         cuda-nvml-dev-$CUDA_PKG_VERSION \         
         cuda-nvgraph-dev-$CUDA_PKG_VERSION \         
         cuda-cusolver-dev-$CUDA_PKG_VERSION \         
         cuda-cublas-dev-$CUDA_PKG_VERSION \         
         cuda-cufft-dev-$CUDA_PKG_VERSION \         
         cuda-curand-dev-$CUDA_PKG_VERSION \         
         cuda-cusparse-dev-$CUDA_PKG_VERSION \         
         cuda-npp-dev-$CUDA_PKG_VERSION \         
         cuda-cudart-dev-$CUDA_PKG_VERSION \         
         cuda-driver-dev-$CUDA_PKG_VERSION \
         cuda-samples-$CUDA_PKG_VERSION && \
     rm -rf /var/lib/apt/lists/*
# dbk -- added cuda-samples-$CUDA_PKG_VERSION
# adding freeglut3-dev and deps for OpenGL support
# Note: this will greatly increase the size of this image!
RUN  apt-get install -y freeglut3-dev   
ENV LIBRARY_PATH /usr/local/cuda/lib64/stubs:${LIBRARY_PATH}

That should do it. I've added some comments and at least tried to give proper attribution to the original listed maintainer.

Building the Dockerfile

In an earlier post I made the modifications above in a a running container and then used the docker commit command to save it as a new image. That is a useful way to make custom images. Here we are going to build a container from "scratch".

Create a directory to do the build in.

mkdir my-cuda8-dev-opengl

Now create a file name Dockerfile in that directory with the contents of our modified Dockerfile. Then do the build.

docker build -t my-cuda8-dev-opengl .

That should pull images layers install all the packages and create the image. You should see "Successfully built someImageID". ( You may get some complaints from debconf during the build you can ignore those.) docker images should now show your new image with the default tag :latest.

You can now start that container, compile and an OpenGL program from the CUDA Samples and run it on your display.

nvidia-docker run -it -v $HOME/docker:/projects -v /tmp/.X11-unix:/tmp/.X11-unix -e DISPLAY=$DISPLAY my-cuda8-dev-opengl

Then in the container you can do,

cd projects/
cp -a /usr/local/cuda/samples .
cd samples/5_Simulations/smokeParticles/

That will compile and run a nice CUDA demo with OpenGL output on your display from a docker container!

Saving an image to Docker Hub

To complete this tutorial it would be good to put our new image on Docker Hub. Accounts are free for public repositories and it can be very convenient to have an account.

I recommend that you go to https://hub.docker.com/ and create an account.

Log in to Docker Hub:


docker login
Add a repo tag to your Image

You will need to modify the tag for the image you created to include your Docker Hub repo name. For this example I am using dbkdoc, my public repo on Docker Hub.

docker tag my-cuda8-dev-opengl:latest dbkdoc/my-cuda8-dev-opengl:latest
Push an Image to Docker Hub

After you have logged in and added your repo tag you can push up your image. Note: it may take several minutes to push your image depending on your upload speed and the image size.

docker push dbkdoc/my-cuda8-dev-opengl:latest

After you push your image you should use a web browser to connect to Docker Hub and add some comments. It is also a very good idea to add a copy the content of your Dockerfile there. The best pratice is to do that and keep your Dockerfiles on GitHub or GitLab and set up automatic builds! (That is having your image rebuilt automatically on Docker Hub when you make git commits.)
Now if you are on another systems or want to share your image it's there ready to grab.

nvidia-docker run -it -v $HOME/docker:/projects -v /tmp/.X11-unix:/tmp/.X11-unix -e DISPLAY=$DISPLAY dbkdoc/my-cuda8-dev-opengl:latest

On a systems that has never used that image or any of the layers that it's build with the above command will pull everything needed. Nice!

This should get you on your way to working productively with Docker on a single-user-workstation. There are MANY more things that can be done so keep exploring.

Happy computing! --dbk

Tags: Docker, NVIDIA-docker, GPU