Docker and NVIDIA-docker on your workstation: Setup User NamespacesWritten on March 10, 2017 by Dr Donald Kinghorn
In this third post about "Docker and NVIDIA-Docker on your Workstation" I will go through configuration of Linux Kernel User-Namespaces for use with Docker. This is the last important system configuration detail needed to fulfill the setup suggested in the first post describing my motivation for this project.
User-Namespaces will allow the configuration of a much more secure and convenient to use as a "Single-User-Docker-Workstation".
It is assumed that you have already installed the host OS, Docker-Engine, and NVIDIA-Docker as described in the "installation post".
Note: In the short time since the "installation" post both Docker-Engine and NVIDIA-Docker have been updated. Docker changed it's version naming scheme. The latest release (as of this writing) is 17.03.0 and they are now on a monthly release cycle. The instructions in the installation post will automatically pull the new version with the normal Ubuntu updates.
NVIDIA-Docker is now at version 1.0.1. You can find the newest releases on GitHub. You will need to manually update using the latest .deb file.
What are Kernel "Namespaces" and what is the User-Namespace?
In general [Linux Namespaces(http://man7.org/linux/man-pages/man7/namespaces.7.html) provide an abstraction for system resources that can give a running process the appearance of using an isolated instance of the resource. There are namespaces for process IDs, network, interprocess communication, user IDs and others. This namespace mechanism along with system "control groups (cgroups)" are major components of containerization.
The User-Namespace is the latest namespace to be utilized by Docker. It was added with version 1.10. The main idea of a user-namespace is that a processes UID (user ID) and GID (group ID) can be different inside and outside of a containers namespace. The significant consequence of this is that a container can have it's root process mapped to a non-privileged user ID on the host. The configuration mechanism for this mapping is given by the subordinate user and group ID files
Currently Docker allows only a single user and group ID to be remapped for containers. This will require starting the Docker service using the
The restriction to a single user re-mapping is not limiting in our case since we are configuring a "Single User Workstation".
Why use User-Namespaces with Docker?
A few experiments should clarify why we want to configure Docker with user-namespaces.
I'm going to start the tiny Alpine Linux in a container running
/bin/sh. I will bind the host /opt directory to /opt in the container. I put a file owned by root on the host in
Note: The command prompt on my host system for this testing is
I was able to delete that file owned by root! Root inside the container could act like root on the host.
...start an Alpine container and run the ping command;
...look for that ping process on my host system
ping in the container is running as root on the host!
Now I'll stop docker and restart the daemon manually using the
--userns-remap flag. I'm remapping to the user Name kinghorn (it wont be my user ID on the host and we'll see why later).
We can repeat experiment 1, but now we are running Docker with user namespace remapping.
Note that this time docker pulled a new copy of Alpine from Docker Hub. I explain that shortly.
Good! Now the files on the host file-system are owned by "nobody" in the container and I can not delete the file.
...lets look at the ping process as before.
The ping process in the container is now owned by user ID 165536 instead of root.
That's much better but who is 165536?
Lets take a look at what's going on in experiment 2.
The first thing to notice is that when I started the Alpine container it pulled a new copy instead of using the copy I had from earlier. If we look in
/var/lib/docker there is now a new directory.
The directory is named "165536.165536" owned by user 165536. This a new directory for Docker files owned by this mystery user 165536. The rest of the experiment showed that this new user could not act as root on the host system and the container processes are owned by this user instead of root. That's almost what we want. Where did this user 165536 come from? Lets look at the
/etc/subuid file. This is the subordinate user ID file used by User-Namespaces.
There are three entries, one for lxd one for root and one for kinghorn. (The entries for lxd and root are junk that the Ubuntu server install put in there. I have no interested in Canonical's LXD. We'll delete that rubbish later.) Each line in this file has a starting UID and a range of additional UIDs that can be used (65536 of them). The subordinate UID for kinghorn is the user ID that was seen in the experiment above. That's because I started the Docker daemon with
How to configure Docker User-Namespaces
There are two things we need to do to configure User-Namespaces for Docker;
Setup the systemd unit files for docker.service to start with the
/etc/subgidwith the actual user ID we want Docker to use for our container programs, -- our own real user ID.
Modifying Docker's systemd configuration
Most people don't like messing with systemd and that is completely understandable. However, if you want to set things up right that's what you need to do. Docker uses systemd. On Ubuntu you may stumble across
/etc/default/docker don't get your hopes up, that's junk. Docker doesn't use it now.
Systemd default service configurations can be overridden by "add-in" files in directories with names like
/etc/system.d/system/"service-name".service.d. Lets create that for the docker service.
Now create the file
/etc/systemd/system/docker.service.d/userns-remap.conf. This file should contain the following;
Notes: The empty
ExecStart= is needed to clear out the systemd default before the changes are made in the next line.
I used "kinghorn" as the remap user name. That's my login account name and I'm using that because I want to "own" what happens in the containers I start up. If you use "default" Docker will create a user named "dockremap" and add an entry in
/etc/subuid. You can use any user you want that has an account on your host system and that has an entry in
subgid. At this time you can only use one user for this with Docker in the future expect this to become more general.
This is the step that will achieve the desired configuration! I want the process that starts in a container to be owned by my user ID from the viewpoint of my host system. To make this happen I will use my host user ID as the first subordinate ID.
/etc/subgid files to contain the following (same content for both files and use "your" host user ID)
Notes: My (kinghorn) user/group ID is 1000. I'm setting the first subordinate ID to that and then configuring the remaining 65535 (65536 - 1) IDs to start with the arbitrary ID 100001. That first ID will be used by the container startup process for the program or environment I'm trying to run. I can bind a subdirectory in my home directory to the container so that any files there can be used in the container. Also, any files I create in the container will be owned by my user on the host.
Restart to be sure the new configuration starts correctly on boot
Check that everything is working as expected.
Starting Alpine again pulled a new image and there is now a new directory 1000.1000 in
/var/lib/docker owned by "kinghorn"
ping in the container and see who owns it on the host.
ping command that I started in the container is now being run by "kinghorn". That is exactly what I want!
Two Important Notes
You may have noticed that shell prompt in the Alpine container,
/ #. That is indicating that the shell is running as root in the container. This is normal for Docker containers. The main process in a container starts in the containers PID-namespace using PID 1 owned by the container root. The big difference with the configuration that was setup above is that now root in the container is just my user account from the perspective of the host. (You need to fully understand that last statement!) Running a program in a container with this configuration is effectively the same as if I had started it from the host. Exactly what I wanted.
You might be thinking "that is a pretty radical configuration change from the default Docker behavior" and you would be correct! If you do this and you want to start a container without using User-Namspaces you can override the configuration when you start a container by using the command-line flag
--userns=hostThat will start up a container in the "normal" way i.e. owned by root.
Test: Compiling a CUDA GPU program on a system "without CUDA installed" by Using Nvidia-docker and Docker
I have created a directory projects/docker-test and put a copy of the CUDA sample program sources in it. I'll bind this directory to a container that has a CUDA setup in it and compile a program. I can run it in the container and then exit the container and the program will still be there owned and runnable by me.
(This is one of my home systems with a GTX980 ... I'll clean up the compiling terminal output a little to make this example more readable.)
I am now out of the container. The program I just compiled in the container is in my directory and I own it.
Note that the performance in and out of the container is essentially identical. Perfect!
In the next post in this Docker series I'll write about using GUI programs running in Docker containers and give more usage information.
Happy computing --dbk