Building container images

As this book attempts to troubleshoot Docker, wouldn't it prove beneficial to reduce our chances for errors that we would have to troubleshoot in the first place? Fortunately for us, the Docker community (and the open source community at large) provides a healthy registry of base (or root) images that dramatically reduce errors and provide more repeatable processes. Searching the Docker Registry, we can find official and automated build statuses for a broad and growing array of container images. The Docker official repositories (https://docs.docker.com/docker-hub/official_repos/) are carefully organized collections of images supported by Docker Inc.-automated repositories that allow you to validate source and content of a particular image also exist.

A major thrust and theme of this chapter will be in basic Docker fundamentals; while they may seem trivial to the experienced container user, following some best practices and levels of standardization will serve us well in avoiding trouble spots in addition to enhancing our abilities to troubleshoot.

Official images from the Docker Registry

Standardization is a major component for repeatable processes. As such, wherever and whenever possible, one should opt for a standard base image as provided in the Docker Hub for the variant Linux distributions (for example, CentOS, Debian, Fedora, RHEL, Ubuntu, and others) or for specific use cases (for example, WordPress applications). Such base images are derived from their respective Linux platform images, and are built specifically for use in containers. Further, standardized base images are well maintained and updated frequently to address security advisories and critical bug fixes.

These base images are built, validated, and supported by Docker Inc. and are easily recognized by their single word names (for example, centos). Additionally, user members of the Docker community also provide and maintain prebuilt images to address certain use cases. Such user images are denoted with the prefix of the Docker Hub username that created them, suffixed with the image name (for example, tutum/centos).

To our great advantage, these standard base images remain ready and are publicly available on the Docker Registry; images can be searched for and retrieved simply using the docker search and docker pull Terminal commands. These will download any image(s) that are not already located on the Docker host. The Docker Registry has become increasingly powerful in providing official base images for which one can use directly, or at least as a readily available starting point toward addressing the needs of your container building.

Note

While this book assumes your familiarity with Docker Hub/Registry and GitHub/Bitbucket, we will dedicate initial coverage of these as your first line of reference for efficient image building for containers. You can visit the official registry of Docker images at https://registry.hub.docker.com/.

The Docker Registry can be searched from your Docker Hub account or directly from the Terminal, as follows:

$ sudo docker search centos

Flags can be applied to your search criteria to filter images for star ratings, automated builds, and many more. To use the official centos image from the registry, from a Terminal:

• $ sudo docker pull centos: This will download the centos image to your host machine.
• $ sudo docker run centos: This will first look for this image localized on your host and, if not found, it will download the image to host. The run parameters for the image will have been defined in its Dockerfile.

User repositories

Further, as we have seen, we are not limited merely to the repositories of official Docker images. Indeed, a wealth of community users (both as inpiduals and from corporate enterprises) have prepared images constructed to meet certain needs. As an example, an ubuntu image is created to run the joomla content management system within a container running on Apache, MySql, and PHP.

Here, we have a user repository with just such an image (namespace/repository name):

Note

Try it out: Practice an image pull and runfrom the Docker Registry from the Terminal.

$ sudo docker pull cloudconsulted/joomla

pulls our base image for a container and $ sudo docker run -d -p 80:80 cloudconsulted/joomla runs our container image and maps port 80 of the host to port 80 of the container.

Point your browser to http://localhost and you will have the build page for a new Joomla website!

Building our own base images

There may be occasion, however, when we need to create custom images to suit our own development and deployment environment. If your use case dictates using a nonstandardized base image, you will need to roll your own image. As with any approach, appropriate planning beforehand is necessary. Before building an image, you should spend adequate time to fully understand the use case your container is meant to address. There isn't much need for a container that cannot run the intended application. Other considerations may include whether the library or binary you are including in the image is reusable, and many more. Once you feel you are done, review your needs and requirements once more and filter out parts that are unnecessary; we do not want to bloat our containers for no good reason.

Using the Docker Registry, you may find automated builds. These builds are pulled from repositories at GitHub/Bitbucket and can, therefore, be forked and modified to your own specifications. Your newly forked repository can then in turn be synced to the Docker Registry with your new image, which can then be pulled and run as needed for your containers.

Note

Try it out: Pull the ubuntu minimal image from the following repository and drop it to your Dockerfile directory to create your own image:

$ sudo docker pull cloudconsulted/ubuntu-dockerbase $ mkdir dockerbuilder $ cd dockerbuilder

Open an editor (vi/vim or nano) and create a new Dockerfile:

$ sudo nano Dockerfile

We will delve into creating good Dockerfiles later as we talk about layered and automated image building. For now, we just want to create our own new base image, only symbolically going through the procedure and location for creating a Dockerfile. For the sake of simplicity, here we are just calling the base image from which we want to build our new image:

FROM cloudconsulted/ubuntu-dockerbase:latest 

Save and close this Dockerfile. We now build our new image locally:

$ sudo docker build -t mynew-ubuntu

Let's check to ensure our new image is listed:

$ sudo docker images

Note our IMAGE ID for mynew-ubuntu, as we will need it shortly:

Create a new public/private repository under your Docker Hub username. I'm adding the new repository here under <namespace><reponame> as cloudconsulted/mynew-ubuntu:

Next, return to the Terminal so that we can tag our new image to push to the new Docker Hub repository under our <namespace>:

$ sudo docker tag 1d4bf9f2c9c0 cloudconsulted/mynew-ubuntu:latest

Ensure that our new image is correctly tagged for <namespace><repository> in our images list:

$ sudo docker images

Also, we will find our newly created image labeled for pushing it to our Docker Hub repository.

Now, let's push the image up to our Docker Hub repository:

$ sudo docker push cloudconsulted/mynew-ubuntu

Then, check the Hub for our new image:

There are essentially two approaches to building your own Docker images:

• Manually constructing layers interactively via bash shell to install necessary applications
• Automating through a Dockerfile that builds the images with all necessary applications

Building images using the scratch repository

Going about building your own container images for Docker is highly dependent on which Linux distribution you intend to package. With such variance, and with the prevalence and growing registry of images already available to us via the Docker Registry, we won't spend much time on such a manual approach.

Here again, we can look in the Docker Registry to provide us with a minimal image to use. A scratch repository has been created from an empty TAR file that can be utilized simply via docker pull. As before, make your Dockerfile according to your parameters, and you have your new image, from scratch.

This process can be even further simplified by making use of available tools, such as supermin (Fedora systems) or debootstrap (Debian systems). Using such tools, the build process for an Ubuntu base image, for example, can be as simple as follows:

$ sudo debootstrap raring raring > /dev/null 
$ sudo tar -c raring -c . |  docker import - raring a29c15f1bf7a 
$ sudo docker run raring cat /etc/lsb-release 
DISTRIB_ID=Ubuntu 
DISTRIB_RELEASE=14.04 
DISTRIB_CODENAME=raring 
DISTRIB_DESCRIPTION="Ubuntu 14.04" 

Building layered images

A core concept and feature of Docker is layered images. One of the most important features of Docker is image layering and the management of image content. A layered approach for container images is very efficient, as you can reference the contents in the image, identifying the layer in a layered image. This is very powerful when building multiple images, using the Docker Registry to push and pull images.

[Image Copyright © Docker, Inc.]

Building layered images using Dockerfiles

Layered images are primarily built using the Dockerfile. In essence, a Dockerfile is a script that automatically builds our containers from a source (base or root) image in the order you need them executed by the Docker daemon, step by step, layer upon layer. These are successive commands (instructions) and arguments enlisted within the file that execute a proscribed set of actions on a base image, with each command constituting a new layer, in order to build a new one. This not only facilitates the organization of our image building but greatly enhances deployments from beginning to end through its simplification. The scripts within a Dockerfile can be presented to the Docker daemon in a range of ways to build new images for our containers.

Dockerfile construction

The first command of a Dockerfile is typically the FROM command. FROM specifies the base image to be pulled. This base image can be located in the public Docker registry (https://www.docker.com/) within a private registry or even a localized Docker image from the host.

Additional layers in a Docker image are populated as per the directives defined in the Dockerfile. Dockerfiles have very handy directives. Every new directive defined in the Dockerfile constitutes a layer in a layered image. With a RUN directive, we can specify a command to be run, with the result of the command as an additional layer in the image.

Tip

It is highly advised to logically group the operations performed in an image and keep the number of layers to a minimum. For example, while trying to install the dependencies for your application, one can install all the dependencies in one RUN directive rather than using N number of directives per dependency.

We will inspect more closely, the aspects of Dockerfiles for automation in a later section, Automated image building. For now, we need to make certain that we grasp the concept and construction of the Dockerfile itself. Let's look specifically at a simple list of commands that can be employed. As we have seen before, our Dockerfile should be created in a working directory containing our existing code (and/or other dependencies, scripts, and others).

Tip

CAUTION: Avoid use of the root [/] directory as root of your source repository. The docker build command makes use of the directory containing your Dockerfile as the build context (including all of its subdirectories). The build context will be sent to the Docker daemon before building the image, which means if you use / as the source repository, the entire contents of your hard drive will get sent to the daemon (and thus to the machine running the daemon). In most cases, it is best to put each Dockerfile in an empty directory. Then, only add the files needed for building the Dockerfile to the directory. To increase the build's performance, a .dockerignore file can be added to the context directory to properly exclude files and directories.

Dockerfile commands and syntax

While simplistic, the order and syntax of our Dockerfile commands are extremely important. Proper attention to details and best practice here will not only help ensure successful automated deployments, but also serve to help in any troubleshooting efforts.

Let's delineate some basic commands and illustrate them directly with a working Dockerfile; our joomla image from before is a good example of a basic layered image build from a Dockerfile.

Note

Our sample joomla base image is located in the public Docker index via

cloudconsulted/joomla.

FROM

A proper Dockerfile begins with defining an image FROM, from which the build process starts. This instruction specifies the base image to be used. It should be the first instruction in Dockerfile, and it is a must for building an image via Dockerfile. You can specify the local image, an image present at the Docker public registry, or image at a private registry.

Common Constructs

FROM <image> 
FROM <image>:<tag> 
FROM <image>@<digest> 

<tag> and <digest> are optional; if you do not specify them, it defaults to latest.

Example Dockerfile from our Joomla Image

Here, we define the base image to be used for the container:

# Image for container base 
FROM ubuntu 

MAINTAINER

This line designates the Author of the built image. This is an optional instruction in Dockerfile; however, one should specify this instruction with the name and/or e-mail address of the author. MAINTAINER details can be placed anywhere you prefer in your Dockerfile, so long as it is always post your FROM command, as they do not constitute any execution but rather a value of a definition (that is, just some additional information).

Common Constructs

MAINTAINER <name><email> 

Example Dockerfile from our Joomla Image

Here, we define the author for this container and image:

# Add name of image author 
MAINTAINER John Wooten <jwooten@cloudconsulted.com> 

ENV

This instruction sets the environment variable in Dockerfile. An environment variable set can be used in subsequent instructions.

Common Constructs

ENV <key> <value> 

The preceding code sets one environment variable <key> with <value>.

ENV <key1>=<value1> <key2>=<value2> 

The preceding instruction sets two environment variables. Use the = sign between key and value of an environment variable and separate two environment key-values with space to define multiple environment variables:

ENV key1="env value with space" 

Use quotes for value having spaces for environment variable.

The following are the points to remember about ENV instructions:

• Use single instruction to define multiple environment variables
• Environment variables are available when you create container from image
• One can review the environment variable from image using docker inspect <image>
• Values of environment variables can be changed at runtime by passing the --env <key>=<value> option to the docker run command

Example Dockerfile from our Joomla Image 

Here, we set the environment variables for Joomla and the Docker image running without an interactive Terminal:

# Set the environment variables 
ENV DEBIAN_FRONTEND noninteractive 
ENV JOOMLA_VERSION 3.4.1 

RUN

This instruction allows you to run commands and yield a layer. The output of the RUN instruction will be a layer built for image under process. Command passed to the RUN instruction runs on the layers built before this instruction; one needs to take care of the orders.

Common Constructs

RUN <command> 

The <command> is executed in a shell -/bin/sh -c shell form.

RUN ["executable", "parameter1", "parameter2"] 
 

In this particular form, you specify the executable and parameters in executable form. Ensure that you pass the absolute path of the executable in the command. This is useful for cases where the base image does not have /bin/sh. You can specify an executable, which could be your only executable in a base image and build the layers on top using it.

This is also useful if you do not want to use the /bin/sh shell. Consider this:

RUN ["/bin/bash", "-c", "echo True!"] 
RUN <command1>;<command2> 

Actually, this is a special form of example, where you specify multiple commands separated by ;. The RUN instruction executes such commands together and builds a single layer for all of the commands specified.

Example Dockerfile from our Joomla Image

Here, we update the package manager and install required dependencies:

# Update package manager and install required dependencies 
RUN apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install -y \ 
    mysql-server \ 
    apache2 \ 
    php5 \ 
    php5-imap \ 
    php5-mcrypt \ 
    php5-gd \ 
    php5-curl \ 
    php5-apcu \ 
    php5-mysqlnd \ 
    supervisor 

Note that we have purposefully written so that new packages are to be added as their own apt-get install lines, following the initial install commands.

This is done so that, should we ever need to add or remove a package, we can do so without requiring to re-install all other packages within our Dockerfile. Obviously, this provides considerable savings in build time, should the need arise.

Note

Docker Cache: Docker will first check against the host's image cache for any matching layers from previous builds. If found, the given build step within the Dockerfile will be skipped to utilize the previous layer, from cache. As such, it is best practice to enlist each of the Dockerfile's apt-get -y install commands on their own.

As we've discussed, the RUN command in a Dockerfile will execute any given command under the context and filesystem of the Docker container, and produce a new image layer with any resulting file system changes. We first run apt-get update to ensure that the repositories and the PPAs of the packages are updated. Then, in separate calls, we instruct the package manager to install MySQL, Apache, PHP, and Supervisor. The -y flag skips interactive confirmation.

With all of our necessary dependencies installed to run our service, we ought to tidy up a bit to give us a cleaner Docker image:

# Clean up any files used by apt-get 
RUN apt-get clean && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/* 

ADD

This information is used to copy files and directories from the local filesystem or files from a remote URL into the image. The source and destination must be specified in ADD instructions.

Common Constructs

ADD  <source_file>  <destination_directory> 

Here the path of <source_file> is relative to the build context. Also, the path of <destination_directory> could either be absolute or relative to the WORKDIR:

ADD  <file1> <file2> <file3> <destination_directory> 

Multiple files, for example, <file1> , <file2>, and <file3>, are copied into <destination_directory>. Note that paths of these source files should be relative to the build context, as follows:

ADD <source_directory> <destination_directory> 

Contents of the <source_directory> are copied into <destination_directory> along with the filesystem metadata; the directory itself is not copied:

ADD text_* /text_files

All the files starting with text_ in the build context directory are copied in the /text_files directory in the container image:

ADD ["filename with space",...,  "<dest>"] 

Filename with a space can be specified in quotes; one needs to use a JSON array to specify the ADD instruction in this case.

The following are the points to remember about ADD instructions:

• All new files and directories that are copied into the container image have UID and GID as 0
• In cases where the source file is a remote URL, the destination file will have a permission of 600
• All the local files referenced in the source of the ADD instruction should be in the build context directory or in its subdirectories
• If the local source file is a supported tar archive then it is unpacked as a directory
• If multiple source files are specified, the destination must be a directory and end with a trailing slash, /
• If a destination does not exist, it will be created along with all the parent directories in the path, if required

Example Dockerfile from our Joomla Image

Here, we download joomla into the Apache web root:

# Download joomla and put it default apache web root 
ADD https://github.com/joomla/joomla-cms/releases/download/$JOOMLA_VERSION/Joomla_$JOOMLA_VERSION-Stable-Full_Package.tar.gz /tmp/joomla/ 
RUN tar -zxvf /tmp/joomla/Joomla_$JOOMLA_VERSION-Stable-Full_Package.tar.gz -C /tmp/joomla/ 
RUN rm -rf /var/www/html/* 
RUN cp -r /tmp/joomla/* /var/www/html/ 
 
# Put default htaccess in place 
RUN mv /var/www/html/htaccess.txt /var/www/html/.htaccess 
 
RUN chown -R www-data:www-data /var/www 
 
# Expose HTTP and MySQL 
EXPOSE 80 3306 

COPY

The COPY command specifies that a file, located at the input path, should be copied from the same directory as the Dockerfile to the output path inside the container.

CMD

The CMD instruction has three forms-a shell form, as default parameters to ENTRYPOINT and the preferred executable form. The main purpose of a CMD is to provide defaults for an executing container. These defaults can either include or omit an executable, the latter of which must specify an ENTRYPOINT instruction as well. If the user specifies arguments to Docker run, then they will override the default specified in CMD. If you would like your container to run the same executable every time, then you should consider using ENTRYPOINT in combination with CMD.

The following are the points to remember:

• Do not to confuse CMD with RUN-RUN will actually execute the command and commit the result, whereas CMD does not execute commands during a build, but instead specifies the intended command for the image
• A Dockerfile can only execute one CMD; if you enlist more than one, only the last CMD will be executed

Example Dockerfile from our Joomla Image

Here, we set up Apache for it to start:

# Use supervisord to start apache / mysql 
COPY supervisord.conf /etc/supervisor/conf.d/supervisord.conf 
CMD ["/usr/bin/supervisord", "-n"] 

The following is the content of our completed Joomla Dockerfile:

FROM ubuntu 
MAINTAINER John Wooten <jwooten@cloudconsulted.com> 
 
ENV DEBIAN_FRONTEND noninteractive 
ENV JOOMLA_VERSION 3.4.1 
 
RUN apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install -y \ 
    mysql-server \ 
    apache2 \ 
    php5 \ 
    php5-imap \ 
    php5-mcrypt \ 
    php5-gd \ 
    php5-curl \ 
    php5-apcu \ 
    php5-mysqlnd \ 
    supervisor 
 
# Clean up any files used by apt-get 
RUN apt-get clean && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/* 
 
# Download joomla and put it default apache web root 
ADD https://github.com/joomla/joomla-cms/releases/download/$JOOMLA_VERSION/Joomla_$JOOMLA_VERSION-Stable-Full_Package.tar.gz /tmp/joomla/ 
RUN tar -zxvf /tmp/joomla/Joomla_$JOOMLA_VERSION-Stable-Full_Package.tar.gz -C /tmp/joomla/ 
RUN rm -rf /var/www/html/* 
RUN cp -r /tmp/joomla/* /var/www/html/ 
 
# Put default htaccess in place 
RUN mv /var/www/html/htaccess.txt /var/www/html/.htaccess 
 
RUN chown -R www-data:www-data /var/www 
 
# Expose HTTP and MySQL 
EXPOSE 80 3306 
 
# Use supervisord to start apache / mysql 
COPY supervisord.conf /etc/supervisor/conf.d/supervisord.conf 
CMD ["/usr/bin/supervisord", "-n"] 

Other common Dockerfile commands are as follows: ENTRYPOINT

An ENTRYPOINT allows you to configure a container that will run as an executable. From Docker's documentation, we will use the provided example; the following will start nginx with its default content, listening on port 80:

docker run -i -t --rm -p 80:80 nginx 

Command-line arguments to docker run <image> will be appended after all elements in an executable form ENTRYPOINT, and will override all elements specified using CMD. This allows arguments to be passed to the entry point, that is, docker run <image> -d will pass the -d argument to the entry point. You can override the ENTRYPOINT instruction using the docker run --entrypoint flag.

LABEL

This instruction specifies the metadata for the image. This image metadata can later be inspected using the docker inspect <image> command. The idea here is to add information about the image in image metadata for easy retrieval. In order to get the metadata from the image, one does not need to create a container from the image (or mount the image on local filesystem), Docker associates metdata data with every Docker image, and it has a predefined structure for it; using LABEL, one can add additional associated metadata describing the image.

The label for the image is a key-value pair. Following are examples of using LABEL in a Dockerfile:

LABEL <key>=<value>  <key>=<value>  <key>=<value> 

This instruction will add three labels to the image. Also, note that it will create one new layer as all the labels are added in a single LABEL instruction:

LABEL  "key"="value with spaces" 

Use quotes in labels if the label value has spaces:

LABEL LongDescription="This label value extends over new \ 
line." 

If the value of the label is long, use backslash to extend the label value to a new line.

LABEL key1=value1 
LABEL key2=value2 

Multiple labels for an image can be defined by separating them by End Of Line (EOL). Note that, in this case, there will be two image layers created for two different LABEL instructions.

Notes about LABEL instructions:

• Labels are collated together as described in Dockerfile and those from the base image specified in the FROM instruction
• If key in labels are repeated, later one will override the earlier defined key's value.
• Try specifying all the labels in a single LABEL instruction to produce an efficient image, thus avoiding unnecessary image layer count
• To view the labels for a built image, use the docker inspect <image> command

WORKDIR

This instruction is used to set the working directory for subsequent RUN, ADD, COPY, CMD, and ENTRYPOINT instructions in Dockerfile.

Define a work directory in Dockerfile, all subsequent relative paths referenced inside the container will be relative to the specified work directory.

The following are examples of using the WORKDIR instruction:

WORKDIR /opt/myapp 

The preceding instruction specifies /opt/myapp as the working directory for subsequent instructions, as follows:

WORKDIR /opt/ 
WORKDIR myapp 
RUN pwd 

The preceding instruction defines the work directory twice. Note that the second WORKDIR will be relative to the first WORKDIR. The result of the pwd command will be /opt/myapp:

ENV SOURCEDIR /opt/src 
WORKDIR $SOURCEDIR/myapp 

Work directory can resolve the environment variables defined earlier. In this example, the WORKDIR instruction can evaluate the SOURCEDIR environment variable and the resultant working directory will be /opt/src/myapp.

USER

This sets the user for running any subsequent RUN, CMD, and ENTRYPOINT instructions. This also sets the user when a container is created and run from the image.

The following instruction sets the user myappuser for the image and container:

USER myappuser 

Notes about USER instructions:

• One can override the user using --user=name|uid[:<group|gid>] in the docker run command for container