Building Docker images that monitor applications

When I add these new features to the NerdDinner Dockerfile and run a container from the image, I'll be able to see the web request and response logs with the docker container logs command, which relays all of the IIS log entries captured by Docker, and I can use environment variables and configuration files to specify API keys and database user credentials. This makes running and administering the legacy ASP.NET application consistent with how I use any other containerized application running on Docker. I can also configure Docker to monitor the container for me, so I can manage any unexpected failures.

Docker provides the ability to monitor an application's health, rather than just checking whether the application process is still running, with the HEALTHCHECK instruction in the Dockerfile. With HEALTHCHECK you tell Docker how to test whether the application is still healthy. The syntax is similar to the RUN and CMD instructions. You pass in a shell command to execute, which should have a return code of 0 if the application is healthy, and 1 if it's not. Docker runs a health check periodically when the container is running and emits status events if the health of a container changes.

The simple definition of healthy for a web application is the ability to respond normally to HTTP requests. Which request you make depends on how thorough you want the check to be. Ideally the request should execute key parts of your application, so you're confident it is all working correctly. But equally, the request should complete quickly and have a minimal compute impact, so processing lots of health checks doesn't affect consumer requests.

A simple health check for any web application is to just use the Invoke-WebRequest PowerShell cmdlet to fetch the home page and check whether the HTTP response code is 200, which means the response was successfully received:

try { 
    $response = iwr http://localhost/ -UseBasicParsing
    if ($response.StatusCode -eq 200) { 
        return 0
    } else {
        return 1
    } 
catch { return 1 }

For a more complex web application, it can be useful to add a new endpoint specifically for health checks. You can add a diagnostic endpoint to APIs and websites that exercises some of the core logic of your app and returns a Boolean result to indicate whether the app is healthy. You can call this endpoint in the Docker health check and check the response content, as well as the status code, in order to give you more confidence that the app is working correctly.

The HEALTHCHECK instruction in the Dockerfile is very simple. You can configure the interval between checks and the number of checks that can fail before the container is considered unhealthy, but to use the default values just specify the test script in HEALTHCHECK CMD. The following example from the Dockerfile for the dockeronwindows/ch03-iis-healthcheck:2e image uses PowerShell to make a GET request to the diagnostics URL and check the response status code:

HEALTHCHECK --interval=5s `
 CMD powershell -command `
    try { `
     $response = iwr http://localhost/diagnostics -UseBasicParsing; `
     if ($response.StatusCode -eq 200) { return 0} `
     else {return 1}; `
    } catch { return 1 }

I've specified an interval for the health check, so Docker will execute this command inside the container every 5 seconds (the default interval is 30 seconds if you don't specify one). The health check is very cheap to run as it's local to the container, so you can have a short interval like this and catch any problems quickly.

The application in this Docker image is an ASP.NET Web API app, which has a diagnostics endpoint, and a controller you can use to toggle the health of the application. The Dockerfile contains a health check, and you can see how Docker uses it when you run a container from that image:

docker container run -d -P --name healthcheck dockeronwindows/ch03-iis-healthcheck:2e

If you run docker container ls after starting that container, you'll see a slightly different output in the status field, similar to Up 3 seconds (health: starting). Docker runs the health check every 5 seconds for this container, so at this point, the check hasn't been run. Wait a little longer and then the status will be something like Up 46 seconds (healthy).

You can check the current health of the API by querying the diagnostics endpoint:

$port = $(docker container port healthcheck).Split(':')[1]
iwr "http://localhost:$port/diagnostics"

In the returned content, you'll see "Status":"GREEN" meaning the API is healthy. This container will stay healthy until I make a call to the controller to toggle the health. I can do that with a POST request that sets the API to return HTTP status 500 for all subsequent requests:

iwr "http://localhost:$port/toggle/unhealthy" -Method Post

Now the application will respond with a 500 response to all the GET requests that the Docker platform makes, which will fail the healthcheck. Docker keeps trying the healthcheck, and if there are three failures in a row then it considers the container to be unhealthy. At this point, the status field in the container list shows Up 3 minutes (unhealthy). Docker doesn't take automatic action on single containers that are unhealthy, so this one is left running and you can still access the API.

Healthchecks are important when you start running containers in a clustered Docker environment (which I cover in Chapter 7, Orchestrating Distributed Solutions with Docker Swarm), and it's good practice to include them in all Dockerfiles. Being able to package an application which the platform can test for health is a very useful feature - it means that wherever you run the app, Docker can keep a check on it.

Now you have all the tools to containerize an ASP.NET application and make it a good Docker citizen, integrating with the platform so it can be monitored and administered in the same way as other containers. A full .NET Framework application running on Windows Server Core can't meet the expectation of running a single process, because of all the necessary background Windows services, but you should still build container images so they run only one logical function and separate any dependencies.