Demystyfing Docker

Learning objectives

• Decide whether a container is the right tool for a given job.
• Download and run pre-built Docker images.
• Describe the stages of the Docker container lifecycle.
• Build simple Dockerfiles for your own projects.

Why docker matters for data science

• Docker creates standardized environments that are:

  1. Reproducible

  2. Portable

  3. Collaborative

  4. Scalable

• Docker allows us to set up infrastructure as code

• Docker enhances reproducibility by creating a reproducible environment all the way down to the operating system - important for highly regulated industries

• Docker allows you to develop your project using an image that may more closely match the production environment - important for Shiny apps and APIs

See An Introduction to Docker for R Users for a guide on reproducibility with Docker for R users

What is Docker?

• An open-source tool for building, sharing, and running software

• Requires a Linux operating system or the Windows Subsystem for Linux (WSL)

• Windows and macOS users download Docker Desktop, which comes with a Linux VM. Linux users are recommended to install the Docker Engine directly.

• Three terms to be familiar with are DockerFile, Docker Image, and Container Instances.

• Note that the names container and instance are often used interchangeably

Specify your environment via a Dockerfile

• Dockerfiles build Docker images

• Dockerfiles are plain text files using FROM, RUN, COPY, and CMD commands

1FROM ubuntu:latest
2COPY my-data.csv /data/data.csv
3RUN ["head", "/data/data.csv"]

1

Declare the base image

2

Copy data.csv from the host’s working directory to the container’s data directory

3

Print the first few rows of data.csv

• Creating your own Dockerfile is optional - many standard Docker images exist on Dockerhub (e.g. rocker/tidyverse)

• Dockerfiles build images and also pull existing images

Docker images are a snapshot of your environment

• Docker images contain the bundled software (e.g. OS, data, packages)

• Docker images can be shared with others via Docker hub

• Docker images can, in theory, be a standalone project

• Other container registries exist (e.g. Azure Container Registry)

Containers are an ephemeral instance of a Docker Image

• By default, changes made to containers are lost on shutdown

• Data can be preserved from instance to instance of the same container using mounted volumes

• Containers are a process that executes the layers of your Dockerfile

Meeting Videos

Cohort 1