Last updated: 2019-10-01

Checks: 2 0

Knit directory: for-future-reference/

This reproducible R Markdown analysis was created with workflowr (version 1.4.0.9001). The Checks tab describes the reproducibility checks that were applied when the results were created. The Past versions tab lists the development history.

R Markdown file: up-to-date

Great! Since the R Markdown file has been committed to the Git repository, you know the exact version of the code that produced these results.

Repository version: b3183a1

Great! You are using Git for version control. Tracking code development and connecting the code version to the results is critical for reproducibility. The version displayed above was the version of the Git repository at the time these results were generated.

Note that you need to be careful to ensure that all relevant files for the analysis have been committed to Git prior to generating the results (you can use wflow_publish or wflow_git_commit). workflowr only checks the R Markdown file, but you know if there are other scripts or data files that it depends on. Below is the status of the Git repository when the results were generated: 


Ignored files:
    Ignored:    .Rhistory
    Ignored:    .Rproj.user/

Untracked files:
    Untracked:  .binder/
    Untracked:  docs/figure/summation.Rmd/example-1.pdf
    Untracked:  docs/figure/summation.Rmd/example-1.svg
    Untracked:  docs/figure/variance.Rmd/

Note that any generated files, e.g. HTML, png, CSS, etc., are not included in this status report because it is ok for generated content to have uncommitted changes.

These are the previous versions of the R Markdown and HTML files. If you’ve configured a remote Git repository (see ?wflow_git_remote), click on the hyperlinks in the table below to view them.

File Version Author Date Message
Rmd b3183a1 John Blischak 2019-10-01 Explain how to obtain Docker image digest.
html 0f8b583 John Blischak 2019-03-19 Build site.
html 8ab13ac John Blischak 2019-02-06 Build site.
Rmd 440a5eb John Blischak 2019-02-06 Add sagemath Docker container and how to mount local directory.
html 84cd2a1 John Blischak 2019-01-29 Build site.
Rmd 86f5c3d John Blischak 2019-01-29 Add notes on using Docker.

Docker is virtualization software. The official documentation is at docs.docker.com.

Terminology:

Getting help:

docker --help
docker <subcommand> --help

Using Docker containers

The name of an image is referred to as a repository. If the repository is from a remote source like DockerHub, it will also have an associated username. Lastly, each version of the image has a specific tag (the default is latest). Thus in the commands below, any instance of <image> can be replaced with the corresponding (username/)repository(:tag) that specifies the desired image.

docker pull <image>
docker pull continuumio/miniconda3
docker pull continuumio/miniconda3:4.5.11
docker image ls
# Alternative:
docker images
docker run --rm <image> <command>
docker container ls
# Alternative:
docker ps
docker stop <container>

You can reference the container by its ID or Name (you have to run docker container ls to obtain this information).

docker run --rm -it <image>
# Depending on the Dockerfile, you may have to specify the executable to run
docker run --rm -it <image> bash

Building a Docker image

docker build -t <name-of-tag> .
# Remove exited containers:
docker ps -aq --no-trunc -f status=exited | xargs docker rm
# Remove untagged images:
docker images -q --filter dangling=true | xargs docker rmi
# Start with the xenial tag of the ubuntu image
FROM ubuntu:xenial

# Add contact info
MAINTAINER <name>, <email>

# Install software
RUN apt-get update && \
    apt-get install -y --no-install-recommends \
    emacs \
    git \
    nano

# Execute code
RUN mkdir /root/project

# Set the working directory
WORKDIR /root/project

# Copy file from local machine to Docker image
COPY file.txt /root/project/

# Copy directory from local machine to Docker image
COPY dir/ /root/project/dir/

# Run this command by default when running an interactive container
CMD [ "/bin/bash" ]

Mounting a local directory in a Docker container

A Docker container can be very useful for running software that requires a complex installation. However, in order for this useful, it needs to be able to read and write from the local host machine. To mount a local directory, use the flag -v.

docker run -it -v ~/Desktop/project:/project ubuntu bash

Note that it is critical that both the local and container paths are absolute paths. Docker will throw an error if the container path is not absolute. However, if the local path is relative, instead of mounting that directory, it will create a new volume and save the files there (use docker volume list to view and docker volume rm <name> to delete).

For Docker containers that are run as root, the newly created files have root permissions. This is annoying, but the only workaround I have found is building your own Docker image that adds a layer creating a local user.

For Docker containers that are run as non-root, you should be able to read and write both in the container and from the local machine with no issues.

Resources:

Specifying a Docker image by its digest

A tag is a moving target. When the image owner updates the image build, they can move the tag to this latest version. While a tag like latest makes it obvious that this is transient, any tag can be updated. To ensure that you are always using the exact same version of an image (e.g. for a CI build), you can specify the digest of the image. Note that this is not the same as the ID number returned by docker image list. Instead you have to use docker image inspect 1. The result is a JSON blob, so you can extract the digest using a format string: {{.RepoDigests}}. This will return the digest for the current version of IMAGE:TAG that you have on your local machine. To get the latest digest, make sure to pull prior to inspection.

docker image inspect -f "{{.RepoDigests}}" IMAGE:TAG

Note that you can also see the digest whenever you pull an image, and you can also obtain it via the DockerHub UI. Once you have the digest, you can refer to it by prefixing with @:

docker run -it --rm IMAGE@DIGEST

Useful images

Launch Jupyter from a Docker container (see my comment on GitHub for details):

docker run -i -t -p 8888:8888 continuumio/miniconda3 /bin/bash -c "/opt/conda/bin/conda install -c conda-forge jupyter -y --quiet && mkdir /opt/notebooks && /opt/conda/bin/jupyter notebook --notebook-dir=/opt/notebooks --ip='0.0.0.0' --port=8888 --no-browser --allow-root"
docker run -p 8888:8888 sagemath/sagemath:latest sage-jupyter

The URL that sage-jupyter returns is rough, e.g. http://0efce28e9a34:8888/?token=791b95e58b652231dc507bb4fd9f2af7ea65b62d2eee4d7b&token=791b95e58b652231dc507bb4fd9f2af7ea65b62d2eee4d7b. First, you have to update the address to be http://localhost:8888. Second, it repeats the same token twice. This still works though.

Note that sage-jupyter is a variable that is controlled by entrypoint.sh. It executes sage -n jupyter --no-browser --ip='0.0.0.0' --port=8888.

Tutorials

  1. Yes, you can use the shortcut docker inspect↩