Yet we lack conventions and technical infrastructure for such openness.

Enter the Research Compendium

The goal of a research compendium is to provide a standard and easily recognizable way for organizing the digital materials of a project to enable others to inspect, reproduce, and extend the research.

R community response

R packages can be used as a research compendium for organising and sharing files!

From raw to analytical data

Separate function definition and application

• When your project is new and shiny, the script file usually contains many lines of directly executated code.

• As it matures, reusable chunks get pulled into their own functions.

• The actual analysis scripts then become relatively short, and use the functions defined in separate R scripts.

Use Rstudio projects

here::here("path", "to", "file")

Follow convention

It’s like agreeing that we will all drive on the left or the right. A hallmark of civilization is following conventions that constrain your behavior a little, in the name of public safety.

Jenny Bryan on Project-oriented workflows

A place for everything, everything in its place.

Benjamin Franklin

Tools

rrtools: Research Compendia in R

The goal of rrtools is to provide instructions, templates, and functions for making a basic compendium suitable for writing reproducible research with R.

rrtools build on tools & conventions for R package development to

• organise files
• manage dependencies
• share code
• document code
• check and test code

rrtools extends and works with a number of R packages:

• devtools: functions for package development

• usethis: automates repetitive tasks that arise during project setup and development

• bookdown: facilitates writing books and long-form articles/reports with R Markdown

Data

Workshop aims and objectives

In this workshop we’ll use materials associated with a published paper (text, data and code) to create a research compendium around it.

By the end of the workshop, you should be able to:

• Be able to Create a Research Compendium to manage and share resources associated with an academic publication.

• Be able to produce a reproducible manuscript from a single rmarkdown document.

• Appreciate the power of convention!

Install packages

Next, let’s install the packages we’ll need, starting with rrtools (if you haven’t got devtools installed, you’ll need to before you can install rrtools from GitHub).

# install.packages("devtools")
devtools::install_github("benmarwick/rrtools")

Installing rrtools imports many of the packages we’ll need today (eg, have a look at the imports section of the DESCRIPTION file).

Imports: devtools, git2r, whisker, rstudioapi, rmarkdown, knitr,
    bookdown, curl, RCurl, jsonlite, methods, httr, usethis, clisymbols,
    crayon, glue, readr (>= 1.1.1)

Now, install some additional packages we’ll need for the workshop.

install.packages(c(
    # source paper analysis
    "ggthemes"
    # bibliographic / publishing
    "citr", "rticles",
    # documentation
    "roxygen2",
    # graphics
    "Cairo"))

Get workshop materials

Today we’ll be working with a subset of materials from the published compendium of code, data, and author’s manuscript:

usethis::use_course(url = "bit.ly/rrtools_wks", destdir = "~/Desktop")

├── README.md <- .......................repo README
├── analysis.R <- ......................analysis underlying paper
├── gillespie.csv <- ...................data
├── paper.pdf <- .......................LaTex pdf of the paper
├── paper.txt <- .......................text body of the paper
├── refs.bib <- ........................bibtex bibliographic file
└── rrtools-wkshp-materials.Rproj <- ...rstudio project file

load library

First we need to load rrtools

library(rrtools) 

This performs a quick check to confirm you have Git installed and configured

If you do, you should see the following output in the console.

✔ Git is installed on this computer, your username is annakrystalli

create compendium

Now we’re ready to create our compendium. We use function rrtools::create_compendium and supply it with a path at which our compendium will be created. The final part of our path becomes the compendium name. Because the function effectively creates a package, only a single string of lowercase alpha characters is accepted as a name. so let’s go for rrcompendium as the final part of our path.

To create rrcompendium in a directory called Documents/workflows/ I use:

rrtools::create_compendium("~/Documents/workflows/rrcompendium")

Go ahead and create a compendium at a location of your choice. Stick with compendium name rrcompendium for ease of following the materials. If the call was successfull you should see the following console output:

✔ Setting active project to '/Users/Anna/Documents/workflows/rrcompendium'
✔ Creating 'R/'
✔ Creating 'man/'
✔ Writing 'DESCRIPTION'
✔ Writing 'NAMESPACE'
✔ Writing 'rrcompendium.Rproj'
✔ Adding '.Rproj.user' to '.gitignore'
✔ Adding '^rrcompendium\\.Rproj$', '^\\.Rproj\\.user$' to '.Rbuildignore'
✔ Opening new project 'rrcompendium' in RStudio
✔ The package rrcompendium has been created
✔ Opening the new compendium in a new RStudio session...

Next, you need to:  ↓ ↓ ↓ 
● Edit the DESCRIPTION file
● Use other 'rrtools' functions to add components to the compendium

and a new Rstudio session launched for the compendium:

Initiate git

We can initialise our compendium with .git using:

usethis::use_git()

N.B. Beware, you may have ended up with two Rstudio sessions of rrcompendium. Make sure to only have one session of a single project at one time to avoid problems.

Inspect templates

.
├── DESCRIPTION <- .............................package metadata
|                                               dependency management
├── NAMESPACE <- ...............................AUTO-GENERATED on build
├── R <- .......................................folder for functions
├── man <- .....................................AUTO-GENERATED on build
└── rrcompendium.Rproj <- ......................rstudio project file

rrtools::create_compendium() creates the bare backbone of infrastructure required for a research compendium. At this point it provides facilities to store general metadata about our compendium (eg bibliographic details to create a citation) and manage dependencies in the DESCRIPTION file and store and document functions in the R/ folder. Together these allow us to manage, install and share functionality associated with our project.

🚦 Update DESCRIPTION file

Let’s update some basic details in the DESCRIPTION file:

Package: rrcompendium
Title: What the Package Does (One Line, Title Case)
Version: 0.0.0.9000
Authors@R: 
    person(given = "First",
           family = "Last",
           role = c("aut", "cre"),
           email = "first.last@example.com")
Description: What the package does (one paragraph)
License: What license it uses
ByteCompile: true
Encoding: UTF-8
LazyData: true

Title: Partial Reproduction of Boettiger Ecology Letters 2018;21:1255–1267 
    with rrtools

Authors@R: 
    person(given = "Anna",
           family = "Krystalli",
           role = c("aut", "cre"),
           email = "annakrystalli@googlemail.com")

Description: This repository contains the research compendium of the 
    partial reproduction of Boettiger Ecology Letters 2018;21:1255–1267. 
    The compendium contains all data, code, and text associated with this       sub-section of the analysis

 usethis::use_mit_license()

✔ Setting License field in DESCRIPTION to 'MIT + file LICENSE'
✔ Writing 'LICENSE.md'
✔ Adding '^LICENSE\\.md$' to '.Rbuildignore'

License: MIT + file LICENSE

Package: rrcompendium
Title: Partial Reproduction of Boettiger Ecology Letters 2018;21:1255–1267 
    with rrtools
Version: 0.0.0.9000
Authors@R: 
    person(given = "Anna",
           family = "Krystalli",
           role = c("aut", "cre"),
           email = "annakrystalli@googlemail.com")
Description: This repository contains the research compendium of the partial 
    reproduction of Boettiger Ecology Letters 2018;21:1255–1267. The compendium 
    contains all data, code, and text associated with this sub-section of the 
    analysis.
License: MIT + file LICENSE
ByteCompile: true
Encoding: UTF-8
LazyData: true

.
├── DESCRIPTION
├── LICENSE
├── LICENSE.md
├── NAMESPACE
├── R
├── man
└── rrcompendium.Rproj

Create README

Every GitHub repository needs a README landing page.

We can create an rrtools README template using:

rrtools::use_readme_rmd()

✔ Creating 'README.Rmd' from template.
✔ Adding 'README.Rmd' to `.Rbuildignore`.
● Modify 'README.Rmd'
✔ Rendering README.Rmd to README.md for GitHub.
✔ Adding code of conduct.
✔ Creating 'CONDUCT.md' from template.
✔ Adding 'CONDUCT.md' to `.Rbuildignore`.
✔ Adding instructions to contributors.
✔ Creating 'CONTRIBUTING.md' from template.
✔ Adding 'CONTRIBUTING.md' to `.Rbuildignore`.

This generates README.Rmd and renders it to README.md, ready to display on GitHub. It contains:

• a template citation to show others how to cite your project.
• license information for the text, figures, code and data in your compendium

---
output: github_document
---

<!-- README.md is generated from README.Rmd. Please edit that file -->

``{r, echo = FALSE}
knitr::opts_chunk$set(
  collapse = TRUE,
  comment = "#>",
  fig.path = "README-"
)
``

# rrcompendium

This repository contains the data and code for our paper:

> Authors, (YYYY). _Title of paper_. Name of journal/book <https://doi.org/xxx/xxx>

Our pre-print is online here:

> Authors, (YYYY). _Title of paper_. Name of journal/book, Accessed 11 Apr 2019. Online at <https://doi.org/xxx/xxx>


### How to cite

Please cite this compendium as:

> Authors, (2019). _Compendium of R code and data for 'Title of paper'_. Accessed 11 Apr 2019. Online at <https://doi.org/xxx/xxx>

### How to download or install

You can download the compendium as a zip from from this URL: </archive/master.zip>

Or you can install this compendium as an R package, rrcompendium, from GitHub with:
### Licenses

**Text and figures :**  [CC-BY-4.0](http://creativecommons.org/licenses/by/4.0/)

**Code :** See the [DESCRIPTION](DESCRIPTION) file

**Data :** [CC-0](http://creativecommons.org/publicdomain/zero/1.0/) attribution requested in reuse

### Contributions

We welcome contributions from everyone. Before you get started, please see our [contributor guidelines](CONTRIBUTING.md). Please note that this project is released with a [Contributor Code of Conduct](CONDUCT.md). By participating in this project you agree to abide by its terms.

The call also adds two other markdown files:

• CONDUCT.md: a code of conduct for users
• CONTRIBUTING.md:: basic instructions for people who want to contribute to our compendium

update README

There’s five main edits we need to make to the template:

This repository contains the data and code for our reproduction paper:

> Krystalli, A, (2018). _Partial Reproduction of Boettiger Ecology Letters 2018;21:1255–1267 with rrtools_.  <https://doi.org/{DOI-to-paper}>

Our reproduction pre-print is online here:

> Krystalli, A, (2018). _Partial Reproduction of Boettiger Ecology Letters 2018;21:1255–1267 with rrtools_, Accessed 11 Apr 2019. Online at <https://doi.org/{DOI-to-compendium}>

Please cite this compendium as:

> Krystalli, A, (2019). _Compendium of R code and data for 'Partial Reproduction of Boettiger Ecology Letters 2018;21:1255–1267 with rrtools'_. Accessed 11 Apr 2019. Online at <https://doi.org/{DOI-to-compendium}>

### How to download or install

You can download the compendium as a zip from from this URL: <https://github.com/annakrystalli/rrcompendium/archive/master.zip>

**Text and figures :**  [CC-BY-4.0](http://creativecommons.org/licenses/by/4.0/), Copyright (c) 2018 Carl Boettiger.

**Code :** See the [DESCRIPTION](DESCRIPTION) file

**Data :** [CC-BY-4.0](http://creativecommons.org/licenses/by/4.0/), Copyright (c) 2018 Carl Boettiger.

🚦 Create GitHub repository

Next, we’ll create a GitHub repository to share our compendium. We’ll make use of our GITHUB_PAT and go for https authentication. We can do this with function:

usethis::use_github(protocol = "https")

✔ Setting active project to '/Users/Anna/Documents/workflows/rrcompendium'
● Check title and description
  Name:        rrcompendium
  Description: Partial Reproduction of Boettiger Ecology Letters 2018;21:1255–1267 with rrtools
  

The function will prompt you to confirm the name and description for your GitHub repo which it parses from our DESCRIPTION file. If everything looks good select the affirmative option.

Are title and description ok?
1: Not now
2: Definitely
3: Nope

If creation of the repo was successfull you should see the following console output:

✔ Creating GitHub repository
✔ Adding GitHub remote
✔ Adding GitHub links to DESCRIPTION
✔ Setting URL field in DESCRIPTION to 'https://github.com/annakrystalli/rrcompendium'
✔ Setting BugReports field in DESCRIPTION to 'https://github.com/annakrystalli/rrcompendium/issues'
✔ Pushing to GitHub and setting remote tracking branch
✔ Opening URL https://github.com/annakrystalli/rrcompendium

.
├── CONDUCT.md
├── CONTRIBUTING.md
├── DESCRIPTION
├── LICENSE
├── LICENSE.md
├── NAMESPACE
├── R
├── README.Rmd
├── README.md
├── man
└── rrcompendium.Rproj

Create analysis

We now need an analysis folder to contain our analysis and paper. We can do this using function rrtools::use_analysis()

The function has three location = options:

• top_level to create a top-level analysis/ directory

• inst to create an inst/ directory (so that all the sub-directories are available after the package is installed)

• vignettes to create a vignettes/ directory (and automatically update the DESCRIPTION).

The default is a top-level analysis/.

rrtools::use_analysis()

✔ Adding bookdown to Imports
✔ Creating 'analysis' directory and contents
✔ Creating 'analysis'
✔ Creating 'analysis/paper'
✔ Creating 'analysis/figures'
✔ Creating 'analysis/templates'
✔ Creating 'analysis/data'
✔ Creating 'analysis/data/raw_data'
✔ Creating 'analysis/data/derived_data'
✔ Creating 'references.bib' from template.
✔ Creating 'paper.Rmd' from template.

Next, you need to:  ↓ ↓ ↓ ↓ 
● Write your article/report/thesis, start at the paper.Rmd file
● Add the citation style library file (csl) to replace the default provided here, see https://github.com/citation-style-language/
● Add bibliographic details of cited items to the 'references.bib' file
● For adding captions & cross-referencing in an Rmd, see https://bookdown.org/yihui/bookdown/
● For adding citations & reference lists in an Rmd, see http://rmarkdown.rstudio.com/authoring_bibliographies_and_citations.html

Note that:
⚠ Your data files are tracked by Git and will be pushed to GitHub

Regardless for location option, the contents of the created sub-directories are the same:

    analysis/
    |
    ├── paper/
    │   ├── paper.Rmd       # this is the main document to edit
    │   └── references.bib  # this contains the reference list information
    
    ├── figures/            # location of the figures produced by the Rmd
    |
    ├── data/
    │   ├── DO-NOT-EDIT-ANY-FILES-IN-HERE-BY-HAND
    │   ├── raw_data/       # data obtained from elsewhere
    │   └── derived_data/   # data generated during the analysis
    |
    └── templates
        ├── journal-of-archaeological-science.csl
        |                   # this sets the style of citations & reference list
        ├── template.docx   # used to style the output of the paper.Rmd
        └── template.Rmd

Let’s inspect

Setup data

analysis/data
├── DO-NOT-EDIT-ANY-FILES-IN-HERE-BY-HAND
├── derived_data
└── raw_data
    └── gillespie.csv

Inspect analysis.R file

Let’s also open analysis.R in the course materials and run the code. The script has some initial setup, then loads the data, recodes one of the columns for plotting and then plots the results of the simulation, which generates figure 1 in paper.pdf.

library(dplyr)
library(readr)
library(ggplot2)
library(ggthemes)
theme_set(theme_grey())

# load-data
data <- read_csv(here::here("gillespie.csv"), col_types = "cdiddd")

# create colour palette
colours <- ptol_pal()(2)

# recode-data
data <- data %>% 
  mutate(system_size = recode(system_size, 
                              large = "A. 1000 total sites", 
                              small = "B. 100 total sites")) 

# plot-gillespie
data %>%
  ggplot(aes(x = time)) + 
  geom_hline(aes(yintercept = mean), lty=2, col=colours[2]) + 
  geom_hline(aes(yintercept = minus_sd), lty=2, col=colours[2]) + 
  geom_hline(aes(yintercept = plus_sd), lty=2, col=colours[2]) + 
  geom_line(aes(y = n), col=colours[1]) +
  facet_wrap(~system_size, scales = "free_y") 

🚦 Create journal article template using rticles

The rticles package is designed to simplify the creation of documents that conform to submission standards. A suite of custom R Markdown templates for popular journals is provided by the package.

rmarkdown::draft(here::here("analysis","paper.Rmd"), template = "elsevier_article", package = "rticles")

analysis/paper
├── elsarticle.cls
├── mybibfile.bib
├── numcompress.sty
└── paper.Rmd

🚦 Update YAML

The YAML header in Paper.Rmd contains document wide metadata and is pre-populated with some fields relevant to an academic publication.

---
title: Short Paper
author:
  - name: Alice Anonymous
    email: alice@example.com
    affiliation: Some Institute of Technology
    footnote: Corresponding Author
  - name: Bob Security
    email: bob@example.com
    affiliation: Another University
address:
  - code: Some Institute of Technology
    address: Department, Street, City, State, Zip
  - code: Another University
    address: Department, Street, City, State, Zip
abstract: |
  This is the abstract.

  It consists of two paragraphs.

journal: "An awesome journal"
date: "2019-04-11"
bibliography: mybibfile.bib
output: rticles::elsevier_article
---

Here we’re going to reproduce paper.pdf as is, so we’ll actually be editing the file with details from the original publication.

First, let’s clear all text BELOW the YAML header (which is delimited by ---. DO NOT delete the YAML header).

Next, let’s open paper.txt from the course material which contains all text from the in paper.pdf. We can use it to complete some of the fields in the YAML header.

title: "From noise to knowledge: how randomness generates novel phenomena and reveals information"

author:
  - name: "Carl Boettiger"
    affiliation: a
    email: "cboettig@berkeley.edu"

address:
  - code: a
    address: "Dept of Environmental Science, Policy, and Management, University of California Berkeley, Berkeley CA 94720-3114, USA"

bibliography: refs.bib

layout:  3p

preamble: |
  \usepackage[nomarkers]{endfloat}
  \linenumbers
  \usepackage{setspace}
  \doublespacing

abstract: |

  # Abstract
  
  Noise, as the term itself suggests, is most often seen a nuisance to ecological insight, a inconvenient reality that must be acknowledged, a haystack that must be stripped away to reveal the processes of interest underneath. Yet despite this well-earned reputation, noise is often interesting in its own right: noise can induce novel phenomena that could not be understood from some underlying determinstic model alone.  Nor is all noise the same, and close examination of differences in frequency, color or magnitude can reveal insights that would otherwise be inaccessible.  Yet with each aspect of stochasticity leading to some new or unexpected behavior, the time is right to move beyond the familiar refrain of "everything is important" (Bjørnstad & Grenfell 2001).  Stochastic phenomena can suggest new ways of inferring process from pattern, and thus spark more dialog between theory and empirical perspectives that best advances the field as a whole. I highlight a few compelling examples, while observing that the study of stochastic phenomena are only beginning to make this translation into empirical inference.  There are rich opportunities at this interface in the years ahead.  

output: rticles::elsevier_article

🚦 Add text

Now let’s add the main body of the paper from paper.txt.

\newpage

# Introduction: Noise the nuisance

To many, stochasticity, or more simply, noise,
is just that -- something which obscures patterns we are

...
...
...
...
...

# Acknowledgements

The author acknowledges feedback and advice from the editor,
Tim Coulson and two anonymous reviewers. This work was supported in
part by  USDA National Institute of Food and Agriculture, Hatch
project CA-B-INS-0162-H.

# References

🚦 Update references

Next we’ll replace the flat citations in the text with real linked citation which can be used to auto-generate formatted inline citations and the references section.

🚦 Update math

For the sake of time today, and not to open this topic too deeply here, I’ve included the following LaTex equation syntax in the text:

\begin{align}
\frac{\mathrm{d} n}{\mathrm{d} t} = \underbrace{c n \left(1 - \frac{n}{N}\right)}_{\textrm{birth}} - \underbrace{e n}_{\textrm{death}}, \label{levins}
\end{align}

that generates equation 1 in the paper.pdf.

\[\begin{align} \frac{\mathrm{d} n}{\mathrm{d} t} = \underbrace{c n \left(1 - \frac{n}{N}\right)}_{\textrm{birth}} - \underbrace{e n}_{\textrm{death}}, \label{levins} \end{align}\]

So you don’t need to edit anything here.

Check Math expressions and Markdown extensions by bookdown for more information.

🚦 Add code

Now that we’ve set up the text for our paper, let’s insert the code to generate figure 1.

    ```{r libraries, include=FALSE}
    
 ```

knitr::opts_chunk$set(echo = FALSE, message=FALSE, warning=FALSE, 
                      dev="cairo_pdf", fig.width=7, fig.height=3.5)

knitr::opts_chunk$set(echo = FALSE, message=FALSE, warning=FALSE, 
                      dev="cairo_pdf", fig.width=7, fig.height=3.5)

library(dplyr)
library(readr)
library(ggplot2)
library(ggthemes)
library(rrcompendium)

theme_set(theme_grey())

# create colour palette
colours <- ptol_pal()(2)

# load-data
data <- read_csv(here::here("gillespie.csv"), col_types = "cdiddd")  

# recode-data
data <- data %>% 
  mutate(system_size = recode(system_size, large = "A. 1000 total sites", small= "B. 100 total sites")) 

# plot-gillespie
data %>%
  ggplot(aes(x = time)) + 
  geom_hline(aes(yintercept = mean), lty=2, col=colours[2]) + 
  geom_hline(aes(yintercept = minus_sd), lty=2, col=colours[2]) + 
  geom_hline(aes(yintercept = plus_sd), lty=2, col=colours[2]) + 
  geom_line(aes(y = n), col=colours[1]) +
  facet_wrap(~system_size, scales = "free_y") 

    ```{r figure1, figure1, fig.cap="Population dynamics from a Gillespie simulation of the Levins model with large (N=1000, panel A) and small (N=100, panel B) number of sites (blue) show relatively weaker effects of demographic noise in the bigger system. Models are otherwise identical, with e = 0.2 and c = 1 (code in appendix A). Theoretical predictions for mean and plus/minus one standard deviation shown in horizontal re dashed lines."} ```

Render final document to pdf

Let’s check our final work by re-knitting to pdf. You should be looking at something that looks a lot like paper.pdf

🚦 Add paper dependencies

Finally, before we’re finished, let’s ensure the dependencies introduced in the paper are included. We can use rrtools::add_dependencies_to_description()

rrtools::add_dependencies_to_description()

This function scans script files (.R, .Rmd, .Rnw, .Rpres, etc.) for external package dependencies indicated by library(), require() or :: and adds those packages to the Imports field in the package DESCRIPTION:

Imports: 
    bookdown,
    dplyr,
    ggplot2 (>= 3.0.0),
    ggthemes (>= 3.5.0),
    here (>= 0.1),
    knitr (>= 1.20),
    rticles (>= 0.6)
    

Final compendium

You can see the resulting rrcompendium here

The complete compendium should contain the following files:

.
├── CONDUCT.md
├── CONTRIBUTING.md
├── DESCRIPTION
├── LICENSE
├── LICENSE.md
├── NAMESPACE
├── R
│   └── process-data.R
├── README.Rmd
├── README.md
├── analysis
│   ├── data
│   │   ├── DO-NOT-EDIT-ANY-FILES-IN-HERE-BY-HAND
│   │   ├── derived_data
│   │   └── raw_data
│   │       └── gillespie.csv
│   ├── figures
│   ├── paper
│   │   ├── elsarticle.cls
│   │   ├── mybibfile.bib
│   │   ├── numcompress.sty
│   │   ├── paper.Rmd
│   │   ├── paper.fff
│   │   ├── paper.pdf
│   │   ├── paper.spl
│   │   ├── paper.tex
│   │   ├── paper_files
│   │   │   └── figure-latex
│   │   │       └── figure1-1.pdf
│   │   └── refs.bib
│   └── templates
│       ├── journal-of-archaeological-science.csl
│       ├── template.Rmd
│       └── template.docx
├── inst
│   └── testdata
│       └── gillespie.csv
├── man
│   └── recode_system_size.Rd
├── rrcompendium.Rproj
└── tests
    ├── testthat
    │   └── test-process-data.R
    └── testthat.R

Session information

sessionInfo()