Last updated: 2020-11-13

Checks: 2 0

Knit directory: rr_tools/

This reproducible R Markdown analysis was created with workflowr (version 1.6.2). The Checks tab describes the reproducibility checks that were applied when the results were created. The Past versions tab lists the development history.

R Markdown file: up-to-date

Great! Since the R Markdown file has been committed to the Git repository, you know the exact version of the code that produced these results.

Repository version: 57a1f25

Great! You are using Git for version control. Tracking code development and connecting the code version to the results is critical for reproducibility.

The results in this page were generated with repository version 57a1f25. See the Past versions tab to see a history of the changes made to the R Markdown and HTML files.

Note that you need to be careful to ensure that all relevant files for the analysis have been committed to Git prior to generating the results (you can use wflow_publish or wflow_git_commit). workflowr only checks the R Markdown file, but you know if there are other scripts or data files that it depends on. Below is the status of the Git repository when the results were generated: 


Ignored files:
    Ignored:    .Rhistory
    Ignored:    .Rproj.user/

Untracked files:
    Untracked:  README.html
    Untracked:  figure/

Note that any generated files, e.g. HTML, png, CSS, etc., are not included in this status report because it is ok for generated content to have uncommitted changes.

These are the previous versions of the repository in which changes were made to the R Markdown (analysis/index.Rmd) and HTML (docs/index.html) files. If you’ve configured a remote Git repository (see ?wflow_git_remote), click on the hyperlinks in the table below to view the files as they were in that past version.

File Version Author Date Message
Rmd 57a1f25 jean997 2020-11-13 add slides
html 3eaf685 jean997 2020-10-22 Build site.
Rmd 3d1a485 jean997 2020-10-22 wflow_publish(“analysis/index.Rmd”)
html 496bc7a jean997 2020-10-22 Build site.
Rmd b1ec62c jean997 2020-10-22 wflow_publish(fl)
html 366d288 jean997 2020-10-22 Build site.
Rmd 3180fcb jean997 2020-10-22 wflow_publish(all = TRUE)
html 3180fcb jean997 2020-10-22 wflow_publish(all = TRUE)
html 971cbb5 jean997 2020-10-21 Build site.
Rmd 34cf544 jean997 2020-10-21 wflow_publish(all = TRUE)
html 34cf544 jean997 2020-10-21 wflow_publish(all = TRUE)
html b0d3bd7 jean997 2020-10-21 Build site.
Rmd de9e202 jean997 2020-10-21 added more words
html 0bb9529 jean997 2020-10-21 Build site.
Rmd 7e6b37c jean997 2020-10-21 wflow_publish(“analysis/index.Rmd”)
html 2ce37af jean997 2020-10-21 Build site.
Rmd 06a1c66 jean997 2020-10-21 wflow_publish(c(“analysis/index.Rmd”, “analysis/file_org.rmd”,
Rmd f3d4d87 jean997 2020-10-21 add content to version_control plus some small edits
Rmd 0809fee jean997 2020-10-21 initial structure
Rmd fc35dfd jean997 2020-10-21 Start workflowr project.

Introduction

The aim of this website is to give an overview of several tools for reproducible statistical and data science research. Many of these tools will also help with workflow, organization, productivity, and your general sense of peace and well being as you go about your work.

Here are some caveats before we get going:

  • This is not an exhaustive list. If you have something that you use and like I would love to know about it and add it in. Feel free to open a GitHub Issue and let me know about it!
  • This website doesn’t attempt to give thorough tutorials. It is more of an aggregation of useful things that have already been well documented others elsewhere. I’ll do my best to supply all the links you need to get going with each thing. Please let me know if you find a broken link of if you’ve found something useful you want to add.
  • I am mostly an R user so some of these tools are fairly R focused but in a lot of cases similar principles apply across languages.

This website was built using workflowr which is covered in the section on Workbooks and Reports.

Here is a set of slides that cover the material here.

What is Reproducibility?

Reproducibility has to do with regenerating results that are reported in scientific literature. There are different ways that results might be regenerated which mean different things about the reliability of the result. We say that:

The result is repeatable if the original investigators can regenerate published results from the original data.

The result is reproducible if different investigators can regenerate the results, potentially using materials and instructions obtained from the original investigators.

The result is replicable if new investigators can reach the same conclusions from a new experiment/sample.

Scientific integrity requires us to pay attention to ensuring all three of these features. Here we are focusing on technical tools and practices that help guarantee repeatability and reproducibility. We won’t address errors in experimental design or data collection.

What Does Reproducible Statistical Research Look Like?

Some results may be nominally reproducible in that authors promise “code and data on reasonable request” but how often can these results actually be reproduced? According to Minocher et al not as often as we would like! What can you do to make sure your results are easily reproducible and you are not scrambling if someone asks you for the results years later?

  • Code is publicly available and well documented.
  • Data is publicly available if possible.
  • Code can be run on a reasonable range of computing systems.
  • Analysis is well documented enough that it can be run by another without help from the authors.

Effort, Pain, and Learning Curves

Each step that you take towards reproducibility will require some modification of your work habits. It is natural to work initially in ways that turn out later to be messy and hard to recreate. This is why it is so common for people to lose files or send the wrong version of something. Learning new tools, installing software, and creating new habits all take up time and can be awkward and uncomfortable. The payoff is in fewer headaches down the road and confidence in the results you report or publish on.

Developing your work habits and your programming skills is an ongoing project. I organized the content below into categories that are about the order that I have adopted different habits but fee free to hop around.

Content

  1. File Names and Directory Structure
  2. Version Control (Git)
    • Git
  3. Code and Programming Practices
    • Data processing practices
    • Programming tips
  4. Workbooks and Reports
    • R Markdown and Sweave with knitr
    • workflowr for project organization
    • Software environments
  5. Pipelines
    • Snakemake
    • DSC for simulations

Resources

A Good Talk by Richard McElreath

Advanced R Book by Hadley Wickham

R Packages Book by Hadley Wickham

Efficient R by Collin Gillespe and Robin Lovelace

Happy Git with R by Jenny Bryan

R Markdown Cookbook by Yihui Xie, Christophe Dervieux, and Emily Riederer

Conda

workflowr developed by John Blischak

Snakemake

DSC