Last updated: 2023-09-21
Checks: 2 0
Knit directory: snakemake_tutorial/
This reproducible R Markdown analysis was created with workflowr (version 1.7.1). The Checks tab describes the reproducibility checks that were applied when the results were created. The Past versions tab lists the development history.
Great! Since the R Markdown file has been committed to the Git repository, you know the exact version of the code that produced these results.
Great! You are using Git for version control. Tracking code development and connecting the code version to the results is critical for reproducibility.
The results in this page were generated with repository version 7957b05. See the Past versions tab to see a history of the changes made to the R Markdown and HTML files.
Note that you need to be careful to ensure that all relevant files for
the analysis have been committed to Git prior to generating the results
(you can use wflow_publish or
wflow_git_commit). workflowr only checks the R Markdown
file, but you know if there are other scripts or data files that it
depends on. Below is the status of the Git repository when the results
were generated:
Ignored files:
Ignored: .Rproj.user/
Note that any generated files, e.g. HTML, png, CSS, etc., are not included in this status report because it is ok for generated content to have uncommitted changes.
These are the previous versions of the repository in which changes were
made to the R Markdown (analysis/introduction.Rmd) and HTML
(docs/introduction.html) files. If you’ve configured a
remote Git repository (see ?wflow_git_remote), click on the
hyperlinks in the table below to view the files as they were in that
past version.
| File | Version | Author | Date | Message |
|---|---|---|---|---|
| Rmd | 7957b05 | Jean Morrison | 2023-09-21 | add installation instructions |
| html | 7957b05 | Jean Morrison | 2023-09-21 | add installation instructions |
A workflow is a series of analytical steps to go from raw data to an analysis result. Often in biostatistics, statistical genetics, or bioinformatics, our workflows include several steps with intermediate outputs. We might need to combine multiple different programs in sequence. If we are unlucky, these programs might even have conflicting dependencies.
For example, to run a genome-wide association study, we need to format phenotype and genotype data, perform quality control checks and filters, compute a genetic relatedness matrix, and then use one of several available packages to perform association tests for each variant.
A natural way to run a workflow is to execute each step one at a time “by hand”. Every analysis process will involve this type of process to some degree, and this is not a bad way to start working through a procdeure.
However! This is not a great way to produce repeatable results that you have high confidence in. The “by hand” strategy is very vulnerable to common errors. For example, you may modify an early data normalization or processing step but forget to replace the file name in all of the downstream steps.
This is also not a scalable strategy. For example, suppose we are running GWAS of protein levels measured by a high-throughput technology. Rather than running association tests for one phenotype, we need to run association tests for thousands of phenotypes. One option would be to write a bash script loops through the genes and submits the job to the cluster. However, writing this code and then monitoring job submission and completion takes time and can be a bit fiddly.
Snakemake is a tool for creating and executing workflows.
A Snakemake workflow is repeatable. This is especially true if you use Conda inside of Snakemake to control the software environment of each step.
A Snakemake workflow is scalable. It is easy to modify a workflow to be performed many times or with multiple options. Snakemake can automate the cluster submission process so you don’t have to manage it.
A Snakemake workflow is human readable. You will have a single document that describes exactly what you did at each step, rather than having scrpts spread over many directories.
To write a Snakemake workflow, we need to describe each step in the procedure using three attributes: input files, output files, and the action (shell command)