Last updated: 2023-09-20
Checks: 1 1
Knit directory: snakemake_tutorial/
This reproducible R Markdown analysis was created with workflowr (version 1.6.2). The Checks tab describes the reproducibility checks that were applied when the results were created. The Past versions tab lists the development history.
The R Markdown is untracked by Git. To know which version of the R
Markdown file created these results, you’ll want to first commit it to
the Git repo. If you’re still working on the analysis, you can ignore
this warning. When you’re finished, you can run
wflow_publish to commit the R Markdown file and build the
HTML.
Great! You are using Git for version control. Tracking code development and connecting the code version to the results is critical for reproducibility.
The results in this page were generated with repository version a32ee26. See the Past versions tab to see a history of the changes made to the R Markdown and HTML files.
Note that you need to be careful to ensure that all relevant files for
the analysis have been committed to Git prior to generating the results
(you can use wflow_publish or
wflow_git_commit). workflowr only checks the R Markdown
file, but you know if there are other scripts or data files that it
depends on. Below is the status of the Git repository when the results
were generated:
Ignored files:
Ignored: .Rproj.user/
Untracked files:
Untracked: analysis/install.Rmd
Untracked: analysis/introduction.Rmd
Unstaged changes:
Modified: analysis/_site.yml
Modified: analysis/index.Rmd
Note that any generated files, e.g. HTML, png, CSS, etc., are not included in this status report because it is ok for generated content to have uncommitted changes.
There are no past versions. Publish this analysis with
wflow_publish() to start tracking its development.
A workflow is a series of analytical steps to go from raw data to an analysis result. Often in biostatistics, statistical genetics, or bioinformatics, our workflows include several steps with intermediate outputs. We might need to combine multiple different programs in sequence. If we are unlucky, these programs might even have conflicting dependencies.
For example, to run a genome-wide association study, we need to format phenotype and genotype data, perform quality control checks and filters, compute a genetic relatedness matrix, and then use one of several available packages to perform association tests for each variant.
A natural way to run a workflow is to execute each step one at a time “by hand”. Every analysis process will involve this type of process to some degree, and this is not a bad way to start working through a procdeure.
However! This is not a great way to produce repeatable results that you have high confidence in. The “by hand” strategy is very vulnerable to common errors. For example, you may modify an early data normalization or processing step but forget to replace the file name in all of the downstream steps.
This is also not a scalable strategy. For example, suppose we are running GWAS of protein levels measured by a high-throughput technology. This means that rather than running association tests for one phenotype, we need to run association tests for thousands of phenotypes. One option would be to write a bash script loops through the genes and submits the job to the cluster. However, writing this code and then monitoring job submission and completion takes time and can be a bit fiddly.
Snakemake is a tool for creating and executing workflows.
A Snakemake workflow is repeatable. This is especially true if you use Conda inside of Snakemake to control the software environment of each step.
A Snakemake workflow is scalable. It is easy to modify a workflow to be performed many times or with multiple options. Snakemake can automate the cluster submission process so you don’t have to manage it.
A Snakemake workflow is human readable. You will have a single document that describes exactly what you did at each step, rather than having scrpts spread over many directories.
We can represent a workflow as a directed graph with nodes for files and analysis steps.