Last updated: 2019-05-06

Checks: 5 1

Knit directory: W_shredder/

This reproducible R Markdown analysis was created with workflowr (version 1.3.0). The Checks tab describes the reproducibility checks that were applied when the results were created. The Past versions tab lists the development history.


The R Markdown file has unstaged changes. To know which version of the R Markdown file created these results, you’ll want to first commit it to the Git repo. If you’re still working on the analysis, you can ignore this warning. When you’re finished, you can run wflow_publish to commit the R Markdown file and build the HTML.

Great job! The global environment was empty. Objects defined in the global environment can affect the analysis in your R Markdown file in unknown ways. For reproduciblity it’s best to always run the code in an empty environment.

The command set.seed(20180716) was run prior to running the code in the R Markdown file. Setting a seed ensures that any results that rely on randomness, e.g. subsampling or permutations, are reproducible.

Great job! Recording the operating system, R version, and package versions is critical for reproducibility.

Nice! There were no cached chunks for this analysis, so you can be confident that you successfully produced the results during this run.

Great! You are using Git for version control. Tracking code development and connecting the code version to the results is critical for reproducibility. The version displayed above was the version of the Git repository at the time these results were generated.

Note that you need to be careful to ensure that all relevant files for the analysis have been committed to Git prior to generating the results (you can use wflow_publish or wflow_git_commit). workflowr only checks the R Markdown file, but you know if there are other scripts or data files that it depends on. Below is the status of the Git repository when the results were generated:


Ignored files:
    Ignored:    .DS_Store
    Ignored:    .Rhistory
    Ignored:    .Rproj.user/
    Ignored:    Proc_B_manuscript/.DS_Store
    Ignored:    data/sim_results/
    Ignored:    docs/figure/

Unstaged changes:
    Modified:   analysis/index.Rmd
    Modified:   analysis/run_model.Rmd

Note that any generated files, e.g. HTML, png, CSS, etc., are not included in this status report because it is ok for generated content to have uncommitted changes.


These are the previous versions of the R Markdown and HTML files. If you’ve configured a remote Git repository (see ?wflow_git_remote), click on the hyperlinks in the table below to view them.

File Version Author Date Message
Rmd 1fca665 lukeholman 2019-04-26 increase SLURM time, and writing
Rmd 0ccc722 Luke Holman 2018-12-28 change to 48h
Rmd a587610 Luke Holman 2018-12-21 bigger chunks
Rmd e882e57 Luke Holman 2018-12-04 single core for faster queueing
Rmd 8a6ce50 Luke Holman 2018-12-03 change resources
Rmd 3b94528 Luke Holman 2018-12-03 Tweaks
Rmd ab69fc7 Luke Holman 2018-11-23 fix else
Rmd e60d0d1 Luke Holman 2018-11-23 bug hunt
Rmd b531be6 Luke Holman 2018-11-23 Tweak handling of parameter set up
Rmd e01c7fe Luke Holman 2018-11-22 bug fix
Rmd 6c2a74e Luke Holman 2018-11-22 Added new slurm script to check paras
Rmd 68469d2 Luke Holman 2018-11-22 fix wd typo
Rmd 79a4634 Luke Holman 2018-11-21 Change counting of old parameters
Rmd 7bf9997 Luke Holman 2018-11-20 added print to find bug
Rmd ed47ecc Luke Holman 2018-11-20 go to single core again
Rmd 9d2045c Luke Holman 2018-11-20 Fix parameter file
Rmd ee185ed Luke Holman 2018-11-19 Extra params
Rmd 64ce9e9 Luke Holman 2018-11-18 Less memory?
Rmd 0e180bc Luke Holman 2018-11-18 Fix memory spec
Rmd a9c7dfb Luke Holman 2018-11-16 More memory pls
Rmd 4c91977 Luke Holman 2018-11-16 Get more computation time
Rmd 4e630f4 Luke Holman 2018-11-15 Make combine_results_files.R
Rmd 2416673 Luke Holman 2018-11-15 missed bracket
Rmd 2c0e001 Luke Holman 2018-11-15 Bug fix
Rmd 4be43ee Luke Holman 2018-11-14 fixing bugs
Rmd b02f7d0 Luke Holman 2018-11-14 more nodes pls
Rmd 1cafc43 Luke Holman 2018-11-14 Added fitness to density calculation
Rmd fb9316c Luke Holman 2018-11-13 Request 32GB memory
Rmd 7560af7 Luke Holman 2018-11-13 change node number
Rmd b0fdeed Luke Holman 2018-11-13 multicore again
Rmd 8983cff Luke Holman 2018-11-13 Fixed chunks
Rmd 710d342 Luke Holman 2018-11-13 cpu = 1
Rmd 5f4f83b Luke Holman 2018-11-13 fixed job chunks
Rmd 9440576 Luke Holman 2018-11-13 change name
Rmd de9e0ff Luke Holman 2018-11-13 Added slurm capacity
Rmd 99e93c7 Luke Holman 2018-11-03 Many bug fixes with density dependence
html 99e93c7 Luke Holman 2018-11-03 Many bug fixes with density dependence

# This bit is for the unimelb cluster, Spartan
working_directory <- "/data/projects/punim0243/W_shredder"
setwd(working_directory)
source_rmd <- function(file){
  options(knitr.duplicate.label = "allow")
  tempR <- tempfile(tmpdir = ".", fileext = ".R")
  on.exit(unlink(tempR))
  knitr::purl(file, output = tempR, quiet = TRUE)
  source(tempR, local = globalenv())
}
source_rmd("analysis/model_functions.Rmd")
Warning: package 'dplyr' was built under R version 3.5.2
Warning: package 'purrr' was built under R version 3.5.2
custom_functions <- ls()

Define the parameter space still not yet run

This is defined in an R script that sets up the parameter space, and runs everything that has not already completed.

source("code/set_up_parameters.R")

To save you searching on Github, here is the entire contents of the file code/set_up_parameters.R:

setwd("/data/projects/punim0243/W_shredder")

#############################################
# Load all custom functions and packages
#############################################
source_rmd <- function(file){
  options(knitr.duplicate.label = "allow")
  tempR <- tempfile(tmpdir = ".", fileext = ".R")
  on.exit(unlink(tempR))
  knitr::purl(file, output = tempR, quiet = TRUE)
  source(tempR, local = globalenv())
}
source_rmd("analysis/model_functions.Rmd")
custom_functions <- ls()

#############################################
# Define the entire parameter space to be run
#############################################
print("Defining parameter space")

parameters <- expand.grid(
  release_size = 20,
  release_strategy = c("one_patch", "all_patches"),
  W_shredding_rate = c(0.50, 0.95, 1), # strength of gene drive in females
  Z_conversion_rate = c(0, 0.5, 0.95), # strength of gene drive in males
  Zr_creation_rate = c(0, 0.001, 0.01, 0.1), # frequency of NHEJ in males
  Zr_mutation_rate = c(0.0, 0.00001),
  Wr_mutation_rate = c(0.0, 0.00001),
  cost_Zdrive_female = c(0.01, 0.1, 0.5, 1), # Cost of Z* to female fecundity
  cost_Zdrive_male = c(0.01, 0.2),  # Cost of Z* to male mating success
  male_migration_prob = c(0.05, 0.5),
  female_migration_prob = c(0.05, 0.5),
  migration_type = c("local", "global"), # do migrants move to next door patch, or a random patch anywhere in the world?
  n_patches = c(2, 20),
  softness = c(0, 0.5, 1),
  male_weighting = c(0.5, 1, 1.5),
  density_dependence_shape = c(0.2, 1, 1.8),
  cost_Wr = 0,   # Assume resistance is not costly for now. Seems pretty obvious how this affects evolution
  cost_Zr = 0,
  cost_A = 0,
  cost_B = 0,
  max_fecundity = c(50, 100),
  carrying_capacity = 10000,
  initial_pop_size = 10000,
  initial_Zdrive = 0,
  initial_Zr = 0.00,
  initial_Wr = 0.00,
  initial_A = c(0, 0.05),
  initial_B = c(0, 0.05),
  realisations = 1, # change to e.g. 1:100 for replication
  generations = 1000,
  burn_in = 50
) %>% filter(!(W_shredding_rate == 0 & Z_conversion_rate == 0)) %>%
  mutate(migration_type = as.character(migration_type),
         release_strategy = as.character(release_strategy))

# Shuffle for even workload across all cores
set.seed(1)
parameters <- parameters[sample(nrow(parameters)), ]

# Set the initial frequency to be the same as the mutation rate for the resistant chromosomes
parameters$initial_Wr <- parameters$Wr_mutation_rate
parameters$initial_Zr <- parameters$Zr_mutation_rate

# No point doing lots of different W_shredding_rate values when cost_Zdrive_female == 1
parameters$W_shredding_rate[parameters$cost_Zdrive_female == 1] <- 1
parameters <- parameters %>% distinct()
num_parameter_spaces <- nrow(parameters)

#############################################################################
# Create a data frame of parameter spaces that have been completed already
# and remove rows from `parameters` that are already finished
#############################################################################

print("Checking previously-completed files...")

completed <- readRDS("data/all_results.rds") %>%
  select(!! names(parameters))
completed <- apply(completed, 1, paste0, collapse = "_")

to_do <- data.frame(row = 1:nrow(parameters),
                    pasted = apply(parameters, 1, paste0, collapse = "_"),
                    stringsAsFactors = FALSE) %>%
  filter(!(pasted %in% completed))

parameters <- parameters[to_do$row, ]
print(paste("Already completed", length(completed), "parameter spaces"))
print(paste("Queing up", nrow(parameters), "model runs"))
rm(to_do)

Now launch lots of SLURM jobs to run the remaining parameter spaces

chunk_size <- 4000
cpus <- 1
sopt <- list(time = '168:00:00',  # max run time per node in hours
             mem  = '32768')     # 32GB memory across all 8 cores

chunks <- split(1:nrow(parameters),
                ceiling(seq_along(1:nrow(parameters))/chunk_size))
number_of_chunks <- length(chunks)


sjob <- slurm_apply(
  f = function(i) {
    try(do_all_parameters(parameters[chunks[[i]],], 
                          over_write = FALSE, 
                          cores = cpus,
                          wd = working_directory))
  },
  params = data.frame(i = 1:length(chunks)),
  add_objects = c("do_all_parameters", 
                  "parameters", "cpus",
                  "working_directory",
                  "chunks", "number_of_chunks",
                  custom_functions),
  jobname = 'W_shredder',
  nodes = number_of_chunks, 
  cpus_per_node = cpus, 
  slurm_options = sopt)

sessionInfo()
R version 3.5.1 (2018-07-02)
Platform: x86_64-apple-darwin15.6.0 (64-bit)
Running under: macOS High Sierra 10.13.6

Matrix products: default
BLAS: /Library/Frameworks/R.framework/Versions/3.5/Resources/lib/libRblas.0.dylib
LAPACK: /Library/Frameworks/R.framework/Versions/3.5/Resources/lib/libRlapack.dylib

locale:
[1] en_AU.UTF-8/en_AU.UTF-8/en_AU.UTF-8/C/en_AU.UTF-8/en_AU.UTF-8

attached base packages:
[1] parallel  stats     graphics  grDevices utils     datasets  methods  
[8] base     

other attached packages:
[1] tibble_2.0.99.9000 readr_1.1.1        rslurm_0.4.0      
[4] Rcpp_1.0.0         reshape2_1.4.3     stringr_1.3.1     
[7] tidyr_0.8.2        purrr_0.3.1        dplyr_0.8.0.1     

loaded via a namespace (and not attached):
 [1] knitr_1.20        whisker_0.3-2     magrittr_1.5     
 [4] workflowr_1.3.0   hms_0.4.2         tidyselect_0.2.5 
 [7] R6_2.4.0          rlang_0.3.1       plyr_1.8.4       
[10] tools_3.5.1       git2r_0.23.0      htmltools_0.3.6  
[13] yaml_2.2.0        rprojroot_1.3-2   digest_0.6.18    
[16] assertthat_0.2.0  crayon_1.3.4      fs_1.2.6         
[19] glue_1.3.0.9000   evaluate_0.11     rmarkdown_1.10   
[22] stringi_1.3.1     compiler_3.5.1    pillar_1.3.1.9000
[25] backports_1.1.2   pkgconfig_2.0.2