Last updated: 2021-07-29

Checks: 2 0

Knit directory: implementGMSinCassava/

This reproducible R Markdown analysis was created with workflowr (version 1.6.2). The Checks tab describes the reproducibility checks that were applied when the results were created. The Past versions tab lists the development history.

R Markdown file: up-to-date

Great! Since the R Markdown file has been committed to the Git repository, you know the exact version of the code that produced these results.

Repository version: bc85a7d

Great! You are using Git for version control. Tracking code development and connecting the code version to the results is critical for reproducibility.

The results in this page were generated with repository version bc85a7d. See the Past versions tab to see a history of the changes made to the R Markdown and HTML files.

Note that you need to be careful to ensure that all relevant files for the analysis have been committed to Git prior to generating the results (you can use wflow_publish or wflow_git_commit). workflowr only checks the R Markdown file, but you know if there are other scripts or data files that it depends on. Below is the status of the Git repository when the results were generated: 


Ignored files:
    Ignored:    .DS_Store
    Ignored:    .Rhistory
    Ignored:    .Rproj.user/
    Ignored:    analysis/.DS_Store
    Ignored:    analysis/accuracies.png
    Ignored:    analysis/fig2.png
    Ignored:    analysis/fig3.png
    Ignored:    analysis/fig4.png
    Ignored:    code/.DS_Store
    Ignored:    data/.DS_Store

Untracked files:
    Untracked:  accuracies.png
    Untracked:  analysis/docs/
    Untracked:  analysis/speedUpPredCrossVar.Rmd
    Untracked:  data/AllChrom_RefPanelAndGSprogeny_ReadyForGP_72719.bed
    Untracked:  data/AllChrom_RefPanelAndGSprogeny_ReadyForGP_72719.bim
    Untracked:  data/AllChrom_RefPanelAndGSprogeny_ReadyForGP_72719.fam
    Untracked:  data/AllChrom_RefPanelAndGSprogeny_ReadyForGP_72719.hap.gz
    Untracked:  data/AllChrom_RefPanelAndGSprogeny_ReadyForGP_72719.log
    Untracked:  data/AllChrom_RefPanelAndGSprogeny_ReadyForGP_72719.nosex
    Untracked:  data/AllChrom_RefPanelAndGSprogeny_ReadyForGP_72719.sample
    Untracked:  data/AllChrom_RefPanelAndGSprogeny_ReadyForGP_72719.vcf.gz
    Untracked:  data/DatabaseDownload_2021May04/
    Untracked:  data/blups_forCrossVal.rds
    Untracked:  data/config.txt
    Untracked:  data/config_mw.txt
    Untracked:  data/dosages_IITA_2021May13.rds
    Untracked:  data/dosages_IITA_filtered_2021May13.rds
    Untracked:  data/genmap_2021May13.rds
    Untracked:  data/haps_IITA_2021May13.rds
    Untracked:  data/haps_IITA_filtered_2021May13.rds
    Untracked:  data/recombFreqMat_1minus2c_2021May13.rds
    Untracked:  fig2.png
    Untracked:  fig3.png
    Untracked:  figure/
    Untracked:  output/Cassava_HMII_V3_Marning_imputation_6-18-21.samples
    Untracked:  output/IITA_CleanedTrialData_2021May10.rds
    Untracked:  output/IITA_ExptDesignsDetected_2021May10.rds
    Untracked:  output/IITA_blupsForModelTraining_twostage_asreml_2021May10.rds
    Untracked:  output/IITA_trials_NOT_identifiable.csv
    Untracked:  output/alphaAssignOutput_iita_pedigree.txt.dams
    Untracked:  output/alphaAssignOutput_iita_pedigree.txt.dams.full
    Untracked:  output/alphaAssignOutput_iita_pedigree.txt.pedigree
    Untracked:  output/alphaAssignOutput_iita_pedigree.txt.pedigree.top
    Untracked:  output/alphaAssignOutput_iita_pedigree.txt.sires
    Untracked:  output/alphaAssignOutput_iita_pedigree.txt.sires.full
    Untracked:  output/crossValPredsA.rds
    Untracked:  output/crossValPredsAD.rds
    Untracked:  output/cvAD_1rep_markerEffects.rds
    Untracked:  output/cvAD_1rep_meanPredAccuracy.rds
    Untracked:  output/cvAD_1rep_parentfolds.rds
    Untracked:  output/cvAD_1rep_predAccuracy.rds
    Untracked:  output/cvAD_1rep_predMeans.rds
    Untracked:  output/cvAD_1rep_predVars.rds
    Untracked:  output/cvAD_1rep_varPredAccuracy.rds
    Untracked:  output/cvAD_5rep5fold_markerEffects.rds
    Untracked:  output/cvAD_5rep5fold_meanPredAccuracy.rds
    Untracked:  output/cvAD_5rep5fold_parentfolds.rds
    Untracked:  output/cvAD_5rep5fold_predMeans.rds
    Untracked:  output/cvAD_5rep5fold_predVars.rds
    Untracked:  output/cvAD_5rep5fold_varPredAccuracy.rds
    Untracked:  output/cvDirDom_5rep5fold_markerEffects.rds
    Untracked:  output/cvDirDom_5rep5fold_meanPredAccuracy.rds
    Untracked:  output/cvDirDom_5rep5fold_parentfolds.rds
    Untracked:  output/cvDirDom_5rep5fold_predMeans.rds
    Untracked:  output/cvDirDom_5rep5fold_predVars.rds
    Untracked:  output/cvDirDom_5rep5fold_varPredAccuracy.rds
    Untracked:  output/cvMeanPredAccuracyA.rds
    Untracked:  output/cvMeanPredAccuracyAD.rds
    Untracked:  output/cvPredMeansA.rds
    Untracked:  output/cvPredMeansAD.rds
    Untracked:  output/cvVarPredAccuracyA.rds
    Untracked:  output/cvVarPredAccuracyAD.rds
    Untracked:  output/genomicMatePredictions_top121parents_ModelAD.csv
    Untracked:  output/genomicMatePredictions_top121parents_ModelAD.rds
    Untracked:  output/genomicMatePredictions_top121parents_ModelDirDom.csv
    Untracked:  output/genomicMatePredictions_top121parents_ModelDirDom.rds
    Untracked:  output/genomicPredictions_ModelAD.csv
    Untracked:  output/genomicPredictions_ModelAD.rds
    Untracked:  output/genomicPredictions_ModelDirDom.csv
    Untracked:  output/genomicPredictions_ModelDirDom.rds
    Untracked:  output/kinship_A_IITA_2021May13.rds
    Untracked:  output/kinship_D_IITA_2021May13.rds
    Untracked:  output/kinship_domGenotypic_IITA_2021July5.rds
    Untracked:  output/markEffsTest.rds
    Untracked:  output/markerEffects.rds
    Untracked:  output/markerEffectsA.rds
    Untracked:  output/markerEffectsAD.rds
    Untracked:  output/maxNOHAV_byStudy.csv
    Untracked:  output/obsCrossMeansAndVars.rds
    Untracked:  output/parentfolds.rds
    Untracked:  output/ped2check_genome.rds
    Untracked:  output/ped2genos.txt
    Untracked:  output/pednames2keep.txt
    Untracked:  output/pednames_Prune100_25_pt25.log
    Untracked:  output/pednames_Prune100_25_pt25.nosex
    Untracked:  output/pednames_Prune100_25_pt25.prune.in
    Untracked:  output/pednames_Prune100_25_pt25.prune.out
    Untracked:  output/potential_dams.txt
    Untracked:  output/potential_sires.txt
    Untracked:  output/predVarTest.rds
    Untracked:  output/samples2keep_IITA_2021May13.txt
    Untracked:  output/samples2keep_IITA_MAFpt01_prune50_25_pt98.log
    Untracked:  output/samples2keep_IITA_MAFpt01_prune50_25_pt98.nosex
    Untracked:  output/samples2keep_IITA_MAFpt01_prune50_25_pt98.prune.in
    Untracked:  output/samples2keep_IITA_MAFpt01_prune50_25_pt98.prune.out
    Untracked:  output/samples2keep_notInPHGdb.txt
    Untracked:  output/test_cvAD_markerEffects.rds
    Untracked:  output/test_cvAD_meanPredAccuracy.rds
    Untracked:  output/test_cvAD_parentfolds.rds
    Untracked:  output/test_cvAD_predAccuracy.rds
    Untracked:  output/test_cvAD_predMeans.rds
    Untracked:  output/test_cvAD_predVars.rds
    Untracked:  output/test_cvAD_varPredAccuracy.rds
    Untracked:  output/top50crosses.csv
    Untracked:  output/verified_ped.txt

Note that any generated files, e.g. HTML, png, CSS, etc., are not included in this status report because it is ok for generated content to have uncommitted changes.

These are the previous versions of the repository in which changes were made to the R Markdown (analysis/index.Rmd) and HTML (docs/index.html) files. If you’ve configured a remote Git repository (see ?wflow_git_remote), click on the hyperlinks in the table below to view the files as they were in that past version.

File Version Author Date Message
Rmd e176b81 wolfemd 2021-07-29 Update the project landing page to reflect the near-final state of things. Ready to publish.
Rmd f8f8a28 wolfemd 2021-07-26 Update analysis-to-do list. Add placeholders and links about PHG-vs-Beagle comparisons to-come.
html 934141c wolfemd 2021-07-14 Build site.
html cc1eb4b wolfemd 2021-07-14 Build site.
Rmd 772750a wolfemd 2021-07-14 DirDom model and selection index calc fully integrated functions.
html 5e45aac wolfemd 2021-06-18 Build site.
html df7a366 wolfemd 2021-06-10 Build site.
Rmd c28400f wolfemd 2021-06-10 github link added
html e66bdad wolfemd 2021-06-10 Build site.
Rmd a8452ba wolfemd 2021-06-10 Initial build of the entire page upon completion of all
Rmd 8a0c50e wolfemd 2021-05-04 Start workflowr project.

We saw in an initial study, promising results regarding the prediction of genetic variance in cassava crosses. In that study, we used a high-quality validated pedigree-based phasing pipeline for the marker data. That pipeline is considerably more involved and may not be implementable on the entire breeding germplasm.

In this project, I am integrating cross-variance predictions into our current genomics-enabled breeding pipeline leveraging available data. I conduct additional tests to assess whether current Beagle imputed-and-phased data and a basic pedigree-validation step are sufficient.

This project should form the template for genomic mate selection in NextGen Cassava.

See the Results here!

We will soon implement in summer 2021 for planning fall crossing nurseries and with new GS C5 DArTseqLD data.

What’s new?

  1. Functions previously used in NextGen GS pipeline (code/gsFunctions.R) migrate to and receive an upgrade (code/gmsFunctions.R) to support genomic mate selection. The runGenomicPredictions() function fits genomic mixed-models and produces GEBV/GETGV for existing germplasm as it did previously, but now also returns marker effects and provides a direct input to the new predictCrosses() function, which handles predicting mate selection criteria.

  2. Support for a directional dominance model to incorporate genome-wide homozygosity-effects (inbreeding) into predictions.

  3. Support for selection indices.

  4. Improved version of predCrossVar functions at code/predCrossVar.R

    • eventually to be integrated into the R package
    • faster unless you want to use the Bayesian approach.

  5. Creation of a single-function interface to accomplish parent-wise cross-validation at code/parentWiseCrossVal.R. Provides estimates of (selection index) accuracy predicting family means and variances.

Analysis steps

  1. Prepare training dataset: Download data from DB, “Clean” and format DB data. Use the standard pipeline to obtain complete breeding trial data for IITA, generate de-regressed BLUPs for downstream analysis.

  2. Get BLUPs combining all trial data: Combine data from all trait-trials to get BLUPs for downstream genomic prediction. Fit mixed-model to multi-trial dataset and extract BLUPs, de-regressed BLUPs and weights. Include two rounds of outlier removal.

  3. Validate the pedigree obtained from cassavabase: Before setting up a cross-validation scheme for predictions that depend on a correct pedigree, add a basic verification step to the pipeline. Not trying to fill unknown relationships or otherwise correct the pedigree. Assess evidence that relationship is correct, remove if incorrect.

  4. Preprocess data files: Prepare haplotype and dosage matrices, GRMs, pedigree and BLUPs, genetic map and recombination frequency matrix, for use in predictions.

  5. Extract and process PHG files: Extract a VCF file from the PHG *.db file produced by Evan Long. Subsequently, prepare haplotype and dosage matrices, GRMs, genetic map and recombination frequency matrix, for use in predictions.

  6. Parent-wise and standard cross-validation:

    • Compute parent-wise cross-validation folds using the validated pedigree. Fit models to get marker effects and make subsequent predictions of cross means and (co)variances.
    • Models “AD” (classic BV+DD partition of additive+dominance) and “DirDom” (genotypic add+dom partition with genome-wide homozygosity effect).
    • Include use of selection index weights to compute index accuracy.
    • Include new models and index predictions in standard fold cross-validation in addition to the parent-wise scheme / function.
    • [Coming soon] Run cross-validation using both Beagle- vs. PHG-imputed and phased marker data to compare their quality.

  7. Genomic predictions:

      1. Standard genomic prediction of individual GEBV and GETGV for all selection candidates using all available data. (B) Predict cross means and variances for genomic mate selection.

    • Include models “AD” and “DirDom”

    • Include prediction of selection index GEBV/GETGV and \(UC^{SI}_{parent}\)/\(UC^{SI}_{variety}\).

    • New functions at gmsFunctions.R in code/

  8. Results: Home for plots and summary tables.

Data and code repository access

CLICK HERE FOR ACCESS TO THE FULL REPOSITORY (select “Guest” credentials when prompted by the Cassavabase FTP server)

or

DOWNLOAD FROM GitHub*

*GitHub only hosts files max 50 Mb.

Guide to key directories and file names

  1. data/: raw data (e.g. unimputed SNP data)
  2. output/: outputs (e.g. imputed SNP data)
  3. analysis/: most code and workflow documented in .Rmd files
  4. docs/: compiled .html, “knitted” from .Rmd
  5. code/: supporting functions sourced in analysis/*.Rmd’s.

FILES OF INTEREST: everything is in the output/ sub-directory (click here and select “Guest” credentials when prompted by the Cassavabase FTP server).

genomicPredictions_ModelDirDom.csv genomicMatePredictions_top121parents_ModelDirDom.csv genomicPredictions_ModelAD.csv genomicMatePredictions_top121parents_ModelAD.csv

Analyses To Do

  1. [In Progress] PHG imputed and phased marker data

  2. Cross-variance prediction is slow, but significant speed gains can be made by using fewer markers for the predictions. Examine the speed benefit vs. accuracy cost trade-off.

  3. Improve mate selection accuracy by…

    • Choice of genetic map? AWC’s map? Sex specific maps?
    • Multi-trait REML models, single-step, heterogeneous error and/or of the other usual possibilities,
    • and/or a return to the compute intense Bayesian posterior distribution approach used in the MS… might be more attractive with fewer markers to make compute faster

  4. Simulation to explore factors impacting estimate of accuracy

    • Impact of phasing switch errors
    • Quality and quantity of data available on parents
    • Relatedness of parents
    • What is the relationship between the true accuracy and estimate of accuracy?