Last updated: 2020-05-28

Checks: 7 0

Knit directory: mr_mash_test/

This reproducible R Markdown analysis was created with workflowr (version 1.6.1). The Checks tab describes the reproducibility checks that were applied when the results were created. The Past versions tab lists the development history.

R Markdown file: up-to-date

Great! Since the R Markdown file has been committed to the Git repository, you know the exact version of the code that produced these results.

Environment: empty

Great job! The global environment was empty. Objects defined in the global environment can affect the analysis in your R Markdown file in unknown ways. For reproduciblity it’s best to always run the code in an empty environment.

Seed: set.seed(20200328)

The command set.seed(20200328) was run prior to running the code in the R Markdown file. Setting a seed ensures that any results that rely on randomness, e.g. subsampling or permutations, are reproducible.

Session information: recorded

Great job! Recording the operating system, R version, and package versions is critical for reproducibility.

Cache: none

Nice! There were no cached chunks for this analysis, so you can be confident that you successfully produced the results during this run.

File paths: relative

Great job! Using relative paths to the files within your workflowr project makes it easier to run your code on other machines.

Repository version: 738798c

Great! You are using Git for version control. Tracking code development and connecting the code version to the results is critical for reproducibility.

The results in this page were generated with repository version 738798c. See the Past versions tab to see a history of the changes made to the R Markdown and HTML files.

Note that you need to be careful to ensure that all relevant files for the analysis have been committed to Git prior to generating the results (you can use wflow_publish or wflow_git_commit). workflowr only checks the R Markdown file, but you know if there are other scripts or data files that it depends on. Below is the status of the Git repository when the results were generated: 


Ignored files:
    Ignored:    .sos/
    Ignored:    code/fit_mr_mash.66662433.err
    Ignored:    code/fit_mr_mash.66662433.out
    Ignored:    dsc/.sos/
    Ignored:    dsc/outfiles/
    Ignored:    output/dsc.html
    Ignored:    output/dsc/
    Ignored:    output/dsc_05_18_20.html
    Ignored:    output/dsc_05_18_20/
    Ignored:    output/dsc_05_24_20.html
    Ignored:    output/dsc_05_24_20/
    Ignored:    output/dsc_OLD.html
    Ignored:    output/dsc_OLD/
    Ignored:    output/dsc_test.html
    Ignored:    output/dsc_test/

Untracked files:
    Untracked:  Rplots.pdf
    Untracked:  code/plot_test.R

Unstaged changes:
    Modified:   dsc/midway2.yml

Note that any generated files, e.g. HTML, png, CSS, etc., are not included in this status report because it is ok for generated content to have uncommitted changes.

These are the previous versions of the repository in which changes were made to the R Markdown (analysis/two_active_responses_issue.Rmd) and HTML (docs/two_active_responses_issue.html) files. If you’ve configured a remote Git repository (see ?wflow_git_remote), click on the hyperlinks in the table below to view the files as they were in that past version.

File Version Author Date Message
Rmd 738798c fmorgante 2020-05-28 Add more investigations
html 5e13912 fmorgante 2020-05-27 Build site.
Rmd 068beb2 fmorgante 2020-05-27 Increase cex
html f2b6bf8 fmorgante 2020-05-27 Build site.
Rmd a626e53 fmorgante 2020-05-27 Increase cex
html 7dad46c fmorgante 2020-05-27 Build site.
Rmd 465b10d fmorgante 2020-05-27 Add case study with only two causal tissues out of five 
library(mr.mash.alpha)
library(glmnet)
Loading required package: Matrix
Loading required package: foreach
Loaded glmnet 2.0-16
###Set options
options(stringsAsFactors = FALSE)

Let’s simulate data with n=600, p=1,000, p_causal=50, r=5, r_causal=2, PVE=0.5, shared effects, independent predictors, and lowly correlated residuals.

###Set seed
set.seed(123)

###Set parameters
n <- 600
p <- 1000
p_causal <- 50
r <- 5
r_causal <- 2

###Simulate V, B, X and Y
out <- mr.mash.alpha:::simulate_mr_mash_data(n, p, p_causal, r, r_causal, intercepts = rep(1, r),
                                             pve=0.5, B_cor=1, B_scale=0.8, X_cor=0, X_scale=0.8, V_cor=0.15)

colnames(out$Y) <- paste0("Y", seq(1, r))
rownames(out$Y) <- paste0("N", seq(1, n))
colnames(out$X) <- paste0("X", seq(1, p))
rownames(out$X) <- paste0("N", seq(1, n))

###Split the data in training and test sets
test_set <- sort(sample(x=c(1:n), size=round(n*0.2), replace=FALSE))
Ytrain <- out$Y[-test_set, ]
Xtrain <- out$X[-test_set, ]
Ytest <- out$Y[test_set, ]
Xtest <- out$X[test_set, ]

head(out$B[out$causal_variables, ])
            [,1]        [,2] [,3] [,4] [,5]
[1,]  0.07316118  0.07316118    0    0    0
[2,]  0.44056603  0.44056603    0    0    0
[3,]  0.31699557  0.31699557    0    0    0
[4,] -0.95215593 -0.95215593    0    0    0
[5,]  0.07038939  0.07038939    0    0    0
[6,] -0.92892961 -0.92892961    0    0    0

We build the mixture prior as usual including zero matrix, identity matrix, rank-1 matrices, low, medium and high heterogeneity matrices, shared matrix, each scaled by a grid from 0.1 to 2.1 in steps of 0.2.

grid <- seq(0.1, 2.1, 0.2)
S0 <- mr.mash.alpha:::compute_cov_canonical(ncol(Ytrain), singletons=TRUE, hetgrid=c(0, 0.25, 0.5, 0.75, 0.99), grid, zeromat=TRUE)

We run glmnet with \(\alpha=1\) to obtain an inital estimate for the regression coefficients to provide to mr.mash, and for comparison.

###Fit grop-lasso to initialize mr.mash
cvfit_glmnet <- cv.glmnet(x=Xtrain, y=Ytrain, family="mgaussian", alpha=1)
coeff_glmnet <- coef(cvfit_glmnet, s="lambda.min")
Bhat_glmnet <- matrix(as.numeric(NA), nrow=p, ncol=r)
for(i in 1:length(coeff_glmnet)){
  Bhat_glmnet[, i] <- as.vector(coeff_glmnet[[i]])[-1]
}

We run mr.mash with EM updates of the mixture weights, updating V, and initializing the regression coefficients with the estimates from glmnet.

###Fit mr.mash
fit_mrmash <- mr.mash(Xtrain, Ytrain, S0, update_w0=TRUE, update_w0_method="EM", 
                    compute_ELBO=TRUE, standardize=TRUE, verbose=FALSE, update_V=TRUE,
                    e=1e-8, ca_update_order="consecutive", mu1_init=Bhat_glmnet)
Processing the inputs... Done!
Fitting the optimization algorithm... Done!
Processing the outputs... Done!
mr.mash successfully executed in 8.703947 minutes!

Let’s now compare the results.

layout(matrix(c(1, 1, 2, 2,
                1, 1, 2, 2,
                0, 3, 3, 0,
                0, 3, 3, 0), 4, 4, byrow = TRUE))

###Plot estimated vs true coeffcients
##mr.mash
plot(out$B, fit_mrmash$mu1, main="mr.mash", xlab="True coefficients", ylab="Estimated coefficients",
     cex=2, cex.lab=2, cex.main=2, cex.axis=2)
##glmnet
plot(out$B, Bhat_glmnet, main="glmnet", xlab="True coefficients", ylab="Estimated coefficients",
     cex=2, cex.lab=2, cex.main=2, cex.axis=2)

###Plot mr.mash vs glmnet estimated coeffcients
colorz <- matrix("black", nrow=p, ncol=r)
zeros <- apply(out$B, 2, function(x) x==0)
for(i in 1:ncol(colorz)){
  colorz[zeros[, i], i] <- "red"
}

plot(Bhat_glmnet, fit_mrmash$mu1, main="mr.mash vs glmnet", 
     xlab="glmnet estimated coefficients", ylab="mr.mash estimated coefficients",
     col=colorz, cex=2, cex.lab=2, cex.main=2, cex.axis=2)
legend("topleft", 
       legend = c("Non-zero", "Zero"), 
       col = c("black", "red"), 
       pch = c(1, 1), 
       horiz = FALSE,
       cex=2)

Version Author Date
5e13912 fmorgante 2020-05-27
f2b6bf8 fmorgante 2020-05-27
7dad46c fmorgante 2020-05-27

As we can see, mr.mash performs pretty poorly. One possibility is that the prior used cannot capture the pattern of sharing present only among the first two tissues. Let’s now test this hypothesis by adding the correspondent prior matrices to the mixture and re-run mr.mash.

###Update the mixture prior
K <- length(S0)
matr <- matrix(c(1, 1, 0, 0, 0,
                 1, 1, 0, 0, 0,
                 0, 0, 0, 0, 0,
                 0, 0, 0, 0, 0,
                 0, 0, 0, 0, 0), 5, 5, byrow=T)

S0_1 <- S0
for(i in 1:length(grid)){
  S0_1[[K+i]] <- matr * grid[i]
}
print(paste("Length of the original mixture:", K))
[1] "Length of the original mixture: 111"
print(paste("Length of the updated mixture:", length(S0_1)))
[1] "Length of the updated mixture: 122"
###Fit mr.mash
fit_mrmash_updated <- mr.mash(Xtrain, Ytrain, S0_1, update_w0=TRUE, update_w0_method="EM", 
                              compute_ELBO=TRUE, standardize=TRUE, verbose=FALSE, update_V=TRUE,
                              e=1e-8, ca_update_order="consecutive", mu1_init=Bhat_glmnet)
Processing the inputs... Done!
Fitting the optimization algorithm... Done!
Processing the outputs... Done!
mr.mash successfully executed in 5.42335 minutes!

Let’s compare the results again.

layout(matrix(c(1, 1, 2, 2,
                1, 1, 2, 2,
                0, 3, 3, 0,
                0, 3, 3, 0), 4, 4, byrow = TRUE))

###Plot estimated vs true coeffcients
##mr.mash
plot(out$B, fit_mrmash_updated$mu1, main="mr.mash", xlab="True coefficients", ylab="Estimated coefficients", cex=2, cex.lab=2, cex.main=2, cex.axis=2)
##glmnet
plot(out$B, Bhat_glmnet, main="glmnet", xlab="True coefficients", ylab="Estimated coefficients",
     cex=2, cex.lab=2, cex.main=2, cex.axis=2)

###Plot mr.mash vs glmnet estimated coeffcients
colorz <- matrix("black", nrow=p, ncol=r)
zeros <- apply(out$B, 2, function(x) x==0)
for(i in 1:ncol(colorz)){
  colorz[zeros[, i], i] <- "red"
}

plot(Bhat_glmnet, fit_mrmash_updated$mu1, main="mr.mash vs glmnet", 
     xlab="glmnet estimated coefficients", ylab="mr.mash estimated coefficients",
     col=colorz, cex=2, cex.lab=2, cex.main=2, cex.axis=2)
legend("topleft", 
       legend = c("Non-zero", "Zero"), 
       col = c("black", "red"), 
       pch = c(1, 1), 
       horiz = FALSE,
       cex=2)

Version Author Date
5e13912 fmorgante 2020-05-27
7dad46c fmorgante 2020-05-27

Now the results look much better!

However, we all thought that the identity matrix (properly scaled) could be able to capture these kinds of effects. So, let’s test this intuition by initializing all the parameters (mu1, V, w0) to the true values and provide a 2-component prior with the zero matrix and an identity scaled by the true variance (i.e., 0.8).

###Update the mixture prior
zeromat <- matrix(0, r, r)
diagmat <- diag(0.8, r)
S0_2 <- list(S0_1=diagmat, S0_2=zeromat)

###Fit mr.mash
fit_mrmash_diag <- mr.mash(Xtrain, Ytrain, S0_2, w0=c((p_causal/p), (1-(p_causal/p))), V=out$V, mu1_init=out$B, 
                          update_w0=TRUE, update_w0_method="EM", compute_ELBO=TRUE, standardize=TRUE,
                          verbose=FALSE, update_V=TRUE, e=1e-8, ca_update_order="consecutive")
Processing the inputs... Done!
Fitting the optimization algorithm... Done!
Processing the outputs... Done!
mr.mash successfully executed in 0.01638254 minutes!
###Plot the results
plot(out$B, fit_mrmash_diag$mu1, main="mr.mash", xlab="True coefficients", ylab="Estimated coefficients", cex=2, cex.lab=2, cex.main=2, cex.axis=2)

Unfortunately, it does not work well, as we hoped. Let’s now try to fix w0 and V instead of updating them.

###Fit mr.mash
fit_mrmash_diag_fix <- mr.mash(Xtrain, Ytrain, S0_2, w0=c((p_causal/p), (1-(p_causal/p))), V=out$V, 
                               mu1_init=out$B, update_w0=FALSE, update_w0_method="EM", compute_ELBO=TRUE, 
                               standardize=TRUE, verbose=FALSE, update_V=FALSE, e=1e-8, 
                               ca_update_order="consecutive")
Processing the inputs... Done!
Fitting the optimization algorithm... Done!
Processing the outputs... Done!
mr.mash successfully executed in 0.007988842 minutes!
###Plot the results
plot(out$B, fit_mrmash_diag_fix$mu1, main="mr.mash", xlab="True coefficients", ylab="Estimated coefficients", cex=2, cex.lab=2, cex.main=2, cex.axis=2)

Still not good. Now, we will try to replace the identity matrix with the true covariance among effects, and update all the model parameters.

###Update the mixture prior
zeromat <- matrix(0, r, r)
truecovmat <- matrix(c(0.8, 0.8, 0, 0, 0,
                       0.8, 0.8, 0, 0, 0,
                       0, 0, 0, 0, 0,
                       0, 0, 0, 0, 0,
                       0, 0, 0, 0, 0), 5, 5, byrow=T)
S0_3 <- list(S0_1=truecovmat, S0_2=zeromat)

###Fit mr.mash
fit_mrmash_truecov <- mr.mash(Xtrain, Ytrain, S0_3, w0=c((p_causal/p), (1-(p_causal/p))), V=out$V, mu1_init=out$B,
                          update_w0=TRUE, update_w0_method="EM", compute_ELBO=TRUE, standardize=TRUE,
                          verbose=FALSE, update_V=TRUE, e=1e-8, ca_update_order="consecutive")
Processing the inputs... Done!
Fitting the optimization algorithm... Done!
Processing the outputs... Done!
mr.mash successfully executed in 0.01350585 minutes!
###Plot the results
plot(out$B, fit_mrmash_truecov$mu1, main="mr.mash", xlab="True coefficients", ylab="Estimated coefficients", cex=2, cex.lab=2, cex.main=2, cex.axis=2)

Now it works well. Finally, let’s try not to update the model parameters.

###Fit mr.mash
fit_mrmash_truecov_fix <- mr.mash(Xtrain, Ytrain, S0_3, w0=c((p_causal/p), (1-(p_causal/p))), V=out$V, 
                                  mu1_init=out$B, update_w0=FALSE, update_w0_method="EM", compute_ELBO=TRUE, 
                                  standardize=TRUE, verbose=FALSE, update_V=FALSE, e=1e-8, 
                                  ca_update_order="consecutive")
Processing the inputs... Done!
Fitting the optimization algorithm... Done!
Processing the outputs... Done!
mr.mash successfully executed in 0.01819323 minutes!
###Plot the results
plot(out$B, fit_mrmash_truecov_fix$mu1, main="mr.mash", xlab="True coefficients", ylab="Estimated coefficients", cex=2, cex.lab=2, cex.main=2, cex.axis=2)

In summary, it looks like we need the proper prior to capture that kind of structure.


sessionInfo()
R version 3.5.1 (2018-07-02)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Scientific Linux 7.4 (Nitrogen)

Matrix products: default
BLAS/LAPACK: /software/openblas-0.2.19-el7-x86_64/lib/libopenblas_haswellp-r0.2.19.so

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C              
 [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8    
 [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                 
 [9] LC_ADDRESS=C               LC_TELEPHONE=C            
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] glmnet_2.0-16        foreach_1.4.4        Matrix_1.2-15       
[4] mr.mash.alpha_0.1-74

loaded via a namespace (and not attached):
 [1] MBSP_1.0           Rcpp_1.0.4.6       compiler_3.5.1    
 [4] later_0.7.5        git2r_0.26.1       workflowr_1.6.1   
 [7] iterators_1.0.10   tools_3.5.1        digest_0.6.25     
[10] evaluate_0.12      lattice_0.20-38    GIGrvg_0.5        
[13] yaml_2.2.1         SparseM_1.77       mvtnorm_1.0-12    
[16] coda_0.19-3        stringr_1.4.0      knitr_1.20        
[19] fs_1.3.1           MatrixModels_0.4-1 rprojroot_1.3-2   
[22] grid_3.5.1         glue_1.4.0         R6_2.4.1          
[25] rmarkdown_1.10     mixsqp_0.3-43      irlba_2.3.3       
[28] magrittr_1.5       whisker_0.3-2      codetools_0.2-15  
[31] backports_1.1.5    promises_1.0.1     htmltools_0.3.6   
[34] matrixStats_0.55.0 mcmc_0.9-6         MASS_7.3-51.1     
[37] httpuv_1.4.5       quantreg_5.36      stringi_1.4.3     
[40] MCMCpack_1.4-4