Last updated: 2026-08-07

Checks: 6 1

Knit directory: coderepo-local/

This reproducible R Markdown analysis was created with workflowr (version 1.7.2). The Checks tab describes the reproducibility checks that were applied when the results were created. The Past versions tab lists the development history.


The R Markdown is untracked by Git. To know which version of the R Markdown file created these results, you’ll want to first commit it to the Git repo. If you’re still working on the analysis, you can ignore this warning. When you’re finished, you can run wflow_publish to commit the R Markdown file and build the HTML.

Great job! The global environment was empty. Objects defined in the global environment can affect the analysis in your R Markdown file in unknown ways. For reproduciblity it’s best to always run the code in an empty environment.

The command set.seed(20251109) was run prior to running the code in the R Markdown file. Setting a seed ensures that any results that rely on randomness, e.g. subsampling or permutations, are reproducible.

Great job! Recording the operating system, R version, and package versions is critical for reproducibility.

Nice! There were no cached chunks for this analysis, so you can be confident that you successfully produced the results during this run.

Great job! Using relative paths to the files within your workflowr project makes it easier to run your code on other machines.

Great! You are using Git for version control. Tracking code development and connecting the code version to the results is critical for reproducibility.

The results in this page were generated with repository version 9fbd863. See the Past versions tab to see a history of the changes made to the R Markdown and HTML files.

Note that you need to be careful to ensure that all relevant files for the analysis have been committed to Git prior to generating the results (you can use wflow_publish or wflow_git_commit). workflowr only checks the R Markdown file, but you know if there are other scripts or data files that it depends on. Below is the status of the Git repository when the results were generated:


Ignored files:
    Ignored:    .DS_Store
    Ignored:    .Rhistory
    Ignored:    .Rproj.user/
    Ignored:    analysis/.DS_Store
    Ignored:    analysis/.Rhistory
    Ignored:    code/.DS_Store
    Ignored:    code/.Rhistory
    Ignored:    data/appendixB/
    Ignored:    data/dynamic_eQTL_real/
    Ignored:    data/toy_example/
    Ignored:    output/dynamic_eQTL_real/
    Ignored:    output/pdf/
    Ignored:    output/revision_simulations/

Untracked files:
    Untracked:  Rplots.pdf
    Untracked:  analysis/revision_balanced_variant_thinning.rmd
    Untracked:  analysis/revision_combined_spiky_genotype_simulation.rmd
    Untracked:  analysis/revision_correlated_error_simulation.rmd
    Untracked:  analysis/revision_functional_testing_simulation.rmd
    Untracked:  analysis/revision_genotype_level_simulation.rmd
    Untracked:  analysis/revision_internal_observational_null_set_pilot.rmd
    Untracked:  analysis/revision_internal_one_variant_per_gene_refit.rmd
    Untracked:  analysis/revision_internal_real_genotype_simulation.rmd
    Untracked:  analysis/revision_internal_targeted_broad_calibration.rmd
    Untracked:  analysis/revision_internal_zero_intercept_correlation.rmd
    Untracked:  analysis/revision_ld_tagging_switch_example.rmd
    Untracked:  code/00_eQTLs_CL.R
    Untracked:  code/01_fash_CL.R
    Untracked:  code/01_fash_CL.sbatch
    Untracked:  code/09_penalty_sensitivity.R
    Untracked:  code/09_penalty_sensitivity.sbatch
    Untracked:  code/revision_simulations/
    Untracked:  fash_prior_comparison_order1_after.pdf
    Untracked:  fash_prior_comparison_order1_after.png
    Untracked:  fash_prior_comparison_order1_before.pdf
    Untracked:  fash_prior_comparison_order1_before.png
    Untracked:  fash_prior_comparison_order2_after.pdf
    Untracked:  fash_prior_comparison_order2_after.png
    Untracked:  fash_prior_comparison_order2_before.pdf
    Untracked:  fash_prior_comparison_order2_before.png

Unstaged changes:
    Modified:   .gitignore
    Modified:   analysis/dynamic_eQTL_real.rmd
    Modified:   analysis/index.Rmd
    Modified:   analysis/nonlinear_dynamic_eQTL_real.Rmd
    Modified:   code/README.md
    Modified:   output/toy_example/toy_example_11.pdf
    Modified:   output/toy_example/toy_example_12.pdf
    Modified:   output/toy_example/toy_example_21.pdf
    Modified:   output/toy_example/toy_example_22.pdf

Note that any generated files, e.g. HTML, png, CSS, etc., are not included in this status report because it is ok for generated content to have uncommitted changes.


There are no past versions. Publish this analysis with wflow_publish() to start tracking its development.


Overview

This simulation evaluates FASH functional testing after global dynamic-eQTL discovery. It uses two non-IWP truth mechanisms matched on sample size, variant classes, genetic main effects, functional labels, and analysis.

For donor \(i\), variant \(j\), and time \(t\),

\[ Y_{ijt} = \alpha_{jt} + G_{ij}\beta_j(t) + X_i^T\gamma_{jt} + \epsilon_{ijt}, \qquad \epsilon_{ijt}\sim N(0,1). \]

For each variant, the minor-allele frequency is drawn from \(p_j\sim\mathrm{Uniform}(0.1,0.5)\) and \(G_{ij}\sim\mathrm{Binomial}(2,p_j)\); monomorphic draws are repeated. The five columns of \(X\) are generated independently from \(N(0,1)\) and standardized across donors. Independently for each covariate, variant, and time, \(\gamma_{kjt}\sim N(0,0.5^2)\). The current simulation sets \(\alpha_{jt}=0\); each fitted regression still includes an intercept. Genotype, covariate, covariate-effect, and noise draws are mutually independent.

seed <- 12345
component_seeds <- revision_component_seeds(seed)

genotype_sim <- simulate_genotype_matrix(
  n_donors = 19,
  n_variants = 1000,
  maf_range = c(0.1, 0.5),
  seed = component_seeds[["genotype"]]
)
covariates <- simulate_covariate_matrix(
  n_donors = 19,
  n_covariates = 5,
  seed = component_seeds[["covariates"]]
)

Each replicate contains 19 donors, 16 time points, five PC-like covariates, 200 dynamic eQTLs, 400 constant eQTLs, and 400 zero-effect variants. For every dynamic candidate,

\[ \beta_j(t)=\mu_j+h_j(t), \qquad \mu_j\sim N(0,1). \]

The dynamic eQTLs are balanced as evenly as possible across six preassigned truth cells: early, middle, or late crossed with switch or non-switch. Each candidate draws a new \(\mu_j\) from \(N(0,1)\); rejection sampling retains it only when the final trajectory \(\mu_j+h_j(t)\) satisfies its assigned cell.

R3A: broad random B-spline effects

The deviation \(h_j(t)\) is a cubic B-spline with six iid \(N(0,1)\) coefficients. It is centered over the 16 observed times and scaled to maximum absolute value 2 before adding \(\mu_j\).

R3B: compact raised-cosine effects

The deviation \(h_j(t)\) contains one, two, or three compact raised-cosine peaks with half-width 1.5. Peak locations vary within the early, middle, and late windows, secondary amplitudes are random, and the deviation is scaled to centered RMS 0.9 before adding \(\mu_j\).

Truth definitions

Early, middle, and late use the same max-contrast functionals as the manuscript. The target contrast must be at least 0.1, and the two non-target contrasts must be at most -0.1.

For the switch threshold \(c=0.25\),

\[ T_{\mathrm{switch}}\{\beta\} = \min\left[ \max_t\{\beta(t):\beta(t)>0\}, \max_t\{|\beta(t)|:\beta(t)<0\} \right]-c. \]

A switch truth must have \(T_{\mathrm{switch}}\geq 0.1\); equivalently, it has both a positive and a negative effect of magnitude at least 0.35. A non-switch truth remains the same sign over the full dense grid. Writing \(R_j=\max_t\beta_j(t)-\min_t\beta_j(t)\), its clearance from zero must satisfy

\[ \min_t|\beta_j(t)|\geq\max(0.1,\,0.1R_j). \]

Thus, a larger-amplitude non-switch curve must remain proportionally farther from zero. These margins make the simulation labels unambiguous.

truth_mechanism <- "random_bspline" # Use "raised_cosine" for R3B.
scenario <- paste0(
  if (truth_mechanism == "random_bspline") "r3a_" else "r3b_",
  "matched_functional_",
  truth_mechanism,
  "_relative_clearance_main_effect"
)
time_grid <- 0:15
evaluation_grid <- seq(0, 15, by = 0.1)
class_probs <- c(
  dynamic_bspline = 0.20,
  constant = 0.40,
  zero = 0.40
)

effect_sim <- simulate_matched_functional_effect_set(
  n_variants = 1000,
  truth_mechanism = truth_mechanism,
  time_grid = time_grid,
  evaluation_grid = evaluation_grid,
  class_probs = class_probs,
  dynamic_main_effect_sd = 1,
  constant_sd = 1,
  bspline_amplitude = 2,
  bspline_df = 6,
  bspline_coefficient_sd = 1,
  cosine_width_half = 1.5,
  cosine_spike_counts = 1:3,
  cosine_relative_amplitude_range = c(0.35, 0.75),
  cosine_target_centered_rms = 0.9,
  switch_threshold = 0.25,
  location_truth_margin = 0.10,
  switch_truth_margin = 0.10,
  non_switch_min_abs = 0.10,
  non_switch_min_range_fraction = 0.10,
  exact_class_counts = TRUE,
  seed = seed,
  class_seed = component_seeds[["classes"]],
  constant_seed = component_seeds[["constant_effects"]],
  shape_seed = component_seeds[["functional_truth"]],
  scenario = scenario
)
expression_sim <- simulate_eqtl_expression_from_genotypes(
  G = genotype_sim$G,
  beta_matrix = effect_sim$beta_matrix,
  time_grid = time_grid,
  covariates = covariates,
  expression_noise_sd = 1,
  covariate_effect_sd = 0.5,
  intercept_sd = 0,
  seed = component_seeds[["expression"]]
)

At each observed time, the genetic effect and standard error are estimated from Y ~ G + PC1 + PC2 + PC3 + PC4 + PC5. The t-statistic correction uses df = 19 - 7 = 12; FASH is fit to these estimates and corrected standard errors.

Example truth curves

Two examples are shown for each of the six functional truth cells. Points and bars are the per-time estimates and two corrected standard errors used by FASH; orange lines are the continuous truths. Shading marks the target timing window.

R3A examples

plot_truth_examples(
  example_curves$random_bspline,
  mechanism_labels[["random_bspline"]]
)
Figure R3.1: Representative broad random B-spline truth curves for each functional target and timing cell. Points and bars show estimated effects and two corrected standard errors, orange lines are continuous truths, and shading marks the target timing window.

Figure R3.1: Representative broad random B-spline truth curves for each functional target and timing cell. Points and bars show estimated effects and two corrected standard errors, orange lines are continuous truths, and shading marks the target timing window.

R3B examples

plot_truth_examples(
  example_curves$raised_cosine,
  mechanism_labels[["raised_cosine"]]
)
Figure R3.2: Representative compact raised-cosine truth curves for each functional target and timing cell. Points and bars show estimated effects and two corrected standard errors, orange lines are continuous truths, and shading marks the target timing window.

Figure R3.2: Representative compact raised-cosine truth curves for each functional target and timing cell. Points and bars show estimated effects and two corrected standard errors, orange lines are continuous truths, and shading marks the target timing window.

Inference procedure

At each nominal level \(\alpha\), the same procedure is applied to FASH-IWP1-Raw and FASH-IWP1-BF:

  1. Apply global dynamic-eQTL FDR control at level \(\alpha\).
  2. For each dynamically selected variant, evaluate the early, middle, late, and switch functionals using posterior samples.
  3. Rank functional lfsr values and call variants whose cumulative FSR is at most \(\alpha\).

Power is the fraction of true positive functionals that are called. Empirical FSR is the fraction of functional calls whose known true functional is non-positive. The figures report five-seed Monte Carlo means and 95% t-based intervals across the full alpha grid.

out <- run_genotype_level_dynamic_eqtl_simulation(
  G = genotype_sim$G,
  time_grid = time_grid,
  covariates = covariates,
  class_probs = class_probs,
  maf_range = c(0.1, 0.5),
  expression_noise_sd = 1,
  covariate_effect_sd = 0.5,
  intercept_sd = 0,
  dynamic_main_effect_sd = 1,
  alpha = 0.05,
  seed = seed,
  num_cores = 4,
  num_basis = 20,
  scenario = scenario,
  save_outputs = FALSE,
  effect_sim = effect_sim,
  expression_sim = expression_sim
)

true_functionals <- effect_sim$true_functionals[
  out$unit_info$variant_id,
  c("early", "middle", "late", "switch"),
  drop = FALSE
]
true_dynamic <- out$unit_info$effect_class == "dynamic_bspline"

raw_results <- evaluate_fash_functional_testing(
  fit = out$fash_fits$fash_iwp1_raw,
  true_functionals = true_functionals,
  evaluation_grid = evaluation_grid,
  alpha_grid = seq(0.005, 0.20, by = 0.005),
  method = "FASH-IWP1-Raw",
  switch_threshold = 0.25,
  true_dynamic = true_dynamic,
  num_cores = 4,
  seed = component_seeds[["functional_posterior"]]
)
bf_results <- evaluate_fash_functional_testing(
  fit = out$fash_fits$fash_iwp1_bf,
  true_functionals = true_functionals,
  evaluation_grid = evaluation_grid,
  alpha_grid = seq(0.005, 0.20, by = 0.005),
  method = "FASH-IWP1-BF",
  switch_threshold = 0.25,
  true_dynamic = true_dynamic,
  num_cores = 4,
  seed = component_seeds[["functional_posterior"]] + 100L
)

R3A: broad random B-spline results

Results at alpha 0.05

knitr::kable(
  r3a_table,
  align = "l",
  caption = paste(
    "Table R3.1: Functional-testing power and empirical false sign rate at",
    "alpha 0.05 under broad random B-spline truth."
  )
)
Table R3.1: Functional-testing power and empirical false sign rate at alpha 0.05 under broad random B-spline truth.
Target Method Mean calls Power (95% MC CI) Empirical FSR (95% MC CI)
Early FASH-IWP1-Raw 14.2 0.212 [0.089, 0.334] 0.000 [0.000, 0.000]
Early FASH-IWP1-BF 12.0 0.179 [0.086, 0.273] 0.000 [0.000, 0.000]
Middle FASH-IWP1-Raw 40.6 0.600 [0.480, 0.720] 0.009 [0.000, 0.024]
Middle FASH-IWP1-BF 39.6 0.585 [0.468, 0.702] 0.009 [0.000, 0.024]
Late FASH-IWP1-Raw 16.8 0.255 [0.156, 0.354] 0.000 [0.000, 0.000]
Late FASH-IWP1-BF 15.4 0.233 [0.155, 0.312] 0.000 [0.000, 0.000]
Switch FASH-IWP1-Raw 93.4 0.891 [0.820, 0.962] 0.036 [0.017, 0.056]
Switch FASH-IWP1-BF 88.4 0.857 [0.791, 0.923] 0.020 [0.000, 0.044]

Power as a function of alpha

plot_mc_functional_curve_grid(
  mc_curve = r3a_alpha,
  metric = "power",
  title = "Functional-testing power: broad random B-spline truth"
)
Figure R3.3: Monte Carlo functional-testing power under broad random B-spline truth. Panels identify early, middle, late, and switch targets; curves identify raw and BF-updated FASH in the in-figure legend, with 95% intervals across five seeds.

Figure R3.3: Monte Carlo functional-testing power under broad random B-spline truth. Panels identify early, middle, late, and switch targets; curves identify raw and BF-updated FASH in the in-figure legend, with 95% intervals across five seeds.

Empirical FSR as a function of alpha

plot_mc_functional_curve_grid(
  mc_curve = r3a_alpha,
  metric = "empirical_fsr",
  title = "Empirical FSR: broad random B-spline truth"
)
Figure R3.4: Monte Carlo empirical false sign rate under broad random B-spline truth. Panels identify functional targets, curves identify raw and BF-updated FASH, and the diagonal reference denotes nominal alpha.

Figure R3.4: Monte Carlo empirical false sign rate under broad random B-spline truth. Panels identify functional targets, curves identify raw and BF-updated FASH, and the diagonal reference denotes nominal alpha.

R3B: compact raised-cosine results

Results at alpha 0.05

knitr::kable(
  r3b_table,
  align = "l",
  caption = paste(
    "Table R3.2: Functional-testing power and empirical false sign rate at",
    "alpha 0.05 under compact raised-cosine truth."
  )
)
Table R3.2: Functional-testing power and empirical false sign rate at alpha 0.05 under compact raised-cosine truth.
Target Method Mean calls Power (95% MC CI) Empirical FSR (95% MC CI)
Early FASH-IWP1-Raw 2.0 0.030 [0.001, 0.059] 0.000 [0.000, 0.000]
Early FASH-IWP1-BF 1.6 0.024 [0.003, 0.045] 0.000 [0.000, 0.000]
Middle FASH-IWP1-Raw 5.0 0.072 [0.047, 0.096] 0.040 [0.000, 0.151]
Middle FASH-IWP1-BF 4.0 0.060 [0.033, 0.086] 0.000 [0.000, 0.000]
Late FASH-IWP1-Raw 1.0 0.015 [0.000, 0.034] 0.000 [0.000, 0.000]
Late FASH-IWP1-BF 0.6 0.009 [0.000, 0.026] 0.000 [0.000, 0.000]
Switch FASH-IWP1-Raw 65.0 0.620 [0.545, 0.694] 0.037 [0.014, 0.060]
Switch FASH-IWP1-BF 52.4 0.511 [0.430, 0.592] 0.015 [0.004, 0.026]

Power as a function of alpha

plot_mc_functional_curve_grid(
  mc_curve = r3b_alpha,
  metric = "power",
  title = "Functional-testing power: compact raised-cosine truth"
)
Figure R3.5: Monte Carlo functional-testing power under compact raised-cosine truth. Panels identify early, middle, late, and switch targets; curves identify raw and BF-updated FASH in the in-figure legend, with 95% intervals across five seeds.

Figure R3.5: Monte Carlo functional-testing power under compact raised-cosine truth. Panels identify early, middle, late, and switch targets; curves identify raw and BF-updated FASH in the in-figure legend, with 95% intervals across five seeds.

Empirical FSR as a function of alpha

plot_mc_functional_curve_grid(
  mc_curve = r3b_alpha,
  metric = "empirical_fsr",
  title = "Empirical FSR: compact raised-cosine truth"
)
Figure R3.6: Monte Carlo empirical false sign rate under compact raised-cosine truth. Panels identify functional targets, curves identify raw and BF-updated FASH, and the diagonal reference denotes nominal alpha.

Figure R3.6: Monte Carlo empirical false sign rate under compact raised-cosine truth. Panels identify functional targets, curves identify raw and BF-updated FASH, and the diagonal reference denotes nominal alpha.

Dynamic-null proportion

The first-stage dynamic-null proportion is \(\pi_0=0.8\). The table reports the fitted constant-component weights for the raw and BF-adjusted IWP fits.

knitr::kable(
  pi0_table,
  align = "l",
  caption = paste(
    "Table R3.3: Estimated first-stage dynamic-null proportions for raw and",
    "BF-updated IWP FASH under both functional truth mechanisms."
  )
)
Table R3.3: Estimated first-stage dynamic-null proportions for raw and BF-updated IWP FASH under both functional truth mechanisms.
Mechanism Fit Mean estimated pi0 (95% MC CI)
R3A: broad random B-spline Raw 0.800 [0.797, 0.803]
R3A: broad random B-spline BF-corrected 0.846 [0.842, 0.850]
R3B: compact raised cosine Raw 0.801 [0.755, 0.847]
R3B: compact raised cosine BF-corrected 0.914 [0.910, 0.918]

Interpretation

For R3A, BF-updated FASH has mean empirical FSR 0.020 or lower for all four targets at \(\alpha=0.05\). Its switch power is 0.857.

For the more localized R3B trajectories, BF-updated FASH makes few timing calls and no false timing calls in these five replicates. Its switch power is 0.511, but its mean switch FSR is 0.015 at nominal \(\alpha=0.05\). In this five-seed pilot, the scale-relative clearance condition separates switch and non-switch truths in both mechanisms without removing the compact transient behavior. These results support functional testing for the stated margin-separated designs; they do not imply that BF adjustment alone guarantees FSR control for every functional hypothesis or every truth geometry.

Reproducibility

The R3 simulation driver uses the shared simulation functions. The page-specific cache validation and formatting are in R3 reporting code. The following command regenerates the paired five-seed cache. It is shown for reproducibility but is intentionally not evaluated during workflowr builds; the table and figure chunks above read the saved cache.

The paired five-seed cache can be regenerated with:

Rscript --vanilla \
  code/revision_simulations/r3_functional_testing/run_matched_functional_testing_mc_replication.R \
  --J 1000 \
  --n-donors 19 \
  --n-covariates 5 \
  --noise-sd 1 \
  --dynamic-main-effect-sd 1 \
  --truth-mechanisms random_bspline,raised_cosine \
  --seed-list 12345,22345,32345,42345,52345 \
  --output-id r3_matched_functional_relative_clearance_main_effect_pilot5 \
  --num-cores 4

Cached replicate metrics, call-level diagnostics, examples, summaries, and figures are stored under:

output/revision_simulations/mc/r3_matched_functional_relative_clearance_main_effect_pilot5/

sessionInfo()
R version 4.5.1 (2025-06-13)
Platform: aarch64-apple-darwin20
Running under: macOS Sequoia 15.6.1

Matrix products: default
BLAS:   /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib 
LAPACK: /Library/Frameworks/R.framework/Versions/4.5-arm64/Resources/lib/libRlapack.dylib;  LAPACK version 3.12.1

locale:
[1] C.UTF-8/C.UTF-8/C.UTF-8/C/C.UTF-8/C.UTF-8

time zone: America/Chicago
tzcode source: internal

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] workflowr_1.7.2

loaded via a namespace (and not attached):
 [1] vctrs_0.7.3       httr_1.4.7        cli_3.6.6         knitr_1.50       
 [5] rlang_1.2.0       xfun_0.55         stringi_1.8.7     otel_0.2.0       
 [9] processx_3.8.6    promises_1.5.0    jsonlite_2.0.0    glue_1.8.1       
[13] rprojroot_2.1.1   git2r_0.36.2      htmltools_0.5.9   httpuv_1.6.16    
[17] ps_1.9.1          sass_0.4.10       rmarkdown_2.30    jquerylib_0.1.4  
[21] tibble_3.3.0      evaluate_1.0.5    fastmap_1.2.0     yaml_2.3.12      
[25] lifecycle_1.0.5   whisker_0.4.1     stringr_1.6.0     compiler_4.5.1   
[29] fs_1.6.6          pkgconfig_2.0.3   Rcpp_1.1.1-1.1    rstudioapi_0.17.1
[33] later_1.4.4       digest_0.6.39     R6_2.6.1          pillar_1.11.1    
[37] callr_3.7.6       magrittr_2.0.5    bslib_0.9.0       tools_4.5.1      
[41] cachem_1.1.0      getPass_0.2-4