Background

I was interested in whether we can try to differentiate the flash likelihood at the “null” ($g_l=g_f=0$).

Let n=p=1. And Y (1 by 1 matrix) be given by factor model $$Y = lf + e$$ where $l,f \sim N(0,s^2)$ and $e \sim N(0,1)$.

Integrating out $l$ and $f$ yields a likelihood for $s$: $$L(s) = \int p(Y| l, f) p(l) p(f) dl df$$

My question is what is the derivative of the likelihood (or log likelihood) with respect to $s$. Particularly what is it at $s=0$ (which is the 0-factor model)?

If we define $h$ to be the log-likelihood for $l,f$ $$h(l,f) = \log p(Y|l,f)$$ then is easy to show that

$$d^2h/dl^2 = f^2/s^2$$ and $$d^2h/dldf = -(Y-2lf)$$

so at $l=f=0$ the hessian is the matrix with 0 on diagonal and $-Y$ on off-diagonal. This matrix has eigenvalues $\pm Y$.

This suggests that the posterior distribution of $(l,f)$ will be approximated by a multivariate normal with inverse covariance matrix with $1/s^2$ on diagonal and $Y$ on off-diagonal. Let’s try with s=0.1.

l = rnorm(100000,0,0.1)
f = rnorm(100000,0,0.1)
y = 2
Lik = dnorm((y-l*f),0,1)
ss = sample(1:100000,prob=Lik,replace=TRUE)
plot(l[ss],f[ss])

So we see the problem - indeed the posterior is approximated by this, but it looks just like the prior… So the ratio of the posterior to the prior will be 1.

sessionInfo()

R version 3.3.2 (2016-10-31)
Platform: x86_64-apple-darwin13.4.0 (64-bit)
Running under: OS X El Capitan 10.11.6

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

loaded via a namespace (and not attached):
 [1] Rcpp_0.12.11      rstudioapi_0.6    knitr_1.15.1     
 [4] magrittr_1.5      workflowr_0.4.0   MASS_7.3-47      
 [7] doParallel_1.0.10 pscl_1.4.9        SQUAREM_2016.8-2 
[10] lattice_0.20-35   foreach_1.4.3     ashr_2.1-19      
[13] stringr_1.2.0     tools_3.3.2       parallel_3.3.2   
[16] grid_3.3.2        git2r_0.18.0      htmltools_0.3.6  
[19] iterators_1.0.8   yaml_2.1.14       rprojroot_1.2    
[22] digest_0.6.12     Matrix_1.2-10     codetools_0.2-15 
[25] evaluate_0.10     rmarkdown_1.5     stringi_1.1.5    
[28] backports_1.0.5   truncnorm_1.0-7