Finding candidate classes

Finding candidate latent classes: the graph enumeration and pruning algorithm

—Special considerations: this portion is likely a high-memory task!—

Now that we have our pairwise fits, we need to figure out what are the remaining candidate latent classes. These can be found using the path enumeration and pruning algorithm. This is a graph-based algorithm which can require a lot of memory (especially when the dimension of the data is large). However, with the memory available, this algorithm runs very quickly – in our experience, in under 5 minutes.

First, load in the previously obtained pairwise fits.

data("fits")

Then, we can obtain a reduced list of candidate latent classes with get_reduced_classes(). This function has 3 arguments:

1. the pairwise fits

2. the dimension of the data

3. the name of an output “LEMON graph format” file (here, called lgf.txt). This is not to be edited by the user, but is produced for underlying C software.

# This finds the dimension of the data directly from the pairwise fits
D <- as.numeric(strsplit(tail(names(fits),1), "_")[[1]][2])

# Get the list of candidate latent classes
red_class <- get_reduced_classes(fits, D, "output/lgf.txt")

Writing LGF file...done!
Finding latent classes...done!

# write the output to a text file
readr::write_tsv(data.frame(red_class), path = "output/red_class.txt", col_names = FALSE)

Each row of red_class corresponds to a candidate latent class across the 3 dimensions. The remaining candidate latent classes are as follows:

red_class

      [,1] [,2] [,3]
 [1,]    1    0    1
 [2,]    1    0    0
 [3,]    1    0   -1
 [4,]    0    1   -1
 [5,]    0    0    1
 [6,]    0    0    0
 [7,]    0    0   -1
 [8,]    0   -1    1
 [9,]    0   -1    0
[10,]   -1    0    1
[11,]   -1    0    0
[12,]   -1   -1    1
[13,]   -1   -1    0

which is a subset of the \(3^D=27\) candidate latent classes. Next, we need to determine the hyperparameters on the priors in our Bayesian Gaussian mixture model. Most important is computing the hyperparameters for the prior on the mixing weights. Some classes (especially when the dimension is larger) will have a mixing weight that will result in a degenerate mixing distribution, and so classes with small enough mixing weights can be further pruned from the model. This is discussed in the next step.

Session information

sessionInfo()

R version 3.5.2 (2018-12-20)
Platform: x86_64-apple-darwin15.6.0 (64-bit)
Running under: macOS Sierra 10.12.6

Matrix products: default
BLAS: /Library/Frameworks/R.framework/Versions/3.5/Resources/lib/libRblas.0.dylib
LAPACK: /Library/Frameworks/R.framework/Versions/3.5/Resources/lib/libRlapack.dylib

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] LIEB_0.1.0

loaded via a namespace (and not attached):
 [1] Rcpp_1.0.1           compiler_3.5.2       pillar_1.3.1        
 [4] git2r_0.24.0         plyr_1.8.4           workflowr_1.3.0     
 [7] iterators_1.0.10     tools_3.5.2          testthat_2.0.1      
[10] digest_0.6.18        lattice_0.20-38      evaluate_0.13       
[13] tibble_2.0.1         pkgconfig_2.0.2      rlang_0.3.1         
[16] igraph_1.2.4         foreach_1.4.4        rstudioapi_0.9.0    
[19] yaml_2.2.0           parallel_3.5.2       mvtnorm_1.0-11      
[22] LaplacesDemon_16.1.1 xfun_0.5             coda_0.19-2         
[25] dplyr_0.8.0.1        stringr_1.4.0        knitr_1.22          
[28] fs_1.2.6             hms_0.4.2            grid_3.5.2          
[31] rprojroot_1.3-2      tidyselect_0.2.5     glue_1.3.0          
[34] R6_2.4.0             JuliaCall_0.16.4     rmarkdown_1.12      
[37] purrr_0.3.1          readr_1.3.1          magrittr_1.5        
[40] whisker_0.3-2        backports_1.1.3      codetools_0.2-16    
[43] htmltools_0.3.6      abind_1.4-5          assertthat_0.2.1    
[46] nimble_0.7.0.1       stringi_1.3.1        doParallel_1.0.14   
[49] crayon_1.3.4