Last updated: 2020-04-24

Checks: 6 1

Knit directory: lieb/

This reproducible R Markdown analysis was created with workflowr (version 1.6.1). The Checks tab describes the reproducibility checks that were applied when the results were created. The Past versions tab lists the development history.

R Markdown file: up-to-date

Great! Since the R Markdown file has been committed to the Git repository, you know the exact version of the code that produced these results.

Environment: empty

Great job! The global environment was empty. Objects defined in the global environment can affect the analysis in your R Markdown file in unknown ways. For reproduciblity it’s best to always run the code in an empty environment.

Seed: set.seed(20190717)

The command set.seed(20190717) was run prior to running the code in the R Markdown file. Setting a seed ensures that any results that rely on randomness, e.g. subsampling or permutations, are reproducible.

Session information: unavailable

Recording the operating system, R version, and package versions is critical for reproducibility. To record the session information, add sessioninfo: “sessionInfo()” to _workflowr.yml. Alternatively, you could use devtools::session_info() or sessioninfo::session_info(). Lastly, you can manually add a code chunk to this file to run any one of these commands and then disable to automatic insertion by changing the workflowr setting to sessioninfo: “”.

Cache: none

Nice! There were no cached chunks for this analysis, so you can be confident that you successfully produced the results during this run.

File paths: relative

Great job! Using relative paths to the files within your workflowr project makes it easier to run your code on other machines.

Repository version: 717e762

Great! You are using Git for version control. Tracking code development and connecting the code version to the results is critical for reproducibility.

The results in this page were generated with repository version 717e762. See the Past versions tab to see a history of the changes made to the R Markdown and HTML files.

Note that you need to be careful to ensure that all relevant files for the analysis have been committed to Git prior to generating the results (you can use wflow_publish or wflow_git_commit). workflowr only checks the R Markdown file, but you know if there are other scripts or data files that it depends on. Below is the status of the Git repository when the results were generated: 


Ignored files:
    Ignored:    .DS_Store
    Ignored:    .Rhistory
    Ignored:    .Rproj.user/
    Ignored:    analysis/.DS_Store
    Ignored:    analysis/.Rhistory
    Ignored:    analysis/pairwise_fitting_cache/
    Ignored:    data/.DS_Store
    Ignored:    output/.DS_Store
    Ignored:    output/chip_shiny/.DS_Store
    Ignored:    output/chip_shiny/Data/.DS_Store
    Ignored:    output/vision_shiny/.DS_Store
    Ignored:    output/vision_shiny/.Rhistory
    Ignored:    output/vision_shiny/Data/.DS_Store

Unstaged changes:
    Modified:   analysis/pairwise_fitting.Rmd
    Modified:   analysis/preprocessing.Rmd

Note that any generated files, e.g. HTML, png, CSS, etc., are not included in this status report because it is ok for generated content to have uncommitted changes.

These are the previous versions of the repository in which changes were made to the R Markdown (analysis/candidate_latent_classes.Rmd) and HTML (docs/candidate_latent_classes.html) files. If you’ve configured a remote Git repository (see ?wflow_git_remote), click on the hyperlinks in the table below to view the files as they were in that past version.

File Version Author Date Message
Rmd 828e725 hillarykoch 2020-04-23 add a preprocessing section
html 828e725 hillarykoch 2020-04-23 add a preprocessing section
html c5fd6fc hillarykoch 2020-04-23 change nav bar to accomodate a menu
html 5d5c3dd hillarykoch 2020-03-07 Build site.
html 7c42345 hillarykoch 2020-02-20 Build site.
html 2a5b7c0 hillarykoch 2020-02-20 Build site.
html 3a9bf9d hillarykoch 2020-02-20 Build site.
html 72e6fec hillarykoch 2020-02-20 Build site.
html 862bc02 hillarykoch 2020-02-07 big updates for mcmc processors
html 8b3e556 hillarykoch 2019-12-03 Build site.
html 39383ac hillarykoch 2019-12-03 Build site.
Rmd 645d408 hillarykoch 2019-12-03 workflowr::wflow_publish(files = “*“)
html e467a51 hillarykoch 2019-08-16 different shiny location
html d58e1a6 hillarykoch 2019-08-16 resource data
html 38fb1c0 hillarykoch 2019-08-16 edit shiny
html d712ef9 hillarykoch 2019-07-18 add hamming
html 3a118c5 hillarykoch 2019-07-18 add hamming
Rmd e15a4f0 hillarykoch 2019-07-18 add hamming
html e15a4f0 hillarykoch 2019-07-18 add hamming
html 228123d hillarykoch 2019-07-17 update up to obtaining the hyperparameters
html c1dc0c1 hillarykoch 2019-07-17 update up to obtaining the hyperparameters
html 674120c hillarykoch 2019-07-17 update about page
html e67a3a2 hillarykoch 2019-07-17 update about page
Rmd 2faf446 hillarykoch 2019-07-17 update about page
html 2faf446 hillarykoch 2019-07-17 update about page
html 2177dfa hillarykoch 2019-07-17 update about page
Rmd 8b30694 hillarykoch 2019-07-17 update about page
html 8b30694 hillarykoch 2019-07-17 update about page
html e36b267 hillarykoch 2019-07-17 update about page
html a991668 hillarykoch 2019-07-17 update about page
html a36d893 hillarykoch 2019-07-17 update about page
html e8e54b7 hillarykoch 2019-07-17 update about page
html f47c013 hillarykoch 2019-07-17 update about page
Rmd 50cf23e hillarykoch 2019-07-17 make skeleton
html 50cf23e hillarykoch 2019-07-17 make skeleton

Finding candidate latent classes: the graph enumeration and pruning algorithm

—Special considerations: this portion is likely a high-memory task!—

Now that we have our pairwise fits, we need to figure out what are the remaining candidate latent classes. These can be found using the path enumeration and pruning algorithm. This is a graph-based algorithm which can require a lot of memory (especially when the dimension of the data is large). However, with the memory available, this algorithm runs very quickly – in our experience, in under 5 minutes.

First, load in the previously obtained pairwise fits.

data("fits")

Then, we can obtain a reduced list of candidate latent classes with get_reduced_classes(). This function has 3 arguments:

  1. the pairwise fits

  2. the dimension of the data

  3. the name of an output “LEMON graph format” file (here, called lgf.txt). This is not to be edited by the user, but is produced for underlying C software.

# This finds the dimension of the data directly from the pairwise fits
D <- as.numeric(strsplit(tail(names(fits),1), "_")[[1]][2])

# Get the list of candidate latent classes
red_class <- get_reduced_classes(fits, D, "output/lgf.txt")
Writing LGF file...done!
Finding latent classes...done!
# write the output to a text file
readr::write_tsv(data.frame(red_class), path = "output/red_class.txt", col_names = FALSE)

Each row of red_class corresponds to a candidate latent class across the 3 dimensions. The remaining candidate latent classes are as follows:

red_class
      [,1] [,2] [,3]
 [1,]    1    0    1
 [2,]    1    0    0
 [3,]    1    0   -1
 [4,]    0    1   -1
 [5,]    0    0    1
 [6,]    0    0    0
 [7,]    0    0   -1
 [8,]    0   -1    1
 [9,]    0   -1    0
[10,]   -1    0    1
[11,]   -1    0    0
[12,]   -1   -1    1
[13,]   -1   -1    0

which is a subset of the \(3^D=27\) candidate latent classes. Next, we need to determine the hyperparameters on the priors in our Bayesian Gaussian mixture model. Most important is computing the hyperparameters for the prior on the mixing weights. Some classes (especially when the dimension is larger) will have a mixing weight that will result in a degenerate mixing distribution, and so classes with small enough mixing weights can be further pruned from the model. This is discussed in the next step.