Last updated: 2019-10-18
Checks: 1 1
Knit directory: mcfa-fit/
This reproducible R Markdown analysis was created with workflowr (version 1.4.0). The Checks tab describes the reproducibility checks that were applied when the results were created. The Past versions tab lists the development history.
The R Markdown file has unstaged changes. To know which version of the R Markdown file created these results, you’ll want to first commit it to the Git repo. If you’re still working on the analysis, you can ignore this warning. When you’re finished, you can run wflow_publish to commit the R Markdown file and build the HTML.
Great! You are using Git for version control. Tracking code development and connecting the code version to the results is critical for reproducibility. The version displayed above was the version of the Git repository at the time these results were generated.
Note that you need to be careful to ensure that all relevant files for the analysis have been committed to Git prior to generating the results (you can use wflow_publish or wflow_git_commit). workflowr only checks the R Markdown file, but you know if there are other scripts or data files that it depends on. Below is the status of the Git repository when the results were generated:
Ignored files:
Ignored: .RData
Ignored: .RDataTmp
Ignored: .Rhistory
Ignored: .Rproj.user/
Untracked files:
Untracked: analysis/est_ulsmv.Rmd
Untracked: analysis/est_wlsmv.Rmd
Unstaged changes:
Modified: analysis/est_mlr.Rmd
Modified: analysis/fit_boxplots.Rmd
Modified: analysis/index.Rmd
Modified: analysis/index_cfi.Rmd
Modified: analysis/index_rmsea.Rmd
Modified: analysis/index_srmr.Rmd
Modified: analysis/index_tli.Rmd
Modified: analysis/roc_analyses.Rmd
Note that any generated files, e.g. HTML, png, CSS, etc., are not included in this status report because it is ok for generated content to have uncommitted changes.
These are the previous versions of the R Markdown and HTML files. If you’ve configured a remote Git repository (see ?wflow_git_remote), click on the hyperlinks in the table below to view them.
| File | Version | Author | Date | Message |
|---|---|---|---|---|
| html | b534b90 | noah-padgett | 2019-09-29 | updated publish |
| Rmd | 6457362 | noah-padgett | 2019-09-29 | general update after first SEM review |
| html | 6457362 | noah-padgett | 2019-09-29 | general update after first SEM review |
The standardized root mean square residual (SRMR) is an aggregate measure of the deviation of the observed correlation matrix to the model implied correlation matrix (Joreskog, 1981). Ideally, the average difference between the observed and expected correlations is minimal, and smaller values represent better fitting models. Generally, SRMR is computed as the standardized difference between the observed correlations and the model implied correlations about variables as shown below. \[ SRMR = \sqrt{\frac{2 \sum_{j=1}^{p} \sum_{k=j}^{i} {\left( \frac{s_{jk} - \hat{\sigma}_{jk} }{ \sqrt{ s_{jj}s_{kk}} } \right)}^2 }{p(p+1)}} \] where \(p\) is the total number of variables in the model, \(s_{jk}\) and \(\sigma_{jk}\) are the sample and model implied, respectively, covariance between the \(j^{th}\) and \(k^{th}\) variables. For the SRMR, generally acceptable values less than .08 are used. However, Hu & Bentler (1999) suggested values less .06 alone or .08 in combination of other indices within recommended ranges are indicative of good fit.
In Mplus, SRMR is estimated slightly differently than shown above. The computation extends the definition above by accounting for the meanstructure, multilevel structure, and categorical nature of the data if applicable. Additionally, in ML-CFA, SRMR is differentiated into two different SRMR indices corresponding to each level’s covariance matrix. The level-1 SRMR is known as SRMR Within (SRMRW). The level-2 SRMR is known as SRMR Between (SRMRB). The computation of each of these measures is roughly equivalent conditional on which covariance matrix is under consideration.
The computation of SRMR under the conditions of categorical and multilevel data are described in detail in (Asparouhov & Muthén, 2018). Here is the idea behind the computation. First, we need to compute the polychoric correlations (\(s_{jk}\)) and the variances (\(s_{jj}\)) of the items which are the sample estimates that are approximated by the model implied correlations (\(\sigma_{jk}\)) and variances (\(\sigma_{jj}\)). This is not a straightforward task and is abstracted away for this discussion (interested reader is referred to Bandalos (2014), Muthén, du Toit & Spisic (1997), and Muthén (1978) for more details). We also need to have the category response probabilities for the observed data (\(p_{ij}\)) and model implied (\(q_{ij}\)) for the \(i^{th}\) category of the \(j^{th}\) item where \(i = 1, ..., l_j\) (number of categories for the \(j^{th}\) item, and \(j=1, ..., m\) (number of items). Then, we can define the SRMR to account for the categorical data by:
\[SRMR = \sqrt{\frac{S}{d}}\] where, \[S = \sum_{j=1}^{m}\sum_{k=1}^{j-1}\left(\frac{s_{jk}}{\sqrt{s_{jj}s_{kk}}} -\frac{\sigma_{jk}}{\sqrt{\sigma_{jj}\sigma_{kk}}}\right)^2 + \sum_{j=1}^{m}\left(\frac{s_{jj} -\sigma_{jj}}{s_{jj}}\right)^2 + \sum_{j=1}^{m}\sum_{i=1}^{l_j}\left(p_{ij} -q_{ij}\right)^2\\ d=m(m+3)/2 + \sum_{j=1}^{m}l_j -2m\] where k is simply an indicator to sum over for items.
The computation of the response probabilities (\(p_{ij}\)) are the only major difference between the above equation for SRMR when the multilevel nature of that is accounted for. This is because the response probabilities for single level SRMR is \[P(Y_j = i) = \Phi\left(\frac{\tau_{ij}}{\sqrt{V_{jj}}}\right) - \Phi\left(\frac{\tau_{(i-1)j}}{\sqrt{V_{jj}}}\right)\] where \(\tau_{ij}\) is the \(i^{th}\) threshold of the \(j^{th}\) item. Note that the thresholds are: \(\tau_{0j}=-\infty, \tau_{1j}=1, ..., \tau_{ij}=i, ...\). In the situation with multilevel data, the response probabilities account for the variance at both levels (\(V_{B, jj}\) and \(V_{W,jj}\)). Therefore, the response probability becomes \[P(Y_j = i) = \Phi\left(\frac{\tau_{ij}}{\sqrt{V_{B, jj}+V_{W, jj}}}\right) - \Phi\left(\frac{\tau_{(i-1)j}}{\sqrt{V_{B, jj}+V_{W, jj}}}\right)\]
The more technical details of the computation of SRMRW and SRMRB in Mplus v8.2 is described more detail in (Asparouhov & Muthén, 2018).
For SRMRW and SRMRB, We align our expectations with other methodologists on the untility of these indices in ML-CFA. We expect that SRMRW will only be sensitive to misspecification of the model for the level-1 (pooled within-group) covariance matrix.
While we expect SRMRB to only be sensitive to misspecification of the model for the level-2 (between-group) covariance matrix. However, given the very drastic difference in sample size for each of these fit indices, we expect that SRMRB will yield much more variable estimates across conditions and replications, particularly when the number of level-2 units is below 100. We expect this decline in performance under smaller sample sizes due to the normalizing factor being reduced and other authors have suggested large values are plausible (Asparouhov & Muthen, 2018).