Last updated: 2019-09-28

Checks: 6 1

Knit directory: mcfa-fit/

This reproducible R Markdown analysis was created with workflowr (version 1.4.0). The Checks tab describes the reproducibility checks that were applied when the results were created. The Past versions tab lists the development history.

R Markdown file: uncommitted changes

The R Markdown file has unstaged changes. To know which version of the R Markdown file created these results, you’ll want to first commit it to the Git repo. If you’re still working on the analysis, you can ignore this warning. When you’re finished, you can run wflow_publish to commit the R Markdown file and build the HTML.

Environment: empty

Great job! The global environment was empty. Objects defined in the global environment can affect the analysis in your R Markdown file in unknown ways. For reproduciblity it’s best to always run the code in an empty environment.

Seed: set.seed(20190507)

The command set.seed(20190507) was run prior to running the code in the R Markdown file. Setting a seed ensures that any results that rely on randomness, e.g. subsampling or permutations, are reproducible.

Session information: recorded

Great job! Recording the operating system, R version, and package versions is critical for reproducibility.

Cache: none

Nice! There were no cached chunks for this analysis, so you can be confident that you successfully produced the results during this run.

File paths: relative

Great job! Using relative paths to the files within your workflowr project makes it easier to run your code on other machines.

Repository version: 52c68f9

Great! You are using Git for version control. Tracking code development and connecting the code version to the results is critical for reproducibility. The version displayed above was the version of the Git repository at the time these results were generated.

Note that you need to be careful to ensure that all relevant files for the analysis have been committed to Git prior to generating the results (you can use wflow_publish or wflow_git_commit). workflowr only checks the R Markdown file, but you know if there are other scripts or data files that it depends on. Below is the status of the Git repository when the results were generated:


Ignored files:
    Ignored:    .RData
    Ignored:    .RDataTmp
    Ignored:    .Rhistory
    Ignored:    .Rproj.user/

Untracked files:
    Untracked:  analysis/est_mlr.Rmd
    Untracked:  analysis/fit_index_stats.Rmd
    Untracked:  analysis/index_cfi.Rmd
    Untracked:  analysis/index_rmsea.Rmd
    Untracked:  analysis/index_srmr.Rmd
    Untracked:  analysis/index_tli.Rmd

Unstaged changes:
    Modified:   analysis/index.Rmd
    Modified:   analysis/roc_analyses.Rmd
    Modified:   analysis/summary_stats_tables.Rmd

Note that any generated files, e.g. HTML, png, CSS, etc., are not included in this status report because it is ok for generated content to have uncommitted changes.

These are the previous versions of the R Markdown and HTML files. If you’ve configured a remote Git repository (see ?wflow_git_remote), click on the hyperlinks in the table below to view them.

File	Version	Author	Date	Message
html	982c8f1	noah-padgett	2019-05-18	roc analyses completed
Rmd	4a2f40d	noah-padgett	2019-05-10	fixed index page and updated roc file
Rmd	3cd3ef6	noah-padgett	2019-05-09	updated analyses

Purpose of this file:

Conduct ROC Analysies
Create ROC Curves
Create Summary tables of ROC analyses

Packages and Set-Up

##Chunk iptions
knitr::opts_chunk$set(out.width="225%")
#setwd('C:/Users/noahp/Dropbox/MCFA Thesis/Code Results')
## Packages
## General Packages
library(tidyverse)

-- Attaching packages ------------------------------------------------------------ tidyverse 1.2.1 --

v ggplot2 3.2.0     v purrr   0.3.2
v tibble  2.1.1     v dplyr   0.8.1
v tidyr   0.8.3     v stringr 1.4.0
v readr   1.3.1     v forcats 0.4.0

-- Conflicts --------------------------------------------------------------- tidyverse_conflicts() --
x dplyr::filter() masks stats::filter()
x dplyr::lag()    masks stats::lag()

library(car)

Loading required package: carData


Attaching package: 'car'

The following object is masked from 'package:dplyr':

    recode

The following object is masked from 'package:purrr':

    some

library(psych)


Attaching package: 'psych'

The following object is masked from 'package:car':

    logit

The following objects are masked from 'package:ggplot2':

    %+%, alpha

# Formatting and Tables
library(kableExtra)


Attaching package: 'kableExtra'

The following object is masked from 'package:dplyr':

    group_rows

library(xtable)
# For plotting
library(ggplot2)
theme_set(theme_bw() + theme(legend.position = 'bottom'))
# Data manipulating
library(dplyr)
# ROC Analysis
library(pROC)

Type 'citation("pROC")' for a citation.


Attaching package: 'pROC'

The following objects are masked from 'package:stats':

    cov, smooth, var

## One global parameter for printing figures
save.fig <- F
## Load up the functions needed for ANOVA and Assumption checking
source('code/r_functions.R')

Data Management

sim_results <- as_tibble(read.table('data/compiled_fit_results.txt', header=T,sep='\t'))
## Next, turn condition into a factor for plotting
sim_results$Condition <- as.factor(sim_results$Condition)
## Next, since TLI is non-normed, any value greater than 1 needs to be rescaled to 1.
sim_results$TLI <- ifelse(sim_results$TLI > 1, 1, sim_results$TLI)
sim_results$TLI <- ifelse(sim_results$TLI < 0, 0, sim_results$TLI)
## Next, summarize the results of the chi-square test of model fit. This is done simply by comparing the p-value to alpha (0.05) and indicating whether the model was flagged as fitting or not.
# Note: if  p < 0.05 then this variable is flagged as 0, and 1 otherwise
sim_results$Chi2_pvalue_decision <- ifelse(sim_results$chisqu_pvalue > 0.05, 1, 0)
# 0 = rejected that these data fit this model
# 1 = failed to reject that these data fit this model

Adding Labels to Conditions

Currently, each condition is kind of like a hidden id that we don’t know what the actual factor is. So, first thing isto create meaningful labels for us to use. Remember, the 72 conditions for the this study were

Level-1 sample size (5, 10, 30)
Level-2 sample size (30, 50, 100, 200)
Observed indicator ICC (.1, .3, .5)
Latent variable ICC (.1, .5)

## level-1 Sample size
ss_l1 <- c(5, 10, 30) ## 6 conditions each
ss_l2 <- c(30, 50, 100, 200) ## 18 condition each
icc_ov <- c(.1, .3, .5) ## 2 conditions each
icc_lv <- c(.1, .5) ## every other condition
nCon <- 72 # number of conditions
nRep <- 500 # number of replications per condition
nMod <- 12 ## numberof estimated models per conditions
## Total number of rows: 432,000
ss_l2 <- c(rep(ss_l2[1], 18*nRep*nMod), rep(ss_l2[2], 18*nRep*nMod), rep(ss_l2[3], 18*nRep*nMod), rep(ss_l2[4], 18*nRep*nMod))
ss_l1 <- rep(c(rep(ss_l1[1],6*nRep*nMod), rep(ss_l1[2],6*nRep*nMod), rep(ss_l1[3],6*nRep*nMod)), 4)
icc_ov <- rep(c(rep(icc_ov[1], 2*nRep*nMod), rep(icc_ov[2], 2*nRep*nMod), rep(icc_ov[3], 2*nRep*nMod)), 12)
icc_lv <- rep(c(rep(icc_lv[1], nRep*nMod), rep(icc_lv[2], nRep*nMod)), 36)
## Force these vectors to be column vectors
ss_l1 <- matrix(ss_l1, ncol=1)
ss_l2 <- matrix(ss_l2, ncol=1)
icc_ov <- matrix(icc_ov, ncol=1)
icc_lv <- matrix(icc_lv, ncol=1)
## Add the labels to the results data frame
sim_results <- sim_results[order(sim_results$Condition),]
sim_results <- cbind(sim_results, ss_l1, ss_l2, icc_ov, icc_lv)
## Force the conditions to be factors
sim_results$ss_l1 <- as.factor(sim_results$ss_l1)
sim_results$ss_l2 <- as.factor(sim_results$ss_l2)
sim_results$icc_ov <- as.factor(sim_results$icc_ov)
sim_results$icc_lv <- as.factor(sim_results$icc_lv)
sim_results$Model <- factor(sim_results$Model, levels = c('C','M1','M2','M12'), ordered = T)

ROC Analysis Labels

Make coding of variables/model specifications for the ROC analyses. The coding of the variables is in order to get the correct specifications labeled for the ROC analyses to come. The coding is as follows:

C = correct model specification versus any misspecification (Model C vs. M1, M2, M12)
CvM1 = when only the level-1 model is misspecified (Model C vs. M1)
CvM2 = when only the level-2 model is misspecified (Model C vs. M2)

## Need to make codes for the ROC analyses outcomes
# first, C vs. M1,M2,M12 - Perfect specification
sim_results$C <- ifelse(sim_results$Model == 'C', 1, 0)
table(sim_results$C)


     0      1 
324000 108000

# second, C vs. M1|M12- correct level 1 model
sim_results$CvM1 <- ifelse(sim_results$Model == 'C', 1, 0)
sim_results$CvM1[sim_results$Model == "M2" | sim_results$Model == "M12"] <- NA
table(sim_results$CvM1)


     0      1 
108000 108000

# third, C vs. M2|M12- correct level 2 model
sim_results$CvM2 <- ifelse(sim_results$Model == 'C', 1, 0)
sim_results$CvM2[sim_results$Model == "M1" | sim_results$Model == "M12"] <- NA
table(sim_results$CvM2)


     0      1 
108000 108000

Introduction to ROC Analysis

ROC stands for Receiver Operating Characteristic. ROC analysis aims to detect the presense of signals in data by looking at how the ability to classify an outcome (usually binary) based on a continuous or ordinal indicator. ROC analyses was orginally used in wartime to help detect the presence of radar signals. However, now ROC analysis is used in many areas including medical and psychology research. It is commonly used as a tool to help make decisions about what tools or methods help classify objects or individuals into specific groups.

In R, a package called pROC (Robin et al., 2011) was built that greatly enhancing the flexibility of using R for ROC analysis. For example, aside from just being able to conduct ROC analysis, one can compute confidences for this curve and conduct specific statistical tests comparing AUCs from the same data.

For the ROC analyses, we conducted them in pieces to build more and more fine grained information about the classification quality of fit indices for detecting a simple type of misspecification. For misspecification, we broke up the ROC analyses into three major chunks.

Detecting any type of misspecification (i.e., C vs. M1-M2-M12),
Detecting misspecified level-1 model (i.e., C vs. M1), and
Detecting misspecified level-2 model (i.e., C vs. M2).

Within each of these major chunks of analyses, we furhter investigated whether classification of correctly specified models was depended upon estimator, level-1 sample size, or level-2 sample size. There are intitally be MANY ROC curve figures and over 1200 ROC analyses.

Setting up the objects to store the individual results so that we can use them all for the figures. First, we run over each condition separately then go into the conditional ROC analyses.

fit_roc <- fit_roc_smooth <- list()
roc_summary <- as.data.frame(matrix(0,ncol=13, nrow=5*3*4*4*5))
colnames(roc_summary) <- c('Classification','Index', 'Estimator', 'Level-2 SS', 
                           'Level-1 SS', 'AUC',
                           'partial-AUC','Smoothed-AUC', 'Threshold',
                           'Specificity','Sensitivity', 'Num-C', 'Num-Mis')
roc_summary_gen <- as.data.frame(matrix(0,ncol=8, nrow=5)*3)
colnames(roc_summary_gen) <- c('Classification','Index', 'AUC',
                           'partial-AUC','Smoothed-AUC', 'Optimal-Threshold',
                           'Specificity','Sensitivity')
# Defining iterators
CLASS <- c('C', 'CvM1','CvM2')
INDEX <- c('CFI', 'TLI', 'RMSEA', 'SRMRW', 'SRMRB')
EST <- c('ALL','MLR', 'ULSMV', 'WLSMV')
SS_L1 <- c('ALL', 5, 10, 30)
SS_L2 <- c('ALL', 30, 50, 100, 200)
## Subset to the usable cases
sim_results <- filter(sim_results, Converge == 1 & Admissible == 1)

Detecting any type of misspecification

General overview over all conditions

ig <- 1 ## counter for roc_summary
j <- 1 ## Which class?
for(index in INDEX){
    ## Print out which iteration so we know what we am looking at
    cat('\n\nROC Analysis in')
    cat('\nIndex:\t', index)
    cat('\nClassification:\t', CLASS[j])
    ## Set up iteration key
    key <- paste0(index,'.',CLASS[j])
    ## Create formula
    model <- as.formula(paste0(CLASS[j], '~', index))
    ## Fit ROC curve
    fit_roc[[key]] <-  roc(model, data=sim_results,
                           plot =TRUE, ci=TRUE, print.auc=TRUE)
    ## Create a plot of "smoothed" curve for plotting
    fit_roc_smooth[[key]] <-  smooth(roc(model, data=sim_results))
    ## Compute partial AUC for specificity .8-1
    p.auc <- auc(fit_roc[[key]], partial.auc = c(1,.8),
                 partial.auc.focus = 'sp', partial.auc.correct = T)
    ## get summary info
    roc_summary_gen[ig, 2] <- index
    roc_summary_gen[ig, 1] <- CLASS[j]
    roc_summary_gen[ig, 3] <- fit_roc[[key]]$auc ## total AUC
    roc_summary_gen[ig, 4] <- p.auc ## corrected partial AUC (.5 is no discrimination)
    roc_summary_gen[ig, 5] <- fit_roc_smooth[[key]]$auc ## smoothed AUC
    roc_summary_gen[ig, 6:8] <- coords(fit_roc[[key]], "best", 
                                   ret=c("threshold", "specificity", 'sensitivity'))
    ## print summary
    cat('\n\nSummary of ROC:\n')
    print(roc_summary_gen[ig, ])
    ## add to summary iterator
    ig <- ig + 1
} ## End loop round index



ROC Analysis in
Index:   CFI
Classification:  C

Setting levels: control = 0, case = 1

Setting direction: controls < cases

Setting levels: control = 0, case = 1

Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
  Classification Index       AUC partial-AUC Smoothed-AUC
1              C   CFI 0.8156696   0.6374748    0.8456461
  Optimal-Threshold Specificity Sensitivity
1          0.977017   0.7024613   0.8549251


ROC Analysis in
Index:   TLI
Classification:  C

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
  Classification Index       AUC partial-AUC Smoothed-AUC
2              C   TLI 0.8154291   0.6374691    0.8453772
  Optimal-Threshold Specificity Sensitivity
2         0.9724203   0.7022691   0.8549251


ROC Analysis in
Index:   RMSEA
Classification:  C

Setting levels: control = 0, case = 1

Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
  Classification Index       AUC partial-AUC Smoothed-AUC
3              C RMSEA 0.8034875   0.6348255    0.8299624
  Optimal-Threshold Specificity Sensitivity
3        0.01504671   0.6849311    0.829003


ROC Analysis in
Index:   SRMRW
Classification:  C

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
  Classification Index       AUC partial-AUC Smoothed-AUC
4              C SRMRW 0.7420696   0.6079951    0.7165381
  Optimal-Threshold Specificity Sensitivity
4        0.03812817   0.7281555   0.7226555


ROC Analysis in
Index:   SRMRB
Classification:  C

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.

Version	Author	Date
982c8f1	noah-padgett	2019-05-18



Summary of ROC:
  Classification Index       AUC partial-AUC Smoothed-AUC
5              C SRMRB 0.5979973   0.5720031    0.6039185
  Optimal-Threshold Specificity Sensitivity
5        0.06685217   0.8038081   0.3519244

kable(roc_summary_gen[1:5,], format = 'html', digits=3) %>%
  kable_styling(full_width = T)

Classification	Index	AUC	partial-AUC	Smoothed-AUC	Optimal-Threshold	Specificity	Sensitivity
C	CFI	0.816	0.637	0.846	0.977	0.702	0.855
C	TLI	0.815	0.637	0.845	0.972	0.702	0.855
C	RMSEA	0.803	0.635	0.830	0.015	0.685	0.829
C	SRMRW	0.742	0.608	0.717	0.038	0.728	0.723
C	SRMRB	0.598	0.572	0.604	0.067	0.804	0.352

print(xtable(roc_summary_gen[1:5,c(2:3,6:8)], digits = 3), booktabs=T,include.rownames = F)

% latex table generated in R 3.6.0 by xtable 1.8-4 package
% Sat Sep 28 15:09:22 2019
\begin{table}[ht]
\centering
\begin{tabular}{lrrrr}
  \toprule
Index & AUC & Optimal-Threshold & Specificity & Sensitivity \\ 
  \midrule
CFI & 0.816 & 0.977 & 0.702 & 0.855 \\ 
  TLI & 0.815 & 0.972 & 0.702 & 0.855 \\ 
  RMSEA & 0.803 & 0.015 & 0.685 & 0.829 \\ 
  SRMRW & 0.742 & 0.038 & 0.728 & 0.723 \\ 
  SRMRB & 0.598 & 0.067 & 0.804 & 0.352 \\ 
   \bottomrule
\end{tabular}
\end{table}

More fine grained information within/across conditions

i <- 1 ## counter for roc_summary
j <- 1 ## Which class?
for(index in INDEX){
  for(est in EST){
    for(s2 in SS_L2){
      for(s1 in SS_L1){
    ## Print out which iteration so we know what we are looking at
    cat('\n\nROC Analysis in')
    cat('\nIndex:\t', index)
    cat('\nClassification:\t', CLASS[j])
    cat('\nEstimation Method:\t', est)
    cat('\nLevel-2 Sample Size:\t', s2)
    cat('\nLevel-1 Sample Size:\t', s1)
    ## Set up iteration key
    key <- paste0(index,'.',CLASS[j],'.',est,'.', s2,'.',s1)
    # Subset data as  needed
    if(est == 'ALL' & s2 == 'ALL' & s1 == 'ALL') mydata <- sim_results
    if(est != 'ALL' & s2 == 'ALL' & s1 == 'ALL'){
      mydata <- filter(sim_results, Estimator == est)
    }
    if(est == 'ALL' & s2 != 'ALL' & s1 == 'ALL'){
      mydata <- filter(sim_results, ss_l2 == s2)
    }
    if(est == 'ALL' & s2 == 'ALL' & s1 != 'ALL'){
      mydata <- filter(sim_results, ss_l1 == s1)
    }
    if(est != 'ALL' & s2 != 'ALL' & s1 == 'ALL'){
      mydata <- filter(sim_results, Estimator == est, ss_l2 == s2)
    }
    if(est != 'ALL' & s2 == 'ALL' & s1 != 'ALL'){
      mydata <- filter(sim_results, Estimator == est, ss_l1 == s1)
    }
    if(est == 'ALL' & s2 != 'ALL' & s1 != 'ALL'){
      mydata <- filter(sim_results, ss_l2 == s2, ss_l1 == s1)
    }
    if(est != 'ALL' & s2 != 'ALL' & s1 != 'ALL'){
      mydata <- filter(sim_results, Estimator == est, ss_l2 == s2, ss_l1 == s1)
    }
    ## Create formula
    model <- as.formula(paste0(CLASS[j], '~', index))
    ## Fit ROC curve
    fit_roc[[key]] <-  roc(model, data=mydata,
                           plot =TRUE, ci=TRUE, print.auc=TRUE)
    ## Create a plot of "smoothed" curve for plotting
    fit_roc_smooth[[key]] <-  smooth(roc(model, data=mydata))
    ## Compute partial AUC for specificity .8-1
    p.auc <- auc(fit_roc[[key]], partial.auc = c(1,.8),
                 partial.auc.focus = 'sp', partial.auc.correct = T)
    ## get summary info
    roc_summary[i, 2] <- index
    roc_summary[i, 1] <- CLASS[j]
    roc_summary[i, 3] <- est ##estimator
    roc_summary[i, 4] <- s2 ## level-2 sample size
    roc_summary[i, 5] <- s1 ## level-1 sample size
    roc_summary[i, 6] <- fit_roc[[key]]$auc ## total AUC
    roc_summary[i, 7] <- p.auc ## corrected partial AUC (.5 is no discrimination)
    roc_summary[i, 8] <- fit_roc_smooth[[key]]$auc ## smoothed AUC
    roc_summary[i, 9:11] <- coords(fit_roc[[key]], "best", 
                                   ret=c("threshold", "specificity", 'sensitivity'))
    
    ## add number of C and number of miss models in analysis
    n.C <- nrow(mydata[ mydata[, CLASS[j]] == 1, ])
    n.M <- nrow(mydata[ mydata[, CLASS[j]] == 0, ])
    roc_summary[i, 12] <- n.C
    roc_summary[i, 13] <- n.M
    
    ## print summary
    cat('\n\nSummary of ROC:\n')
    print(roc_summary[i, ])
    ## add to summary iterator
    i <- i + 1
      } ## end loop around ss l1
    } ## End loop around ss l2
  } ## End loop around estimator
} ## End loop round index



ROC Analysis in
Index:   CFI
Classification:  C
Estimation Method:   ALL
Level-2 Sample Size:     ALL
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1

Setting direction: controls < cases

Setting levels: control = 0, case = 1

Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
  Classification Index Estimator Level-2 SS Level-1 SS       AUC
1              C   CFI       ALL        ALL        ALL 0.8156696
  partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C Num-Mis
1   0.6374748    0.8456461  0.977017   0.7024613   0.8549251 83510  223868


ROC Analysis in
Index:   CFI
Classification:  C
Estimation Method:   ALL
Level-2 Sample Size:     ALL
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
  Classification Index Estimator Level-2 SS Level-1 SS       AUC
2              C   CFI       ALL        ALL          5 0.7841525
  partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C Num-Mis
2   0.6492092    0.7994017 0.9706449   0.7493731   0.7515043 23460   63082


ROC Analysis in
Index:   CFI
Classification:  C
Estimation Method:   ALL
Level-2 Sample Size:     ALL
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
  Classification Index Estimator Level-2 SS Level-1 SS      AUC
3              C   CFI       ALL        ALL         10 0.841686
  partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C Num-Mis
3   0.6718204    0.8595385  0.971517   0.7176224   0.8728285 28208   75749


ROC Analysis in
Index:   CFI
Classification:  C
Estimation Method:   ALL
Level-2 Sample Size:     ALL
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
  Classification Index Estimator Level-2 SS Level-1 SS       AUC
4              C   CFI       ALL        ALL         30 0.8285149
  partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C Num-Mis
4   0.6116796    0.8089186 0.9855897    0.642779   0.9444758 31842   85037


ROC Analysis in
Index:   CFI
Classification:  C
Estimation Method:   ALL
Level-2 Sample Size:     30
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
  Classification Index Estimator Level-2 SS Level-1 SS       AUC
5              C   CFI       ALL         30        ALL 0.6788112
  partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C Num-Mis
5   0.5436979    0.7129602  0.967782   0.5872267   0.7389633 16157   43767


ROC Analysis in
Index:   CFI
Classification:  C
Estimation Method:   ALL
Level-2 Sample Size:     30
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
  Classification Index Estimator Level-2 SS Level-1 SS       AUC
6              C   CFI       ALL         30          5 0.6360975
  partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C Num-Mis
6   0.5462611    0.6402178 0.8987014   0.4900958   0.7186791  3994   10825


ROC Analysis in
Index:   CFI
Classification:  C
Estimation Method:   ALL
Level-2 Sample Size:     30
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
  Classification Index Estimator Level-2 SS Level-1 SS       AUC
7              C   CFI       ALL         30         10 0.7075085
  partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C Num-Mis
7   0.5602248    0.7211285 0.9593211   0.5974631   0.7280453  5297   14595


ROC Analysis in
Index:   CFI
Classification:  C
Estimation Method:   ALL
Level-2 Sample Size:     30
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
  Classification Index Estimator Level-2 SS Level-1 SS       AUC
8              C   CFI       ALL         30         30 0.7057042
  partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C Num-Mis
8   0.5352397    0.6391868 0.9709715   0.4671608   0.9429071  6866   18347


ROC Analysis in
Index:   CFI
Classification:  C
Estimation Method:   ALL
Level-2 Sample Size:     50
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
  Classification Index Estimator Level-2 SS Level-1 SS       AUC
9              C   CFI       ALL         50        ALL 0.7736153
  partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C Num-Mis
9    0.597039    0.8021961 0.9703202   0.6415235   0.8260217 19330   52168


ROC Analysis in
Index:   CFI
Classification:  C
Estimation Method:   ALL
Level-2 Sample Size:     50
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
   Classification Index Estimator Level-2 SS Level-1 SS      AUC
10              C   CFI       ALL         50          5 0.719876
   partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
10   0.5846029     0.727214 0.9564927   0.6542392   0.6803653  5037
   Num-Mis
10   13894


ROC Analysis in
Index:   CFI
Classification:  C
Estimation Method:   ALL
Level-2 Sample Size:     50
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
   Classification Index Estimator Level-2 SS Level-1 SS       AUC
11              C   CFI       ALL         50         10 0.8181039
   partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
11   0.6416087    0.8302473 0.9653674   0.6832394   0.8432974  6490
   Num-Mis
11   17559


ROC Analysis in
Index:   CFI
Classification:  C
Estimation Method:   ALL
Level-2 Sample Size:     50
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
   Classification Index Estimator Level-2 SS Level-1 SS       AUC
12              C   CFI       ALL         50         30 0.7949367
   partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
12   0.5795823    0.7632032 0.9834219   0.5886556   0.9689863  7803
   Num-Mis
12   20715


ROC Analysis in
Index:   CFI
Classification:  C
Estimation Method:   ALL
Level-2 Sample Size:     100
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
   Classification Index Estimator Level-2 SS Level-1 SS       AUC
13              C   CFI       ALL        100        ALL 0.8636507
   partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
13   0.6842599    0.8843868 0.9770006   0.7225692   0.9053775 22743
   Num-Mis
13   60844


ROC Analysis in
Index:   CFI
Classification:  C
Estimation Method:   ALL
Level-2 Sample Size:     100
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
   Classification Index Estimator Level-2 SS Level-1 SS       AUC
14              C   CFI       ALL        100          5 0.8495029
   partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
14   0.6870669    0.8592479 0.9672977   0.7231201   0.8453831  6487
   Num-Mis
14   17448


ROC Analysis in
Index:   CFI
Classification:  C
Estimation Method:   ALL
Level-2 Sample Size:     100
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
   Classification Index Estimator Level-2 SS Level-1 SS       AUC
15              C   CFI       ALL        100         10 0.8894004
   partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
15   0.7245151    0.9010436 0.9739699    0.741448   0.9457463  7852
   Num-Mis
15   20843


ROC Analysis in
Index:   CFI
Classification:  C
Estimation Method:   ALL
Level-2 Sample Size:     100
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
   Classification Index Estimator Level-2 SS Level-1 SS       AUC
16              C   CFI       ALL        100         30 0.8648419
   partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
16   0.6585577    0.8412548 0.9910256   0.7011041   0.9710852  8404
   Num-Mis
16   22553


ROC Analysis in
Index:   CFI
Classification:  C
Estimation Method:   ALL
Level-2 Sample Size:     200
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
   Classification Index Estimator Level-2 SS Level-1 SS      AUC
17              C   CFI       ALL        200        ALL 0.912001
   partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
17   0.7665049    0.9217971 0.9802153   0.7635082   0.9645174 25280
   Num-Mis
17   67089


ROC Analysis in
Index:   CFI
Classification:  C
Estimation Method:   ALL
Level-2 Sample Size:     200
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
   Classification Index Estimator Level-2 SS Level-1 SS       AUC
18              C   CFI       ALL        200          5 0.9036681
   partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
18   0.7466247    0.9136351 0.9771976   0.7871384   0.9398136  7942
   Num-Mis
18   20915


ROC Analysis in
Index:   CFI
Classification:  C
Estimation Method:   ALL
Level-2 Sample Size:     200
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
   Classification Index Estimator Level-2 SS Level-1 SS       AUC
19              C   CFI       ALL        200         10 0.9158995
   partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
19   0.7721002    0.9132568 0.9834954   0.7888098   0.9625394  8569
   Num-Mis
19   22752


ROC Analysis in
Index:   CFI
Classification:  C
Estimation Method:   ALL
Level-2 Sample Size:     200
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
   Classification Index Estimator Level-2 SS Level-1 SS       AUC
20              C   CFI       ALL        200         30 0.9306547
   partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
20    0.811288    0.9233155 0.9943931   0.7966015   0.9647622  8769
   Num-Mis
20   23422


ROC Analysis in
Index:   CFI
Classification:  C
Estimation Method:   MLR
Level-2 Sample Size:     ALL
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
   Classification Index Estimator Level-2 SS Level-1 SS       AUC
21              C   CFI       MLR        ALL        ALL 0.8405242
   partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
21   0.7102525    0.8403844  0.966923   0.7464126   0.8343994 30018
   Num-Mis
21   85647


ROC Analysis in
Index:   CFI
Classification:  C
Estimation Method:   MLR
Level-2 Sample Size:     ALL
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
   Classification Index Estimator Level-2 SS Level-1 SS       AUC
22              C   CFI       MLR        ALL          5 0.7732137
   partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
22   0.6776147    0.7767226  0.970673   0.8143822   0.6633874  8945
   Num-Mis
22   24572


ROC Analysis in
Index:   CFI
Classification:  C
Estimation Method:   MLR
Level-2 Sample Size:     ALL
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
   Classification Index Estimator Level-2 SS Level-1 SS       AUC
23              C   CFI       MLR        ALL         10 0.8535188
   partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
23   0.7169985    0.8557453  0.966923   0.7723467   0.8157582 10090
   Num-Mis
23   28785


ROC Analysis in
Index:   CFI
Classification:  C
Estimation Method:   MLR
Level-2 Sample Size:     ALL
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
   Classification Index Estimator Level-2 SS Level-1 SS       AUC
24              C   CFI       MLR        ALL         30 0.8971159
   partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
24   0.7417206    0.8913021 0.9768896   0.7333849   0.9326231 10983
   Num-Mis
24   32290


ROC Analysis in
Index:   CFI
Classification:  C
Estimation Method:   MLR
Level-2 Sample Size:     30
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
   Classification Index Estimator Level-2 SS Level-1 SS       AUC
25              C   CFI       MLR         30        ALL 0.7465924
   partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
25   0.6388114    0.7392327  0.940446   0.7263918   0.6975377  6295
   Num-Mis
25   17854


ROC Analysis in
Index:   CFI
Classification:  C
Estimation Method:   MLR
Level-2 Sample Size:     30
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
   Classification Index Estimator Level-2 SS Level-1 SS       AUC
26              C   CFI       MLR         30          5 0.6657763
   partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
26   0.5693258    0.6643642 0.8875494   0.6725047   0.5886443  1726
   Num-Mis
26    4739


ROC Analysis in
Index:   CFI
Classification:  C
Estimation Method:   MLR
Level-2 Sample Size:     30
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
   Classification Index Estimator Level-2 SS Level-1 SS       AUC
27              C   CFI       MLR         30         10 0.7741805
   partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
27   0.6221154    0.7728544 0.9179975   0.6116255    0.815482  2054
   Num-Mis
27    5832


ROC Analysis in
Index:   CFI
Classification:  C
Estimation Method:   MLR
Level-2 Sample Size:     30
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
   Classification Index Estimator Level-2 SS Level-1 SS       AUC
28              C   CFI       MLR         30         30 0.8734683
   partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
28     0.68565     0.876253 0.9572286   0.6933956   0.9662028  2515
   Num-Mis
28    7283


ROC Analysis in
Index:   CFI
Classification:  C
Estimation Method:   MLR
Level-2 Sample Size:     50
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
   Classification Index Estimator Level-2 SS Level-1 SS       AUC
29              C   CFI       MLR         50        ALL 0.8275218
   partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
29   0.6742161    0.8274696 0.9560915   0.7131802   0.8352049  7124
   Num-Mis
29   20417


ROC Analysis in
Index:   CFI
Classification:  C
Estimation Method:   MLR
Level-2 Sample Size:     50
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
   Classification Index Estimator Level-2 SS Level-1 SS       AUC
30              C   CFI       MLR         50          5 0.7471177
   partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
30   0.6075988    0.7481035 0.9289858   0.6118099   0.7627953  2032
   Num-Mis
30    5724


ROC Analysis in
Index:   CFI
Classification:  C
Estimation Method:   MLR
Level-2 Sample Size:     50
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
   Classification Index Estimator Level-2 SS Level-1 SS       AUC
31              C   CFI       MLR         50         10 0.8594728
   partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
31   0.6898751    0.8590964  0.956094   0.7341679   0.8667784  2387
   Num-Mis
31    6790


ROC Analysis in
Index:   CFI
Classification:  C
Estimation Method:   MLR
Level-2 Sample Size:     50
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
   Classification Index Estimator Level-2 SS Level-1 SS       AUC
32              C   CFI       MLR         50         30 0.8939235
   partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
32   0.7193833    0.8892199 0.9789563   0.7388334   0.9693161  2705
   Num-Mis
32    7903


ROC Analysis in
Index:   CFI
Classification:  C
Estimation Method:   MLR
Level-2 Sample Size:     100
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
   Classification Index Estimator Level-2 SS Level-1 SS       AUC
33              C   CFI       MLR        100        ALL 0.8925726
   partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
33    0.735141    0.8981704  0.973744   0.7497926   0.9183928  8014
   Num-Mis
33   22901


ROC Analysis in
Index:   CFI
Classification:  C
Estimation Method:   MLR
Level-2 Sample Size:     100
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
   Classification Index Estimator Level-2 SS Level-1 SS       AUC
34              C   CFI       MLR        100          5 0.8656921
   partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
34   0.7053162     0.866902 0.9594625   0.7001657   0.8973935  2417
   Num-Mis
34    6637


ROC Analysis in
Index:   CFI
Classification:  C
Estimation Method:   MLR
Level-2 Sample Size:     100
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
   Classification Index Estimator Level-2 SS Level-1 SS     AUC
35              C   CFI       MLR        100         10 0.90639
   partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
35   0.7480822    0.9005239 0.9739088   0.7676083    0.967029  2760
   Num-Mis
35    7823


ROC Analysis in
Index:   CFI
Classification:  C
Estimation Method:   MLR
Level-2 Sample Size:     100
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
   Classification Index Estimator Level-2 SS Level-1 SS       AUC
36              C   CFI       MLR        100         30 0.9182261
   partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
36   0.7754821    0.9123145 0.9906919   0.7916124   0.9700388  2837
   Num-Mis
36    8441


ROC Analysis in
Index:   CFI
Classification:  C
Estimation Method:   MLR
Level-2 Sample Size:     200
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
   Classification Index Estimator Level-2 SS Level-1 SS       AUC
37              C   CFI       MLR        200        ALL 0.9174918
   partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
37    0.777691    0.9114843 0.9808534    0.767191   0.9730926  8585
   Num-Mis
37   24475


ROC Analysis in
Index:   CFI
Classification:  C
Estimation Method:   MLR
Level-2 Sample Size:     200
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
   Classification Index Estimator Level-2 SS Level-1 SS       AUC
38              C   CFI       MLR        200          5 0.9113465
   partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
38   0.7610442    0.9063362  0.976339   0.7870717   0.9577617  2770
   Num-Mis
38    7472


ROC Analysis in
Index:   CFI
Classification:  C
Estimation Method:   MLR
Level-2 Sample Size:     200
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
   Classification Index Estimator Level-2 SS Level-1 SS       AUC
39              C   CFI       MLR        200         10 0.9233906
   partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
39   0.7882468     0.915123 0.9864381   0.8094724    0.975424  2889
   Num-Mis
39    8340


ROC Analysis in
Index:   CFI
Classification:  C
Estimation Method:   MLR
Level-2 Sample Size:     200
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
   Classification Index Estimator Level-2 SS Level-1 SS      AUC
40              C   CFI       MLR        200         30 0.937833
   partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
40   0.8273678    0.9319807  0.994956   0.8237331   0.9856459  2926
   Num-Mis
40    8663


ROC Analysis in
Index:   CFI
Classification:  C
Estimation Method:   ULSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
   Classification Index Estimator Level-2 SS Level-1 SS       AUC
41              C   CFI     ULSMV        ALL        ALL 0.7798031
   partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
41   0.5866553    0.8412403 0.9734562   0.6276138   0.8884364 27515
   Num-Mis
41   71584


ROC Analysis in
Index:   CFI
Classification:  C
Estimation Method:   ULSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
   Classification Index Estimator Level-2 SS Level-1 SS       AUC
42              C   CFI     ULSMV        ALL          5 0.7752069
   partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
42   0.6176273    0.8065222 0.9667542    0.697038   0.7937759  7500
   Num-Mis
42   19987


ROC Analysis in
Index:   CFI
Classification:  C
Estimation Method:   ULSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
   Classification Index Estimator Level-2 SS Level-1 SS       AUC
43              C   CFI     ULSMV        ALL         10 0.8181419
   partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
43   0.6245776    0.8541478 0.9702485   0.6865481   0.8881149  9333
   Num-Mis
43   24300


ROC Analysis in
Index:   CFI
Classification:  C
Estimation Method:   ULSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
   Classification Index Estimator Level-2 SS Level-1 SS       AUC
44              C   CFI     ULSMV        ALL         30 0.7621806
   partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
44   0.5610763    0.7254941 0.9874081   0.5312305   0.9750047 10682
   Num-Mis
44   27297


ROC Analysis in
Index:   CFI
Classification:  C
Estimation Method:   ULSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
   Classification Index Estimator Level-2 SS Level-1 SS       AUC
45              C   CFI     ULSMV         30        ALL 0.6278113
   partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
45   0.5238918    0.7196159 0.9823718   0.4308202   0.8209128  5208
   Num-Mis
45   13635


ROC Analysis in
Index:   CFI
Classification:  C
Estimation Method:   ULSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
   Classification Index Estimator Level-2 SS Level-1 SS       AUC
46              C   CFI     ULSMV         30          5 0.6168203
   partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
46   0.5310738     0.637815 0.9806827   0.6326595   0.5747899  1203
   Num-Mis
46    3200


ROC Analysis in
Index:   CFI
Classification:  C
Estimation Method:   ULSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
   Classification Index Estimator Level-2 SS Level-1 SS       AUC
47              C   CFI     ULSMV         30         10 0.6650954
   partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
47   0.5349914    0.7255216 0.9775778   0.4980545   0.7949913  1719
   Num-Mis
47    4632


ROC Analysis in
Index:   CFI
Classification:  C
Estimation Method:   ULSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
   Classification Index Estimator Level-2 SS Level-1 SS       AUC
48              C   CFI     ULSMV         30         30 0.6141232
   partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
48   0.5160821    0.4807183 0.9876449   0.2677925   0.9750656  2286
   Num-Mis
48    5803


ROC Analysis in
Index:   CFI
Classification:  C
Estimation Method:   ULSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
   Classification Index Estimator Level-2 SS Level-1 SS       AUC
49              C   CFI     ULSMV         50        ALL 0.7121332
   partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
49   0.5538408    0.7704547 0.9734506   0.5522569   0.8256765  6356
   Num-Mis
49   16505


ROC Analysis in
Index:   CFI
Classification:  C
Estimation Method:   ULSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
   Classification Index Estimator Level-2 SS Level-1 SS       AUC
50              C   CFI     ULSMV         50          5 0.6856774
   partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
50   0.5640503    0.6994536 0.9674736   0.6466775   0.6388715  1595
   Num-Mis
50    4319


ROC Analysis in
Index:   CFI
Classification:  C
Estimation Method:   ULSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
   Classification Index Estimator Level-2 SS Level-1 SS       AUC
51              C   CFI     ULSMV         50         10 0.7742024
   partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
51   0.6032993    0.8011223 0.9703256   0.6588002   0.8044298  2122
   Num-Mis
51    5551


ROC Analysis in
Index:   CFI
Classification:  C
Estimation Method:   ULSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
   Classification Index Estimator Level-2 SS Level-1 SS       AUC
52              C   CFI     ULSMV         50         30 0.6942227
   partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
52   0.5346903    0.6433136 0.9886184   0.4165787   0.9742327  2639
   Num-Mis
52    6635


ROC Analysis in
Index:   CFI
Classification:  C
Estimation Method:   ULSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
   Classification Index Estimator Level-2 SS Level-1 SS       AUC
53              C   CFI     ULSMV        100        ALL 0.8271769
   partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
53   0.6338422     0.855105 0.9705003   0.6666496   0.9060964  7529
   Num-Mis
53   19571


ROC Analysis in
Index:   CFI
Classification:  C
Estimation Method:   ULSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
   Classification Index Estimator Level-2 SS Level-1 SS       AUC
54              C   CFI     ULSMV        100          5 0.8294573
   partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
54   0.6705091    0.8404087 0.9651241   0.7086176    0.822736  2076
   Num-Mis
54    5570


ROC Analysis in
Index:   CFI
Classification:  C
Estimation Method:   ULSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
   Classification Index Estimator Level-2 SS Level-1 SS       AUC
55              C   CFI     ULSMV        100         10 0.8744946
   partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
55   0.7109626     0.885863 0.9702372   0.7421748    0.908362  2619
   Num-Mis
55    6741


ROC Analysis in
Index:   CFI
Classification:  C
Estimation Method:   ULSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
   Classification Index Estimator Level-2 SS Level-1 SS       AUC
56              C   CFI     ULSMV        100         30 0.7976433
   partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
56   0.5841222    0.7396644 0.9888685   0.5831956   0.9671842  2834
   Num-Mis
56    7260


ROC Analysis in
Index:   CFI
Classification:  C
Estimation Method:   ULSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
   Classification Index Estimator Level-2 SS Level-1 SS       AUC
57              C   CFI     ULSMV        200        ALL 0.9111187
   partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
57   0.7638665    0.9186153 0.9750099   0.7849861   0.9566611  8422
   Num-Mis
57   21873


ROC Analysis in
Index:   CFI
Classification:  C
Estimation Method:   ULSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
   Classification Index Estimator Level-2 SS Level-1 SS       AUC
58              C   CFI     ULSMV        200          5 0.8983734
   partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
58   0.7380819    0.9081969  0.966701   0.7663091    0.950495  2626
   Num-Mis
58    6898


ROC Analysis in
Index:   CFI
Classification:  C
Estimation Method:   ULSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
   Classification Index Estimator Level-2 SS Level-1 SS       AUC
59              C   CFI     ULSMV        200         10 0.9122291
   partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
59   0.7605967    0.9066453  0.970083   0.7959599   0.9707623  2873
   Num-Mis
59    7376


ROC Analysis in
Index:   CFI
Classification:  C
Estimation Method:   ULSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
   Classification Index Estimator Level-2 SS Level-1 SS      AUC
60              C   CFI     ULSMV        200         30 0.929114
   partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
60   0.8052503    0.9211932 0.9872994    0.794315   0.9750257  2923
   Num-Mis
60    7599


ROC Analysis in
Index:   CFI
Classification:  C
Estimation Method:   WLSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
   Classification Index Estimator Level-2 SS Level-1 SS       AUC
61              C   CFI     WLSMV        ALL        ALL 0.8357895
   partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
61   0.6489059    0.8709323 0.9824695   0.6821233    0.896391 25977
   Num-Mis
61   66637


ROC Analysis in
Index:   CFI
Classification:  C
Estimation Method:   WLSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
   Classification Index Estimator Level-2 SS Level-1 SS       AUC
62              C   CFI     WLSMV        ALL          5 0.8182967
   partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
62   0.6556458     0.835024  0.971518    0.703868   0.8380231  7015
   Num-Mis
62   18523


ROC Analysis in
Index:   CFI
Classification:  C
Estimation Method:   WLSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
   Classification Index Estimator Level-2 SS Level-1 SS      AUC
63              C   CFI     WLSMV        ALL         10 0.865767
   partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
63   0.6855386    0.8802575 0.9823036   0.7244042   0.9024474  8785
   Num-Mis
63   22664


ROC Analysis in
Index:   CFI
Classification:  C
Estimation Method:   WLSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
   Classification Index Estimator Level-2 SS Level-1 SS       AUC
64              C   CFI     WLSMV        ALL         30 0.8412149
   partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
64   0.6249339    0.7965717 0.9950584   0.6692731   0.9510661 10177
   Num-Mis
64   25450


ROC Analysis in
Index:   CFI
Classification:  C
Estimation Method:   WLSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
   Classification Index Estimator Level-2 SS Level-1 SS      AUC
65              C   CFI     WLSMV         30        ALL 0.705109
   partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
65   0.5521048     0.757926 0.9826179   0.5457666   0.8114224  4654
   Num-Mis
65   12278


ROC Analysis in
Index:   CFI
Classification:  C
Estimation Method:   WLSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
   Classification Index Estimator Level-2 SS Level-1 SS       AUC
66              C   CFI     WLSMV         30          5 0.6751482
   partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
66    0.558398    0.6803462 0.9364349   0.4547051   0.8068506  1065
   Num-Mis
66    2886


ROC Analysis in
Index:   CFI
Classification:  C
Estimation Method:   WLSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
   Classification Index Estimator Level-2 SS Level-1 SS       AUC
67              C   CFI     WLSMV         30         10 0.7628488
   partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
67   0.5875928    0.7762223  0.981013   0.6241822   0.8044619  1524
   Num-Mis
67    4131


ROC Analysis in
Index:   CFI
Classification:  C
Estimation Method:   WLSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
   Classification Index Estimator Level-2 SS Level-1 SS       AUC
68              C   CFI     WLSMV         30         30 0.6995549
   partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
68   0.5374463    0.5643921 0.9866408    0.397453   0.9772397  2065
   Num-Mis
68    5261


ROC Analysis in
Index:   CFI
Classification:  C
Estimation Method:   WLSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
   Classification Index Estimator Level-2 SS Level-1 SS       AUC
69              C   CFI     WLSMV         50        ALL 0.8015764
   partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
69   0.6250641    0.8299277 0.9798783   0.6469894   0.8480342  5850
   Num-Mis
69   15246


ROC Analysis in
Index:   CFI
Classification:  C
Estimation Method:   WLSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
   Classification Index Estimator Level-2 SS Level-1 SS       AUC
70              C   CFI     WLSMV         50          5 0.7473706
   partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
70   0.5957039    0.7514255 0.9637368   0.6177616   0.7687943  1410
   Num-Mis
70    3851


ROC Analysis in
Index:   CFI
Classification:  C
Estimation Method:   WLSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
   Classification Index Estimator Level-2 SS Level-1 SS       AUC
71              C   CFI     WLSMV         50         10 0.8428596
   partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
71   0.6632875    0.8409166 0.9759444   0.6878114   0.8798587  1981
   Num-Mis
71    5218


ROC Analysis in
Index:   CFI
Classification:  C
Estimation Method:   WLSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
   Classification Index Estimator Level-2 SS Level-1 SS       AUC
72              C   CFI     WLSMV         50         30 0.8281302
   partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
72   0.6122097    0.7710663 0.9922327   0.6302412   0.9658398  2459
   Num-Mis
72    6177


ROC Analysis in
Index:   CFI
Classification:  C
Estimation Method:   WLSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
   Classification Index Estimator Level-2 SS Level-1 SS       AUC
73              C   CFI     WLSMV        100        ALL 0.8760433
   partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
73   0.7031344    0.8913777 0.9788709   0.7077618   0.9363889  7200
   Num-Mis
73   18372


ROC Analysis in
Index:   CFI
Classification:  C
Estimation Method:   WLSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
   Classification Index Estimator Level-2 SS Level-1 SS     AUC
74              C   CFI     WLSMV        100          5 0.85872
   partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
74   0.6899487    0.8623435 0.9715529   0.7111238   0.8796389  1994
   Num-Mis
74    5241


ROC Analysis in
Index:   CFI
Classification:  C
Estimation Method:   WLSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
   Classification Index Estimator Level-2 SS Level-1 SS       AUC
75              C   CFI     WLSMV        100         10 0.8926447
   partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
75   0.7212663    0.8822644 0.9829252   0.7456601   0.9470279  2473
   Num-Mis
75    6279


ROC Analysis in
Index:   CFI
Classification:  C
Estimation Method:   WLSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
   Classification Index Estimator Level-2 SS Level-1 SS       AUC
76              C   CFI     WLSMV        100         30 0.8890305
   partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
76   0.7060544    0.8628855 0.9943983    0.734676   0.9773143  2733
   Num-Mis
76    6852


ROC Analysis in
Index:   CFI
Classification:  C
Estimation Method:   WLSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
   Classification Index Estimator Level-2 SS Level-1 SS       AUC
77              C   CFI     WLSMV        200        ALL 0.9095945
   partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
77   0.7608657    0.8981726 0.9864019   0.7562798   0.9597486  8273
   Num-Mis
77   20741


ROC Analysis in
Index:   CFI
Classification:  C
Estimation Method:   WLSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
   Classification Index Estimator Level-2 SS Level-1 SS       AUC
78              C   CFI     WLSMV        200          5 0.9038556
   partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
78   0.7424261    0.8965061 0.9786908   0.7686784   0.9622938  2546
   Num-Mis
78    6545


ROC Analysis in
Index:   CFI
Classification:  C
Estimation Method:   WLSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
   Classification Index Estimator Level-2 SS Level-1 SS       AUC
79              C   CFI     WLSMV        200         10 0.9170796
   partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
79   0.7719748    0.9070947 0.9880208   0.7832575   0.9811186  2807
   Num-Mis
79    7036


ROC Analysis in
Index:   CFI
Classification:  C
Estimation Method:   WLSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
   Classification Index Estimator Level-2 SS Level-1 SS       AUC
80              C   CFI     WLSMV        200         30 0.9295252
   partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
80   0.8060368    0.9189402 0.9972288   0.8076816   0.9691781  2920
   Num-Mis
80    7160


ROC Analysis in
Index:   TLI
Classification:  C
Estimation Method:   ALL
Level-2 Sample Size:     ALL
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
   Classification Index Estimator Level-2 SS Level-1 SS       AUC
81              C   TLI       ALL        ALL        ALL 0.8154291
   partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
81   0.6374691    0.8453772 0.9724203   0.7022691   0.8549251 83510
   Num-Mis
81  223868


ROC Analysis in
Index:   TLI
Classification:  C
Estimation Method:   ALL
Level-2 Sample Size:     ALL
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
   Classification Index Estimator Level-2 SS Level-1 SS      AUC
82              C   TLI       ALL        ALL          5 0.783663
   partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
82   0.6491645     0.798945 0.9647189   0.7485637   0.7518457 23460
   Num-Mis
82   63082


ROC Analysis in
Index:   TLI
Classification:  C
Estimation Method:   ALL
Level-2 Sample Size:     ALL
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
   Classification Index Estimator Level-2 SS Level-1 SS       AUC
83              C   TLI       ALL        ALL         10 0.8414227
   partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
83   0.6718036    0.8592732  0.965769   0.7172263   0.8729703 28208
   Num-Mis
83   75749


ROC Analysis in
Index:   TLI
Classification:  C
Estimation Method:   ALL
Level-2 Sample Size:     ALL
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
   Classification Index Estimator Level-2 SS Level-1 SS      AUC
84              C   TLI       ALL        ALL         30 0.828488
   partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
84   0.6116796    0.8095334 0.9826968   0.6425909   0.9445387 31842
   Num-Mis
84   85037


ROC Analysis in
Index:   TLI
Classification:  C
Estimation Method:   ALL
Level-2 Sample Size:     30
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
   Classification Index Estimator Level-2 SS Level-1 SS       AUC
85              C   TLI       ALL         30        ALL 0.6783674
   partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
85   0.5436979    0.7126667  0.961426   0.5872725   0.7385913 16157
   Num-Mis
85   43767


ROC Analysis in
Index:   TLI
Classification:  C
Estimation Method:   ALL
Level-2 Sample Size:     30
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
   Classification Index Estimator Level-2 SS Level-1 SS       AUC
86              C   TLI       ALL         30          5 0.6351997
   partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
86   0.5462489    0.6393891 0.8781324   0.4871199   0.7196874  3994
   Num-Mis
86   10825


ROC Analysis in
Index:   TLI
Classification:  C
Estimation Method:   ALL
Level-2 Sample Size:     30
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
   Classification Index Estimator Level-2 SS Level-1 SS       AUC
87              C   TLI       ALL         30         10 0.7069393
   partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
87   0.5602248    0.7206438 0.9511859   0.5967089   0.7280453  5297
   Num-Mis
87   14595


ROC Analysis in
Index:   TLI
Classification:  C
Estimation Method:   ALL
Level-2 Sample Size:     30
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
   Classification Index Estimator Level-2 SS Level-1 SS      AUC
88              C   TLI       ALL         30         30 0.705614
   partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
88   0.5352397    0.6395661 0.9651658   0.4665068   0.9429071  6866
   Num-Mis
88   18347


ROC Analysis in
Index:   TLI
Classification:  C
Estimation Method:   ALL
Level-2 Sample Size:     50
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
   Classification Index Estimator Level-2 SS Level-1 SS       AUC
89              C   TLI       ALL         50        ALL 0.7732497
   partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
89   0.5970366    0.8019055 0.9644069   0.6410827   0.8259183 19330
   Num-Mis
89   52168


ROC Analysis in
Index:   TLI
Classification:  C
Estimation Method:   ALL
Level-2 Sample Size:     50
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
   Classification Index Estimator Level-2 SS Level-1 SS       AUC
90              C   TLI       ALL         50          5 0.7191502
   partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
90   0.5845689    0.7265479 0.9476751   0.6525838   0.6807624  5037
   Num-Mis
90   13894


ROC Analysis in
Index:   TLI
Classification:  C
Estimation Method:   ALL
Level-2 Sample Size:     50
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
   Classification Index Estimator Level-2 SS Level-1 SS       AUC
91              C   TLI       ALL         50         10 0.8177403
   partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
91   0.6415712    0.8299024 0.9584409   0.6822712   0.8432974  6490
   Num-Mis
91   17559


ROC Analysis in
Index:   TLI
Classification:  C
Estimation Method:   ALL
Level-2 Sample Size:     50
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
   Classification Index Estimator Level-2 SS Level-1 SS       AUC
92              C   TLI       ALL         50         30 0.7949029
   partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
92   0.5795823    0.7637446 0.9801062   0.5883659   0.9689863  7803
   Num-Mis
92   20715


ROC Analysis in
Index:   TLI
Classification:  C
Estimation Method:   ALL
Level-2 Sample Size:     100
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
   Classification Index Estimator Level-2 SS Level-1 SS       AUC
93              C   TLI       ALL        100        ALL 0.8635247
   partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
93   0.6842496    0.8842822 0.9724008   0.7223555   0.9053775 22743
   Num-Mis
93   60844


ROC Analysis in
Index:   TLI
Classification:  C
Estimation Method:   ALL
Level-2 Sample Size:     100
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
   Classification Index Estimator Level-2 SS Level-1 SS       AUC
94              C   TLI       ALL        100          5 0.8491838
   partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
94   0.6869993    0.8589971 0.9607595    0.722547   0.8453831  6487
   Num-Mis
94   17448


ROC Analysis in
Index:   TLI
Classification:  C
Estimation Method:   ALL
Level-2 Sample Size:     100
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
   Classification Index Estimator Level-2 SS Level-1 SS       AUC
95              C   TLI       ALL        100         10 0.8893161
   partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
95   0.7244966    0.9009564 0.9683183    0.739241    0.947784  7852
   Num-Mis
95   20843


ROC Analysis in
Index:   TLI
Classification:  C
Estimation Method:   ALL
Level-2 Sample Size:     100
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
   Classification Index Estimator Level-2 SS Level-1 SS       AUC
96              C   TLI       ALL        100         30 0.8648348
   partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
96   0.6585559    0.8415035 0.9892307   0.7010597   0.9710852  8404
   Num-Mis
96   22553


ROC Analysis in
Index:   TLI
Classification:  C
Estimation Method:   ALL
Level-2 Sample Size:     200
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
   Classification Index Estimator Level-2 SS Level-1 SS       AUC
97              C   TLI       ALL        200        ALL 0.9119842
   partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
97   0.7665023    0.9219697 0.9762583   0.7634635   0.9645174 25280
   Num-Mis
97   67089


ROC Analysis in
Index:   TLI
Classification:  C
Estimation Method:   ALL
Level-2 Sample Size:     200
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
   Classification Index Estimator Level-2 SS Level-1 SS      AUC
98              C   TLI       ALL        200          5 0.903624
   partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
98   0.7466085    0.9138568 0.9723597   0.7860387    0.940821  7942
   Num-Mis
98   20915


ROC Analysis in
Index:   TLI
Classification:  C
Estimation Method:   ALL
Level-2 Sample Size:     200
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
   Classification Index Estimator Level-2 SS Level-1 SS       AUC
99              C   TLI       ALL        200         10 0.9158909
   partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
99   0.7720997    0.9134396 0.9801945   0.7888098   0.9625394  8569
   Num-Mis
99   22752


ROC Analysis in
Index:   TLI
Classification:  C
Estimation Method:   ALL
Level-2 Sample Size:     200
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
100              C   TLI       ALL        200         30 0.9306531
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
100   0.8112868    0.9234488 0.9932717   0.7966015   0.9647622  8769
    Num-Mis
100   23422


ROC Analysis in
Index:   TLI
Classification:  C
Estimation Method:   MLR
Level-2 Sample Size:     ALL
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS      AUC
101              C   TLI       MLR        ALL        ALL 0.840151
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
101   0.7102432    0.8399849 0.9614361   0.7502423   0.8304684 30018
    Num-Mis
101   85647


ROC Analysis in
Index:   TLI
Classification:  C
Estimation Method:   MLR
Level-2 Sample Size:     ALL
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
102              C   TLI       MLR        ALL          5 0.7724962
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
102   0.6775671    0.7760302  0.964785    0.814016   0.6634992  8945
    Num-Mis
102   24572


ROC Analysis in
Index:   TLI
Classification:  C
Estimation Method:   MLR
Level-2 Sample Size:     ALL
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
103              C   TLI       MLR        ALL         10 0.8531109
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
103   0.7169863    0.8552883 0.9603077   0.7721383   0.8157582 10090
    Num-Mis
103   28785


ROC Analysis in
Index:   TLI
Classification:  C
Estimation Method:   MLR
Level-2 Sample Size:     ALL
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
104              C   TLI       MLR        ALL         30 0.8971116
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
104   0.7417206    0.8913148 0.9722676   0.7333849   0.9326231 10983
    Num-Mis
104   32290


ROC Analysis in
Index:   TLI
Classification:  C
Estimation Method:   MLR
Level-2 Sample Size:     30
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
105              C   TLI       MLR         30        ALL 0.7456672
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
105   0.6387466    0.7383243 0.9284345   0.7249356   0.6978554  6295
    Num-Mis
105   17854


ROC Analysis in
Index:   TLI
Classification:  C
Estimation Method:   MLR
Level-2 Sample Size:     30
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
106              C   TLI       MLR         30          5 0.6641781
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
106   0.5690729    0.6627573 0.8668112   0.6782022   0.5816918  1726
    Num-Mis
106    4739


ROC Analysis in
Index:   TLI
Classification:  C
Estimation Method:   MLR
Level-2 Sample Size:     30
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
107              C   TLI       MLR         30         10 0.7731483
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
107   0.6218885    0.7717603 0.9015849   0.6097394    0.815482  2054
    Num-Mis
107    5832


ROC Analysis in
Index:   TLI
Classification:  C
Estimation Method:   MLR
Level-2 Sample Size:     30
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
108              C   TLI       MLR         30         30 0.8734357
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
108   0.6856441    0.8762171 0.9486743   0.6932583   0.9662028  2515
    Num-Mis
108    7283


ROC Analysis in
Index:   TLI
Classification:  C
Estimation Method:   MLR
Level-2 Sample Size:     50
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
109              C   TLI       MLR         50        ALL 0.8270287
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
109   0.6741931    0.8269389 0.9473098   0.7126904   0.8352049  7124
    Num-Mis
109   20417


ROC Analysis in
Index:   TLI
Classification:  C
Estimation Method:   MLR
Level-2 Sample Size:     50
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
110              C   TLI       MLR         50          5 0.7461638
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
110   0.6074848    0.7471875 0.9147829   0.6100629   0.7627953  2032
    Num-Mis
110    5724


ROC Analysis in
Index:   TLI
Classification:  C
Estimation Method:   MLR
Level-2 Sample Size:     50
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
111              C   TLI       MLR         50         10 0.8591524
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
111   0.6898017    0.8587962 0.9473128   0.7337261   0.8667784  2387
    Num-Mis
111    6790


ROC Analysis in
Index:   TLI
Classification:  C
Estimation Method:   MLR
Level-2 Sample Size:     50
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
112              C   TLI       MLR         50         30 0.8939232
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
112   0.7193833    0.8892162 0.9747476   0.7388334   0.9693161  2705
    Num-Mis
112    7903


ROC Analysis in
Index:   TLI
Classification:  C
Estimation Method:   MLR
Level-2 Sample Size:     100
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
113              C   TLI       MLR        100        ALL 0.8925104
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
113   0.7351364    0.8982355 0.9684928   0.7497489   0.9183928  8014
    Num-Mis
113   22901


ROC Analysis in
Index:   TLI
Classification:  C
Estimation Method:   MLR
Level-2 Sample Size:     100
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
114              C   TLI       MLR        100          5 0.8654518
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
114   0.7052545    0.8667904  0.951355   0.6994124   0.8973935  2417
    Num-Mis
114    6637


ROC Analysis in
Index:   TLI
Classification:  C
Estimation Method:   MLR
Level-2 Sample Size:     100
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
115              C   TLI       MLR        100         10 0.9063845
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
115   0.7480809    0.9005234 0.9686906   0.7676083    0.967029  2760
    Num-Mis
115    7823


ROC Analysis in
Index:   TLI
Classification:  C
Estimation Method:   MLR
Level-2 Sample Size:     100
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
116              C   TLI       MLR        100         30 0.9182261
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
116   0.7754821    0.9123058 0.9888302   0.7916124   0.9700388  2837
    Num-Mis
116    8441


ROC Analysis in
Index:   TLI
Classification:  C
Estimation Method:   MLR
Level-2 Sample Size:     200
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS      AUC
117              C   TLI       MLR        200        ALL 0.917491
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
117    0.777691    0.9114787 0.9770241    0.767191   0.9730926  8585
    Num-Mis
117   24475


ROC Analysis in
Index:   TLI
Classification:  C
Estimation Method:   MLR
Level-2 Sample Size:     200
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
118              C   TLI       MLR        200          5 0.9113393
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
118    0.761041    0.9063551 0.9720403   0.7898822   0.9548736  2770
    Num-Mis
118    7472


ROC Analysis in
Index:   TLI
Classification:  C
Estimation Method:   MLR
Level-2 Sample Size:     200
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
119              C   TLI       MLR        200         10 0.9233906
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
119   0.7882467    0.9151245 0.9837258   0.8094724    0.975424  2889
    Num-Mis
119    8340


ROC Analysis in
Index:   TLI
Classification:  C
Estimation Method:   MLR
Level-2 Sample Size:     200
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS      AUC
120              C   TLI       MLR        200         30 0.937833
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
120   0.8273679    0.9319869 0.9939472   0.8237331   0.9856459  2926
    Num-Mis
120    8663


ROC Analysis in
Index:   TLI
Classification:  C
Estimation Method:   ULSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
121              C   TLI     ULSMV        ALL        ALL 0.7796192
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
121   0.5866553    0.8410919 0.9681483   0.6272923   0.8884364 27515
    Num-Mis
121   71584


ROC Analysis in
Index:   TLI
Classification:  C
Estimation Method:   ULSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
122              C   TLI     ULSMV        ALL          5 0.7748064
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
122   0.6176125    0.8061621 0.9601091    0.696186   0.7937759  7500
    Num-Mis
122   19987


ROC Analysis in
Index:   TLI
Classification:  C
Estimation Method:   ULSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
123              C   TLI     ULSMV        ALL         10 0.8179194
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
123   0.6245731     0.853973  0.964404   0.6863835   0.8877934  9333
    Num-Mis
123   24300


ROC Analysis in
Index:   TLI
Classification:  C
Estimation Method:   ULSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
124              C   TLI     ULSMV        ALL         30 0.7621651
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
124   0.5610763    0.7266502 0.9848898   0.5310474   0.9750047 10682
    Num-Mis
124   27297


ROC Analysis in
Index:   TLI
Classification:  C
Estimation Method:   ULSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
125              C   TLI     ULSMV         30        ALL 0.6276549
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
125   0.5238918    0.7199324 0.9787259   0.4303788   0.8211053  5208
    Num-Mis
125   13635


ROC Analysis in
Index:   TLI
Classification:  C
Estimation Method:   ULSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
126              C   TLI     ULSMV         30          5 0.6163422
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
126   0.5310738    0.6376665 0.9768321   0.6323437   0.5747899  1203
    Num-Mis
126    3200


ROC Analysis in
Index:   TLI
Classification:  C
Estimation Method:   ULSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
127              C   TLI     ULSMV         30         10 0.6648603
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
127   0.5349914    0.7254904 0.9730934   0.4980545   0.7949913  1719
    Num-Mis
127    4632


ROC Analysis in
Index:   TLI
Classification:  C
Estimation Method:   ULSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
128              C   TLI     ULSMV         30         30 0.6141127
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
128   0.5160821     0.481824 0.9851739   0.2676202   0.9750656  2286
    Num-Mis
128    5803


ROC Analysis in
Index:   TLI
Classification:  C
Estimation Method:   ULSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS     AUC
129              C   TLI     ULSMV         50        ALL 0.71187
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
129   0.5538408    0.7703794 0.9681446   0.5517722   0.8256765  6356
    Num-Mis
129   16505


ROC Analysis in
Index:   TLI
Classification:  C
Estimation Method:   ULSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
130              C   TLI     ULSMV         50          5 0.6850962
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
130   0.5640503    0.6989685 0.9591129   0.6376476   0.6470219  1595
    Num-Mis
130    4319


ROC Analysis in
Index:   TLI
Classification:  C
Estimation Method:   ULSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
131              C   TLI     ULSMV         50         10 0.7738167
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
131   0.6032981    0.8008191 0.9659009   0.6633039    0.799246  2122
    Num-Mis
131    5551


ROC Analysis in
Index:   TLI
Classification:  C
Estimation Method:   ULSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
132              C   TLI     ULSMV         50         30 0.6942064
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
132   0.5346903    0.6446052 0.9863421   0.4165787   0.9742327  2639
    Num-Mis
132    6635


ROC Analysis in
Index:   TLI
Classification:  C
Estimation Method:   ULSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
133              C   TLI     ULSMV        100        ALL 0.8269825
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
133   0.6338397    0.8549978 0.9646003   0.6660876   0.9060964  7529
    Num-Mis
133   19571


ROC Analysis in
Index:   TLI
Classification:  C
Estimation Method:   ULSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
134              C   TLI     ULSMV        100          5 0.8290449
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
134   0.6704392    0.8401224 0.9580856   0.7075404   0.8232177  2076
    Num-Mis
134    5570


ROC Analysis in
Index:   TLI
Classification:  C
Estimation Method:   ULSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
135              C   TLI     ULSMV        100         10 0.8742862
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
135   0.7109224    0.8857096 0.9646792   0.7429165   0.9072165  2619
    Num-Mis
135    6741


ROC Analysis in
Index:   TLI
Classification:  C
Estimation Method:   ULSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
136              C   TLI     ULSMV        100         30 0.7976238
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
136   0.5841222    0.7401276 0.9866422   0.5831956   0.9671842  2834
    Num-Mis
136    7260


ROC Analysis in
Index:   TLI
Classification:  C
Estimation Method:   ULSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
137              C   TLI     ULSMV        200        ALL 0.9110594
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
137   0.7638469    0.9187087 0.9701152   0.7850775   0.9564237  8422
    Num-Mis
137   21873


ROC Analysis in
Index:   TLI
Classification:  C
Estimation Method:   ULSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
138              C   TLI     ULSMV        200          5 0.8982645
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
138   0.7380337    0.9083132 0.9600457   0.7657292    0.950495  2626
    Num-Mis
138    6898


ROC Analysis in
Index:   TLI
Classification:  C
Estimation Method:   ULSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
139              C   TLI     ULSMV        200         10 0.9121889
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
139   0.7605823    0.9067807 0.9648719   0.7966377     0.96937  2873
    Num-Mis
139    7376


ROC Analysis in
Index:   TLI
Classification:  C
Estimation Method:   ULSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
140              C   TLI     ULSMV        200         30 0.9291059
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
140   0.8052431     0.921359 0.9847592   0.7941834   0.9750257  2923
    Num-Mis
140    7599


ROC Analysis in
Index:   TLI
Classification:  C
Estimation Method:   WLSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
141              C   TLI     WLSMV        ALL        ALL 0.8356371
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
141   0.6489019    0.8707985 0.9789634   0.6819731    0.896391 25977
    Num-Mis
141   66637


ROC Analysis in
Index:   TLI
Classification:  C
Estimation Method:   WLSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
142              C   TLI     WLSMV        ALL          5 0.8179283
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
142   0.6556043    0.8346784 0.9658217   0.7034893   0.8380231  7015
    Num-Mis
142   18523


ROC Analysis in
Index:   TLI
Classification:  C
Estimation Method:   WLSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS      AUC
143              C   TLI     WLSMV        ALL         10 0.865665
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
143   0.6855238     0.880286 0.9787957   0.7244042   0.9022197  8785
    Num-Mis
143   22664


ROC Analysis in
Index:   TLI
Classification:  C
Estimation Method:   WLSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
144              C   TLI     WLSMV        ALL         30 0.8412064
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
144   0.6249339    0.7966459   0.99407   0.6691552   0.9510661 10177
    Num-Mis
144   25450


ROC Analysis in
Index:   TLI
Classification:  C
Estimation Method:   WLSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
145              C   TLI     WLSMV         30        ALL 0.7048094
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
145   0.5521048     0.757882 0.9794614   0.5471559   0.8094828  4654
    Num-Mis
145   12278


ROC Analysis in
Index:   TLI
Classification:  C
Estimation Method:   WLSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
146              C   TLI     WLSMV         30          5 0.6744182
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
146   0.5583961    0.6797326 0.9237219   0.4525983   0.8068506  1065
    Num-Mis
146    2886


ROC Analysis in
Index:   TLI
Classification:  C
Estimation Method:   WLSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
147              C   TLI     WLSMV         30         10 0.7624958
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
147   0.5875928    0.7761803 0.9775887   0.6266053   0.8011811  1524
    Num-Mis
147    4131


ROC Analysis in
Index:   TLI
Classification:  C
Estimation Method:   WLSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
148              C   TLI     WLSMV         30         30 0.6995207
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
148   0.5374463    0.5649521 0.9839689   0.3972629   0.9772397  2065
    Num-Mis
148    5261


ROC Analysis in
Index:   TLI
Classification:  C
Estimation Method:   WLSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
149              C   TLI     WLSMV         50        ALL 0.8012832
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
149   0.6250625    0.8297733 0.9760418   0.6484324   0.8463248  5850
    Num-Mis
149   15246


ROC Analysis in
Index:   TLI
Classification:  C
Estimation Method:   WLSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
150              C   TLI     WLSMV         50          5 0.7467564
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
150    0.595668    0.7508619 0.9564871   0.6172423   0.7687943  1410
    Num-Mis
150    3851


ROC Analysis in
Index:   TLI
Classification:  C
Estimation Method:   WLSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
151              C   TLI     WLSMV         50         10 0.8426485
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
151   0.6632623    0.8409157 0.9711333   0.6870448   0.8798587  1981
    Num-Mis
151    5218


ROC Analysis in
Index:   TLI
Classification:  C
Estimation Method:   WLSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
152              C   TLI     WLSMV         50         30 0.8281087
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
152   0.6122097    0.7712856 0.9906337    0.629108   0.9666531  2459
    Num-Mis
152    6177


ROC Analysis in
Index:   TLI
Classification:  C
Estimation Method:   WLSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
153              C   TLI     WLSMV        100        ALL 0.8759726
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
153   0.7031322    0.8915281 0.9746451   0.7077074   0.9363889  7200
    Num-Mis
153   18372


ROC Analysis in
Index:   TLI
Classification:  C
Estimation Method:   WLSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
154              C   TLI     WLSMV        100          5 0.8584825
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
154   0.6898969    0.8622438 0.9658635   0.7107422   0.8796389  1994
    Num-Mis
154    5241


ROC Analysis in
Index:   TLI
Classification:  C
Estimation Method:   WLSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
155              C   TLI     WLSMV        100         10 0.8926364
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
155   0.7212664    0.8822935 0.9795102   0.7456601   0.9470279  2473
    Num-Mis
155    6279


ROC Analysis in
Index:   TLI
Classification:  C
Estimation Method:   WLSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
156              C   TLI     WLSMV        100         30 0.8890305
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
156   0.7060544    0.8628855 0.9932779    0.734676   0.9773143  2733
    Num-Mis
156    6852


ROC Analysis in
Index:   TLI
Classification:  C
Estimation Method:   WLSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
157              C   TLI     WLSMV        200        ALL 0.9095922
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
157   0.7608658    0.8982065 0.9836823   0.7562798   0.9597486  8273
    Num-Mis
157   20741


ROC Analysis in
Index:   TLI
Classification:  C
Estimation Method:   WLSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
158              C   TLI     WLSMV        200          5 0.9038393
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
158   0.7424206    0.8965787 0.9744289   0.7685256   0.9622938  2546
    Num-Mis
158    6545


ROC Analysis in
Index:   TLI
Classification:  C
Estimation Method:   WLSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
159              C   TLI     WLSMV        200         10 0.9170796
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
159   0.7719749     0.907095  0.985625   0.7832575   0.9811186  2807
    Num-Mis
159    7036


ROC Analysis in
Index:   TLI
Classification:  C
Estimation Method:   WLSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
160              C   TLI     WLSMV        200         30 0.9295252
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
160   0.8060368    0.9189383 0.9966745   0.8076816   0.9691781  2920
    Num-Mis
160    7160


ROC Analysis in
Index:   RMSEA
Classification:  C
Estimation Method:   ALL
Level-2 Sample Size:     ALL
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1

Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
161              C RMSEA       ALL        ALL        ALL 0.8034875
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
161   0.6348255    0.8299624 0.01504671   0.6849311    0.829003 83510
    Num-Mis
161  223868


ROC Analysis in
Index:   RMSEA
Classification:  C
Estimation Method:   ALL
Level-2 Sample Size:     ALL
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
162              C RMSEA       ALL        ALL          5 0.7762955
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
162   0.6444889    0.7899123 0.0190612   0.7223615    0.756924 23460
    Num-Mis
162   63082


ROC Analysis in
Index:   RMSEA
Classification:  C
Estimation Method:   ALL
Level-2 Sample Size:     ALL
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
163              C RMSEA       ALL        ALL         10 0.8315651
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
163   0.6680355    0.8474112 0.01570972   0.7250954   0.8356024 28208
    Num-Mis
163   75749


ROC Analysis in
Index:   RMSEA
Classification:  C
Estimation Method:   ALL
Level-2 Sample Size:     ALL
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
164              C RMSEA       ALL        ALL         30 0.8163124
    partial-AUC Smoothed-AUC   Threshold Specificity Sensitivity Num-C
164   0.6116796    0.7983801 0.008849246   0.6651222   0.8691351 31842
    Num-Mis
164   85037


ROC Analysis in
Index:   RMSEA
Classification:  C
Estimation Method:   ALL
Level-2 Sample Size:     30
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
165              C RMSEA       ALL         30        ALL 0.6636033
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
165   0.5437242    0.6877334 0.02017816   0.5494106   0.7335069 16157
    Num-Mis
165   43767


ROC Analysis in
Index:   RMSEA
Classification:  C
Estimation Method:   ALL
Level-2 Sample Size:     30
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
166              C RMSEA       ALL         30          5 0.6136518
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
166   0.5462427    0.6143865 0.02115398   0.6951548   0.4766826  3994
    Num-Mis
166   10825


ROC Analysis in
Index:   RMSEA
Classification:  C
Estimation Method:   ALL
Level-2 Sample Size:     30
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
167              C RMSEA       ALL         30         10 0.6812898
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
167   0.5602248    0.6797374 0.01823289   0.6227631   0.6570349  5297
    Num-Mis
167   14595


ROC Analysis in
Index:   RMSEA
Classification:  C
Estimation Method:   ALL
Level-2 Sample Size:     30
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
168              C RMSEA       ALL         30         30 0.6908905
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
168   0.5352397    0.6147019 0.02235183   0.3983758   0.9577629  6866
    Num-Mis
168   18347


ROC Analysis in
Index:   RMSEA
Classification:  C
Estimation Method:   ALL
Level-2 Sample Size:     50
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
169              C RMSEA       ALL         50        ALL 0.7581603
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
169   0.5962506    0.7779765 0.01627179   0.6356579   0.7764615 19330
    Num-Mis
169   52168


ROC Analysis in
Index:   RMSEA
Classification:  C
Estimation Method:   ALL
Level-2 Sample Size:     50
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
170              C RMSEA       ALL         50          5 0.7015911
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
170   0.5824211    0.7035449 0.02250211   0.6350223   0.6700417  5037
    Num-Mis
170   13894


ROC Analysis in
Index:   RMSEA
Classification:  C
Estimation Method:   ALL
Level-2 Sample Size:     50
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
171              C RMSEA       ALL         50         10 0.8011259
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
171   0.6345452     0.802155 0.01954048   0.6527707   0.8191063  6490
    Num-Mis
171   17559


ROC Analysis in
Index:   RMSEA
Classification:  C
Estimation Method:   ALL
Level-2 Sample Size:     50
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
172              C RMSEA       ALL         50         30 0.7807615
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
172   0.5795823    0.7343268 0.0131862   0.5508086     0.93259  7803
    Num-Mis
172   20715


ROC Analysis in
Index:   RMSEA
Classification:  C
Estimation Method:   ALL
Level-2 Sample Size:     100
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
173              C RMSEA       ALL        100        ALL 0.8541674
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
173   0.6781382    0.8623878 0.01574057   0.6975544   0.8917909 22743
    Num-Mis
173   60844


ROC Analysis in
Index:   RMSEA
Classification:  C
Estimation Method:   ALL
Level-2 Sample Size:     100
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
174              C RMSEA       ALL        100          5 0.8424137
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
174   0.6800951    0.8431596 0.02110738   0.6756075   0.8697395  6487
    Num-Mis
174   17448


ROC Analysis in
Index:   RMSEA
Classification:  C
Estimation Method:   ALL
Level-2 Sample Size:     100
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
175              C RMSEA       ALL        100         10 0.8859381
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
175   0.7158007    0.8766896 0.01664405   0.7200979   0.9430718  7852
    Num-Mis
175   20843


ROC Analysis in
Index:   RMSEA
Classification:  C
Estimation Method:   ALL
Level-2 Sample Size:     100
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
176              C RMSEA       ALL        100         30 0.8587824
    partial-AUC Smoothed-AUC   Threshold Specificity Sensitivity Num-C
176   0.6556541     0.824537 0.009659838   0.6874473   0.9613279  8404
    Num-Mis
176   22553


ROC Analysis in
Index:   RMSEA
Classification:  C
Estimation Method:   ALL
Level-2 Sample Size:     200
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
177              C RMSEA       ALL        200        ALL 0.9050678
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
177   0.7577637    0.8968035 0.01424754   0.7361117   0.9551028 25280
    Num-Mis
177   67089


ROC Analysis in
Index:   RMSEA
Classification:  C
Estimation Method:   ALL
Level-2 Sample Size:     200
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
178              C RMSEA       ALL        200          5 0.9020957
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
178   0.7417587    0.8956199 0.01743768   0.7640928   0.9529086  7942
    Num-Mis
178   20915


ROC Analysis in
Index:   RMSEA
Classification:  C
Estimation Method:   ALL
Level-2 Sample Size:     200
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
179              C RMSEA       ALL        200         10 0.9183942
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
179   0.7764974    0.9094456 0.01312592   0.7899525   0.9709418  8569
    Num-Mis
179   22752


ROC Analysis in
Index:   RMSEA
Classification:  C
Estimation Method:   ALL
Level-2 Sample Size:     200
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
180              C RMSEA       ALL        200         30 0.9293193
    partial-AUC Smoothed-AUC   Threshold Specificity Sensitivity Num-C
180    0.808054    0.9215232 0.006501705   0.8062505   0.9511917  8769
    Num-Mis
180   23422


ROC Analysis in
Index:   RMSEA
Classification:  C
Estimation Method:   MLR
Level-2 Sample Size:     ALL
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS      AUC
181              C RMSEA       MLR        ALL        ALL 0.829566
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
181   0.7055359    0.8290363 0.0235647   0.7346784   0.8425278 30018
    Num-Mis
181   85647


ROC Analysis in
Index:   RMSEA
Classification:  C
Estimation Method:   MLR
Level-2 Sample Size:     ALL
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
182              C RMSEA       MLR        ALL          5 0.7584874
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
182   0.6710397    0.7619003 0.0230764   0.7821504   0.6841811  8945
    Num-Mis
182   24572


ROC Analysis in
Index:   RMSEA
Classification:  C
Estimation Method:   MLR
Level-2 Sample Size:     ALL
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
183              C RMSEA       MLR        ALL         10 0.8427428
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
183   0.7113189    0.8437997 0.02464492   0.7452145   0.8400396 10090
    Num-Mis
183   28785


ROC Analysis in
Index:   RMSEA
Classification:  C
Estimation Method:   MLR
Level-2 Sample Size:     ALL
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
184              C RMSEA       MLR        ALL         30 0.8958457
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
184   0.7383457    0.8937592 0.01936476   0.7293589   0.9439133 10983
    Num-Mis
184   32290


ROC Analysis in
Index:   RMSEA
Classification:  C
Estimation Method:   MLR
Level-2 Sample Size:     30
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
185              C RMSEA       MLR         30        ALL 0.7251254
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
185   0.6336021    0.7204234 0.02941995   0.7407864   0.6598888  6295
    Num-Mis
185   17854


ROC Analysis in
Index:   RMSEA
Classification:  C
Estimation Method:   MLR
Level-2 Sample Size:     30
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
186              C RMSEA       MLR         30          5 0.6532608
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
186   0.5623187    0.6545669 0.05136179   0.5397763   0.6784473  1726
    Num-Mis
186    4739


ROC Analysis in
Index:   RMSEA
Classification:  C
Estimation Method:   MLR
Level-2 Sample Size:     30
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
187              C RMSEA       MLR         30         10 0.7547477
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
187   0.6117622    0.7556622 0.03496651   0.6423182   0.7439143  2054
    Num-Mis
187    5832


ROC Analysis in
Index:   RMSEA
Classification:  C
Estimation Method:   MLR
Level-2 Sample Size:     30
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
188              C RMSEA       MLR         30         30 0.8703716
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
188   0.6788433    0.8732965 0.02555647   0.6995744   0.9606362  2515
    Num-Mis
188    7283


ROC Analysis in
Index:   RMSEA
Classification:  C
Estimation Method:   MLR
Level-2 Sample Size:     50
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
189              C RMSEA       MLR         50        ALL 0.8140085
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
189   0.6693127    0.8133423 0.0255375   0.7162659    0.814009  7124
    Num-Mis
189   20417


ROC Analysis in
Index:   RMSEA
Classification:  C
Estimation Method:   MLR
Level-2 Sample Size:     50
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
190              C RMSEA       MLR         50          5 0.7301571
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
190   0.5993609    0.7307453 0.03434206   0.5931167   0.7519685  2032
    Num-Mis
190    5724


ROC Analysis in
Index:   RMSEA
Classification:  C
Estimation Method:   MLR
Level-2 Sample Size:     50
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
191              C RMSEA       MLR         50         10 0.8501789
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
191   0.6796767    0.8508748 0.02600274   0.7248895   0.8491831  2387
    Num-Mis
191    6790


ROC Analysis in
Index:   RMSEA
Classification:  C
Estimation Method:   MLR
Level-2 Sample Size:     50
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
192              C RMSEA       MLR         50         30 0.8924885
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
192   0.7153052     0.887166 0.01802007   0.7355435   0.9789279  2705
    Num-Mis
192    7903


ROC Analysis in
Index:   RMSEA
Classification:  C
Estimation Method:   MLR
Level-2 Sample Size:     100
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
193              C RMSEA       MLR        100        ALL 0.8892145
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
193   0.7294182    0.8923975 0.02018153   0.7469106   0.9162715  8014
    Num-Mis
193   22901


ROC Analysis in
Index:   RMSEA
Classification:  C
Estimation Method:   MLR
Level-2 Sample Size:     100
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
194              C RMSEA       MLR        100          5 0.8593737
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
194   0.6951153    0.8591584 0.02401142   0.7168902   0.8709144  2417
    Num-Mis
194    6637


ROC Analysis in
Index:   RMSEA
Classification:  C
Estimation Method:   MLR
Level-2 Sample Size:     100
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
195              C RMSEA       MLR        100         10 0.9034069
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
195   0.7400265    0.8965552 0.01993603   0.7621117   0.9681159  2760
    Num-Mis
195    7823


ROC Analysis in
Index:   RMSEA
Classification:  C
Estimation Method:   MLR
Level-2 Sample Size:     100
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
196              C RMSEA       MLR        100         30 0.9166951
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
196   0.7715651    0.9106089 0.01208103   0.7788177   0.9816708  2837
    Num-Mis
196    8441


ROC Analysis in
Index:   RMSEA
Classification:  C
Estimation Method:   MLR
Level-2 Sample Size:     200
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
197              C RMSEA       MLR        200        ALL 0.9153103
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
197   0.7717454    0.9081077 0.01747611   0.7593871   0.9793826  8585
    Num-Mis
197   24475


ROC Analysis in
Index:   RMSEA
Classification:  C
Estimation Method:   MLR
Level-2 Sample Size:     200
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS      AUC
198              C RMSEA       MLR        200          5 0.908106
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
198   0.7506671    0.9004179 0.01873213   0.7815846   0.9642599  2770
    Num-Mis
198    7472


ROC Analysis in
Index:   RMSEA
Classification:  C
Estimation Method:   MLR
Level-2 Sample Size:     200
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
199              C RMSEA       MLR        200         10 0.9209784
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
199   0.7813742    0.9121518 0.01436028   0.8005995   0.9809623  2889
    Num-Mis
199    8340


ROC Analysis in
Index:   RMSEA
Classification:  C
Estimation Method:   MLR
Level-2 Sample Size:     200
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
200              C RMSEA       MLR        200         30 0.9368052
    partial-AUC Smoothed-AUC   Threshold Specificity Sensitivity Num-C
200   0.8245361    0.9301385 0.007803385   0.8320443    0.975393  2926
    Num-Mis
200    8663


ROC Analysis in
Index:   RMSEA
Classification:  C
Estimation Method:   ULSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
201              C RMSEA     ULSMV        ALL        ALL 0.7704426
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
201   0.5866553    0.8066763 0.01185995   0.5961506   0.8550909 27515
    Num-Mis
201   71584


ROC Analysis in
Index:   RMSEA
Classification:  C
Estimation Method:   ULSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
202              C RMSEA     ULSMV        ALL          5 0.7838138
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
202   0.6167202    0.8072159 0.01690443   0.6560417   0.8151463  7500
    Num-Mis
202   19987


ROC Analysis in
Index:   RMSEA
Classification:  C
Estimation Method:   ULSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
203              C RMSEA     ULSMV        ALL         10 0.8171633
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
203   0.6243439     0.827912 0.01201753    0.692352   0.8431036  9333
    Num-Mis
203   24300


ROC Analysis in
Index:   RMSEA
Classification:  C
Estimation Method:   ULSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
204              C RMSEA     ULSMV        ALL         30 0.7538387
    partial-AUC Smoothed-AUC   Threshold Specificity Sensitivity Num-C
204   0.5610763    0.7079309 0.005718623   0.5515258   0.9045123 10682
    Num-Mis
204   27297


ROC Analysis in
Index:   RMSEA
Classification:  C
Estimation Method:   ULSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
205              C RMSEA     ULSMV         30        ALL 0.6279445
    partial-AUC Smoothed-AUC   Threshold Specificity Sensitivity Num-C
205   0.5238918    0.6754195 0.008175147   0.4483266   0.7712305  5208
    Num-Mis
205   13635


ROC Analysis in
Index:   RMSEA
Classification:  C
Estimation Method:   ULSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
206              C RMSEA     ULSMV         30          5 0.6206373
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
206   0.5310738    0.6348165 0.01115568   0.6193936    0.587395  1203
    Num-Mis
206    3200


ROC Analysis in
Index:   RMSEA
Classification:  C
Estimation Method:   ULSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
207              C RMSEA     ULSMV         30         10 0.6629961
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
207   0.5349914    0.6659627 0.01378362   0.4280156   0.8375073  1719
    Num-Mis
207    4632


ROC Analysis in
Index:   RMSEA
Classification:  C
Estimation Method:   ULSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1

Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in auc.roc(fit_roc[[key]], partial.auc = c(1, 0.8),
partial.auc.focus = "sp", : Partial AUC correction not defined for ROC
curves below the diagonal.

Warning in auc.roc(fit_roc[[key]], partial.auc = c(1, 0.8),
partial.auc.focus = "sp", : An upcoming version of pROC will set the
'transpose' argument to FALSE by default. Set transpose = TRUE explicitly
to keep the current behavior, or transpose = FALSE to adopt the new one
and silence this warning. Type help(coords_transpose) for additional
information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS      AUC
208              C RMSEA     ULSMV         30         30 0.391064
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
208          NA    0.5638286      -Inf           0           1  2286
    Num-Mis
208    5803


ROC Analysis in
Index:   RMSEA
Classification:  C
Estimation Method:   ULSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1

Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
209              C RMSEA     ULSMV         50        ALL 0.7060246
    partial-AUC Smoothed-AUC   Threshold Specificity Sensitivity Num-C
209   0.5538408    0.7283917 0.009163775   0.5896395    0.749056  6356
    Num-Mis
209   16505


ROC Analysis in
Index:   RMSEA
Classification:  C
Estimation Method:   ULSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
210              C RMSEA     ULSMV         50          5 0.6861974
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
210   0.5640503    0.6901852 0.01583009   0.6144941    0.660815  1595
    Num-Mis
210    4319


ROC Analysis in
Index:   RMSEA
Classification:  C
Estimation Method:   ULSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
211              C RMSEA     ULSMV         50         10 0.7701517
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
211   0.6028589    0.7609079 0.01398014   0.6141236   0.8020735  2122
    Num-Mis
211    5551


ROC Analysis in
Index:   RMSEA
Classification:  C
Estimation Method:   ULSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
212              C RMSEA     ULSMV         50         30 0.6875125
    partial-AUC Smoothed-AUC   Threshold Specificity Sensitivity Num-C
212   0.5346903    0.5874597 0.006639893   0.4298417   0.9067829  2639
    Num-Mis
212    6635


ROC Analysis in
Index:   RMSEA
Classification:  C
Estimation Method:   ULSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
213              C RMSEA     ULSMV        100        ALL 0.8126657
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
213   0.6323699    0.8102203 0.01150308   0.6706351   0.8391553  7529
    Num-Mis
213   19571


ROC Analysis in
Index:   RMSEA
Classification:  C
Estimation Method:   ULSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
214              C RMSEA     ULSMV        100          5 0.8293623
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
214   0.6671431    0.8303387 0.01689238   0.6998205   0.8261079  2076
    Num-Mis
214    5570


ROC Analysis in
Index:   RMSEA
Classification:  C
Estimation Method:   ULSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
215              C RMSEA     ULSMV        100         10 0.8703102
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
215   0.7062107    0.8633388 0.01378626   0.7070168   0.8999618  2619
    Num-Mis
215    6741


ROC Analysis in
Index:   RMSEA
Classification:  C
Estimation Method:   ULSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
216              C RMSEA     ULSMV        100         30 0.7891287
    partial-AUC Smoothed-AUC   Threshold Specificity Sensitivity Num-C
216   0.5841222    0.7149693 0.006433063   0.5742424   0.9244884  2834
    Num-Mis
216    7260


ROC Analysis in
Index:   RMSEA
Classification:  C
Estimation Method:   ULSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
217              C RMSEA     ULSMV        200        ALL 0.8926374
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
217   0.7518892    0.8887084 0.01302992   0.6894802   0.9469247  8422
    Num-Mis
217   21873


ROC Analysis in
Index:   RMSEA
Classification:  C
Estimation Method:   ULSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
218              C RMSEA     ULSMV        200          5 0.8964808
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
218   0.7371824    0.8950219 0.01458533   0.7839954    0.907083  2626
    Num-Mis
218    6898


ROC Analysis in
Index:   RMSEA
Classification:  C
Estimation Method:   ULSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
219              C RMSEA     ULSMV        200         10 0.9161194
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
219   0.7739693    0.9092792 0.0117016   0.8002983    0.943961  2873
    Num-Mis
219    7376


ROC Analysis in
Index:   RMSEA
Classification:  C
Estimation Method:   ULSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
220              C RMSEA     ULSMV        200         30 0.9257449
    partial-AUC Smoothed-AUC   Threshold Specificity Sensitivity Num-C
220   0.8069256    0.9254495 0.005540083   0.8048427   0.9291823  2923
    Num-Mis
220    7599


ROC Analysis in
Index:   RMSEA
Classification:  C
Estimation Method:   WLSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS     AUC
221              C RMSEA     WLSMV        ALL        ALL 0.83241
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
221   0.6481465    0.8644167 0.01381277   0.6886103   0.8722798 25977
    Num-Mis
221   66637


ROC Analysis in
Index:   RMSEA
Classification:  C
Estimation Method:   WLSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
222              C RMSEA     WLSMV        ALL          5 0.8203322
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
222   0.6546275    0.8365857 0.01855636   0.7188531   0.8185973  7015
    Num-Mis
222   18523


ROC Analysis in
Index:   RMSEA
Classification:  C
Estimation Method:   WLSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
223              C RMSEA     WLSMV        ALL         10 0.8664979
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
223      0.6852    0.8763196 0.01603517   0.7127538   0.9149687  8785
    Num-Mis
223   22664


ROC Analysis in
Index:   RMSEA
Classification:  C
Estimation Method:   WLSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS      AUC
224              C RMSEA     WLSMV        ALL         30 0.838768
    partial-AUC Smoothed-AUC   Threshold Specificity Sensitivity Num-C
224   0.6249339    0.8040076 0.007553795   0.6721807   0.9364253 10177
    Num-Mis
224   25450


ROC Analysis in
Index:   RMSEA
Classification:  C
Estimation Method:   WLSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
225              C RMSEA     WLSMV         30        ALL 0.6995061
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
225   0.5521048    0.7432134 0.01332833   0.5454397   0.7862069  4654
    Num-Mis
225   12278


ROC Analysis in
Index:   RMSEA
Classification:  C
Estimation Method:   WLSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
226              C RMSEA     WLSMV         30          5 0.6738844
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
226    0.558385    0.6790694 0.02123193   0.6141152   0.6517602  1065
    Num-Mis
226    2886


ROC Analysis in
Index:   RMSEA
Classification:  C
Estimation Method:   WLSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
227              C RMSEA     WLSMV         30         10 0.7589749
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
227   0.5875928    0.7673096 0.01803989   0.5517325   0.8628609  1524
    Num-Mis
227    4131


ROC Analysis in
Index:   RMSEA
Classification:  C
Estimation Method:   WLSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
228              C RMSEA     WLSMV         30         30 0.6942752
    partial-AUC Smoothed-AUC   Threshold Specificity Sensitivity Num-C
228   0.5374463    0.5727365 0.007649619   0.4455427   0.8953995  2065
    Num-Mis
228    5261


ROC Analysis in
Index:   RMSEA
Classification:  C
Estimation Method:   WLSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS      AUC
229              C RMSEA     WLSMV         50        ALL 0.789643
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
229   0.6246102    0.8109341 0.01059523   0.7182212   0.7358974  5850
    Num-Mis
229   15246


ROC Analysis in
Index:   RMSEA
Classification:  C
Estimation Method:   WLSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
230              C RMSEA     WLSMV         50          5 0.7426834
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
230   0.5950975    0.7469586  0.020484   0.6437289   0.7269504  1410
    Num-Mis
230    3851


ROC Analysis in
Index:   RMSEA
Classification:  C
Estimation Method:   WLSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
231              C RMSEA     WLSMV         50         10 0.8415311
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
231   0.6627301     0.838395 0.01849509   0.6636642   0.8965169  1981
    Num-Mis
231    5218


ROC Analysis in
Index:   RMSEA
Classification:  C
Estimation Method:   WLSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
232              C RMSEA     WLSMV         50         30 0.8207603
    partial-AUC Smoothed-AUC   Threshold Specificity Sensitivity Num-C
232   0.6122097    0.7819184 0.008736422   0.6174518   0.9337129  2459
    Num-Mis
232    6177


ROC Analysis in
Index:   RMSEA
Classification:  C
Estimation Method:   WLSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
233              C RMSEA     WLSMV        100        ALL 0.8726419
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
233   0.7027779    0.8857631 0.01533626   0.7232745   0.9102778  7200
    Num-Mis
233   18372


ROC Analysis in
Index:   RMSEA
Classification:  C
Estimation Method:   WLSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
234              C RMSEA     WLSMV        100          5 0.8583778
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
234   0.6896264    0.8632436 0.02109871   0.6727724   0.9137412  1994
    Num-Mis
234    5241


ROC Analysis in
Index:   RMSEA
Classification:  C
Estimation Method:   WLSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
235              C RMSEA     WLSMV        100         10 0.8927432
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
235   0.7216711    0.8821242 0.01663043   0.7276636   0.9652244  2473
    Num-Mis
235    6279


ROC Analysis in
Index:   RMSEA
Classification:  C
Estimation Method:   WLSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
236              C RMSEA     WLSMV        100         30 0.8886035
    partial-AUC Smoothed-AUC   Threshold Specificity Sensitivity Num-C
236   0.7057847    0.8594164 0.008578598   0.7368651   0.9718258  2733
    Num-Mis
236    6852


ROC Analysis in
Index:   RMSEA
Classification:  C
Estimation Method:   WLSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS      AUC
237              C RMSEA     WLSMV        200        ALL 0.909503
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
237   0.7613046     0.897834 0.01412005   0.7519406   0.9628913  8273
    Num-Mis
237   20741


ROC Analysis in
Index:   RMSEA
Classification:  C
Estimation Method:   WLSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
238              C RMSEA     WLSMV        200          5 0.9038272
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
238   0.7419997    0.8950343 0.01739685   0.7706646   0.9611155  2546
    Num-Mis
238    6545


ROC Analysis in
Index:   RMSEA
Classification:  C
Estimation Method:   WLSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS     AUC
239              C RMSEA     WLSMV        200         10 0.91829
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
239   0.7750943    0.9079796 0.01329112   0.7862422   0.9793374  2807
    Num-Mis
239    7036


ROC Analysis in
Index:   RMSEA
Classification:  C
Estimation Method:   WLSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
240              C RMSEA     WLSMV        200         30 0.9296943
    partial-AUC Smoothed-AUC   Threshold Specificity Sensitivity Num-C
240   0.8061857    0.9186347 0.006356727   0.8110335   0.9650685  2920
    Num-Mis
240    7160


ROC Analysis in
Index:   SRMRW
Classification:  C
Estimation Method:   ALL
Level-2 Sample Size:     ALL
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
241              C SRMRW       ALL        ALL        ALL 0.7420696
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
241   0.6079951    0.7165381 0.03812817   0.7281555   0.7226555 83510
    Num-Mis
241  223868


ROC Analysis in
Index:   SRMRW
Classification:  C
Estimation Method:   ALL
Level-2 Sample Size:     ALL
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
242              C SRMRW       ALL        ALL          5 0.6849087
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
242   0.6060076    0.6727791 0.04642137   0.7411522    0.581445 23460
    Num-Mis
242   63082


ROC Analysis in
Index:   SRMRW
Classification:  C
Estimation Method:   ALL
Level-2 Sample Size:     ALL
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
243              C SRMRW       ALL        ALL         10 0.7621713
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
243   0.6100403    0.7405188 0.04074856   0.6956786   0.7541658 28208
    Num-Mis
243   75749


ROC Analysis in
Index:   SRMRW
Classification:  C
Estimation Method:   ALL
Level-2 Sample Size:     ALL
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
244              C SRMRW       ALL        ALL         30 0.8121917
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
244   0.6053864    0.7919885 0.03519755   0.6606654   0.9397023 31842
    Num-Mis
244   85037


ROC Analysis in
Index:   SRMRW
Classification:  C
Estimation Method:   ALL
Level-2 Sample Size:     30
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
245              C SRMRW       ALL         30        ALL 0.6558303
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
245   0.5895854    0.6468735 0.0442164   0.7701042   0.4976438 16157
    Num-Mis
245   43767


ROC Analysis in
Index:   SRMRW
Classification:  C
Estimation Method:   ALL
Level-2 Sample Size:     30
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
246              C SRMRW       ALL         30          5 0.6197713
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
246   0.5542263     0.616773 0.0722534   0.7022226   0.4794555  3994
    Num-Mis
246   10825


ROC Analysis in
Index:   SRMRW
Classification:  C
Estimation Method:   ALL
Level-2 Sample Size:     30
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
247              C SRMRW       ALL         30         10 0.7063382
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
247   0.5849286    0.6931489 0.05528005   0.6284539   0.7012276  5297
    Num-Mis
247   14595


ROC Analysis in
Index:   SRMRW
Classification:  C
Estimation Method:   ALL
Level-2 Sample Size:     30
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
248              C SRMRW       ALL         30         30 0.7787395
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
248   0.5930131    0.7544901 0.03862758   0.6613615   0.8722692  6866
    Num-Mis
248   18347


ROC Analysis in
Index:   SRMRW
Classification:  C
Estimation Method:   ALL
Level-2 Sample Size:     50
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
249              C SRMRW       ALL         50        ALL 0.7152942
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
249   0.6008614    0.7003068 0.04221478    0.725253   0.6387998 19330
    Num-Mis
249   52168


ROC Analysis in
Index:   SRMRW
Classification:  C
Estimation Method:   ALL
Level-2 Sample Size:     50
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
250              C SRMRW       ALL         50          5 0.6848287
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
250   0.5775882    0.6808346 0.06155785   0.6419318   0.6354973  5037
    Num-Mis
250   13894


ROC Analysis in
Index:   SRMRW
Classification:  C
Estimation Method:   ALL
Level-2 Sample Size:     50
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
251              C SRMRW       ALL         50         10 0.7698945
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
251   0.5985872    0.7528981 0.04674038    0.632667   0.8343606  6490
    Num-Mis
251   17559


ROC Analysis in
Index:   SRMRW
Classification:  C
Estimation Method:   ALL
Level-2 Sample Size:     50
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
252              C SRMRW       ALL         50         30 0.8048348
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
252   0.5961738    0.7898199 0.03503936   0.6677287   0.9261822  7803
    Num-Mis
252   20715


ROC Analysis in
Index:   SRMRW
Classification:  C
Estimation Method:   ALL
Level-2 Sample Size:     100
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
253              C SRMRW       ALL        100        ALL 0.7910858
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
253   0.6048468    0.7766529 0.03828041   0.6979817   0.8057864 22743
    Num-Mis
253   60844


ROC Analysis in
Index:   SRMRW
Classification:  C
Estimation Method:   ALL
Level-2 Sample Size:     100
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
254              C SRMRW       ALL        100          5 0.7818985
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
254   0.6078964    0.7753532 0.04795395   0.6545736   0.8086943  6487
    Num-Mis
254   17448


ROC Analysis in
Index:   SRMRW
Classification:  C
Estimation Method:   ALL
Level-2 Sample Size:     100
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
255              C SRMRW       ALL        100         10 0.8167037
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
255   0.6036522     0.816031 0.03751988   0.6624286   0.9316098  7852
    Num-Mis
255   20843


ROC Analysis in
Index:   SRMRW
Classification:  C
Estimation Method:   ALL
Level-2 Sample Size:     100
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
256              C SRMRW       ALL        100         30 0.8224686
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
256   0.6012271    0.8318707 0.03240573   0.6641688   0.9727511  8404
    Num-Mis
256   22553


ROC Analysis in
Index:   SRMRW
Classification:  C
Estimation Method:   ALL
Level-2 Sample Size:     200
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
257              C SRMRW       ALL        200        ALL 0.8284972
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
257   0.6097959    0.8353881 0.03614138    0.654459   0.9628956 25280
    Num-Mis
257   67089


ROC Analysis in
Index:   SRMRW
Classification:  C
Estimation Method:   ALL
Level-2 Sample Size:     200
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
258              C SRMRW       ALL        200          5 0.8299269
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
258   0.6204966    0.8407181 0.03969059    0.649486   0.9496349  7942
    Num-Mis
258   20915


ROC Analysis in
Index:   SRMRW
Classification:  C
Estimation Method:   ALL
Level-2 Sample Size:     200
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
259              C SRMRW       ALL        200         10 0.8328916
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
259   0.6122962    0.8564588 0.03296658   0.6631065   0.9855292  8569
    Num-Mis
259   22752


ROC Analysis in
Index:   SRMRW
Classification:  C
Estimation Method:   ALL
Level-2 Sample Size:     200
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
260              C SRMRW       ALL        200         30 0.8303881
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
260   0.6080897    0.8487977 0.02448299   0.6794467   0.9824381  8769
    Num-Mis
260   23422


ROC Analysis in
Index:   SRMRW
Classification:  C
Estimation Method:   MLR
Level-2 Sample Size:     ALL
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
261              C SRMRW       MLR        ALL        ALL 0.7536143
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
261   0.6069411    0.7283724 0.03622468   0.7198618   0.7733693 30018
    Num-Mis
261   85647


ROC Analysis in
Index:   SRMRW
Classification:  C
Estimation Method:   MLR
Level-2 Sample Size:     ALL
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
262              C SRMRW       MLR        ALL          5 0.6982292
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
262   0.6128848    0.6939973 0.04029701   0.7791389   0.5643376  8945
    Num-Mis
262   24572


ROC Analysis in
Index:   SRMRW
Classification:  C
Estimation Method:   MLR
Level-2 Sample Size:     ALL
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
263              C SRMRW       MLR        ALL         10 0.7949675
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
263   0.6106717    0.7890972 0.0379248   0.6865034   0.8222002 10090
    Num-Mis
263   28785


ROC Analysis in
Index:   SRMRW
Classification:  C
Estimation Method:   MLR
Level-2 Sample Size:     ALL
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
264              C SRMRW       MLR        ALL         30 0.8315099
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
264   0.6111303    0.8402009 0.03061211   0.6608548   0.9984522 10983
    Num-Mis
264   32290


ROC Analysis in
Index:   SRMRW
Classification:  C
Estimation Method:   MLR
Level-2 Sample Size:     30
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
265              C SRMRW       MLR         30        ALL 0.6740344
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
265    0.596409    0.6708338 0.04167803    0.738378   0.5677522  6295
    Num-Mis
265   17854


ROC Analysis in
Index:   SRMRW
Classification:  C
Estimation Method:   MLR
Level-2 Sample Size:     30
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
266              C SRMRW       MLR         30          5 0.7044725
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
266   0.5692619    0.7066543 0.07115479   0.4861785   0.8273465  1726
    Num-Mis
266    4739


ROC Analysis in
Index:   SRMRW
Classification:  C
Estimation Method:   MLR
Level-2 Sample Size:     30
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
267              C SRMRW       MLR         30         10 0.7952682
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
267   0.5968844    0.7986915 0.04805392   0.6335734   0.8695229  2054
    Num-Mis
267    5832


ROC Analysis in
Index:   SRMRW
Classification:  C
Estimation Method:   MLR
Level-2 Sample Size:     30
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
268              C SRMRW       MLR         30         30 0.8261635
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
268    0.604362    0.8338246 0.03282434   0.6516545   0.9988072  2515
    Num-Mis
268    7283


ROC Analysis in
Index:   SRMRW
Classification:  C
Estimation Method:   MLR
Level-2 Sample Size:     50
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
269              C SRMRW       MLR         50        ALL 0.7368275
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
269   0.6056583    0.7302753 0.0374622   0.7444776    0.682201  7124
    Num-Mis
269   20417


ROC Analysis in
Index:   SRMRW
Classification:  C
Estimation Method:   MLR
Level-2 Sample Size:     50
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
270              C SRMRW       MLR         50          5 0.7684037
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
270   0.5967831    0.7739567 0.05631633     0.57058    0.855315  2032
    Num-Mis
270    5724


ROC Analysis in
Index:   SRMRW
Classification:  C
Estimation Method:   MLR
Level-2 Sample Size:     50
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
271              C SRMRW       MLR         50         10 0.8249209
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
271   0.6102729     0.843791 0.04067005   0.6290133   0.9652283  2387
    Num-Mis
271    6790


ROC Analysis in
Index:   SRMRW
Classification:  C
Estimation Method:   MLR
Level-2 Sample Size:     50
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
272              C SRMRW       MLR         50         30 0.8310602
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
272   0.6094651    0.8329195 0.02888454   0.6596229           1  2705
    Num-Mis
272    7903


ROC Analysis in
Index:   SRMRW
Classification:  C
Estimation Method:   MLR
Level-2 Sample Size:     100
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
273              C SRMRW       MLR        100        ALL 0.8163754
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
273   0.6063046    0.8207423 0.03681337   0.6722414    0.903419  8014
    Num-Mis
273   22901


ROC Analysis in
Index:   SRMRW
Classification:  C
Estimation Method:   MLR
Level-2 Sample Size:     100
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
274              C SRMRW       MLR        100          5 0.8311379
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
274    0.625175    0.8458183 0.04285063   0.6317613   0.9557302  2417
    Num-Mis
274    6637


ROC Analysis in
Index:   SRMRW
Classification:  C
Estimation Method:   MLR
Level-2 Sample Size:     100
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
275              C SRMRW       MLR        100         10 0.8337692
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
275   0.6155993    0.8390561 0.03213872   0.6507734   0.9996377  2760
    Num-Mis
275    7823


ROC Analysis in
Index:   SRMRW
Classification:  C
Estimation Method:   MLR
Level-2 Sample Size:     100
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
276              C SRMRW       MLR        100         30 0.8352022
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
276    0.615121    0.8372068 0.02504541   0.6636654           1  2837
    Num-Mis
276    8441


ROC Analysis in
Index:   SRMRW
Classification:  C
Estimation Method:   MLR
Level-2 Sample Size:     200
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
277              C SRMRW       MLR        200        ALL 0.8319834
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
277   0.6100624    0.8610259 0.03163202   0.6613687    0.993477  8585
    Num-Mis
277   24475


ROC Analysis in
Index:   SRMRW
Classification:  C
Estimation Method:   MLR
Level-2 Sample Size:     200
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
278              C SRMRW       MLR        200          5 0.8499731
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
278   0.6404459    0.8683116 0.03163202   0.6668897   0.9797834  2770
    Num-Mis
278    7472


ROC Analysis in
Index:   SRMRW
Classification:  C
Estimation Method:   MLR
Level-2 Sample Size:     200
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
279              C SRMRW       MLR        200         10 0.8438976
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
279    0.630975    0.8478955 0.02412356   0.6552758   0.9996539  2889
    Num-Mis
279    8340


ROC Analysis in
Index:   SRMRW
Classification:  C
Estimation Method:   MLR
Level-2 Sample Size:     200
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
280              C SRMRW       MLR        200         30 0.8397861
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
280   0.6230404    0.8420465 0.01332344   0.6643195           1  2926
    Num-Mis
280    8663


ROC Analysis in
Index:   SRMRW
Classification:  C
Estimation Method:   ULSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
281              C SRMRW     ULSMV        ALL        ALL 0.7400924
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
281   0.6332536    0.7148733 0.04450407   0.7412921   0.7147273 27515
    Num-Mis
281   71584


ROC Analysis in
Index:   SRMRW
Classification:  C
Estimation Method:   ULSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
282              C SRMRW     ULSMV        ALL          5 0.6717838
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
282   0.6148532     0.659926 0.05010638   0.7878013   0.5389342  7500
    Num-Mis
282   19987


ROC Analysis in
Index:   SRMRW
Classification:  C
Estimation Method:   ULSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
283              C SRMRW     ULSMV        ALL         10 0.7515538
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
283   0.6336956    0.7287313 0.0454097   0.7365193   0.7095702  9333
    Num-Mis
283   24300


ROC Analysis in
Index:   SRMRW
Classification:  C
Estimation Method:   ULSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
284              C SRMRW     ULSMV        ALL         30 0.8194977
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
284   0.6435882    0.7894792 0.04238427    0.674946    0.903857 10682
    Num-Mis
284   27297


ROC Analysis in
Index:   SRMRW
Classification:  C
Estimation Method:   ULSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
285              C SRMRW     ULSMV         30        ALL 0.6416738
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
285   0.5898321    0.6357768 0.05236474   0.7890401   0.4585018  5208
    Num-Mis
285   13635


ROC Analysis in
Index:   SRMRW
Classification:  C
Estimation Method:   ULSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
286              C SRMRW     ULSMV         30          5 0.6072774
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
286   0.5367276    0.6038865 0.09583593    0.585597   0.5781513  1203
    Num-Mis
286    3200


ROC Analysis in
Index:   SRMRW
Classification:  C
Estimation Method:   ULSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
287              C SRMRW     ULSMV         30         10 0.6906158
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
287   0.5805256    0.6814862 0.06939408   0.5927367   0.7134537  1719
    Num-Mis
287    4632


ROC Analysis in
Index:   SRMRW
Classification:  C
Estimation Method:   ULSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
288              C SRMRW     ULSMV         30         30 0.7584197
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
288   0.5949327    0.7344563 0.04699101   0.6972256   0.7725284  2286
    Num-Mis
288    5803


ROC Analysis in
Index:   SRMRW
Classification:  C
Estimation Method:   ULSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS      AUC
289              C SRMRW     ULSMV         50        ALL 0.708731
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
289   0.6135276    0.6965719 0.04759392   0.7704332   0.5951857  6356
    Num-Mis
289   16505


ROC Analysis in
Index:   SRMRW
Classification:  C
Estimation Method:   ULSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
290              C SRMRW     ULSMV         50          5 0.6896468
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
290   0.5729085    0.6859405 0.07462283   0.6200509   0.6664577  1595
    Num-Mis
290    4319


ROC Analysis in
Index:   SRMRW
Classification:  C
Estimation Method:   ULSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
291              C SRMRW     ULSMV         50         10 0.7640923
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
291   0.6038402    0.7461111 0.05410996    0.681679   0.7700283  2122
    Num-Mis
291    5551


ROC Analysis in
Index:   SRMRW
Classification:  C
Estimation Method:   ULSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
292              C SRMRW     ULSMV         50         30 0.8119673
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
292   0.6063541     0.791038 0.04200769   0.7017332   0.8791209  2639
    Num-Mis
292    6635


ROC Analysis in
Index:   SRMRW
Classification:  C
Estimation Method:   ULSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
293              C SRMRW     ULSMV        100        ALL 0.8029562
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
293   0.6326752    0.7853914 0.04582409   0.6890808   0.8346394  7529
    Num-Mis
293   19571


ROC Analysis in
Index:   SRMRW
Classification:  C
Estimation Method:   ULSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
294              C SRMRW     ULSMV        100          5 0.7911353
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
294   0.6092846    0.7828776 0.05604707   0.6658887   0.8179191  2076
    Num-Mis
294    5570


ROC Analysis in
Index:   SRMRW
Classification:  C
Estimation Method:   ULSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
295              C SRMRW     ULSMV        100         10 0.8338888
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
295   0.6159484     0.827946 0.0431894   0.6990061    0.912562  2619
    Num-Mis
295    6741


ROC Analysis in
Index:   SRMRW
Classification:  C
Estimation Method:   ULSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
296              C SRMRW     ULSMV        100         30 0.8459244
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
296   0.6137699     0.849298 0.03458862   0.7064738   0.9424841  2834
    Num-Mis
296    7260


ROC Analysis in
Index:   SRMRW
Classification:  C
Estimation Method:   ULSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
297              C SRMRW     ULSMV        200        ALL 0.8549602
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
297   0.6547264    0.8570827 0.04155507   0.6638321   0.9709095  8422
    Num-Mis
297   21873


ROC Analysis in
Index:   SRMRW
Classification:  C
Estimation Method:   ULSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
298              C SRMRW     ULSMV        200          5 0.8434775
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
298   0.6186314    0.8519016 0.0429225    0.709336   0.9348819  2626
    Num-Mis
298    6898


ROC Analysis in
Index:   SRMRW
Classification:  C
Estimation Method:   ULSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
299              C SRMRW     ULSMV        200         10 0.8558613
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
299   0.6222299    0.8690998 0.03296658   0.7383406   0.9568395  2873
    Num-Mis
299    7376


ROC Analysis in
Index:   SRMRW
Classification:  C
Estimation Method:   ULSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
300              C SRMRW     ULSMV        200         30 0.8603917
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
300   0.6222538     0.876575 0.02448299   0.7777339   0.9473144  2923
    Num-Mis
300    7599


ROC Analysis in
Index:   SRMRW
Classification:  C
Estimation Method:   WLSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
301              C SRMRW     WLSMV        ALL        ALL 0.7503818
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
301   0.5943789    0.7269196 0.04426785   0.6705759   0.8131572 25977
    Num-Mis
301   66637


ROC Analysis in
Index:   SRMRW
Classification:  C
Estimation Method:   WLSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS     AUC
302              C SRMRW     WLSMV        ALL          5 0.69631
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
302   0.6039903    0.6867659 0.04794503   0.7466054   0.6100557  7015
    Num-Mis
302   18523


ROC Analysis in
Index:   SRMRW
Classification:  C
Estimation Method:   WLSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
303              C SRMRW     WLSMV        ALL         10 0.7764872
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
303   0.6035735     0.770975 0.04480228   0.6638129   0.8162777  8785
    Num-Mis
303   22664


ROC Analysis in
Index:   SRMRW
Classification:  C
Estimation Method:   WLSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
304              C SRMRW     WLSMV        ALL         30 0.8064904
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
304   0.5976192    0.8208333 0.03862758   0.6000393   0.9969539 10177
    Num-Mis
304   25450


ROC Analysis in
Index:   SRMRW
Classification:  C
Estimation Method:   WLSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
305              C SRMRW     WLSMV         30        ALL 0.6823063
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
305   0.5900579    0.6730593 0.05020983   0.7379863   0.5846983  4654
    Num-Mis
305   12278


ROC Analysis in
Index:   SRMRW
Classification:  C
Estimation Method:   WLSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
306              C SRMRW     WLSMV         30          5 0.6791432
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
306   0.5556997    0.6796207 0.08499703      0.5625   0.7136061  1065
    Num-Mis
306    2886


ROC Analysis in
Index:   SRMRW
Classification:  C
Estimation Method:   WLSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
307              C SRMRW     WLSMV         30         10 0.7771438
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
307    0.584555    0.7810763 0.0598836   0.5975285   0.8713911  1524
    Num-Mis
307    4131


ROC Analysis in
Index:   SRMRW
Classification:  C
Estimation Method:   WLSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
308              C SRMRW     WLSMV         30         30 0.8048669
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
308    0.589698    0.8141293 0.04156411   0.5968447   0.9985472  2065
    Num-Mis
308    5261


ROC Analysis in
Index:   SRMRW
Classification:  C
Estimation Method:   WLSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS      AUC
309              C SRMRW     WLSMV         50        ALL 0.733456
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
309   0.5942119     0.723037 0.04697696   0.6890988   0.7384615  5850
    Num-Mis
309   15246


ROC Analysis in
Index:   SRMRW
Classification:  C
Estimation Method:   WLSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
310              C SRMRW     WLSMV         50          5 0.7447866
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
310   0.5807084    0.7480352 0.06913859   0.5567385   0.8340426  1410
    Num-Mis
310    3851


ROC Analysis in
Index:   SRMRW
Classification:  C
Estimation Method:   WLSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
311              C SRMRW     WLSMV         50         10 0.8082646
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
311   0.5969294    0.8276145 0.04916411   0.6092373   0.9540636  1981
    Num-Mis
311    5218


ROC Analysis in
Index:   SRMRW
Classification:  C
Estimation Method:   WLSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
312              C SRMRW     WLSMV         50         30 0.8105979
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
312   0.5967244    0.8137959 0.03665462   0.6025579           1  2459
    Num-Mis
312    6177


ROC Analysis in
Index:   SRMRW
Classification:  C
Estimation Method:   WLSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
313              C SRMRW     WLSMV        100        ALL 0.7970927
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
313   0.5943237    0.8022243 0.04486383   0.6273133   0.9101389  7200
    Num-Mis
313   18372


ROC Analysis in
Index:   SRMRW
Classification:  C
Estimation Method:   WLSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
314              C SRMRW     WLSMV        100          5 0.8116055
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
314   0.6064463     0.823631 0.05248121   0.5901546   0.9513541  1994
    Num-Mis
314    5241


ROC Analysis in
Index:   SRMRW
Classification:  C
Estimation Method:   WLSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS     AUC
315              C SRMRW     WLSMV        100         10 0.81844
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
315   0.6067244    0.8245313 0.03910348   0.5988215   0.9983825  2473
    Num-Mis
315    6279


ROC Analysis in
Index:   SRMRW
Classification:  C
Estimation Method:   WLSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
316              C SRMRW     WLSMV        100         30 0.8176188
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
316   0.6079322    0.8214763  0.024264   0.5998249           1  2733
    Num-Mis
316    6852


ROC Analysis in
Index:   SRMRW
Classification:  C
Estimation Method:   WLSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
317              C SRMRW     WLSMV        200        ALL 0.8128751
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
317    0.600899    0.8440027 0.0378596    0.607396   0.9920222  8273
    Num-Mis
317   20741


ROC Analysis in
Index:   SRMRW
Classification:  C
Estimation Method:   WLSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
318              C SRMRW     WLSMV        200          5 0.8329951
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
318   0.6206273    0.8505907 0.0378596   0.6348358    0.974077  2546
    Num-Mis
318    6545


ROC Analysis in
Index:   SRMRW
Classification:  C
Estimation Method:   WLSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS      AUC
319              C SRMRW     WLSMV        200         10 0.838318
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
319   0.6299049     0.845613 0.02525087     0.62834   0.9804061  2807
    Num-Mis
319    7036


ROC Analysis in
Index:   SRMRW
Classification:  C
Estimation Method:   WLSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
320              C SRMRW     WLSMV        200         30 0.8331659
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
320   0.6280602    0.8383261 0.01459169   0.6043296   0.9924658  2920
    Num-Mis
320    7160


ROC Analysis in
Index:   SRMRB
Classification:  C
Estimation Method:   ALL
Level-2 Sample Size:     ALL
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
321              C SRMRB       ALL        ALL        ALL 0.5979973
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
321   0.5720031    0.6039185 0.06685217   0.8038081   0.3519244 83510
    Num-Mis
321  223868


ROC Analysis in
Index:   SRMRB
Classification:  C
Estimation Method:   ALL
Level-2 Sample Size:     ALL
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
322              C SRMRB       ALL        ALL          5 0.5667398
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
322   0.5504085    0.5703466 0.07404096   0.8082209   0.3073017 23460
    Num-Mis
322   63082


ROC Analysis in
Index:   SRMRB
Classification:  C
Estimation Method:   ALL
Level-2 Sample Size:     ALL
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
323              C SRMRB       ALL        ALL         10 0.5963925
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
323   0.5709592    0.6026725 0.06874302   0.7924715   0.3580798 28208
    Num-Mis
323   75749


ROC Analysis in
Index:   SRMRB
Classification:  C
Estimation Method:   ALL
Level-2 Sample Size:     ALL
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
324              C SRMRB       ALL        ALL         30 0.6299666
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
324    0.591615    0.6395258 0.06306209   0.7930077   0.4038063 31842
    Num-Mis
324   85037


ROC Analysis in
Index:   SRMRB
Classification:  C
Estimation Method:   ALL
Level-2 Sample Size:     30
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
325              C SRMRB       ALL         30        ALL 0.5654934
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
325   0.5418092    0.5673981 0.1190256   0.7734005   0.3258309 16157
    Num-Mis
325   43767


ROC Analysis in
Index:   SRMRB
Classification:  C
Estimation Method:   ALL
Level-2 Sample Size:     30
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
326              C SRMRB       ALL         30          5 0.5407246
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
326   0.5159361    0.5375736 0.1512696   0.6444713   0.4257625  3994
    Num-Mis
326   10825


ROC Analysis in
Index:   SRMRB
Classification:  C
Estimation Method:   ALL
Level-2 Sample Size:     30
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
327              C SRMRB       ALL         30         10 0.5579602
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
327    0.532301    0.5565794 0.1269688   0.7090161   0.3875354  5297
    Num-Mis
327   14595


ROC Analysis in
Index:   SRMRB
Classification:  C
Estimation Method:   ALL
Level-2 Sample Size:     30
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
328              C SRMRB       ALL         30         30 0.5982512
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
328   0.5636455    0.6028177 0.1097485   0.7983867   0.3528983  6866
    Num-Mis
328   18347


ROC Analysis in
Index:   SRMRB
Classification:  C
Estimation Method:   ALL
Level-2 Sample Size:     50
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
329              C SRMRB       ALL         50        ALL 0.5973111
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
329   0.5629657    0.6004473 0.09574117   0.7612904   0.3832385 19330
    Num-Mis
329   52168


ROC Analysis in
Index:   SRMRB
Classification:  C
Estimation Method:   ALL
Level-2 Sample Size:     50
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
330              C SRMRB       ALL         50          5 0.5616972
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
330   0.5264937    0.5585045 0.1231687   0.5885994   0.5114155  5037
    Num-Mis
330   13894


ROC Analysis in
Index:   SRMRB
Classification:  C
Estimation Method:   ALL
Level-2 Sample Size:     50
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
331              C SRMRB       ALL         50         10 0.5884641
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
331   0.5562378    0.5876739 0.09793443   0.7629706   0.3832049  6490
    Num-Mis
331   17559


ROC Analysis in
Index:   SRMRB
Classification:  C
Estimation Method:   ALL
Level-2 Sample Size:     50
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
332              C SRMRB       ALL         50         30 0.6450205
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
332   0.5947747    0.6544628 0.0864975   0.7987931   0.4059977  7803
    Num-Mis
332   20715


ROC Analysis in
Index:   SRMRB
Classification:  C
Estimation Method:   ALL
Level-2 Sample Size:     100
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
333              C SRMRB       ALL        100        ALL 0.6476984
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
333   0.5983153    0.6505286 0.07399184   0.7208928   0.4969881 22743
    Num-Mis
333   60844


ROC Analysis in
Index:   SRMRB
Classification:  C
Estimation Method:   ALL
Level-2 Sample Size:     100
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
334              C SRMRB       ALL        100          5 0.5961615
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
334   0.5561124    0.5919026 0.08800472   0.6629986   0.4943734  6487
    Num-Mis
334   17448


ROC Analysis in
Index:   SRMRB
Classification:  C
Estimation Method:   ALL
Level-2 Sample Size:     100
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
335              C SRMRB       ALL        100         10 0.6482542
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
335     0.60291    0.6505693 0.07395368   0.7443746   0.4910851  7852
    Num-Mis
335   20843


ROC Analysis in
Index:   SRMRB
Classification:  C
Estimation Method:   ALL
Level-2 Sample Size:     100
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
336              C SRMRB       ALL        100         30 0.7146355
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
336    0.639413    0.7343572 0.06651756   0.7571498   0.5302237  8404
    Num-Mis
336   22553


ROC Analysis in
Index:   SRMRB
Classification:  C
Estimation Method:   ALL
Level-2 Sample Size:     200
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
337              C SRMRB       ALL        200        ALL 0.7005905
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
337   0.6303214    0.7035266 0.05745011   0.6916335   0.6033623 25280
    Num-Mis
337   67089


ROC Analysis in
Index:   SRMRB
Classification:  C
Estimation Method:   ALL
Level-2 Sample Size:     200
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
338              C SRMRB       ALL        200          5 0.6444102
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
338   0.5902023    0.6402612 0.06567525   0.6932345   0.5457064  7942
    Num-Mis
338   20915


ROC Analysis in
Index:   SRMRB
Classification:  C
Estimation Method:   ALL
Level-2 Sample Size:     200
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
339              C SRMRB       ALL        200         10 0.7109956
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
339   0.6452634    0.7227255 0.05376805   0.7823488   0.5304003  8569
    Num-Mis
339   22752


ROC Analysis in
Index:   SRMRB
Classification:  C
Estimation Method:   ALL
Level-2 Sample Size:     200
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
340              C SRMRB       ALL        200         30 0.7783228
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
340   0.6761218    0.8047894 0.05437703   0.6343609   0.7422739  8769
    Num-Mis
340   23422


ROC Analysis in
Index:   SRMRB
Classification:  C
Estimation Method:   MLR
Level-2 Sample Size:     ALL
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
341              C SRMRB       MLR        ALL        ALL 0.5999392
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
341   0.5774277    0.6052659 0.08255746   0.8001565   0.3660804 30018
    Num-Mis
341   85647


ROC Analysis in
Index:   SRMRB
Classification:  C
Estimation Method:   MLR
Level-2 Sample Size:     ALL
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
342              C SRMRB       MLR        ALL          5 0.5727901
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
342   0.5626343    0.5786412 0.09286638   0.8075859   0.3253214  8945
    Num-Mis
342   24572


ROC Analysis in
Index:   SRMRB
Classification:  C
Estimation Method:   MLR
Level-2 Sample Size:     ALL
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
343              C SRMRB       MLR        ALL         10 0.5995167
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
343   0.5762846    0.6056247 0.08255029   0.7991662   0.3625372 10090
    Num-Mis
343   28785


ROC Analysis in
Index:   SRMRB
Classification:  C
Estimation Method:   MLR
Level-2 Sample Size:     ALL
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
344              C SRMRB       MLR        ALL         30 0.6357916
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
344   0.5970014    0.6449865 0.0821276   0.7463301   0.4597105 10983
    Num-Mis
344   32290


ROC Analysis in
Index:   SRMRB
Classification:  C
Estimation Method:   MLR
Level-2 Sample Size:     30
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
345              C SRMRB       MLR         30        ALL 0.5651321
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
345   0.5502432    0.5673757 0.1433468   0.7926515    0.324861  6295
    Num-Mis
345   17854


ROC Analysis in
Index:   SRMRB
Classification:  C
Estimation Method:   MLR
Level-2 Sample Size:     30
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS      AUC
346              C SRMRB       MLR         30          5 0.540921
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
346   0.5264263    0.5402602 0.1711401   0.7170289   0.3707995  1726
    Num-Mis
346    4739


ROC Analysis in
Index:   SRMRB
Classification:  C
Estimation Method:   MLR
Level-2 Sample Size:     30
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
347              C SRMRB       MLR         30         10 0.5639811
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
347   0.5449064    0.5641817 0.1494552   0.7532579   0.3651412  2054
    Num-Mis
347    5832


ROC Analysis in
Index:   SRMRB
Classification:  C
Estimation Method:   MLR
Level-2 Sample Size:     30
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
348              C SRMRB       MLR         30         30 0.6084072
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
348   0.5788332     0.611994 0.1345366   0.7992585   0.3781312  2515
    Num-Mis
348    7283


ROC Analysis in
Index:   SRMRB
Classification:  C
Estimation Method:   MLR
Level-2 Sample Size:     50
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
349              C SRMRB       MLR         50        ALL 0.6096485
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
349   0.5754623    0.6100036 0.1190734    0.743547   0.4347277  7124
    Num-Mis
349   20417


ROC Analysis in
Index:   SRMRB
Classification:  C
Estimation Method:   MLR
Level-2 Sample Size:     50
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
350              C SRMRB       MLR         50          5 0.5791245
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
350   0.5482595    0.5772302 0.1395699   0.6783718   0.4635827  2032
    Num-Mis
350    5724


ROC Analysis in
Index:   SRMRB
Classification:  C
Estimation Method:   MLR
Level-2 Sample Size:     50
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
351              C SRMRB       MLR         50         10 0.6114047
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
351   0.5808254    0.6099724  0.121023   0.7466863   0.4537076  2387
    Num-Mis
351    6790


ROC Analysis in
Index:   SRMRB
Classification:  C
Estimation Method:   MLR
Level-2 Sample Size:     50
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
352              C SRMRB       MLR         50         30 0.6640058
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
352   0.6078505    0.6693377 0.1098324   0.7565481   0.4868762  2705
    Num-Mis
352    7903


ROC Analysis in
Index:   SRMRB
Classification:  C
Estimation Method:   MLR
Level-2 Sample Size:     100
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
353              C SRMRB       MLR        100        ALL 0.6728429
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
353   0.6165356    0.6710538 0.09250532   0.6812803   0.5834789  8014
    Num-Mis
353   22901


ROC Analysis in
Index:   SRMRB
Classification:  C
Estimation Method:   MLR
Level-2 Sample Size:     100
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
354              C SRMRB       MLR        100          5 0.6334326
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
354   0.5936268    0.6292419 0.1022958   0.7262317   0.5225486  2417
    Num-Mis
354    6637


ROC Analysis in
Index:   SRMRB
Classification:  C
Estimation Method:   MLR
Level-2 Sample Size:     100
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
355              C SRMRB       MLR        100         10 0.6776192
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
355    0.630761    0.6752214 0.08613431   0.8060846   0.4905797  2760
    Num-Mis
355    7823


ROC Analysis in
Index:   SRMRB
Classification:  C
Estimation Method:   MLR
Level-2 Sample Size:     100
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
356              C SRMRB       MLR        100         30 0.7463851
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
356   0.6557591    0.7656546 0.0835318   0.7231371   0.6122665  2837
    Num-Mis
356    8441


ROC Analysis in
Index:   SRMRB
Classification:  C
Estimation Method:   MLR
Level-2 Sample Size:     200
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
357              C SRMRB       MLR        200        ALL 0.7317993
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
357   0.6572444    0.7318819 0.06865594   0.7075792   0.6462434  8585
    Num-Mis
357   24475


ROC Analysis in
Index:   SRMRB
Classification:  C
Estimation Method:   MLR
Level-2 Sample Size:     200
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
358              C SRMRB       MLR        200          5 0.6862881
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
358   0.6429182    0.6822383 0.07429766   0.7786403   0.5429603  2770
    Num-Mis
358    7472


ROC Analysis in
Index:   SRMRB
Classification:  C
Estimation Method:   MLR
Level-2 Sample Size:     200
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
359              C SRMRB       MLR        200         10 0.7412693
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
359   0.6751296    0.7554207 0.06556097   0.7822542   0.5801315  2889
    Num-Mis
359    8340


ROC Analysis in
Index:   SRMRB
Classification:  C
Estimation Method:   MLR
Level-2 Sample Size:     200
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
360              C SRMRB       MLR        200         30 0.8187628
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
360    0.708878    0.8410292 0.06409246   0.6775944   0.7624744  2926
    Num-Mis
360    8663


ROC Analysis in
Index:   SRMRB
Classification:  C
Estimation Method:   ULSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS      AUC
361              C SRMRB     ULSMV        ALL        ALL 0.595275
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
361   0.5802469    0.6056278 0.05553207   0.8380857   0.3243636 27515
    Num-Mis
361   71584


ROC Analysis in
Index:   SRMRB
Classification:  C
Estimation Method:   ULSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
362              C SRMRB     ULSMV        ALL          5 0.5629772
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
362   0.5550101    0.5701621 0.06366312   0.8168696   0.3003873  7500
    Num-Mis
362   19987


ROC Analysis in
Index:   SRMRB
Classification:  C
Estimation Method:   ULSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
363              C SRMRB     ULSMV        ALL         10 0.5939354
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
363   0.5799016    0.6046258 0.05487045   0.8515683   0.3085414  9333
    Num-Mis
363   24300


ROC Analysis in
Index:   SRMRB
Classification:  C
Estimation Method:   ULSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
364              C SRMRB     ULSMV        ALL         30 0.6255954
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
364   0.6014553    0.6369965 0.05437358   0.8003077   0.4030144 10682
    Num-Mis
364   27297


ROC Analysis in
Index:   SRMRB
Classification:  C
Estimation Method:   ULSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
365              C SRMRB     ULSMV         30        ALL 0.5666281
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
365    0.549246     0.573127 0.1057217   0.7338727   0.3814751  5208
    Num-Mis
365   13635


ROC Analysis in
Index:   SRMRB
Classification:  C
Estimation Method:   ULSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
366              C SRMRB     ULSMV         30          5 0.5446504
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
366   0.5173438    0.5439842 0.1414411   0.4740998   0.6033613  1203
    Num-Mis
366    3200


ROC Analysis in
Index:   SRMRB
Classification:  C
Estimation Method:   ULSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
367              C SRMRB     ULSMV         30         10 0.5616261
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
367   0.5353728    0.5609572 0.1077406   0.7163856   0.4030285  1719
    Num-Mis
367    4632


ROC Analysis in
Index:   SRMRB
Classification:  C
Estimation Method:   ULSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
368              C SRMRB     ULSMV         30         30 0.5929264
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
368   0.5775139    0.6010889 0.09602858   0.7718422    0.399825  2286
    Num-Mis
368    5803


ROC Analysis in
Index:   SRMRB
Classification:  C
Estimation Method:   ULSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS      AUC
369              C SRMRB     ULSMV         50        ALL 0.595777
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
369   0.5760245    0.6036062 0.08121042   0.7950924   0.3659534  6356
    Num-Mis
369   16505


ROC Analysis in
Index:   SRMRB
Classification:  C
Estimation Method:   ULSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS      AUC
370              C SRMRB     ULSMV         50          5 0.556805
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
370   0.5306214    0.5554038 0.1039867   0.5934244   0.5065831  1595
    Num-Mis
370    4319


ROC Analysis in
Index:   SRMRB
Classification:  C
Estimation Method:   ULSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
371              C SRMRB     ULSMV         50         10 0.5911639
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
371   0.5661654    0.5929175 0.08580086   0.7472527   0.4245994  2122
    Num-Mis
371    5551


ROC Analysis in
Index:   SRMRB
Classification:  C
Estimation Method:   ULSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
372              C SRMRB     ULSMV         50         30 0.6395081
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
372   0.6195915    0.6525729 0.07174356   0.8482291   0.3937097  2639
    Num-Mis
372    6635


ROC Analysis in
Index:   SRMRB
Classification:  C
Estimation Method:   ULSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
373              C SRMRB     ULSMV        100        ALL 0.6486804
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
373   0.6198267    0.6548291 0.05954407   0.8431353    0.397928  7529
    Num-Mis
373   19571


ROC Analysis in
Index:   SRMRB
Classification:  C
Estimation Method:   ULSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
374              C SRMRB     ULSMV        100          5 0.5974677
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
374   0.5694477    0.5951798 0.07361416   0.7145422   0.4638728  2076
    Num-Mis
374    5570


ROC Analysis in
Index:   SRMRB
Classification:  C
Estimation Method:   ULSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
375              C SRMRB     ULSMV        100         10 0.6518764
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
375   0.6335019    0.6564262 0.06298478   0.7955793   0.4894998  2619
    Num-Mis
375    6741


ROC Analysis in
Index:   SRMRB
Classification:  C
Estimation Method:   ULSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
376              C SRMRB     ULSMV        100         30 0.7101368
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
376   0.6727809    0.7311361 0.05298483   0.8949036   0.4414255  2834
    Num-Mis
376    7260


ROC Analysis in
Index:   SRMRB
Classification:  C
Estimation Method:   ULSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
377              C SRMRB     ULSMV        200        ALL 0.7012045
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
377   0.6558506    0.7068131 0.04706463   0.7933983   0.5162669  8422
    Num-Mis
377   21873


ROC Analysis in
Index:   SRMRB
Classification:  C
Estimation Method:   ULSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
378              C SRMRB     ULSMV        200          5 0.6456838
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
378    0.614426    0.6436794 0.05343983   0.7974775   0.4615385  2626
    Num-Mis
378    6898


ROC Analysis in
Index:   SRMRB
Classification:  C
Estimation Method:   ULSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
379              C SRMRB     ULSMV        200         10 0.7140128
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
379   0.6852347    0.7267892 0.04448186   0.8756779   0.4918204  2873
    Num-Mis
379    7376


ROC Analysis in
Index:   SRMRB
Classification:  C
Estimation Method:   ULSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
380              C SRMRB     ULSMV        200         30 0.7794902
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
380   0.7134323    0.8075134 0.03848547   0.9345967   0.4673281  2923
    Num-Mis
380    7599


ROC Analysis in
Index:   SRMRB
Classification:  C
Estimation Method:   WLSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
381              C SRMRB     WLSMV        ALL        ALL 0.6012829
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
381   0.5783985    0.6085354 0.06202429   0.8221488   0.3451835 25977
    Num-Mis
381   66637


ROC Analysis in
Index:   SRMRB
Classification:  C
Estimation Method:   WLSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
382              C SRMRB     WLSMV        ALL          5 0.5695962
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
382   0.5580033    0.5761923 0.06853553    0.822667   0.3019569  7015
    Num-Mis
382   18523


ROC Analysis in
Index:   SRMRB
Classification:  C
Estimation Method:   WLSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
383              C SRMRB     WLSMV        ALL         10 0.5980224
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
383   0.5770231    0.6060279 0.06287979   0.8180936    0.342288  8785
    Num-Mis
383   22664


ROC Analysis in
Index:   SRMRB
Classification:  C
Estimation Method:   WLSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
384              C SRMRB     WLSMV        ALL         30 0.6311219
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
384   0.5960239    0.6404518 0.05978157   0.7983104   0.4071927 10177
    Num-Mis
384   25450


ROC Analysis in
Index:   SRMRB
Classification:  C
Estimation Method:   WLSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
385              C SRMRB     WLSMV         30        ALL 0.5755795
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
385   0.5510474    0.5794118 0.1167339   0.7933148   0.3252155  4654
    Num-Mis
385   12278


ROC Analysis in
Index:   SRMRB
Classification:  C
Estimation Method:   WLSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
386              C SRMRB     WLSMV         30          5 0.5506345
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
386   0.5183637    0.5483111  0.152358   0.5379213   0.5547098  1065
    Num-Mis
386    2886


ROC Analysis in
Index:   SRMRB
Classification:  C
Estimation Method:   WLSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS      AUC
387              C SRMRB     WLSMV         30         10 0.572526
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
387    0.540511    0.5700238 0.1224092   0.7361279   0.3910761  1524
    Num-Mis
387    4131


ROC Analysis in
Index:   SRMRB
Classification:  C
Estimation Method:   WLSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
388              C SRMRB     WLSMV         30         30 0.6083328
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
388   0.5765605    0.6137784 0.1092953   0.8224672   0.3564165  2065
    Num-Mis
388    5261


ROC Analysis in
Index:   SRMRB
Classification:  C
Estimation Method:   WLSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
389              C SRMRB     WLSMV         50        ALL 0.6134245
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
389   0.5789396    0.6176619 0.09582017   0.7304211   0.4504274  5850
    Num-Mis
389   15246


ROC Analysis in
Index:   SRMRB
Classification:  C
Estimation Method:   WLSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
390              C SRMRB     WLSMV         50          5 0.5764801
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
390   0.5340945    0.5732459 0.1118638   0.6421709   0.4829787  1410
    Num-Mis
390    3851


ROC Analysis in
Index:   SRMRB
Classification:  C
Estimation Method:   WLSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
391              C SRMRB     WLSMV         50         10 0.5981841
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
391   0.5674741    0.5975887 0.09571916   0.7740514   0.4088844  1981
    Num-Mis
391    5218


ROC Analysis in
Index:   SRMRB
Classification:  C
Estimation Method:   WLSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
392              C SRMRB     WLSMV         50         30 0.6714189
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
392   0.6196634    0.6808685 0.08583831   0.8193298   0.4375763  2459
    Num-Mis
392    6177


ROC Analysis in
Index:   SRMRB
Classification:  C
Estimation Method:   WLSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS      AUC
393              C SRMRB     WLSMV        100        ALL 0.669509
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
393   0.6206045    0.6701034 0.07118502   0.7281733   0.5336111  7200
    Num-Mis
393   18372


ROC Analysis in
Index:   SRMRB
Classification:  C
Estimation Method:   WLSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
394              C SRMRB     WLSMV        100          5 0.6084689
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
394   0.5726185    0.6032819 0.07983166   0.7355467   0.4623872  1994
    Num-Mis
394    5241


ROC Analysis in
Index:   SRMRB
Classification:  C
Estimation Method:   WLSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
395              C SRMRB     WLSMV        100         10 0.6678274
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
395   0.6268994    0.6670371 0.07167246   0.7550565   0.5301254  2473
    Num-Mis
395    6279


ROC Analysis in
Index:   SRMRB
Classification:  C
Estimation Method:   WLSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
396              C SRMRB     WLSMV        100         30 0.7537523
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
396   0.6735256    0.7744439 0.06389182   0.7952423    0.557629  2733
    Num-Mis
396    6852


ROC Analysis in
Index:   SRMRB
Classification:  C
Estimation Method:   WLSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
397              C SRMRB     WLSMV        200        ALL 0.7228849
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
397   0.6510054    0.7222226 0.05439942    0.699243   0.6429348  8273
    Num-Mis
397   20741


ROC Analysis in
Index:   SRMRB
Classification:  C
Estimation Method:   WLSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS     AUC
398              C SRMRB     WLSMV        200          5 0.66117
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
398   0.6157789    0.6573175 0.06066784   0.7271199   0.5475255  2546
    Num-Mis
398    6545


ROC Analysis in
Index:   SRMRB
Classification:  C
Estimation Method:   WLSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
399              C SRMRB     WLSMV        200         10 0.7363002
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
399   0.6728677    0.7450122 0.05176924   0.7946276   0.5771286  2807
    Num-Mis
399    7036


ROC Analysis in
Index:   SRMRB
Classification:  C
Estimation Method:   WLSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.

Version	Author	Date
982c8f1	noah-padgett	2019-05-18



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS      AUC
400              C SRMRB     WLSMV        200         30 0.813997
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
400   0.7074172    0.8380453 0.04835387   0.7351955   0.6982877  2920
    Num-Mis
400    7160

kable(roc_summary[1:400, ], format = 'html', digits=3) %>%
  kable_styling(full_width = T)

Classification	Index	Estimator	Level-2 SS	Level-1 SS	AUC	partial-AUC	Smoothed-AUC	Threshold	Specificity	Sensitivity	Num-C	Num-Mis
C	CFI	ALL	ALL	ALL	0.816	0.637	0.846	0.977	0.702	0.855	83510	223868
C	CFI	ALL	ALL	5	0.784	0.649	0.799	0.971	0.749	0.752	23460	63082
C	CFI	ALL	ALL	10	0.842	0.672	0.860	0.972	0.718	0.873	28208	75749
C	CFI	ALL	ALL	30	0.829	0.612	0.809	0.986	0.643	0.944	31842	85037
C	CFI	ALL	30	ALL	0.679	0.544	0.713	0.968	0.587	0.739	16157	43767
C	CFI	ALL	30	5	0.636	0.546	0.640	0.899	0.490	0.719	3994	10825
C	CFI	ALL	30	10	0.708	0.560	0.721	0.959	0.597	0.728	5297	14595
C	CFI	ALL	30	30	0.706	0.535	0.639	0.971	0.467	0.943	6866	18347
C	CFI	ALL	50	ALL	0.774	0.597	0.802	0.970	0.642	0.826	19330	52168
C	CFI	ALL	50	5	0.720	0.585	0.727	0.956	0.654	0.680	5037	13894
C	CFI	ALL	50	10	0.818	0.642	0.830	0.965	0.683	0.843	6490	17559
C	CFI	ALL	50	30	0.795	0.580	0.763	0.983	0.589	0.969	7803	20715
C	CFI	ALL	100	ALL	0.864	0.684	0.884	0.977	0.723	0.905	22743	60844
C	CFI	ALL	100	5	0.850	0.687	0.859	0.967	0.723	0.845	6487	17448
C	CFI	ALL	100	10	0.889	0.725	0.901	0.974	0.741	0.946	7852	20843
C	CFI	ALL	100	30	0.865	0.659	0.841	0.991	0.701	0.971	8404	22553
C	CFI	ALL	200	ALL	0.912	0.767	0.922	0.980	0.764	0.965	25280	67089
C	CFI	ALL	200	5	0.904	0.747	0.914	0.977	0.787	0.940	7942	20915
C	CFI	ALL	200	10	0.916	0.772	0.913	0.983	0.789	0.963	8569	22752
C	CFI	ALL	200	30	0.931	0.811	0.923	0.994	0.797	0.965	8769	23422
C	CFI	MLR	ALL	ALL	0.841	0.710	0.840	0.967	0.746	0.834	30018	85647
C	CFI	MLR	ALL	5	0.773	0.678	0.777	0.971	0.814	0.663	8945	24572
C	CFI	MLR	ALL	10	0.854	0.717	0.856	0.967	0.772	0.816	10090	28785
C	CFI	MLR	ALL	30	0.897	0.742	0.891	0.977	0.733	0.933	10983	32290
C	CFI	MLR	30	ALL	0.747	0.639	0.739	0.940	0.726	0.698	6295	17854
C	CFI	MLR	30	5	0.666	0.569	0.664	0.888	0.673	0.589	1726	4739
C	CFI	MLR	30	10	0.774	0.622	0.773	0.918	0.612	0.815	2054	5832
C	CFI	MLR	30	30	0.873	0.686	0.876	0.957	0.693	0.966	2515	7283
C	CFI	MLR	50	ALL	0.828	0.674	0.827	0.956	0.713	0.835	7124	20417
C	CFI	MLR	50	5	0.747	0.608	0.748	0.929	0.612	0.763	2032	5724
C	CFI	MLR	50	10	0.859	0.690	0.859	0.956	0.734	0.867	2387	6790
C	CFI	MLR	50	30	0.894	0.719	0.889	0.979	0.739	0.969	2705	7903
C	CFI	MLR	100	ALL	0.893	0.735	0.898	0.974	0.750	0.918	8014	22901
C	CFI	MLR	100	5	0.866	0.705	0.867	0.959	0.700	0.897	2417	6637
C	CFI	MLR	100	10	0.906	0.748	0.901	0.974	0.768	0.967	2760	7823
C	CFI	MLR	100	30	0.918	0.775	0.912	0.991	0.792	0.970	2837	8441
C	CFI	MLR	200	ALL	0.917	0.778	0.911	0.981	0.767	0.973	8585	24475
C	CFI	MLR	200	5	0.911	0.761	0.906	0.976	0.787	0.958	2770	7472
C	CFI	MLR	200	10	0.923	0.788	0.915	0.986	0.809	0.975	2889	8340
C	CFI	MLR	200	30	0.938	0.827	0.932	0.995	0.824	0.986	2926	8663
C	CFI	ULSMV	ALL	ALL	0.780	0.587	0.841	0.973	0.628	0.888	27515	71584
C	CFI	ULSMV	ALL	5	0.775	0.618	0.807	0.967	0.697	0.794	7500	19987
C	CFI	ULSMV	ALL	10	0.818	0.625	0.854	0.970	0.687	0.888	9333	24300
C	CFI	ULSMV	ALL	30	0.762	0.561	0.725	0.987	0.531	0.975	10682	27297
C	CFI	ULSMV	30	ALL	0.628	0.524	0.720	0.982	0.431	0.821	5208	13635
C	CFI	ULSMV	30	5	0.617	0.531	0.638	0.981	0.633	0.575	1203	3200
C	CFI	ULSMV	30	10	0.665	0.535	0.726	0.978	0.498	0.795	1719	4632
C	CFI	ULSMV	30	30	0.614	0.516	0.481	0.988	0.268	0.975	2286	5803
C	CFI	ULSMV	50	ALL	0.712	0.554	0.770	0.973	0.552	0.826	6356	16505
C	CFI	ULSMV	50	5	0.686	0.564	0.699	0.967	0.647	0.639	1595	4319
C	CFI	ULSMV	50	10	0.774	0.603	0.801	0.970	0.659	0.804	2122	5551
C	CFI	ULSMV	50	30	0.694	0.535	0.643	0.989	0.417	0.974	2639	6635
C	CFI	ULSMV	100	ALL	0.827	0.634	0.855	0.971	0.667	0.906	7529	19571
C	CFI	ULSMV	100	5	0.829	0.671	0.840	0.965	0.709	0.823	2076	5570
C	CFI	ULSMV	100	10	0.874	0.711	0.886	0.970	0.742	0.908	2619	6741
C	CFI	ULSMV	100	30	0.798	0.584	0.740	0.989	0.583	0.967	2834	7260
C	CFI	ULSMV	200	ALL	0.911	0.764	0.919	0.975	0.785	0.957	8422	21873
C	CFI	ULSMV	200	5	0.898	0.738	0.908	0.967	0.766	0.950	2626	6898
C	CFI	ULSMV	200	10	0.912	0.761	0.907	0.970	0.796	0.971	2873	7376
C	CFI	ULSMV	200	30	0.929	0.805	0.921	0.987	0.794	0.975	2923	7599
C	CFI	WLSMV	ALL	ALL	0.836	0.649	0.871	0.982	0.682	0.896	25977	66637
C	CFI	WLSMV	ALL	5	0.818	0.656	0.835	0.972	0.704	0.838	7015	18523
C	CFI	WLSMV	ALL	10	0.866	0.686	0.880	0.982	0.724	0.902	8785	22664
C	CFI	WLSMV	ALL	30	0.841	0.625	0.797	0.995	0.669	0.951	10177	25450
C	CFI	WLSMV	30	ALL	0.705	0.552	0.758	0.983	0.546	0.811	4654	12278
C	CFI	WLSMV	30	5	0.675	0.558	0.680	0.936	0.455	0.807	1065	2886
C	CFI	WLSMV	30	10	0.763	0.588	0.776	0.981	0.624	0.804	1524	4131
C	CFI	WLSMV	30	30	0.700	0.537	0.564	0.987	0.397	0.977	2065	5261
C	CFI	WLSMV	50	ALL	0.802	0.625	0.830	0.980	0.647	0.848	5850	15246
C	CFI	WLSMV	50	5	0.747	0.596	0.751	0.964	0.618	0.769	1410	3851
C	CFI	WLSMV	50	10	0.843	0.663	0.841	0.976	0.688	0.880	1981	5218
C	CFI	WLSMV	50	30	0.828	0.612	0.771	0.992	0.630	0.966	2459	6177
C	CFI	WLSMV	100	ALL	0.876	0.703	0.891	0.979	0.708	0.936	7200	18372
C	CFI	WLSMV	100	5	0.859	0.690	0.862	0.972	0.711	0.880	1994	5241
C	CFI	WLSMV	100	10	0.893	0.721	0.882	0.983	0.746	0.947	2473	6279
C	CFI	WLSMV	100	30	0.889	0.706	0.863	0.994	0.735	0.977	2733	6852
C	CFI	WLSMV	200	ALL	0.910	0.761	0.898	0.986	0.756	0.960	8273	20741
C	CFI	WLSMV	200	5	0.904	0.742	0.897	0.979	0.769	0.962	2546	6545
C	CFI	WLSMV	200	10	0.917	0.772	0.907	0.988	0.783	0.981	2807	7036
C	CFI	WLSMV	200	30	0.930	0.806	0.919	0.997	0.808	0.969	2920	7160
C	TLI	ALL	ALL	ALL	0.815	0.637	0.845	0.972	0.702	0.855	83510	223868
C	TLI	ALL	ALL	5	0.784	0.649	0.799	0.965	0.749	0.752	23460	63082
C	TLI	ALL	ALL	10	0.841	0.672	0.859	0.966	0.717	0.873	28208	75749
C	TLI	ALL	ALL	30	0.828	0.612	0.810	0.983	0.643	0.945	31842	85037
C	TLI	ALL	30	ALL	0.678	0.544	0.713	0.961	0.587	0.739	16157	43767
C	TLI	ALL	30	5	0.635	0.546	0.639	0.878	0.487	0.720	3994	10825
C	TLI	ALL	30	10	0.707	0.560	0.721	0.951	0.597	0.728	5297	14595
C	TLI	ALL	30	30	0.706	0.535	0.640	0.965	0.467	0.943	6866	18347
C	TLI	ALL	50	ALL	0.773	0.597	0.802	0.964	0.641	0.826	19330	52168
C	TLI	ALL	50	5	0.719	0.585	0.727	0.948	0.653	0.681	5037	13894
C	TLI	ALL	50	10	0.818	0.642	0.830	0.958	0.682	0.843	6490	17559
C	TLI	ALL	50	30	0.795	0.580	0.764	0.980	0.588	0.969	7803	20715
C	TLI	ALL	100	ALL	0.864	0.684	0.884	0.972	0.722	0.905	22743	60844
C	TLI	ALL	100	5	0.849	0.687	0.859	0.961	0.723	0.845	6487	17448
C	TLI	ALL	100	10	0.889	0.724	0.901	0.968	0.739	0.948	7852	20843
C	TLI	ALL	100	30	0.865	0.659	0.842	0.989	0.701	0.971	8404	22553
C	TLI	ALL	200	ALL	0.912	0.767	0.922	0.976	0.763	0.965	25280	67089
C	TLI	ALL	200	5	0.904	0.747	0.914	0.972	0.786	0.941	7942	20915
C	TLI	ALL	200	10	0.916	0.772	0.913	0.980	0.789	0.963	8569	22752
C	TLI	ALL	200	30	0.931	0.811	0.923	0.993	0.797	0.965	8769	23422
C	TLI	MLR	ALL	ALL	0.840	0.710	0.840	0.961	0.750	0.830	30018	85647
C	TLI	MLR	ALL	5	0.772	0.678	0.776	0.965	0.814	0.663	8945	24572
C	TLI	MLR	ALL	10	0.853	0.717	0.855	0.960	0.772	0.816	10090	28785
C	TLI	MLR	ALL	30	0.897	0.742	0.891	0.972	0.733	0.933	10983	32290
C	TLI	MLR	30	ALL	0.746	0.639	0.738	0.928	0.725	0.698	6295	17854
C	TLI	MLR	30	5	0.664	0.569	0.663	0.867	0.678	0.582	1726	4739
C	TLI	MLR	30	10	0.773	0.622	0.772	0.902	0.610	0.815	2054	5832
C	TLI	MLR	30	30	0.873	0.686	0.876	0.949	0.693	0.966	2515	7283
C	TLI	MLR	50	ALL	0.827	0.674	0.827	0.947	0.713	0.835	7124	20417
C	TLI	MLR	50	5	0.746	0.607	0.747	0.915	0.610	0.763	2032	5724
C	TLI	MLR	50	10	0.859	0.690	0.859	0.947	0.734	0.867	2387	6790
C	TLI	MLR	50	30	0.894	0.719	0.889	0.975	0.739	0.969	2705	7903
C	TLI	MLR	100	ALL	0.893	0.735	0.898	0.968	0.750	0.918	8014	22901
C	TLI	MLR	100	5	0.865	0.705	0.867	0.951	0.699	0.897	2417	6637
C	TLI	MLR	100	10	0.906	0.748	0.901	0.969	0.768	0.967	2760	7823
C	TLI	MLR	100	30	0.918	0.775	0.912	0.989	0.792	0.970	2837	8441
C	TLI	MLR	200	ALL	0.917	0.778	0.911	0.977	0.767	0.973	8585	24475
C	TLI	MLR	200	5	0.911	0.761	0.906	0.972	0.790	0.955	2770	7472
C	TLI	MLR	200	10	0.923	0.788	0.915	0.984	0.809	0.975	2889	8340
C	TLI	MLR	200	30	0.938	0.827	0.932	0.994	0.824	0.986	2926	8663
C	TLI	ULSMV	ALL	ALL	0.780	0.587	0.841	0.968	0.627	0.888	27515	71584
C	TLI	ULSMV	ALL	5	0.775	0.618	0.806	0.960	0.696	0.794	7500	19987
C	TLI	ULSMV	ALL	10	0.818	0.625	0.854	0.964	0.686	0.888	9333	24300
C	TLI	ULSMV	ALL	30	0.762	0.561	0.727	0.985	0.531	0.975	10682	27297
C	TLI	ULSMV	30	ALL	0.628	0.524	0.720	0.979	0.430	0.821	5208	13635
C	TLI	ULSMV	30	5	0.616	0.531	0.638	0.977	0.632	0.575	1203	3200
C	TLI	ULSMV	30	10	0.665	0.535	0.725	0.973	0.498	0.795	1719	4632
C	TLI	ULSMV	30	30	0.614	0.516	0.482	0.985	0.268	0.975	2286	5803
C	TLI	ULSMV	50	ALL	0.712	0.554	0.770	0.968	0.552	0.826	6356	16505
C	TLI	ULSMV	50	5	0.685	0.564	0.699	0.959	0.638	0.647	1595	4319
C	TLI	ULSMV	50	10	0.774	0.603	0.801	0.966	0.663	0.799	2122	5551
C	TLI	ULSMV	50	30	0.694	0.535	0.645	0.986	0.417	0.974	2639	6635
C	TLI	ULSMV	100	ALL	0.827	0.634	0.855	0.965	0.666	0.906	7529	19571
C	TLI	ULSMV	100	5	0.829	0.670	0.840	0.958	0.708	0.823	2076	5570
C	TLI	ULSMV	100	10	0.874	0.711	0.886	0.965	0.743	0.907	2619	6741
C	TLI	ULSMV	100	30	0.798	0.584	0.740	0.987	0.583	0.967	2834	7260
C	TLI	ULSMV	200	ALL	0.911	0.764	0.919	0.970	0.785	0.956	8422	21873
C	TLI	ULSMV	200	5	0.898	0.738	0.908	0.960	0.766	0.950	2626	6898
C	TLI	ULSMV	200	10	0.912	0.761	0.907	0.965	0.797	0.969	2873	7376
C	TLI	ULSMV	200	30	0.929	0.805	0.921	0.985	0.794	0.975	2923	7599
C	TLI	WLSMV	ALL	ALL	0.836	0.649	0.871	0.979	0.682	0.896	25977	66637
C	TLI	WLSMV	ALL	5	0.818	0.656	0.835	0.966	0.703	0.838	7015	18523
C	TLI	WLSMV	ALL	10	0.866	0.686	0.880	0.979	0.724	0.902	8785	22664
C	TLI	WLSMV	ALL	30	0.841	0.625	0.797	0.994	0.669	0.951	10177	25450
C	TLI	WLSMV	30	ALL	0.705	0.552	0.758	0.979	0.547	0.809	4654	12278
C	TLI	WLSMV	30	5	0.674	0.558	0.680	0.924	0.453	0.807	1065	2886
C	TLI	WLSMV	30	10	0.762	0.588	0.776	0.978	0.627	0.801	1524	4131
C	TLI	WLSMV	30	30	0.700	0.537	0.565	0.984	0.397	0.977	2065	5261
C	TLI	WLSMV	50	ALL	0.801	0.625	0.830	0.976	0.648	0.846	5850	15246
C	TLI	WLSMV	50	5	0.747	0.596	0.751	0.956	0.617	0.769	1410	3851
C	TLI	WLSMV	50	10	0.843	0.663	0.841	0.971	0.687	0.880	1981	5218
C	TLI	WLSMV	50	30	0.828	0.612	0.771	0.991	0.629	0.967	2459	6177
C	TLI	WLSMV	100	ALL	0.876	0.703	0.892	0.975	0.708	0.936	7200	18372
C	TLI	WLSMV	100	5	0.858	0.690	0.862	0.966	0.711	0.880	1994	5241
C	TLI	WLSMV	100	10	0.893	0.721	0.882	0.980	0.746	0.947	2473	6279
C	TLI	WLSMV	100	30	0.889	0.706	0.863	0.993	0.735	0.977	2733	6852
C	TLI	WLSMV	200	ALL	0.910	0.761	0.898	0.984	0.756	0.960	8273	20741
C	TLI	WLSMV	200	5	0.904	0.742	0.897	0.974	0.769	0.962	2546	6545
C	TLI	WLSMV	200	10	0.917	0.772	0.907	0.986	0.783	0.981	2807	7036
C	TLI	WLSMV	200	30	0.930	0.806	0.919	0.997	0.808	0.969	2920	7160
C	RMSEA	ALL	ALL	ALL	0.803	0.635	0.830	0.015	0.685	0.829	83510	223868
C	RMSEA	ALL	ALL	5	0.776	0.644	0.790	0.019	0.722	0.757	23460	63082
C	RMSEA	ALL	ALL	10	0.832	0.668	0.847	0.016	0.725	0.836	28208	75749
C	RMSEA	ALL	ALL	30	0.816	0.612	0.798	0.009	0.665	0.869	31842	85037
C	RMSEA	ALL	30	ALL	0.664	0.544	0.688	0.020	0.549	0.734	16157	43767
C	RMSEA	ALL	30	5	0.614	0.546	0.614	0.021	0.695	0.477	3994	10825
C	RMSEA	ALL	30	10	0.681	0.560	0.680	0.018	0.623	0.657	5297	14595
C	RMSEA	ALL	30	30	0.691	0.535	0.615	0.022	0.398	0.958	6866	18347
C	RMSEA	ALL	50	ALL	0.758	0.596	0.778	0.016	0.636	0.776	19330	52168
C	RMSEA	ALL	50	5	0.702	0.582	0.704	0.023	0.635	0.670	5037	13894
C	RMSEA	ALL	50	10	0.801	0.635	0.802	0.020	0.653	0.819	6490	17559
C	RMSEA	ALL	50	30	0.781	0.580	0.734	0.013	0.551	0.933	7803	20715
C	RMSEA	ALL	100	ALL	0.854	0.678	0.862	0.016	0.698	0.892	22743	60844
C	RMSEA	ALL	100	5	0.842	0.680	0.843	0.021	0.676	0.870	6487	17448
C	RMSEA	ALL	100	10	0.886	0.716	0.877	0.017	0.720	0.943	7852	20843
C	RMSEA	ALL	100	30	0.859	0.656	0.825	0.010	0.687	0.961	8404	22553
C	RMSEA	ALL	200	ALL	0.905	0.758	0.897	0.014	0.736	0.955	25280	67089
C	RMSEA	ALL	200	5	0.902	0.742	0.896	0.017	0.764	0.953	7942	20915
C	RMSEA	ALL	200	10	0.918	0.776	0.909	0.013	0.790	0.971	8569	22752
C	RMSEA	ALL	200	30	0.929	0.808	0.922	0.007	0.806	0.951	8769	23422
C	RMSEA	MLR	ALL	ALL	0.830	0.706	0.829	0.024	0.735	0.843	30018	85647
C	RMSEA	MLR	ALL	5	0.758	0.671	0.762	0.023	0.782	0.684	8945	24572
C	RMSEA	MLR	ALL	10	0.843	0.711	0.844	0.025	0.745	0.840	10090	28785
C	RMSEA	MLR	ALL	30	0.896	0.738	0.894	0.019	0.729	0.944	10983	32290
C	RMSEA	MLR	30	ALL	0.725	0.634	0.720	0.029	0.741	0.660	6295	17854
C	RMSEA	MLR	30	5	0.653	0.562	0.655	0.051	0.540	0.678	1726	4739
C	RMSEA	MLR	30	10	0.755	0.612	0.756	0.035	0.642	0.744	2054	5832
C	RMSEA	MLR	30	30	0.870	0.679	0.873	0.026	0.700	0.961	2515	7283
C	RMSEA	MLR	50	ALL	0.814	0.669	0.813	0.026	0.716	0.814	7124	20417
C	RMSEA	MLR	50	5	0.730	0.599	0.731	0.034	0.593	0.752	2032	5724
C	RMSEA	MLR	50	10	0.850	0.680	0.851	0.026	0.725	0.849	2387	6790
C	RMSEA	MLR	50	30	0.892	0.715	0.887	0.018	0.736	0.979	2705	7903
C	RMSEA	MLR	100	ALL	0.889	0.729	0.892	0.020	0.747	0.916	8014	22901
C	RMSEA	MLR	100	5	0.859	0.695	0.859	0.024	0.717	0.871	2417	6637
C	RMSEA	MLR	100	10	0.903	0.740	0.897	0.020	0.762	0.968	2760	7823
C	RMSEA	MLR	100	30	0.917	0.772	0.911	0.012	0.779	0.982	2837	8441
C	RMSEA	MLR	200	ALL	0.915	0.772	0.908	0.017	0.759	0.979	8585	24475
C	RMSEA	MLR	200	5	0.908	0.751	0.900	0.019	0.782	0.964	2770	7472
C	RMSEA	MLR	200	10	0.921	0.781	0.912	0.014	0.801	0.981	2889	8340
C	RMSEA	MLR	200	30	0.937	0.825	0.930	0.008	0.832	0.975	2926	8663
C	RMSEA	ULSMV	ALL	ALL	0.770	0.587	0.807	0.012	0.596	0.855	27515	71584
C	RMSEA	ULSMV	ALL	5	0.784	0.617	0.807	0.017	0.656	0.815	7500	19987
C	RMSEA	ULSMV	ALL	10	0.817	0.624	0.828	0.012	0.692	0.843	9333	24300
C	RMSEA	ULSMV	ALL	30	0.754	0.561	0.708	0.006	0.552	0.905	10682	27297
C	RMSEA	ULSMV	30	ALL	0.628	0.524	0.675	0.008	0.448	0.771	5208	13635
C	RMSEA	ULSMV	30	5	0.621	0.531	0.635	0.011	0.619	0.587	1203	3200
C	RMSEA	ULSMV	30	10	0.663	0.535	0.666	0.014	0.428	0.838	1719	4632
C	RMSEA	ULSMV	30	30	0.391	NA	0.564	-Inf	0.000	1.000	2286	5803
C	RMSEA	ULSMV	50	ALL	0.706	0.554	0.728	0.009	0.590	0.749	6356	16505
C	RMSEA	ULSMV	50	5	0.686	0.564	0.690	0.016	0.614	0.661	1595	4319
C	RMSEA	ULSMV	50	10	0.770	0.603	0.761	0.014	0.614	0.802	2122	5551
C	RMSEA	ULSMV	50	30	0.688	0.535	0.587	0.007	0.430	0.907	2639	6635
C	RMSEA	ULSMV	100	ALL	0.813	0.632	0.810	0.012	0.671	0.839	7529	19571
C	RMSEA	ULSMV	100	5	0.829	0.667	0.830	0.017	0.700	0.826	2076	5570
C	RMSEA	ULSMV	100	10	0.870	0.706	0.863	0.014	0.707	0.900	2619	6741
C	RMSEA	ULSMV	100	30	0.789	0.584	0.715	0.006	0.574	0.924	2834	7260
C	RMSEA	ULSMV	200	ALL	0.893	0.752	0.889	0.013	0.689	0.947	8422	21873
C	RMSEA	ULSMV	200	5	0.896	0.737	0.895	0.015	0.784	0.907	2626	6898
C	RMSEA	ULSMV	200	10	0.916	0.774	0.909	0.012	0.800	0.944	2873	7376
C	RMSEA	ULSMV	200	30	0.926	0.807	0.925	0.006	0.805	0.929	2923	7599
C	RMSEA	WLSMV	ALL	ALL	0.832	0.648	0.864	0.014	0.689	0.872	25977	66637
C	RMSEA	WLSMV	ALL	5	0.820	0.655	0.837	0.019	0.719	0.819	7015	18523
C	RMSEA	WLSMV	ALL	10	0.866	0.685	0.876	0.016	0.713	0.915	8785	22664
C	RMSEA	WLSMV	ALL	30	0.839	0.625	0.804	0.008	0.672	0.936	10177	25450
C	RMSEA	WLSMV	30	ALL	0.700	0.552	0.743	0.013	0.545	0.786	4654	12278
C	RMSEA	WLSMV	30	5	0.674	0.558	0.679	0.021	0.614	0.652	1065	2886
C	RMSEA	WLSMV	30	10	0.759	0.588	0.767	0.018	0.552	0.863	1524	4131
C	RMSEA	WLSMV	30	30	0.694	0.537	0.573	0.008	0.446	0.895	2065	5261
C	RMSEA	WLSMV	50	ALL	0.790	0.625	0.811	0.011	0.718	0.736	5850	15246
C	RMSEA	WLSMV	50	5	0.743	0.595	0.747	0.020	0.644	0.727	1410	3851
C	RMSEA	WLSMV	50	10	0.842	0.663	0.838	0.018	0.664	0.897	1981	5218
C	RMSEA	WLSMV	50	30	0.821	0.612	0.782	0.009	0.617	0.934	2459	6177
C	RMSEA	WLSMV	100	ALL	0.873	0.703	0.886	0.015	0.723	0.910	7200	18372
C	RMSEA	WLSMV	100	5	0.858	0.690	0.863	0.021	0.673	0.914	1994	5241
C	RMSEA	WLSMV	100	10	0.893	0.722	0.882	0.017	0.728	0.965	2473	6279
C	RMSEA	WLSMV	100	30	0.889	0.706	0.859	0.009	0.737	0.972	2733	6852
C	RMSEA	WLSMV	200	ALL	0.910	0.761	0.898	0.014	0.752	0.963	8273	20741
C	RMSEA	WLSMV	200	5	0.904	0.742	0.895	0.017	0.771	0.961	2546	6545
C	RMSEA	WLSMV	200	10	0.918	0.775	0.908	0.013	0.786	0.979	2807	7036
C	RMSEA	WLSMV	200	30	0.930	0.806	0.919	0.006	0.811	0.965	2920	7160
C	SRMRW	ALL	ALL	ALL	0.742	0.608	0.717	0.038	0.728	0.723	83510	223868
C	SRMRW	ALL	ALL	5	0.685	0.606	0.673	0.046	0.741	0.581	23460	63082
C	SRMRW	ALL	ALL	10	0.762	0.610	0.741	0.041	0.696	0.754	28208	75749
C	SRMRW	ALL	ALL	30	0.812	0.605	0.792	0.035	0.661	0.940	31842	85037
C	SRMRW	ALL	30	ALL	0.656	0.590	0.647	0.044	0.770	0.498	16157	43767
C	SRMRW	ALL	30	5	0.620	0.554	0.617	0.072	0.702	0.479	3994	10825
C	SRMRW	ALL	30	10	0.706	0.585	0.693	0.055	0.628	0.701	5297	14595
C	SRMRW	ALL	30	30	0.779	0.593	0.754	0.039	0.661	0.872	6866	18347
C	SRMRW	ALL	50	ALL	0.715	0.601	0.700	0.042	0.725	0.639	19330	52168
C	SRMRW	ALL	50	5	0.685	0.578	0.681	0.062	0.642	0.635	5037	13894
C	SRMRW	ALL	50	10	0.770	0.599	0.753	0.047	0.633	0.834	6490	17559
C	SRMRW	ALL	50	30	0.805	0.596	0.790	0.035	0.668	0.926	7803	20715
C	SRMRW	ALL	100	ALL	0.791	0.605	0.777	0.038	0.698	0.806	22743	60844
C	SRMRW	ALL	100	5	0.782	0.608	0.775	0.048	0.655	0.809	6487	17448
C	SRMRW	ALL	100	10	0.817	0.604	0.816	0.038	0.662	0.932	7852	20843
C	SRMRW	ALL	100	30	0.822	0.601	0.832	0.032	0.664	0.973	8404	22553
C	SRMRW	ALL	200	ALL	0.828	0.610	0.835	0.036	0.654	0.963	25280	67089
C	SRMRW	ALL	200	5	0.830	0.620	0.841	0.040	0.649	0.950	7942	20915
C	SRMRW	ALL	200	10	0.833	0.612	0.856	0.033	0.663	0.986	8569	22752
C	SRMRW	ALL	200	30	0.830	0.608	0.849	0.024	0.679	0.982	8769	23422
C	SRMRW	MLR	ALL	ALL	0.754	0.607	0.728	0.036	0.720	0.773	30018	85647
C	SRMRW	MLR	ALL	5	0.698	0.613	0.694	0.040	0.779	0.564	8945	24572
C	SRMRW	MLR	ALL	10	0.795	0.611	0.789	0.038	0.687	0.822	10090	28785
C	SRMRW	MLR	ALL	30	0.832	0.611	0.840	0.031	0.661	0.998	10983	32290
C	SRMRW	MLR	30	ALL	0.674	0.596	0.671	0.042	0.738	0.568	6295	17854
C	SRMRW	MLR	30	5	0.704	0.569	0.707	0.071	0.486	0.827	1726	4739
C	SRMRW	MLR	30	10	0.795	0.597	0.799	0.048	0.634	0.870	2054	5832
C	SRMRW	MLR	30	30	0.826	0.604	0.834	0.033	0.652	0.999	2515	7283
C	SRMRW	MLR	50	ALL	0.737	0.606	0.730	0.037	0.744	0.682	7124	20417
C	SRMRW	MLR	50	5	0.768	0.597	0.774	0.056	0.571	0.855	2032	5724
C	SRMRW	MLR	50	10	0.825	0.610	0.844	0.041	0.629	0.965	2387	6790
C	SRMRW	MLR	50	30	0.831	0.609	0.833	0.029	0.660	1.000	2705	7903
C	SRMRW	MLR	100	ALL	0.816	0.606	0.821	0.037	0.672	0.903	8014	22901
C	SRMRW	MLR	100	5	0.831	0.625	0.846	0.043	0.632	0.956	2417	6637
C	SRMRW	MLR	100	10	0.834	0.616	0.839	0.032	0.651	1.000	2760	7823
C	SRMRW	MLR	100	30	0.835	0.615	0.837	0.025	0.664	1.000	2837	8441
C	SRMRW	MLR	200	ALL	0.832	0.610	0.861	0.032	0.661	0.993	8585	24475
C	SRMRW	MLR	200	5	0.850	0.640	0.868	0.032	0.667	0.980	2770	7472
C	SRMRW	MLR	200	10	0.844	0.631	0.848	0.024	0.655	1.000	2889	8340
C	SRMRW	MLR	200	30	0.840	0.623	0.842	0.013	0.664	1.000	2926	8663
C	SRMRW	ULSMV	ALL	ALL	0.740	0.633	0.715	0.045	0.741	0.715	27515	71584
C	SRMRW	ULSMV	ALL	5	0.672	0.615	0.660	0.050	0.788	0.539	7500	19987
C	SRMRW	ULSMV	ALL	10	0.752	0.634	0.729	0.045	0.737	0.710	9333	24300
C	SRMRW	ULSMV	ALL	30	0.819	0.644	0.789	0.042	0.675	0.904	10682	27297
C	SRMRW	ULSMV	30	ALL	0.642	0.590	0.636	0.052	0.789	0.459	5208	13635
C	SRMRW	ULSMV	30	5	0.607	0.537	0.604	0.096	0.586	0.578	1203	3200
C	SRMRW	ULSMV	30	10	0.691	0.581	0.681	0.069	0.593	0.713	1719	4632
C	SRMRW	ULSMV	30	30	0.758	0.595	0.734	0.047	0.697	0.773	2286	5803
C	SRMRW	ULSMV	50	ALL	0.709	0.614	0.697	0.048	0.770	0.595	6356	16505
C	SRMRW	ULSMV	50	5	0.690	0.573	0.686	0.075	0.620	0.666	1595	4319
C	SRMRW	ULSMV	50	10	0.764	0.604	0.746	0.054	0.682	0.770	2122	5551
C	SRMRW	ULSMV	50	30	0.812	0.606	0.791	0.042	0.702	0.879	2639	6635
C	SRMRW	ULSMV	100	ALL	0.803	0.633	0.785	0.046	0.689	0.835	7529	19571
C	SRMRW	ULSMV	100	5	0.791	0.609	0.783	0.056	0.666	0.818	2076	5570
C	SRMRW	ULSMV	100	10	0.834	0.616	0.828	0.043	0.699	0.913	2619	6741
C	SRMRW	ULSMV	100	30	0.846	0.614	0.849	0.035	0.706	0.942	2834	7260
C	SRMRW	ULSMV	200	ALL	0.855	0.655	0.857	0.042	0.664	0.971	8422	21873
C	SRMRW	ULSMV	200	5	0.843	0.619	0.852	0.043	0.709	0.935	2626	6898
C	SRMRW	ULSMV	200	10	0.856	0.622	0.869	0.033	0.738	0.957	2873	7376
C	SRMRW	ULSMV	200	30	0.860	0.622	0.877	0.024	0.778	0.947	2923	7599
C	SRMRW	WLSMV	ALL	ALL	0.750	0.594	0.727	0.044	0.671	0.813	25977	66637
C	SRMRW	WLSMV	ALL	5	0.696	0.604	0.687	0.048	0.747	0.610	7015	18523
C	SRMRW	WLSMV	ALL	10	0.776	0.604	0.771	0.045	0.664	0.816	8785	22664
C	SRMRW	WLSMV	ALL	30	0.806	0.598	0.821	0.039	0.600	0.997	10177	25450
C	SRMRW	WLSMV	30	ALL	0.682	0.590	0.673	0.050	0.738	0.585	4654	12278
C	SRMRW	WLSMV	30	5	0.679	0.556	0.680	0.085	0.562	0.714	1065	2886
C	SRMRW	WLSMV	30	10	0.777	0.585	0.781	0.060	0.598	0.871	1524	4131
C	SRMRW	WLSMV	30	30	0.805	0.590	0.814	0.042	0.597	0.999	2065	5261
C	SRMRW	WLSMV	50	ALL	0.733	0.594	0.723	0.047	0.689	0.738	5850	15246
C	SRMRW	WLSMV	50	5	0.745	0.581	0.748	0.069	0.557	0.834	1410	3851
C	SRMRW	WLSMV	50	10	0.808	0.597	0.828	0.049	0.609	0.954	1981	5218
C	SRMRW	WLSMV	50	30	0.811	0.597	0.814	0.037	0.603	1.000	2459	6177
C	SRMRW	WLSMV	100	ALL	0.797	0.594	0.802	0.045	0.627	0.910	7200	18372
C	SRMRW	WLSMV	100	5	0.812	0.606	0.824	0.052	0.590	0.951	1994	5241
C	SRMRW	WLSMV	100	10	0.818	0.607	0.825	0.039	0.599	0.998	2473	6279
C	SRMRW	WLSMV	100	30	0.818	0.608	0.821	0.024	0.600	1.000	2733	6852
C	SRMRW	WLSMV	200	ALL	0.813	0.601	0.844	0.038	0.607	0.992	8273	20741
C	SRMRW	WLSMV	200	5	0.833	0.621	0.851	0.038	0.635	0.974	2546	6545
C	SRMRW	WLSMV	200	10	0.838	0.630	0.846	0.025	0.628	0.980	2807	7036
C	SRMRW	WLSMV	200	30	0.833	0.628	0.838	0.015	0.604	0.992	2920	7160
C	SRMRB	ALL	ALL	ALL	0.598	0.572	0.604	0.067	0.804	0.352	83510	223868
C	SRMRB	ALL	ALL	5	0.567	0.550	0.570	0.074	0.808	0.307	23460	63082
C	SRMRB	ALL	ALL	10	0.596	0.571	0.603	0.069	0.792	0.358	28208	75749
C	SRMRB	ALL	ALL	30	0.630	0.592	0.640	0.063	0.793	0.404	31842	85037
C	SRMRB	ALL	30	ALL	0.565	0.542	0.567	0.119	0.773	0.326	16157	43767
C	SRMRB	ALL	30	5	0.541	0.516	0.538	0.151	0.644	0.426	3994	10825
C	SRMRB	ALL	30	10	0.558	0.532	0.557	0.127	0.709	0.388	5297	14595
C	SRMRB	ALL	30	30	0.598	0.564	0.603	0.110	0.798	0.353	6866	18347
C	SRMRB	ALL	50	ALL	0.597	0.563	0.600	0.096	0.761	0.383	19330	52168
C	SRMRB	ALL	50	5	0.562	0.526	0.559	0.123	0.589	0.511	5037	13894
C	SRMRB	ALL	50	10	0.588	0.556	0.588	0.098	0.763	0.383	6490	17559
C	SRMRB	ALL	50	30	0.645	0.595	0.654	0.086	0.799	0.406	7803	20715
C	SRMRB	ALL	100	ALL	0.648	0.598	0.651	0.074	0.721	0.497	22743	60844
C	SRMRB	ALL	100	5	0.596	0.556	0.592	0.088	0.663	0.494	6487	17448
C	SRMRB	ALL	100	10	0.648	0.603	0.651	0.074	0.744	0.491	7852	20843
C	SRMRB	ALL	100	30	0.715	0.639	0.734	0.067	0.757	0.530	8404	22553
C	SRMRB	ALL	200	ALL	0.701	0.630	0.704	0.057	0.692	0.603	25280	67089
C	SRMRB	ALL	200	5	0.644	0.590	0.640	0.066	0.693	0.546	7942	20915
C	SRMRB	ALL	200	10	0.711	0.645	0.723	0.054	0.782	0.530	8569	22752
C	SRMRB	ALL	200	30	0.778	0.676	0.805	0.054	0.634	0.742	8769	23422
C	SRMRB	MLR	ALL	ALL	0.600	0.577	0.605	0.083	0.800	0.366	30018	85647
C	SRMRB	MLR	ALL	5	0.573	0.563	0.579	0.093	0.808	0.325	8945	24572
C	SRMRB	MLR	ALL	10	0.600	0.576	0.606	0.083	0.799	0.363	10090	28785
C	SRMRB	MLR	ALL	30	0.636	0.597	0.645	0.082	0.746	0.460	10983	32290
C	SRMRB	MLR	30	ALL	0.565	0.550	0.567	0.143	0.793	0.325	6295	17854
C	SRMRB	MLR	30	5	0.541	0.526	0.540	0.171	0.717	0.371	1726	4739
C	SRMRB	MLR	30	10	0.564	0.545	0.564	0.149	0.753	0.365	2054	5832
C	SRMRB	MLR	30	30	0.608	0.579	0.612	0.135	0.799	0.378	2515	7283
C	SRMRB	MLR	50	ALL	0.610	0.575	0.610	0.119	0.744	0.435	7124	20417
C	SRMRB	MLR	50	5	0.579	0.548	0.577	0.140	0.678	0.464	2032	5724
C	SRMRB	MLR	50	10	0.611	0.581	0.610	0.121	0.747	0.454	2387	6790
C	SRMRB	MLR	50	30	0.664	0.608	0.669	0.110	0.757	0.487	2705	7903
C	SRMRB	MLR	100	ALL	0.673	0.617	0.671	0.093	0.681	0.583	8014	22901
C	SRMRB	MLR	100	5	0.633	0.594	0.629	0.102	0.726	0.523	2417	6637
C	SRMRB	MLR	100	10	0.678	0.631	0.675	0.086	0.806	0.491	2760	7823
C	SRMRB	MLR	100	30	0.746	0.656	0.766	0.084	0.723	0.612	2837	8441
C	SRMRB	MLR	200	ALL	0.732	0.657	0.732	0.069	0.708	0.646	8585	24475
C	SRMRB	MLR	200	5	0.686	0.643	0.682	0.074	0.779	0.543	2770	7472
C	SRMRB	MLR	200	10	0.741	0.675	0.755	0.066	0.782	0.580	2889	8340
C	SRMRB	MLR	200	30	0.819	0.709	0.841	0.064	0.678	0.762	2926	8663
C	SRMRB	ULSMV	ALL	ALL	0.595	0.580	0.606	0.056	0.838	0.324	27515	71584
C	SRMRB	ULSMV	ALL	5	0.563	0.555	0.570	0.064	0.817	0.300	7500	19987
C	SRMRB	ULSMV	ALL	10	0.594	0.580	0.605	0.055	0.852	0.309	9333	24300
C	SRMRB	ULSMV	ALL	30	0.626	0.601	0.637	0.054	0.800	0.403	10682	27297
C	SRMRB	ULSMV	30	ALL	0.567	0.549	0.573	0.106	0.734	0.381	5208	13635
C	SRMRB	ULSMV	30	5	0.545	0.517	0.544	0.141	0.474	0.603	1203	3200
C	SRMRB	ULSMV	30	10	0.562	0.535	0.561	0.108	0.716	0.403	1719	4632
C	SRMRB	ULSMV	30	30	0.593	0.578	0.601	0.096	0.772	0.400	2286	5803
C	SRMRB	ULSMV	50	ALL	0.596	0.576	0.604	0.081	0.795	0.366	6356	16505
C	SRMRB	ULSMV	50	5	0.557	0.531	0.555	0.104	0.593	0.507	1595	4319
C	SRMRB	ULSMV	50	10	0.591	0.566	0.593	0.086	0.747	0.425	2122	5551
C	SRMRB	ULSMV	50	30	0.640	0.620	0.653	0.072	0.848	0.394	2639	6635
C	SRMRB	ULSMV	100	ALL	0.649	0.620	0.655	0.060	0.843	0.398	7529	19571
C	SRMRB	ULSMV	100	5	0.597	0.569	0.595	0.074	0.715	0.464	2076	5570
C	SRMRB	ULSMV	100	10	0.652	0.634	0.656	0.063	0.796	0.489	2619	6741
C	SRMRB	ULSMV	100	30	0.710	0.673	0.731	0.053	0.895	0.441	2834	7260
C	SRMRB	ULSMV	200	ALL	0.701	0.656	0.707	0.047	0.793	0.516	8422	21873
C	SRMRB	ULSMV	200	5	0.646	0.614	0.644	0.053	0.797	0.462	2626	6898
C	SRMRB	ULSMV	200	10	0.714	0.685	0.727	0.044	0.876	0.492	2873	7376
C	SRMRB	ULSMV	200	30	0.779	0.713	0.808	0.038	0.935	0.467	2923	7599
C	SRMRB	WLSMV	ALL	ALL	0.601	0.578	0.609	0.062	0.822	0.345	25977	66637
C	SRMRB	WLSMV	ALL	5	0.570	0.558	0.576	0.069	0.823	0.302	7015	18523
C	SRMRB	WLSMV	ALL	10	0.598	0.577	0.606	0.063	0.818	0.342	8785	22664
C	SRMRB	WLSMV	ALL	30	0.631	0.596	0.640	0.060	0.798	0.407	10177	25450
C	SRMRB	WLSMV	30	ALL	0.576	0.551	0.579	0.117	0.793	0.325	4654	12278
C	SRMRB	WLSMV	30	5	0.551	0.518	0.548	0.152	0.538	0.555	1065	2886
C	SRMRB	WLSMV	30	10	0.573	0.541	0.570	0.122	0.736	0.391	1524	4131
C	SRMRB	WLSMV	30	30	0.608	0.577	0.614	0.109	0.822	0.356	2065	5261
C	SRMRB	WLSMV	50	ALL	0.613	0.579	0.618	0.096	0.730	0.450	5850	15246
C	SRMRB	WLSMV	50	5	0.576	0.534	0.573	0.112	0.642	0.483	1410	3851
C	SRMRB	WLSMV	50	10	0.598	0.567	0.598	0.096	0.774	0.409	1981	5218
C	SRMRB	WLSMV	50	30	0.671	0.620	0.681	0.086	0.819	0.438	2459	6177
C	SRMRB	WLSMV	100	ALL	0.670	0.621	0.670	0.071	0.728	0.534	7200	18372
C	SRMRB	WLSMV	100	5	0.608	0.573	0.603	0.080	0.736	0.462	1994	5241
C	SRMRB	WLSMV	100	10	0.668	0.627	0.667	0.072	0.755	0.530	2473	6279
C	SRMRB	WLSMV	100	30	0.754	0.674	0.774	0.064	0.795	0.558	2733	6852
C	SRMRB	WLSMV	200	ALL	0.723	0.651	0.722	0.054	0.699	0.643	8273	20741
C	SRMRB	WLSMV	200	5	0.661	0.616	0.657	0.061	0.727	0.548	2546	6545
C	SRMRB	WLSMV	200	10	0.736	0.673	0.745	0.052	0.795	0.577	2807	7036
C	SRMRB	WLSMV	200	30	0.814	0.707	0.838	0.048	0.735	0.698	2920	7160

Detecting Misspecification at Level-1

General overview over all conditions

j <- 2 ## Which class?
for(index in INDEX){
    ## Print out which iteration so we know what we am looking at
    cat('\n\nROC Analysis in')
    cat('\nIndex:\t', index)
    cat('\nClassification:\t', CLASS[j])
    ## Set up iteration key
    key <- paste0(index,'.',CLASS[j])
    ## Create formula
    model <- as.formula(paste0(CLASS[j], '~', index))
    ## Fit ROC curve
    fit_roc[[key]] <-  roc(model, data=sim_results,
                           plot =TRUE, ci=TRUE, print.auc=TRUE)
    ## Create a plot of "smoothed" curve for plotting
    fit_roc_smooth[[key]] <-  smooth(roc(model, data=sim_results))
    ## Compute partial AUC for specificity .8-1
    p.auc <- auc(fit_roc[[key]], partial.auc = c(1,.8),
                 partial.auc.focus = 'sp', partial.auc.correct = T)
    ## get summary info
    roc_summary_gen[ig, 2] <- index
    roc_summary_gen[ig, 1] <- CLASS[j]
    roc_summary_gen[ig, 3] <- fit_roc[[key]]$auc ## total AUC
    roc_summary_gen[ig, 4] <- p.auc ## corrected partial AUC (.5 is no discrimination)
    roc_summary_gen[ig, 5] <- fit_roc_smooth[[key]]$auc ## smoothed AUC
    roc_summary_gen[ig, 6:8] <- coords(fit_roc[[key]], "best", 
                                   ret=c("threshold", "specificity", 'sensitivity'))
    ## print summary
    cat('\n\nSummary of ROC:\n')
    print(roc_summary_gen[ig, ])
    ## add to summary iterator
    ig <- ig + 1
} ## End loop round index



ROC Analysis in
Index:   CFI
Classification:  CvM1

Setting levels: control = 0, case = 1

Setting direction: controls < cases

Setting levels: control = 0, case = 1

Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
  Classification Index       AUC partial-AUC Smoothed-AUC
6           CvM1   CFI 0.8942411   0.7896568    0.9008448
  Optimal-Threshold Specificity Sensitivity
6         0.9707385   0.8682327    0.883183


ROC Analysis in
Index:   TLI
Classification:  CvM1

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
  Classification Index       AUC partial-AUC Smoothed-AUC
7           CvM1   TLI 0.8942411   0.7896568    0.9008143
  Optimal-Threshold Specificity Sensitivity
7         0.9648862   0.8682327    0.883183


ROC Analysis in
Index:   RMSEA
Classification:  CvM1

Setting levels: control = 0, case = 1

Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
  Classification Index       AUC partial-AUC Smoothed-AUC
8           CvM1 RMSEA 0.8776767   0.7534801    0.8844573
  Optimal-Threshold Specificity Sensitivity
8         0.0169548   0.8114027   0.8589859


ROC Analysis in
Index:   SRMRW
Classification:  CvM1

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
  Classification Index       AUC partial-AUC Smoothed-AUC
9           CvM1 SRMRW 0.8725866    0.869644    0.8879885
  Optimal-Threshold Specificity Sensitivity
9        0.03824319   0.9745978   0.7240689


ROC Analysis in
Index:   SRMRB
Classification:  CvM1

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.

Version	Author	Date
982c8f1	noah-padgett	2019-05-18



Summary of ROC:
   Classification Index       AUC partial-AUC Smoothed-AUC
10           CvM1 SRMRB 0.5373652   0.5405211    0.5473238
   Optimal-Threshold Specificity Sensitivity
10        0.05688385   0.8393742   0.2433847

kable(roc_summary_gen[1:5,], format = 'html', digits=3) %>%
  kable_styling(full_width = T)

Classification	Index	AUC	partial-AUC	Smoothed-AUC	Optimal-Threshold	Specificity	Sensitivity
C	CFI	0.816	0.637	0.846	0.977	0.702	0.855
C	TLI	0.815	0.637	0.845	0.972	0.702	0.855
C	RMSEA	0.803	0.635	0.830	0.015	0.685	0.829
C	SRMRW	0.742	0.608	0.717	0.038	0.728	0.723
C	SRMRB	0.598	0.572	0.604	0.067	0.804	0.352

print(xtable(roc_summary_gen[1:5,c(2:3,6:8)], digits = 3), booktabs=T,include.rownames = F)

% latex table generated in R 3.6.0 by xtable 1.8-4 package
% Sat Sep 28 17:58:06 2019
\begin{table}[ht]
\centering
\begin{tabular}{lrrrr}
  \toprule
Index & AUC & Optimal-Threshold & Specificity & Sensitivity \\ 
  \midrule
CFI & 0.816 & 0.977 & 0.702 & 0.855 \\ 
  TLI & 0.815 & 0.972 & 0.702 & 0.855 \\ 
  RMSEA & 0.803 & 0.015 & 0.685 & 0.829 \\ 
  SRMRW & 0.742 & 0.038 & 0.728 & 0.723 \\ 
  SRMRB & 0.598 & 0.067 & 0.804 & 0.352 \\ 
   \bottomrule
\end{tabular}
\end{table}

More fine grained information within/across conditions

i <- 401
j <- 2 ## Which class?
for(index in INDEX){
  for(est in EST){
    for(s2 in SS_L2){
      for(s1 in SS_L1){
    ## Print out which iteration so we know what we are looking at
    cat('\n\nROC Analysis in')
    cat('\nIndex:\t', index)
    cat('\nClassification:\t', CLASS[j])
    cat('\nEstimation Method:\t', est)
    cat('\nLevel-2 Sample Size:\t', s2)
    cat('\nLevel-1 Sample Size:\t', s1)
    ## Set up iteration key
    key <- paste0(index,'.',CLASS[j],'.',est,'.', s2,'.',s1)
    # Subset data as  needed
    if(est == 'ALL' & s2 == 'ALL' & s1 == 'ALL') mydata <- sim_results
    if(est != 'ALL' & s2 == 'ALL' & s1 == 'ALL'){
      mydata <- filter(sim_results, Estimator == est)
    }
    if(est == 'ALL' & s2 != 'ALL' & s1 == 'ALL'){
      mydata <- filter(sim_results, ss_l2 == s2)
    }
    if(est == 'ALL' & s2 == 'ALL' & s1 != 'ALL'){
      mydata <- filter(sim_results, ss_l1 == s1)
    }
    if(est != 'ALL' & s2 != 'ALL' & s1 == 'ALL'){
      mydata <- filter(sim_results, Estimator == est, ss_l2 == s2)
    }
    if(est != 'ALL' & s2 == 'ALL' & s1 != 'ALL'){
      mydata <- filter(sim_results, Estimator == est, ss_l1 == s1)
    }
    if(est == 'ALL' & s2 != 'ALL' & s1 != 'ALL'){
      mydata <- filter(sim_results, ss_l2 == s2, ss_l1 == s1)
    }
    if(est != 'ALL' & s2 != 'ALL' & s1 != 'ALL'){
      mydata <- filter(sim_results, Estimator == est, ss_l2 == s2, ss_l1 == s1)
    }
    ## Create formula
    model <- as.formula(paste0(CLASS[j], '~', index))
    ## Fit ROC curve
    fit_roc[[key]] <-  roc(model, data=mydata,
                           plot =TRUE, ci=TRUE, print.auc=TRUE)
    ## Create a plot of "smoothed" curve for plotting
    fit_roc_smooth[[key]] <-  tryCatch(smooth(roc(model, data=mydata)),
                                       error = function(e) NA)
                                      
    ## Compute partial AUC for specificity .8-1
    p.auc <- auc(fit_roc[[key]], partial.auc = c(1,.8),
                 partial.auc.focus = 'sp', partial.auc.correct = T)
    ## get summary info
    roc_summary[i, 2] <- index
    roc_summary[i, 1] <- CLASS[j]
    roc_summary[i, 3] <- est ##estimator
    roc_summary[i, 4] <- s2 ## level-2 sample size
    roc_summary[i, 5] <- s1 ## level-1 sample size
    roc_summary[i, 6] <- fit_roc[[key]]$auc ## total AUC
    roc_summary[i, 7] <- p.auc ## corrected partial AUC (.5 is no discrimination)
    if(is.na(fit_roc_smooth[[key]]) == T){
      roc_summary[i, 8] <- NA
    } else {
      roc_summary[i, 8] <- fit_roc_smooth[[key]]$auc ## smoothed AUC
    }
    roc_summary[i, 9:11] <- coords(fit_roc[[key]], "best", 
                                   ret=c("threshold", "specificity", 'sensitivity'))
    
    ## add number of C and number of miss models in analysis
    n.C <- nrow(mydata[ mydata[, CLASS[j]] == 1, ])
    n.M <- nrow(mydata[ mydata[, CLASS[j]] == 0, ])
    roc_summary[i, 12] <- n.C
    roc_summary[i, 13] <- n.M
    
    ## print summary
    cat('\n\nSummary of ROC:\n')
    print(roc_summary[i, ])
    ## add to summary iterator
    i <- i + 1
      } ## end loop around ss l1
    } ## End loop around ss l2
  } ## End loop around estimator
} ## End loop round index



ROC Analysis in
Index:   CFI
Classification:  CvM1
Estimation Method:   ALL
Level-2 Sample Size:     ALL
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1

Setting direction: controls < cases

Setting levels: control = 0, case = 1

Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
401           CvM1   CFI       ALL        ALL        ALL 0.8942411
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity  Num-C
401   0.7896568    0.9008448 0.9707385   0.8682327    0.883183 241960
    Num-Mis
401  223868


ROC Analysis in
Index:   CFI
Classification:  CvM1
Estimation Method:   ALL
Level-2 Sample Size:     ALL
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
402           CvM1   CFI       ALL        ALL          5 0.8456471
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
402   0.7693923    0.8464149 0.9680536   0.8682343   0.7659284 68483
    Num-Mis
402   63082


ROC Analysis in
Index:   CFI
Classification:  CvM1
Estimation Method:   ALL
Level-2 Sample Size:     ALL
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
403           CvM1   CFI       ALL        ALL         10 0.9265018
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
403   0.8554348    0.9220163 0.9675898   0.9019288   0.8890662 81868
    Num-Mis
403   75749


ROC Analysis in
Index:   CFI
Classification:  CvM1
Estimation Method:   ALL
Level-2 Sample Size:     ALL
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
404           CvM1   CFI       ALL        ALL         30 0.9077412
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
404   0.7447199      0.91154  0.976402   0.8356945   0.9757553 91609
    Num-Mis
404   85037


ROC Analysis in
Index:   CFI
Classification:  CvM1
Estimation Method:   ALL
Level-2 Sample Size:     30
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
405           CvM1   CFI       ALL         30        ALL 0.7387235
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
405   0.5719083    0.7701139 0.9626317   0.6915349   0.7607267 47053
    Num-Mis
405   43767


ROC Analysis in
Index:   CFI
Classification:  CvM1
Estimation Method:   ALL
Level-2 Sample Size:     30
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
406           CvM1   CFI       ALL         30          5 0.6614421
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
406   0.5605278    0.6656344 0.9214189    0.619389   0.6327199 11584
    Num-Mis
406   10825


ROC Analysis in
Index:   CFI
Classification:  CvM1
Estimation Method:   ALL
Level-2 Sample Size:     30
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
407           CvM1   CFI       ALL         30         10 0.7732855
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
407   0.6118027    0.7867282 0.9520732   0.6747349   0.7688385 15551
    Num-Mis
407   14595


ROC Analysis in
Index:   CFI
Classification:  CvM1
Estimation Method:   ALL
Level-2 Sample Size:     30
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
408           CvM1   CFI       ALL         30         30 0.7825584
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
408   0.5584719    0.7416804 0.9647869   0.6268178   0.9685406 19918
    Num-Mis
408   18347


ROC Analysis in
Index:   CFI
Classification:  CvM1
Estimation Method:   ALL
Level-2 Sample Size:     50
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
409           CvM1   CFI       ALL         50        ALL 0.8506211
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
409   0.7024575    0.8667305 0.9703262   0.8082932     0.82597 56136
    Num-Mis
409   52168


ROC Analysis in
Index:   CFI
Classification:  CvM1
Estimation Method:   ALL
Level-2 Sample Size:     50
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
410           CvM1   CFI       ALL         50          5 0.7700341
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
410   0.6364384    0.7753967 0.9607316   0.7629468   0.6541592 14818
    Num-Mis
410   13894


ROC Analysis in
Index:   CFI
Classification:  CvM1
Estimation Method:   ALL
Level-2 Sample Size:     50
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
411           CvM1   CFI       ALL         50         10 0.9051174
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
411   0.7953663     0.903227  0.959175    0.816683   0.8872111 18954
    Num-Mis
411   17559


ROC Analysis in
Index:   CFI
Classification:  CvM1
Estimation Method:   ALL
Level-2 Sample Size:     50
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
412           CvM1   CFI       ALL         50         30 0.8775863
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
412   0.6626137    0.8735651 0.9803301   0.7842054   0.9811611 22364
    Num-Mis
412   20715


ROC Analysis in
Index:   CFI
Classification:  CvM1
Estimation Method:   ALL
Level-2 Sample Size:     100
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS      AUC
413           CvM1   CFI       ALL        100        ALL 0.950266
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
413    0.878201    0.9503397 0.9707376   0.9093476    0.933474 65882
    Num-Mis
413   60844


ROC Analysis in
Index:   CFI
Classification:  CvM1
Estimation Method:   ALL
Level-2 Sample Size:     100
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
414           CvM1   CFI       ALL        100          5 0.9259115
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
414   0.8418075    0.9233488 0.9670375   0.8665186   0.8466163 18983
    Num-Mis
414   17448


ROC Analysis in
Index:   CFI
Classification:  CvM1
Estimation Method:   ALL
Level-2 Sample Size:     100
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS      AUC
415           CvM1   CFI       ALL        100         10 0.982225
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
415   0.9606691    0.9730033 0.9709627   0.9558824   0.9593734 22711
    Num-Mis
415   20843


ROC Analysis in
Index:   CFI
Classification:  CvM1
Estimation Method:   ALL
Level-2 Sample Size:     100
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
416           CvM1   CFI       ALL        100         30 0.9375299
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
416   0.8265154    0.9297539 0.9819622   0.8939282   0.9950024 24188
    Num-Mis
416   22553


ROC Analysis in
Index:   CFI
Classification:  CvM1
Estimation Method:   ALL
Level-2 Sample Size:     200
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
417           CvM1   CFI       ALL        200        ALL 0.9970231
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
417   0.9925265    0.9937743 0.9749699   0.9823409   0.9791535 72889
    Num-Mis
417   67089


ROC Analysis in
Index:   CFI
Classification:  CvM1
Estimation Method:   ALL
Level-2 Sample Size:     200
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
418           CvM1   CFI       ALL        200          5 0.9934448
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
418   0.9845821    0.9892718 0.9708699   0.9696128   0.9665072 23098
    Num-Mis
418   20915


ROC Analysis in
Index:   CFI
Classification:  CvM1
Estimation Method:   ALL
Level-2 Sample Size:     200
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
419           CvM1   CFI       ALL        200         10 0.9993397
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
419   0.9983872    0.9987116  0.970083   0.9920528   0.9901972 24652
    Num-Mis
419   22752


ROC Analysis in
Index:   CFI
Classification:  CvM1
Estimation Method:   ALL
Level-2 Sample Size:     200
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
420           CvM1   CFI       ALL        200         30 0.9981706
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
420   0.9949328     0.997855 0.9872994   0.9907828   0.9916752 25139
    Num-Mis
420   23422


ROC Analysis in
Index:   CFI
Classification:  CvM1
Estimation Method:   MLR
Level-2 Sample Size:     ALL
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
421           CvM1   CFI       MLR        ALL        ALL 0.9322555
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
421   0.9220184    0.9188806 0.9564152   0.9483419   0.8731095 89822
    Num-Mis
421   85647


ROC Analysis in
Index:   CFI
Classification:  CvM1
Estimation Method:   MLR
Level-2 Sample Size:     ALL
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
422           CvM1   CFI       MLR        ALL          5 0.8319261
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
422   0.7993272    0.8315971   0.96279   0.9069904   0.7043041 26722
    Num-Mis
422   24572


ROC Analysis in
Index:   CFI
Classification:  CvM1
Estimation Method:   MLR
Level-2 Sample Size:     ALL
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
423           CvM1   CFI       MLR        ALL         10 0.9466965
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
423   0.9288553    0.9401724 0.9559649   0.9549539   0.8660059 30195
    Num-Mis
423   28785


ROC Analysis in
Index:   CFI
Classification:  CvM1
Estimation Method:   MLR
Level-2 Sample Size:     ALL
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
424           CvM1   CFI       MLR        ALL         30 0.9997934
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
424   0.9994262    0.9997744 0.9559119   0.9947917   0.9932623 32905
    Num-Mis
424   32290


ROC Analysis in
Index:   CFI
Classification:  CvM1
Estimation Method:   MLR
Level-2 Sample Size:     30
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
425           CvM1   CFI       MLR         30        ALL 0.8290347
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
425   0.7978814    0.8215607 0.9435555   0.9180786   0.6840349 18778
    Num-Mis
425   17854


ROC Analysis in
Index:   CFI
Classification:  CvM1
Estimation Method:   MLR
Level-2 Sample Size:     30
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
426           CvM1   CFI       MLR         30          5 0.6986976
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
426   0.5974753    0.6982967 0.8889698   0.7215373   0.5816918  5086
    Num-Mis
426    4739


ROC Analysis in
Index:   CFI
Classification:  CvM1
Estimation Method:   MLR
Level-2 Sample Size:     30
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
427           CvM1   CFI       MLR         30         10 0.8598081
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
427   0.7513297    0.8578347 0.9187879   0.7511442   0.8106134  6138
    Num-Mis
427    5832


ROC Analysis in
Index:   CFI
Classification:  CvM1
Estimation Method:   MLR
Level-2 Sample Size:     30
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
428           CvM1   CFI       MLR         30         30 0.9980708
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
428    0.994641    0.9979653 0.9489178   0.9728164   0.9872763  7554
    Num-Mis
428    7283


ROC Analysis in
Index:   CFI
Classification:  CvM1
Estimation Method:   MLR
Level-2 Sample Size:     50
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
429           CvM1   CFI       MLR         50        ALL 0.9262014
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
429   0.8822931     0.913531 0.9545155   0.9205607   0.8422235 21335
    Num-Mis
429   20417


ROC Analysis in
Index:   CFI
Classification:  CvM1
Estimation Method:   MLR
Level-2 Sample Size:     50
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
430           CvM1   CFI       MLR         50          5 0.7946051
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
430   0.6692239    0.7953592 0.9290285   0.6814106   0.7623031  6083
    Num-Mis
430    5724


ROC Analysis in
Index:   CFI
Classification:  CvM1
Estimation Method:   MLR
Level-2 Sample Size:     50
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
431           CvM1   CFI       MLR         50         10 0.9605345
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
431   0.9103955    0.9594576 0.9530421   0.9073244   0.8864684  7170
    Num-Mis
431    6790


ROC Analysis in
Index:   CFI
Classification:  CvM1
Estimation Method:   MLR
Level-2 Sample Size:     50
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
432           CvM1   CFI       MLR         50         30 0.9999925
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
432   0.9999793    0.9999754 0.9660635   0.9980206   0.9996303  8082
    Num-Mis
432    7903


ROC Analysis in
Index:   CFI
Classification:  CvM1
Estimation Method:   MLR
Level-2 Sample Size:     100
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS      AUC
433           CvM1   CFI       MLR        100        ALL 0.990454
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
433   0.9773952    0.9844113 0.9618129   0.9587076   0.9599451 24013
    Num-Mis
433   22901


ROC Analysis in
Index:   CFI
Classification:  CvM1
Estimation Method:   MLR
Level-2 Sample Size:     100
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS      AUC
434           CvM1   CFI       MLR        100          5 0.945666
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
434   0.8769856    0.9456985 0.9587612   0.8474388    0.902772  7258
    Num-Mis
434    6637


ROC Analysis in
Index:   CFI
Classification:  CvM1
Estimation Method:   MLR
Level-2 Sample Size:     100
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
435           CvM1   CFI       MLR        100         10 0.9993947
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
435   0.9983187    0.9994504 0.9659251   0.9880648   0.9905797  8237
    Num-Mis
435    7823


ROC Analysis in
Index:   CFI
Classification:  CvM1
Estimation Method:   MLR
Level-2 Sample Size:     100
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in ci.auc.roc(roc, ...): ci.auc() of a ROC curve with AUC == 1 is
always 1-1 and can be misleading.

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS AUC partial-AUC
436           CvM1   CFI       MLR        100         30   1           1
    Smoothed-AUC Threshold Specificity Sensitivity Num-C Num-Mis
436           NA  0.967426           1           1  8518    8441


ROC Analysis in
Index:   CFI
Classification:  CvM1
Estimation Method:   MLR
Level-2 Sample Size:     200
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
437           CvM1   CFI       MLR        200        ALL 0.9997541
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
437    0.999317    0.9997498 0.9685923   0.9944324   0.9953407 25696
    Num-Mis
437   24475


ROC Analysis in
Index:   CFI
Classification:  CvM1
Estimation Method:   MLR
Level-2 Sample Size:     200
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
438           CvM1   CFI       MLR        200          5 0.9980997
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
438   0.9947215      0.99789 0.9707632   0.9845917   0.9808664  8295
    Num-Mis
438    7472


ROC Analysis in
Index:   CFI
Classification:  CvM1
Estimation Method:   MLR
Level-2 Sample Size:     200
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in ci.auc.roc(roc, ...): ci.auc() of a ROC curve with AUC == 1 is
always 1-1 and can be misleading.

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS AUC partial-AUC
439           CvM1   CFI       MLR        200         10   1           1
    Smoothed-AUC Threshold Specificity Sensitivity Num-C Num-Mis
439           NA 0.9682202           1           1  8650    8340


ROC Analysis in
Index:   CFI
Classification:  CvM1
Estimation Method:   MLR
Level-2 Sample Size:     200
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in ci.auc.roc(roc, ...): ci.auc() of a ROC curve with AUC == 1 is
always 1-1 and can be misleading.

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS AUC partial-AUC
440           CvM1   CFI       MLR        200         30   1           1
    Smoothed-AUC Threshold Specificity Sensitivity Num-C Num-Mis
440           NA 0.9692736           1           1  8751    8663


ROC Analysis in
Index:   CFI
Classification:  CvM1
Estimation Method:   ULSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
441           CvM1   CFI     ULSMV        ALL        ALL 0.8220301
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
441   0.6199448    0.8714369 0.9774024    0.750117   0.8742182 77713
    Num-Mis
441   71584


ROC Analysis in
Index:   CFI
Classification:  CvM1
Estimation Method:   ULSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
442           CvM1   CFI     ULSMV        ALL          5 0.8186528
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
442   0.6965808    0.8352832 0.9667913   0.7888797   0.7933752 21448
    Num-Mis
442   19987


ROC Analysis in
Index:   CFI
Classification:  CvM1
Estimation Method:   ULSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
443           CvM1   CFI     ULSMV        ALL         10 0.8728878
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
443    0.723139    0.8923976 0.9702485   0.8095634   0.8881149 26416
    Num-Mis
443   24300


ROC Analysis in
Index:   CFI
Classification:  CvM1
Estimation Method:   ULSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
444           CvM1   CFI     ULSMV        ALL         30 0.7922338
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
444   0.5723193     0.792785 0.9872387   0.6430504   0.9753791 29849
    Num-Mis
444   27297


ROC Analysis in
Index:   CFI
Classification:  CvM1
Estimation Method:   ULSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
445           CvM1   CFI     ULSMV         30        ALL 0.6489314
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
445   0.5289806    0.7513845 0.9793255   0.4817073   0.8299634 14727
    Num-Mis
445   13635


ROC Analysis in
Index:   CFI
Classification:  CvM1
Estimation Method:   ULSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
446           CvM1   CFI     ULSMV         30          5 0.6266319
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
446   0.5359382    0.6490378  0.966612   0.6060606   0.6218487  3399
    Num-Mis
446    3200


ROC Analysis in
Index:   CFI
Classification:  CvM1
Estimation Method:   ULSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
447           CvM1   CFI     ULSMV         30         10 0.6906809
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
447    0.542908    0.7603944 0.9775123   0.5593816   0.7949913  4926
    Num-Mis
447    4632


ROC Analysis in
Index:   CFI
Classification:  CvM1
Estimation Method:   ULSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
448           CvM1   CFI     ULSMV         30         30 0.6310332
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
448   0.5186999    0.5154008 0.9874228   0.3366924   0.9755031  6402
    Num-Mis
448    5803


ROC Analysis in
Index:   CFI
Classification:  CvM1
Estimation Method:   ULSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
449           CvM1   CFI     ULSMV         50        ALL 0.7346183
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
449   0.5619858    0.7987331 0.9715841   0.6129881   0.8340151 17918
    Num-Mis
449   16505


ROC Analysis in
Index:   CFI
Classification:  CvM1
Estimation Method:   ULSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS      AUC
450           CvM1   CFI     ULSMV         50          5 0.712862
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
450   0.5859783    0.7274272 0.9618627   0.6674124   0.6695925  4573
    Num-Mis
450    4319


ROC Analysis in
Index:   CFI
Classification:  CvM1
Estimation Method:   ULSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
451           CvM1   CFI     ULSMV         50         10 0.8130939
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
451   0.6443693    0.8366097  0.970552   0.7480122    0.803016  6038
    Num-Mis
451    5551


ROC Analysis in
Index:   CFI
Classification:  CvM1
Estimation Method:   ULSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
452           CvM1   CFI     ULSMV         50         30 0.7008154
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
452   0.5346987    0.6743767 0.9880069   0.4819522   0.9765063  7307
    Num-Mis
452    6635


ROC Analysis in
Index:   CFI
Classification:  CvM1
Estimation Method:   ULSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
453           CvM1   CFI     ULSMV        100        ALL 0.8753835
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
453   0.7110234    0.9008906 0.9706737   0.7812124   0.9051667 21277
    Num-Mis
453   19571


ROC Analysis in
Index:   CFI
Classification:  CvM1
Estimation Method:   ULSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
454           CvM1   CFI     ULSMV        100          5 0.8798768
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
454   0.7679274    0.8835196 0.9665027   0.8091831   0.8131021  5969
    Num-Mis
454    5570


ROC Analysis in
Index:   CFI
Classification:  CvM1
Estimation Method:   ULSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
455           CvM1   CFI     ULSMV        100         10 0.9459836
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
455   0.8847264    0.9434985 0.9702017   0.8852792    0.908362  7390
    Num-Mis
455    6741


ROC Analysis in
Index:   CFI
Classification:  CvM1
Estimation Method:   ULSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS      AUC
456           CvM1   CFI     ULSMV        100         30 0.823498
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
456   0.5958182    0.7893048 0.9836196   0.6741728   0.9816514  7918
    Num-Mis
456    7260


ROC Analysis in
Index:   CFI
Classification:  CvM1
Estimation Method:   ULSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
457           CvM1   CFI     ULSMV        200        ALL 0.9889358
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
457    0.973161    0.9862303 0.9752983   0.9538745   0.9555925 23791
    Num-Mis
457   21873


ROC Analysis in
Index:   CFI
Classification:  CvM1
Estimation Method:   ULSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
458           CvM1   CFI     ULSMV        200          5 0.9825903
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
458   0.9606567    0.9794635 0.9681439   0.9335647   0.9440213  7507
    Num-Mis
458    6898


ROC Analysis in
Index:   CFI
Classification:  CvM1
Estimation Method:   ULSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
459           CvM1   CFI     ULSMV        200         10 0.9964108
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
459   0.9911202    0.9959018  0.970083   0.9766804   0.9707623  8062
    Num-Mis
459    7376


ROC Analysis in
Index:   CFI
Classification:  CvM1
Estimation Method:   ULSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
460           CvM1   CFI     ULSMV        200         30 0.9931552
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
460   0.9812515    0.9929759 0.9872994   0.9717391   0.9750257  8222
    Num-Mis
460    7599


ROC Analysis in
Index:   CFI
Classification:  CvM1
Estimation Method:   WLSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
461           CvM1   CFI     WLSMV        ALL        ALL 0.9448689
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
461   0.8770901    0.9466497 0.9770158   0.9059147   0.9228518 74425
    Num-Mis
461   66637


ROC Analysis in
Index:   CFI
Classification:  CvM1
Estimation Method:   WLSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
462           CvM1   CFI     WLSMV        ALL          5 0.9096198
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
462   0.8307082     0.906213  0.969595   0.8816193   0.8497358 20313
    Num-Mis
462   18523


ROC Analysis in
Index:   CFI
Classification:  CvM1
Estimation Method:   WLSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
463           CvM1   CFI     WLSMV        ALL         10 0.9766076
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
463   0.9431138    0.9703289 0.9730338   0.9342594   0.9532157 25257
    Num-Mis
463   22664


ROC Analysis in
Index:   CFI
Classification:  CvM1
Estimation Method:   WLSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS     AUC
464           CvM1   CFI     WLSMV        ALL         30 0.95035
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
464   0.8620834    0.9385691 0.9875561   0.8976669   0.9920409 28855
    Num-Mis
464   25450


ROC Analysis in
Index:   CFI
Classification:  CvM1
Estimation Method:   WLSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
465           CvM1   CFI     WLSMV         30        ALL 0.8035081
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
465   0.6315897    0.8442203 0.9826179   0.7320475   0.8114224 13548
    Num-Mis
465   12278


ROC Analysis in
Index:   CFI
Classification:  CvM1
Estimation Method:   WLSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
466           CvM1   CFI     WLSMV         30          5 0.7340084
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
466   0.6001856    0.7381871 0.9301306   0.5125149   0.8392008  3099
    Num-Mis
466    2886


ROC Analysis in
Index:   CFI
Classification:  CvM1
Estimation Method:   WLSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS      AUC
467           CvM1   CFI     WLSMV         30         10 0.885788
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
467   0.7646601    0.8936167 0.9810663   0.8354756   0.8038058  4487
    Num-Mis
467    4131


ROC Analysis in
Index:   CFI
Classification:  CvM1
Estimation Method:   WLSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
468           CvM1   CFI     WLSMV         30         30 0.8006895
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
468   0.5838078    0.7227762   0.98645   0.6121701   0.9786925  5962
    Num-Mis
468    5261


ROC Analysis in
Index:   CFI
Classification:  CvM1
Estimation Method:   WLSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
469           CvM1   CFI     WLSMV         50        ALL 0.9255219
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
469   0.8514703    0.9298416 0.9795861   0.8839307   0.8494017 16883
    Num-Mis
469   15246


ROC Analysis in
Index:   CFI
Classification:  CvM1
Estimation Method:   WLSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
470           CvM1   CFI     WLSMV         50          5 0.8363186
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
470   0.6984912    0.8401812 0.9636885    0.767061   0.7687943  4162
    Num-Mis
470    3851


ROC Analysis in
Index:   CFI
Classification:  CvM1
Estimation Method:   WLSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
471           CvM1   CFI     WLSMV         50         10 0.9717358
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
471   0.9303107     0.971447  0.971662   0.9139711   0.9162039  5746
    Num-Mis
471    5218


ROC Analysis in
Index:   CFI
Classification:  CvM1
Estimation Method:   WLSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
472           CvM1   CFI     WLSMV         50         30 0.9717436
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
472   0.9215099    0.9691112 0.9908881   0.9199278   0.9772265  6975
    Num-Mis
472    6177


ROC Analysis in
Index:   CFI
Classification:  CvM1
Estimation Method:   WLSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS     AUC
473           CvM1   CFI     WLSMV        100        ALL 0.99169
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
473   0.9808589    0.9883524 0.9716048   0.9654618   0.9645833 20592
    Num-Mis
473   18372


ROC Analysis in
Index:   CFI
Classification:  CvM1
Estimation Method:   WLSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
474           CvM1   CFI     WLSMV        100          5 0.9604013
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
474    0.904812    0.9598942 0.9685833   0.8897904   0.9042126  5756
    Num-Mis
474    5241


ROC Analysis in
Index:   CFI
Classification:  CvM1
Estimation Method:   WLSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
475           CvM1   CFI     WLSMV        100         10 0.9994351
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
475    0.998431    0.9993112 0.9726976   0.9868106   0.9935301  7084
    Num-Mis
475    6279


ROC Analysis in
Index:   CFI
Classification:  CvM1
Estimation Method:   WLSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in ci.auc.roc(roc, ...): ci.auc() of a ROC curve with AUC == 1 is
always 1-1 and can be misleading.

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS AUC partial-AUC
476           CvM1   CFI     WLSMV        100         30   1           1
    Smoothed-AUC Threshold Specificity Sensitivity Num-C Num-Mis
476           NA 0.9843855           1           1  7752    6852


ROC Analysis in
Index:   CFI
Classification:  CvM1
Estimation Method:   WLSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
477           CvM1   CFI     WLSMV        200        ALL 0.9998578
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
477    0.999605    0.9998255 0.9713042   0.9942979   0.9962529 23402
    Num-Mis
477   20741


ROC Analysis in
Index:   CFI
Classification:  CvM1
Estimation Method:   WLSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
478           CvM1   CFI     WLSMV        200          5 0.9987928
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
478   0.9966467    0.9986255 0.9713042   0.9832869    0.987824  7296
    Num-Mis
478    6545


ROC Analysis in
Index:   CFI
Classification:  CvM1
Estimation Method:   WLSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in ci.auc.roc(roc, ...): ci.auc() of a ROC curve with AUC == 1 is
always 1-1 and can be misleading.

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS AUC partial-AUC
479           CvM1   CFI     WLSMV        200         10   1           1
    Smoothed-AUC Threshold Specificity Sensitivity Num-C Num-Mis
479           NA 0.9769484           1           1  7940    7036


ROC Analysis in
Index:   CFI
Classification:  CvM1
Estimation Method:   WLSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in ci.auc.roc(roc, ...): ci.auc() of a ROC curve with AUC == 1 is
always 1-1 and can be misleading.

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS AUC partial-AUC
480           CvM1   CFI     WLSMV        200         30   1           1
    Smoothed-AUC Threshold Specificity Sensitivity Num-C Num-Mis
480           NA 0.9793391           1           1  8166    7160


ROC Analysis in
Index:   TLI
Classification:  CvM1
Estimation Method:   ALL
Level-2 Sample Size:     ALL
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
481           CvM1   TLI       ALL        ALL        ALL 0.8942411
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity  Num-C
481   0.7896568    0.9008143 0.9648862   0.8682327    0.883183 241960
    Num-Mis
481  223868


ROC Analysis in
Index:   TLI
Classification:  CvM1
Estimation Method:   ALL
Level-2 Sample Size:     ALL
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
482           CvM1   TLI       ALL        ALL          5 0.8456471
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
482   0.7693923    0.8464268 0.9616643   0.8682343   0.7659284 68483
    Num-Mis
482   63082


ROC Analysis in
Index:   TLI
Classification:  CvM1
Estimation Method:   ALL
Level-2 Sample Size:     ALL
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
483           CvM1   TLI       ALL        ALL         10 0.9265018
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
483   0.8554348    0.9220312 0.9611078   0.9019288   0.8890662 81868
    Num-Mis
483   75749


ROC Analysis in
Index:   TLI
Classification:  CvM1
Estimation Method:   ALL
Level-2 Sample Size:     ALL
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
484           CvM1   TLI       ALL        ALL         30 0.9077412
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
484   0.7447199    0.9115011 0.9716824   0.8356945   0.9757553 91609
    Num-Mis
484   85037


ROC Analysis in
Index:   TLI
Classification:  CvM1
Estimation Method:   ALL
Level-2 Sample Size:     30
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
485           CvM1   TLI       ALL         30        ALL 0.7387235
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
485   0.5719083    0.7701203  0.955158   0.6915349   0.7607267 47053
    Num-Mis
485   43767


ROC Analysis in
Index:   TLI
Classification:  CvM1
Estimation Method:   ALL
Level-2 Sample Size:     30
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
486           CvM1   TLI       ALL         30          5 0.6614421
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
486   0.5605278    0.6656346 0.9057027    0.619389   0.6327199 11584
    Num-Mis
486   10825


ROC Analysis in
Index:   TLI
Classification:  CvM1
Estimation Method:   ALL
Level-2 Sample Size:     30
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
487           CvM1   TLI       ALL         30         10 0.7732855
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
487   0.6118027    0.7867282 0.9424879   0.6747349   0.7688385 15551
    Num-Mis
487   14595


ROC Analysis in
Index:   TLI
Classification:  CvM1
Estimation Method:   ALL
Level-2 Sample Size:     30
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
488           CvM1   TLI       ALL         30         30 0.7825584
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
488   0.5584719    0.7416804 0.9577443   0.6268178   0.9685406 19918
    Num-Mis
488   18347


ROC Analysis in
Index:   TLI
Classification:  CvM1
Estimation Method:   ALL
Level-2 Sample Size:     50
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
489           CvM1   TLI       ALL         50        ALL 0.8506211
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
489   0.7024575    0.8667261 0.9643914   0.8082932     0.82597 56136
    Num-Mis
489   52168


ROC Analysis in
Index:   TLI
Classification:  CvM1
Estimation Method:   ALL
Level-2 Sample Size:     50
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
490           CvM1   TLI       ALL         50          5 0.7700342
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
490   0.6364385    0.7753913 0.9528779   0.7629468   0.6541592 14818
    Num-Mis
490   13894


ROC Analysis in
Index:   TLI
Classification:  CvM1
Estimation Method:   ALL
Level-2 Sample Size:     50
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
491           CvM1   TLI       ALL         50         10 0.9051174
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
491   0.7953663    0.9032339 0.9510101    0.816683   0.8872111 18954
    Num-Mis
491   17559


ROC Analysis in
Index:   TLI
Classification:  CvM1
Estimation Method:   ALL
Level-2 Sample Size:     50
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
492           CvM1   TLI       ALL         50         30 0.8775863
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
492   0.6626137    0.8735622 0.9763962   0.7842054   0.9811611 22364
    Num-Mis
492   20715


ROC Analysis in
Index:   TLI
Classification:  CvM1
Estimation Method:   ALL
Level-2 Sample Size:     100
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS      AUC
493           CvM1   TLI       ALL        100        ALL 0.950266
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
493    0.878201    0.9503303 0.9648851   0.9093476    0.933474 65882
    Num-Mis
493   60844


ROC Analysis in
Index:   TLI
Classification:  CvM1
Estimation Method:   ALL
Level-2 Sample Size:     100
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
494           CvM1   TLI       ALL        100          5 0.9259115
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
494   0.8418075     0.923349  0.960445   0.8665186   0.8466163 18983
    Num-Mis
494   17448


ROC Analysis in
Index:   TLI
Classification:  CvM1
Estimation Method:   ALL
Level-2 Sample Size:     100
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
495           CvM1   TLI       ALL        100         10 0.9822249
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
495    0.960669    0.9730014 0.9651552   0.9558824   0.9593734 22711
    Num-Mis
495   20843


ROC Analysis in
Index:   TLI
Classification:  CvM1
Estimation Method:   ALL
Level-2 Sample Size:     100
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
496           CvM1   TLI       ALL        100         30 0.9375299
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
496   0.8265154    0.9297481 0.9783547   0.8939282   0.9950024 24188
    Num-Mis
496   22553


ROC Analysis in
Index:   TLI
Classification:  CvM1
Estimation Method:   ALL
Level-2 Sample Size:     200
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
497           CvM1   TLI       ALL        200        ALL 0.9970231
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
497   0.9925265    0.9937718 0.9699639   0.9823409   0.9791535 72889
    Num-Mis
497   67089


ROC Analysis in
Index:   TLI
Classification:  CvM1
Estimation Method:   ALL
Level-2 Sample Size:     200
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
498           CvM1   TLI       ALL        200          5 0.9934448
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
498   0.9845821    0.9892739 0.9650439   0.9696128   0.9665072 23098
    Num-Mis
498   20915


ROC Analysis in
Index:   TLI
Classification:  CvM1
Estimation Method:   ALL
Level-2 Sample Size:     200
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
499           CvM1   TLI       ALL        200         10 0.9993397
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
499   0.9983872    0.9987113 0.9640996   0.9920528   0.9901972 24652
    Num-Mis
499   22752


ROC Analysis in
Index:   TLI
Classification:  CvM1
Estimation Method:   ALL
Level-2 Sample Size:     200
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
500           CvM1   TLI       ALL        200         30 0.9981706
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
500   0.9949328    0.9978544 0.9847592   0.9907828   0.9916752 25139
    Num-Mis
500   23422


ROC Analysis in
Index:   TLI
Classification:  CvM1
Estimation Method:   MLR
Level-2 Sample Size:     ALL
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
501           CvM1   TLI       MLR        ALL        ALL 0.9322555
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
501   0.9220184    0.9188767 0.9476982   0.9483419   0.8731095 89822
    Num-Mis
501   85647


ROC Analysis in
Index:   TLI
Classification:  CvM1
Estimation Method:   MLR
Level-2 Sample Size:     ALL
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
502           CvM1   TLI       MLR        ALL          5 0.8319262
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
502   0.7993272    0.8315975  0.955348   0.9069904   0.7043041 26722
    Num-Mis
502   24572


ROC Analysis in
Index:   TLI
Classification:  CvM1
Estimation Method:   MLR
Level-2 Sample Size:     ALL
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
503           CvM1   TLI       MLR        ALL         10 0.9466965
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
503   0.9288553    0.9401712 0.9471578   0.9549539   0.8660059 30195
    Num-Mis
503   28785


ROC Analysis in
Index:   TLI
Classification:  CvM1
Estimation Method:   MLR
Level-2 Sample Size:     ALL
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
504           CvM1   TLI       MLR        ALL         30 0.9997934
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
504   0.9994262    0.9997744 0.9470943   0.9947917   0.9932623 32905
    Num-Mis
504   32290


ROC Analysis in
Index:   TLI
Classification:  CvM1
Estimation Method:   MLR
Level-2 Sample Size:     30
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
505           CvM1   TLI       MLR         30        ALL 0.8290348
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
505   0.7978814    0.8215602 0.9322666   0.9180786   0.6840349 18778
    Num-Mis
505   17854


ROC Analysis in
Index:   TLI
Classification:  CvM1
Estimation Method:   MLR
Level-2 Sample Size:     30
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
506           CvM1   TLI       MLR         30          5 0.6986979
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
506   0.5974753     0.698296 0.8667637   0.7215373   0.5816918  5086
    Num-Mis
506    4739


ROC Analysis in
Index:   TLI
Classification:  CvM1
Estimation Method:   MLR
Level-2 Sample Size:     30
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
507           CvM1   TLI       MLR         30         10 0.8598081
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
507   0.7513297    0.8578347 0.9025455   0.7511442   0.8106134  6138
    Num-Mis
507    5832


ROC Analysis in
Index:   TLI
Classification:  CvM1
Estimation Method:   MLR
Level-2 Sample Size:     30
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
508           CvM1   TLI       MLR         30         30 0.9980708
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
508    0.994641    0.9979653 0.9387013   0.9728164   0.9872763  7554
    Num-Mis
508    7283


ROC Analysis in
Index:   TLI
Classification:  CvM1
Estimation Method:   MLR
Level-2 Sample Size:     50
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
509           CvM1   TLI       MLR         50        ALL 0.9262014
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
509   0.8822932    0.9135346 0.9454186   0.9205607   0.8422235 21335
    Num-Mis
509   20417


ROC Analysis in
Index:   TLI
Classification:  CvM1
Estimation Method:   MLR
Level-2 Sample Size:     50
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
510           CvM1   TLI       MLR         50          5 0.7946052
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
510   0.6692243    0.7953619 0.9148342   0.6814106   0.7623031  6083
    Num-Mis
510    5724


ROC Analysis in
Index:   TLI
Classification:  CvM1
Estimation Method:   MLR
Level-2 Sample Size:     50
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
511           CvM1   TLI       MLR         50         10 0.9605345
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
511   0.9103955    0.9594576 0.9436506   0.9073244   0.8864684  7170
    Num-Mis
511    6790


ROC Analysis in
Index:   TLI
Classification:  CvM1
Estimation Method:   MLR
Level-2 Sample Size:     50
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
512           CvM1   TLI       MLR         50         30 0.9999925
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
512   0.9999793    0.9999754 0.9592762   0.9980206   0.9996303  8082
    Num-Mis
512    7903


ROC Analysis in
Index:   TLI
Classification:  CvM1
Estimation Method:   MLR
Level-2 Sample Size:     100
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS      AUC
513           CvM1   TLI       MLR        100        ALL 0.990454
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
513   0.9773952    0.9844106 0.9541755   0.9587076   0.9599451 24013
    Num-Mis
513   22901


ROC Analysis in
Index:   TLI
Classification:  CvM1
Estimation Method:   MLR
Level-2 Sample Size:     100
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS      AUC
514           CvM1   TLI       MLR        100          5 0.945666
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
514   0.8769856    0.9456985 0.9505134   0.8474388    0.902772  7258
    Num-Mis
514    6637


ROC Analysis in
Index:   TLI
Classification:  CvM1
Estimation Method:   MLR
Level-2 Sample Size:     100
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
515           CvM1   TLI       MLR        100         10 0.9993947
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
515   0.9983187    0.9994504 0.9591101   0.9880648   0.9905797  8237
    Num-Mis
515    7823


ROC Analysis in
Index:   TLI
Classification:  CvM1
Estimation Method:   MLR
Level-2 Sample Size:     100
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in ci.auc.roc(roc, ...): ci.auc() of a ROC curve with AUC == 1 is
always 1-1 and can be misleading.

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS AUC partial-AUC
516           CvM1   TLI       MLR        100         30   1           1
    Smoothed-AUC Threshold Specificity Sensitivity Num-C Num-Mis
516           NA 0.9609112           1           1  8518    8441


ROC Analysis in
Index:   TLI
Classification:  CvM1
Estimation Method:   MLR
Level-2 Sample Size:     200
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
517           CvM1   TLI       MLR        200        ALL 0.9997541
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
517    0.999317    0.9997498 0.9623108   0.9944324   0.9953407 25696
    Num-Mis
517   24475


ROC Analysis in
Index:   TLI
Classification:  CvM1
Estimation Method:   MLR
Level-2 Sample Size:     200
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
518           CvM1   TLI       MLR        200          5 0.9980997
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
518   0.9947215    0.9978898 0.9649158   0.9845917   0.9808664  8295
    Num-Mis
518    7472


ROC Analysis in
Index:   TLI
Classification:  CvM1
Estimation Method:   MLR
Level-2 Sample Size:     200
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in ci.auc.roc(roc, ...): ci.auc() of a ROC curve with AUC == 1 is
always 1-1 and can be misleading.

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS AUC partial-AUC
519           CvM1   TLI       MLR        200         10   1           1
    Smoothed-AUC Threshold Specificity Sensitivity Num-C Num-Mis
519           NA 0.9618642           1           1  8650    8340


ROC Analysis in
Index:   TLI
Classification:  CvM1
Estimation Method:   MLR
Level-2 Sample Size:     200
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in ci.auc.roc(roc, ...): ci.auc() of a ROC curve with AUC == 1 is
always 1-1 and can be misleading.

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS AUC partial-AUC
520           CvM1   TLI       MLR        200         30   1           1
    Smoothed-AUC Threshold Specificity Sensitivity Num-C Num-Mis
520           NA 0.9631283           1           1  8751    8663


ROC Analysis in
Index:   TLI
Classification:  CvM1
Estimation Method:   ULSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
521           CvM1   TLI     ULSMV        ALL        ALL 0.8220301
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
521   0.6199448    0.8714382 0.9728829    0.750117   0.8742182 77713
    Num-Mis
521   71584


ROC Analysis in
Index:   TLI
Classification:  CvM1
Estimation Method:   ULSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
522           CvM1   TLI     ULSMV        ALL          5 0.8186528
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
522   0.6965808    0.8352935 0.9601495   0.7888797   0.7933752 21448
    Num-Mis
522   19987


ROC Analysis in
Index:   TLI
Classification:  CvM1
Estimation Method:   ULSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
523           CvM1   TLI     ULSMV        ALL         10 0.8728878
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
523    0.723139    0.8924318 0.9642983   0.8095634   0.8881149 26416
    Num-Mis
523   24300


ROC Analysis in
Index:   TLI
Classification:  CvM1
Estimation Method:   ULSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
524           CvM1   TLI     ULSMV        ALL         30 0.7922338
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
524   0.5723193    0.7927906 0.9846864   0.6430504   0.9753791 29849
    Num-Mis
524   27297


ROC Analysis in
Index:   TLI
Classification:  CvM1
Estimation Method:   ULSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
525           CvM1   TLI     ULSMV         30        ALL 0.6489314
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
525   0.5289806    0.7513845 0.9751906   0.4817073   0.8299634 14727
    Num-Mis
525   13635


ROC Analysis in
Index:   TLI
Classification:  CvM1
Estimation Method:   ULSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
526           CvM1   TLI     ULSMV         30          5 0.6266319
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
526   0.5359382    0.6490378 0.9599345   0.6060606   0.6218487  3399
    Num-Mis
526    3200


ROC Analysis in
Index:   TLI
Classification:  CvM1
Estimation Method:   ULSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
527           CvM1   TLI     ULSMV         30         10 0.6906809
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
527    0.542908    0.7603944 0.9730148   0.5593816   0.7949913  4926
    Num-Mis
527    4632


ROC Analysis in
Index:   TLI
Classification:  CvM1
Estimation Method:   ULSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
528           CvM1   TLI     ULSMV         30         30 0.6310332
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
528   0.5186999    0.5154008 0.9849074   0.3366924   0.9755031  6402
    Num-Mis
528    5803


ROC Analysis in
Index:   TLI
Classification:  CvM1
Estimation Method:   ULSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
529           CvM1   TLI     ULSMV         50        ALL 0.7346183
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
529   0.5619858    0.7987331 0.9659009   0.6129881   0.8340151 17918
    Num-Mis
529   16505


ROC Analysis in
Index:   TLI
Classification:  CvM1
Estimation Method:   ULSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS      AUC
530           CvM1   TLI     ULSMV         50          5 0.712862
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
530   0.5859783    0.7274272 0.9542352   0.6674124   0.6695925  4573
    Num-Mis
530    4319


ROC Analysis in
Index:   TLI
Classification:  CvM1
Estimation Method:   ULSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
531           CvM1   TLI     ULSMV         50         10 0.8130939
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
531   0.6443693    0.8366097 0.9646625   0.7480122    0.803016  6038
    Num-Mis
531    5551


ROC Analysis in
Index:   TLI
Classification:  CvM1
Estimation Method:   ULSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
532           CvM1   TLI     ULSMV         50         30 0.7008154
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
532   0.5346987    0.6743767 0.9856083   0.4819522   0.9765063  7307
    Num-Mis
532    6635


ROC Analysis in
Index:   TLI
Classification:  CvM1
Estimation Method:   ULSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
533           CvM1   TLI     ULSMV        100        ALL 0.8753835
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
533   0.7110234    0.9008906 0.9648084   0.7812124   0.9051667 21277
    Num-Mis
533   19571


ROC Analysis in
Index:   TLI
Classification:  CvM1
Estimation Method:   ULSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
534           CvM1   TLI     ULSMV        100          5 0.8798768
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
534   0.7679274    0.8835196 0.9598032   0.8091831   0.8131021  5969
    Num-Mis
534    5570


ROC Analysis in
Index:   TLI
Classification:  CvM1
Estimation Method:   ULSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
535           CvM1   TLI     ULSMV        100         10 0.9459836
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
535   0.8847264    0.9434985  0.964242   0.8852792    0.908362  7390
    Num-Mis
535    6741


ROC Analysis in
Index:   TLI
Classification:  CvM1
Estimation Method:   ULSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS      AUC
536           CvM1   TLI     ULSMV        100         30 0.823498
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
536   0.5958182    0.7893048 0.9803435   0.6741728   0.9816514  7918
    Num-Mis
536    7260


ROC Analysis in
Index:   TLI
Classification:  CvM1
Estimation Method:   ULSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
537           CvM1   TLI     ULSMV        200        ALL 0.9889358
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
537    0.973161    0.9862316 0.9703579   0.9538745   0.9555925 23791
    Num-Mis
537   21873


ROC Analysis in
Index:   TLI
Classification:  CvM1
Estimation Method:   ULSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
538           CvM1   TLI     ULSMV        200          5 0.9825903
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
538   0.9606567    0.9794652 0.9617727   0.9335647   0.9440213  7507
    Num-Mis
538    6898


ROC Analysis in
Index:   TLI
Classification:  CvM1
Estimation Method:   ULSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
539           CvM1   TLI     ULSMV        200         10 0.9964108
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
539   0.9911202    0.9959017 0.9640996   0.9766804   0.9707623  8062
    Num-Mis
539    7376


ROC Analysis in
Index:   TLI
Classification:  CvM1
Estimation Method:   ULSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
540           CvM1   TLI     ULSMV        200         30 0.9931552
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
540   0.9812515    0.9929759 0.9847592   0.9717391   0.9750257  8222
    Num-Mis
540    7599


ROC Analysis in
Index:   TLI
Classification:  CvM1
Estimation Method:   WLSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
541           CvM1   TLI     WLSMV        ALL        ALL 0.9448689
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
541   0.8770901    0.9466312  0.972419   0.9059147   0.9228518 74425
    Num-Mis
541   66637


ROC Analysis in
Index:   TLI
Classification:  CvM1
Estimation Method:   WLSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
542           CvM1   TLI     WLSMV        ALL          5 0.9096199
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
542   0.8307083    0.9062115  0.963514   0.8816193   0.8497358 20313
    Num-Mis
542   18523


ROC Analysis in
Index:   TLI
Classification:  CvM1
Estimation Method:   WLSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
543           CvM1   TLI     WLSMV        ALL         10 0.9766076
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
543   0.9431138     0.970323 0.9676405   0.9342594   0.9532157 25257
    Num-Mis
543   22664


ROC Analysis in
Index:   TLI
Classification:  CvM1
Estimation Method:   WLSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS     AUC
544           CvM1   TLI     WLSMV        ALL         30 0.95035
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
544   0.8620834    0.9385603 0.9850673   0.8976669   0.9920409 28855
    Num-Mis
544   25450


ROC Analysis in
Index:   TLI
Classification:  CvM1
Estimation Method:   WLSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
545           CvM1   TLI     WLSMV         30        ALL 0.8035081
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
545   0.6315897    0.8442203 0.9791415   0.7320475   0.8114224 13548
    Num-Mis
545   12278


ROC Analysis in
Index:   TLI
Classification:  CvM1
Estimation Method:   WLSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
546           CvM1   TLI     WLSMV         30          5 0.7340084
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
546   0.6001856    0.7381871 0.9161567   0.5125149   0.8392008  3099
    Num-Mis
546    2886


ROC Analysis in
Index:   TLI
Classification:  CvM1
Estimation Method:   WLSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS      AUC
547           CvM1   TLI     WLSMV         30         10 0.885788
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
547   0.7646601    0.8936167 0.9772796   0.8354756   0.8038058  4487
    Num-Mis
547    4131


ROC Analysis in
Index:   TLI
Classification:  CvM1
Estimation Method:   WLSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
548           CvM1   TLI     WLSMV         30         30 0.8006895
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
548   0.5838078    0.7227762   0.98374   0.6121701   0.9786925  5962
    Num-Mis
548    5261


ROC Analysis in
Index:   TLI
Classification:  CvM1
Estimation Method:   WLSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
549           CvM1   TLI     WLSMV         50        ALL 0.9255219
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
549   0.8514703    0.9298396 0.9755033   0.8839307   0.8494017 16883
    Num-Mis
549   15246


ROC Analysis in
Index:   TLI
Classification:  CvM1
Estimation Method:   WLSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
550           CvM1   TLI     WLSMV         50          5 0.8363186
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
550   0.6984912    0.8401812 0.9564262    0.767061   0.7687943  4162
    Num-Mis
550    3851


ROC Analysis in
Index:   TLI
Classification:  CvM1
Estimation Method:   WLSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
551           CvM1   TLI     WLSMV         50         10 0.9717358
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
551   0.9303107     0.971447 0.9659944   0.9139711   0.9162039  5746
    Num-Mis
551    5218


ROC Analysis in
Index:   TLI
Classification:  CvM1
Estimation Method:   WLSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
552           CvM1   TLI     WLSMV         50         30 0.9717436
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
552   0.9215099    0.9691112 0.9890657   0.9199278   0.9772265  6975
    Num-Mis
552    6177


ROC Analysis in
Index:   TLI
Classification:  CvM1
Estimation Method:   WLSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS     AUC
553           CvM1   TLI     WLSMV        100        ALL 0.99169
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
553   0.9808589    0.9883539 0.9659258   0.9654618   0.9645833 20592
    Num-Mis
553   18372


ROC Analysis in
Index:   TLI
Classification:  CvM1
Estimation Method:   WLSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
554           CvM1   TLI     WLSMV        100          5 0.9604013
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
554    0.904812    0.9598942    0.9623   0.8897904   0.9042126  5756
    Num-Mis
554    5241


ROC Analysis in
Index:   TLI
Classification:  CvM1
Estimation Method:   WLSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
555           CvM1   TLI     WLSMV        100         10 0.9994351
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
555    0.998431    0.9993112 0.9672371   0.9868106   0.9935301  7084
    Num-Mis
555    6279


ROC Analysis in
Index:   TLI
Classification:  CvM1
Estimation Method:   WLSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in ci.auc.roc(roc, ...): ci.auc() of a ROC curve with AUC == 1 is
always 1-1 and can be misleading.

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS AUC partial-AUC
556           CvM1   TLI     WLSMV        100         30   1           1
    Smoothed-AUC Threshold Specificity Sensitivity Num-C Num-Mis
556           NA 0.9812626           1           1  7752    6852


ROC Analysis in
Index:   TLI
Classification:  CvM1
Estimation Method:   WLSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
557           CvM1   TLI     WLSMV        200        ALL 0.9998578
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
557    0.999605    0.9998255 0.9655651   0.9942979   0.9962529 23402
    Num-Mis
557   20741


ROC Analysis in
Index:   TLI
Classification:  CvM1
Estimation Method:   WLSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
558           CvM1   TLI     WLSMV        200          5 0.9987928
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
558   0.9966467    0.9986255 0.9655651   0.9832869    0.987824  7296
    Num-Mis
558    6545


ROC Analysis in
Index:   TLI
Classification:  CvM1
Estimation Method:   WLSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in ci.auc.roc(roc, ...): ci.auc() of a ROC curve with AUC == 1 is
always 1-1 and can be misleading.

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS AUC partial-AUC
559           CvM1   TLI     WLSMV        200         10   1           1
    Smoothed-AUC Threshold Specificity Sensitivity Num-C Num-Mis
559           NA 0.9723381           1           1  7940    7036


ROC Analysis in
Index:   TLI
Classification:  CvM1
Estimation Method:   WLSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in ci.auc.roc(roc, ...): ci.auc() of a ROC curve with AUC == 1 is
always 1-1 and can be misleading.

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS AUC partial-AUC
560           CvM1   TLI     WLSMV        200         30   1           1
    Smoothed-AUC Threshold Specificity Sensitivity Num-C Num-Mis
560           NA 0.9752069           1           1  8166    7160


ROC Analysis in
Index:   RMSEA
Classification:  CvM1
Estimation Method:   ALL
Level-2 Sample Size:     ALL
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1

Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
561           CvM1 RMSEA       ALL        ALL        ALL 0.8776767
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity  Num-C
561   0.7534801    0.8844573 0.0169548   0.8114027   0.8589859 241960
    Num-Mis
561  223868


ROC Analysis in
Index:   RMSEA
Classification:  CvM1
Estimation Method:   ALL
Level-2 Sample Size:     ALL
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
562           CvM1 RMSEA       ALL        ALL          5 0.8331061
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
562    0.749103    0.8333871 0.01961916   0.8242569    0.767422 68483
    Num-Mis
562   63082


ROC Analysis in
Index:   RMSEA
Classification:  CvM1
Estimation Method:   ALL
Level-2 Sample Size:     ALL
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
563           CvM1 RMSEA       ALL        ALL         10 0.9102621
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
563   0.8196951    0.9074572 0.01716517   0.8547496   0.8610934 81868
    Num-Mis
563   75749


ROC Analysis in
Index:   RMSEA
Classification:  CvM1
Estimation Method:   ALL
Level-2 Sample Size:     ALL
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
564           CvM1 RMSEA       ALL        ALL         30 0.8910933
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
564   0.7136866    0.8932595 0.01381252   0.7712307   0.9429056 91609
    Num-Mis
564   85037


ROC Analysis in
Index:   RMSEA
Classification:  CvM1
Estimation Method:   ALL
Level-2 Sample Size:     30
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
565           CvM1 RMSEA       ALL         30        ALL 0.7199243
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
565   0.5719083     0.744975 0.02250199   0.6318823   0.7684152 47053
    Num-Mis
565   43767


ROC Analysis in
Index:   RMSEA
Classification:  CvM1
Estimation Method:   ALL
Level-2 Sample Size:     30
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
566           CvM1 RMSEA       ALL         30          5 0.6323019
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
566   0.5587825    0.6350303 0.02290212   0.7032419   0.5006302 11584
    Num-Mis
566   10825


ROC Analysis in
Index:   RMSEA
Classification:  CvM1
Estimation Method:   ALL
Level-2 Sample Size:     30
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
567           CvM1 RMSEA       ALL         30         10 0.7395346
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
567   0.6047076    0.7450225 0.01975351   0.6945597    0.682153 15551
    Num-Mis
567   14595


ROC Analysis in
Index:   RMSEA
Classification:  CvM1
Estimation Method:   ALL
Level-2 Sample Size:     30
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
568           CvM1 RMSEA       ALL         30         30 0.7653955
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
568   0.5584719    0.7137023 0.02291648    0.570916   0.9628605 19918
    Num-Mis
568   18347


ROC Analysis in
Index:   RMSEA
Classification:  CvM1
Estimation Method:   ALL
Level-2 Sample Size:     50
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
569           CvM1 RMSEA       ALL         50        ALL 0.8317013
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
569   0.6749309    0.8447561 0.01989944   0.7179404   0.8386446 56136
    Num-Mis
569   52168


ROC Analysis in
Index:   RMSEA
Classification:  CvM1
Estimation Method:   ALL
Level-2 Sample Size:     50
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
570           CvM1 RMSEA       ALL         50          5 0.7455559
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
570   0.6252503    0.7494548 0.02297401   0.6953562   0.6809609 14818
    Num-Mis
570   13894


ROC Analysis in
Index:   RMSEA
Classification:  CvM1
Estimation Method:   ALL
Level-2 Sample Size:     50
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
571           CvM1 RMSEA       ALL         50         10 0.8816433
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
571   0.7457119    0.8831442 0.01994737   0.7947007   0.8275809 18954
    Num-Mis
571   17559


ROC Analysis in
Index:   RMSEA
Classification:  CvM1
Estimation Method:   ALL
Level-2 Sample Size:     50
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
572           CvM1 RMSEA       ALL         50         30 0.8602482
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
572    0.649907    0.8349762 0.01458665     0.69987   0.9593746 22364
    Num-Mis
572   20715


ROC Analysis in
Index:   RMSEA
Classification:  CvM1
Estimation Method:   ALL
Level-2 Sample Size:     100
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
573           CvM1 RMSEA       ALL        100        ALL 0.9357911
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
573   0.8376339    0.9391419 0.01689071   0.8550127   0.9106978 65882
    Num-Mis
573   60844


ROC Analysis in
Index:   RMSEA
Classification:  CvM1
Estimation Method:   ALL
Level-2 Sample Size:     100
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
574           CvM1 RMSEA       ALL        100          5 0.9134745
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
574    0.812646    0.9146538 0.02109814   0.8051292   0.8695853 18983
    Num-Mis
574   17448


ROC Analysis in
Index:   RMSEA
Classification:  CvM1
Estimation Method:   ALL
Level-2 Sample Size:     100
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
575           CvM1 RMSEA       ALL        100         10 0.9721312
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
575   0.9228384    0.9731054 0.01701878   0.8977273    0.949567 22711
    Num-Mis
575   20843


ROC Analysis in
Index:   RMSEA
Classification:  CvM1
Estimation Method:   ALL
Level-2 Sample Size:     100
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
576           CvM1 RMSEA       ALL        100         30 0.9283979
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
576   0.8011053    0.9064362 0.01173214   0.8501994   0.9907187 24188
    Num-Mis
576   22553


ROC Analysis in
Index:   RMSEA
Classification:  CvM1
Estimation Method:   ALL
Level-2 Sample Size:     200
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
577           CvM1 RMSEA       ALL        200        ALL 0.9839081
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
577   0.9553906    0.9862981 0.01456073   0.9272587   0.9595332 72889
    Num-Mis
577   67089


ROC Analysis in
Index:   RMSEA
Classification:  CvM1
Estimation Method:   ALL
Level-2 Sample Size:     200
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
578           CvM1 RMSEA       ALL        200          5 0.9888932
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
578   0.9694291    0.9900941 0.01772568   0.9383574   0.9581969 23098
    Num-Mis
578   20915


ROC Analysis in
Index:   RMSEA
Classification:  CvM1
Estimation Method:   ALL
Level-2 Sample Size:     200
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
579           CvM1 RMSEA       ALL        200         10 0.9972609
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
579   0.9923915    0.9977032 0.01343915   0.9700105   0.9756098 24652
    Num-Mis
579   22752


ROC Analysis in
Index:   RMSEA
Classification:  CvM1
Estimation Method:   ALL
Level-2 Sample Size:     200
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
580           CvM1 RMSEA       ALL        200         30 0.9901609
    partial-AUC Smoothed-AUC   Threshold Specificity Sensitivity Num-C
580   0.9726691    0.9907097 0.007265142    0.929949    0.976166 25139
    Num-Mis
580   23422


ROC Analysis in
Index:   RMSEA
Classification:  CvM1
Estimation Method:   MLR
Level-2 Sample Size:     ALL
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
581           CvM1 RMSEA       MLR        ALL        ALL 0.9175217
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
581   0.9134692    0.9054988 0.02484184   0.9500832   0.8559531 89822
    Num-Mis
581   85647


ROC Analysis in
Index:   RMSEA
Classification:  CvM1
Estimation Method:   MLR
Level-2 Sample Size:     ALL
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
582           CvM1 RMSEA       MLR        ALL          5 0.8086688
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
582   0.7884557     0.811621 0.02382187   0.8953642   0.6954723 26722
    Num-Mis
582   24572


ROC Analysis in
Index:   RMSEA
Classification:  CvM1
Estimation Method:   MLR
Level-2 Sample Size:     ALL
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
583           CvM1 RMSEA       MLR        ALL         10 0.9321771
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
583   0.9172251    0.9277269 0.02483705   0.9549539   0.8429138 30195
    Num-Mis
583   28785


ROC Analysis in
Index:   RMSEA
Classification:  CvM1
Estimation Method:   MLR
Level-2 Sample Size:     ALL
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
584           CvM1 RMSEA       MLR        ALL         30 0.9994934
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
584   0.9987256    0.9993864 0.02512648   0.9938272    0.988983 32905
    Num-Mis
584   32290


ROC Analysis in
Index:   RMSEA
Classification:  CvM1
Estimation Method:   MLR
Level-2 Sample Size:     30
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
585           CvM1 RMSEA       MLR         30        ALL 0.7992365
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
585   0.7792552    0.7974507 0.02963195   0.9028114   0.6629071 18778
    Num-Mis
585   17854


ROC Analysis in
Index:   RMSEA
Classification:  CvM1
Estimation Method:   MLR
Level-2 Sample Size:     30
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
586           CvM1 RMSEA       MLR         30          5 0.6781101
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
586   0.5853558    0.6802025 0.04819875   0.6519217   0.6048667  5086
    Num-Mis
586    4739


ROC Analysis in
Index:   RMSEA
Classification:  CvM1
Estimation Method:   MLR
Level-2 Sample Size:     30
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
587           CvM1 RMSEA       MLR         30         10 0.8336335
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
587   0.7232312     0.836271 0.03352234    0.798627   0.7039922  6138
    Num-Mis
587    5832


ROC Analysis in
Index:   RMSEA
Classification:  CvM1
Estimation Method:   MLR
Level-2 Sample Size:     30
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
588           CvM1 RMSEA       MLR         30         30 0.9965964
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
588    0.990713     0.996369 0.02707205   0.9674688   0.9809145  7554
    Num-Mis
588    7283


ROC Analysis in
Index:   RMSEA
Classification:  CvM1
Estimation Method:   MLR
Level-2 Sample Size:     50
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS      AUC
589           CvM1 RMSEA       MLR         50        ALL 0.908262
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
589   0.8686897    0.8965276 0.02616199   0.9118595    0.822712 21335
    Num-Mis
589   20417


ROC Analysis in
Index:   RMSEA
Classification:  CvM1
Estimation Method:   MLR
Level-2 Sample Size:     50
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
590           CvM1 RMSEA       MLR         50          5 0.7709793
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
590   0.6550652      0.77167 0.0343688   0.6569038   0.7524606  6083
    Num-Mis
590    5724


ROC Analysis in
Index:   RMSEA
Classification:  CvM1
Estimation Method:   MLR
Level-2 Sample Size:     50
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
591           CvM1 RMSEA       MLR         50         10 0.9493677
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
591   0.8885922    0.9494476 0.02710695   0.8769307   0.8772518  7170
    Num-Mis
591    6790


ROC Analysis in
Index:   RMSEA
Classification:  CvM1
Estimation Method:   MLR
Level-2 Sample Size:     50
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
592           CvM1 RMSEA       MLR         50         30 0.9999753
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
592   0.9999313    0.9998976 0.02196645   0.9972288   0.9996303  8082
    Num-Mis
592    7903


ROC Analysis in
Index:   RMSEA
Classification:  CvM1
Estimation Method:   MLR
Level-2 Sample Size:     100
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
593           CvM1 RMSEA       MLR        100        ALL 0.9880868
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
593   0.9733345    0.9821241 0.02387894   0.9530571   0.9580734 24013
    Num-Mis
593   22901


ROC Analysis in
Index:   RMSEA
Classification:  CvM1
Estimation Method:   MLR
Level-2 Sample Size:     100
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
594           CvM1 RMSEA       MLR        100          5 0.9369407
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
594   0.8607828    0.9378649 0.02405459   0.8569042   0.8713281  7258
    Num-Mis
594    6637


ROC Analysis in
Index:   RMSEA
Classification:  CvM1
Estimation Method:   MLR
Level-2 Sample Size:     100
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
595           CvM1 RMSEA       MLR        100         10 0.9991256
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
595    0.997571    0.9991547 0.02226483   0.9816709    0.990942  8237
    Num-Mis
595    7823


ROC Analysis in
Index:   RMSEA
Classification:  CvM1
Estimation Method:   MLR
Level-2 Sample Size:     100
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in ci.auc.roc(roc, ...): ci.auc() of a ROC curve with AUC == 1 is
always 1-1 and can be misleading.

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS AUC partial-AUC
596           CvM1 RMSEA       MLR        100         30   1           1
    Smoothed-AUC Threshold Specificity Sensitivity Num-C Num-Mis
596           NA 0.0199181           1           1  8518    8441


ROC Analysis in
Index:   RMSEA
Classification:  CvM1
Estimation Method:   MLR
Level-2 Sample Size:     200
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS      AUC
597           CvM1 RMSEA       MLR        200        ALL 0.999729
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
597   0.9992471    0.9997574 0.02017256   0.9959261   0.9933605 25696
    Num-Mis
597   24475


ROC Analysis in
Index:   RMSEA
Classification:  CvM1
Estimation Method:   MLR
Level-2 Sample Size:     200
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
598           CvM1 RMSEA       MLR        200          5 0.9978201
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
598   0.9939448    0.9978241 0.02017256   0.9845917   0.9794224  8295
    Num-Mis
598    7472


ROC Analysis in
Index:   RMSEA
Classification:  CvM1
Estimation Method:   MLR
Level-2 Sample Size:     200
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in ci.auc.roc(roc, ...): ci.auc() of a ROC curve with AUC == 1 is
always 1-1 and can be misleading.

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS AUC partial-AUC
599           CvM1 RMSEA       MLR        200         10   1           1
    Smoothed-AUC  Threshold Specificity Sensitivity Num-C Num-Mis
599           NA 0.02013616           1           1  8650    8340


ROC Analysis in
Index:   RMSEA
Classification:  CvM1
Estimation Method:   MLR
Level-2 Sample Size:     200
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in ci.auc.roc(roc, ...): ci.auc() of a ROC curve with AUC == 1 is
always 1-1 and can be misleading.

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS AUC partial-AUC
600           CvM1 RMSEA       MLR        200         30   1           1
    Smoothed-AUC Threshold Specificity Sensitivity Num-C Num-Mis
600           NA 0.0180532           1           1  8751    8663


ROC Analysis in
Index:   RMSEA
Classification:  CvM1
Estimation Method:   ULSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
601           CvM1 RMSEA     ULSMV        ALL        ALL 0.8091759
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
601   0.6190646    0.8409622 0.01193518   0.6768835   0.8565455 77713
    Num-Mis
601   71584


ROC Analysis in
Index:   RMSEA
Classification:  CvM1
Estimation Method:   ULSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS      AUC
602           CvM1 RMSEA     ULSMV        ALL          5 0.826962
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
602   0.6850525    0.8409997 0.01538462   0.7704564   0.7776145 21448
    Num-Mis
602   19987


ROC Analysis in
Index:   RMSEA
Classification:  CvM1
Estimation Method:   ULSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
603           CvM1 RMSEA     ULSMV        ALL         10 0.8669512
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
603   0.7017283    0.8783837 0.01306755   0.7589744   0.8704319 26416
    Num-Mis
603   24300


ROC Analysis in
Index:   RMSEA
Classification:  CvM1
Estimation Method:   ULSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
604           CvM1 RMSEA     ULSMV        ALL         30 0.7784221
    partial-AUC Smoothed-AUC   Threshold Specificity Sensitivity Num-C
604   0.5723193    0.7453667 0.006383387   0.5873309   0.9244523 29849
    Num-Mis
604   27297


ROC Analysis in
Index:   RMSEA
Classification:  CvM1
Estimation Method:   ULSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
605           CvM1 RMSEA     ULSMV         30        ALL 0.6510594
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
605   0.5289806    0.7047751 0.01035087        0.46   0.8033892 14727
    Num-Mis
605   13635


ROC Analysis in
Index:   RMSEA
Classification:  CvM1
Estimation Method:   ULSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
606           CvM1 RMSEA     ULSMV         30          5 0.6328283
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
606   0.5359382    0.6469652 0.01182432   0.6292929   0.5957983  3399
    Num-Mis
606    3200


ROC Analysis in
Index:   RMSEA
Classification:  CvM1
Estimation Method:   ULSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
607           CvM1 RMSEA     ULSMV         30         10 0.6877889
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
607    0.542908    0.6928414 0.01378362   0.4694308   0.8375073  4926
    Num-Mis
607    4632


ROC Analysis in
Index:   RMSEA
Classification:  CvM1
Estimation Method:   ULSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1

Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in auc.roc(fit_roc[[key]], partial.auc = c(1, 0.8),
partial.auc.focus = "sp", : Partial AUC correction not defined for ROC
curves below the diagonal.

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
608           CvM1 RMSEA     ULSMV         30         30 0.3748022
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
608          NA    0.5636005      -Inf           0           1  6402
    Num-Mis
608    5803


ROC Analysis in
Index:   RMSEA
Classification:  CvM1
Estimation Method:   ULSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1

Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
609           CvM1 RMSEA     ULSMV         50        ALL 0.7292783
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
609   0.5619858    0.7552531 0.01189535   0.5891159   0.8001888 17918
    Num-Mis
609   16505


ROC Analysis in
Index:   RMSEA
Classification:  CvM1
Estimation Method:   ULSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
610           CvM1 RMSEA     ULSMV         50          5 0.7118657
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
610   0.5859568    0.7178021 0.01609399    0.650261   0.6658307  4573
    Num-Mis
610    4319


ROC Analysis in
Index:   RMSEA
Classification:  CvM1
Estimation Method:   ULSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
611           CvM1 RMSEA     ULSMV         50         10 0.8033953
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
611   0.6353173    0.7990178 0.01402397   0.6678899   0.8025448  6038
    Num-Mis
611    5551


ROC Analysis in
Index:   RMSEA
Classification:  CvM1
Estimation Method:   ULSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
612           CvM1 RMSEA     ULSMV         50         30 0.6927826
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
612   0.5346987    0.5838606 0.00928592   0.4224708   0.9568018  7307
    Num-Mis
612    6635


ROC Analysis in
Index:   RMSEA
Classification:  CvM1
Estimation Method:   ULSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
613           CvM1 RMSEA     ULSMV        100        ALL 0.8543715
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
613   0.6870591    0.8579546 0.01090788    0.765413   0.8266702 21277
    Num-Mis
613   19571


ROC Analysis in
Index:   RMSEA
Classification:  CvM1
Estimation Method:   ULSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS     AUC
614           CvM1 RMSEA     ULSMV        100          5 0.87579
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
614   0.7497353    0.8778585 0.01689934   0.7722123   0.8261079  5969
    Num-Mis
614    5570


ROC Analysis in
Index:   RMSEA
Classification:  CvM1
Estimation Method:   ULSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
615           CvM1 RMSEA     ULSMV        100         10 0.9326295
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
615   0.8377181    0.9339422 0.01352483   0.8177665   0.8915617  7390
    Num-Mis
615    6741


ROC Analysis in
Index:   RMSEA
Classification:  CvM1
Estimation Method:   ULSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS      AUC
616           CvM1 RMSEA     ULSMV        100         30 0.809251
    partial-AUC Smoothed-AUC   Threshold Specificity Sensitivity Num-C
616   0.5958182    0.7376006 0.009317277   0.5942096   0.9837685  7918
    Num-Mis
616    7260


ROC Analysis in
Index:   RMSEA
Classification:  CvM1
Estimation Method:   ULSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
617           CvM1 RMSEA     ULSMV        200        ALL 0.9578394
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
617   0.8950521    0.9622529 0.01306541   0.8214945   0.9472809 23791
    Num-Mis
617   21873


ROC Analysis in
Index:   RMSEA
Classification:  CvM1
Estimation Method:   ULSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
618           CvM1 RMSEA     ULSMV        200          5 0.9757116
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
618   0.9391125    0.9778463 0.01458533   0.9261279    0.907083  7507
    Num-Mis
618    6898


ROC Analysis in
Index:   RMSEA
Classification:  CvM1
Estimation Method:   ULSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS      AUC
619           CvM1 RMSEA     ULSMV        200         10 0.992833
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
619   0.9804206    0.9943414 0.01136246   0.9730224   0.9345632  8062
    Num-Mis
619    7376


ROC Analysis in
Index:   RMSEA
Classification:  CvM1
Estimation Method:   ULSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
620           CvM1 RMSEA     ULSMV        200         30 0.9764892
    partial-AUC Smoothed-AUC   Threshold Specificity Sensitivity Num-C
620   0.9374609    0.9778234 0.005540083   0.9034783   0.9291823  8222
    Num-Mis
620    7599


ROC Analysis in
Index:   RMSEA
Classification:  CvM1
Estimation Method:   WLSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
621           CvM1 RMSEA     WLSMV        ALL        ALL 0.9402416
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
621   0.8607007    0.9435131 0.01669161   0.8787345   0.9138004 74425
    Num-Mis
621   66637


ROC Analysis in
Index:   RMSEA
Classification:  CvM1
Estimation Method:   WLSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
622           CvM1 RMSEA     WLSMV        ALL          5 0.9121844
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
622   0.8286473    0.9087568 0.02119764   0.8582118   0.8698757 20313
    Num-Mis
622   18523


ROC Analysis in
Index:   RMSEA
Classification:  CvM1
Estimation Method:   WLSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
623           CvM1 RMSEA     WLSMV        ALL         10 0.9767306
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
623   0.9396679    0.9729551 0.01960808   0.9210144     0.96107 25257
    Num-Mis
623   22664


ROC Analysis in
Index:   RMSEA
Classification:  CvM1
Estimation Method:   WLSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
624           CvM1 RMSEA     WLSMV        ALL         30 0.9468727
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
624   0.8525401    0.9446462 0.01082446   0.8728588   0.9791687 28855
    Num-Mis
624   25450


ROC Analysis in
Index:   RMSEA
Classification:  CvM1
Estimation Method:   WLSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
625           CvM1 RMSEA     WLSMV         30        ALL 0.7958901
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
625   0.6312104    0.8310732 0.01375456   0.7068249   0.7939655 13548
    Num-Mis
625   12278


ROC Analysis in
Index:   RMSEA
Classification:  CvM1
Estimation Method:   WLSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
626           CvM1 RMSEA     WLSMV         30          5 0.7324048
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
626   0.6006128    0.7369558 0.02129088   0.6996424   0.6517602  3099
    Num-Mis
626    2886


ROC Analysis in
Index:   RMSEA
Classification:  CvM1
Estimation Method:   WLSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
627           CvM1 RMSEA     WLSMV         30         10 0.8804723
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
627   0.7568446    0.8870534 0.01803989   0.7540703   0.8628609  4487
    Num-Mis
627    4131


ROC Analysis in
Index:   RMSEA
Classification:  CvM1
Estimation Method:   WLSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
628           CvM1 RMSEA     WLSMV         30         30 0.7926892
    partial-AUC Smoothed-AUC   Threshold Specificity Sensitivity Num-C
628   0.5838078    0.7278708 0.008211438   0.6165689    0.903632  5962
    Num-Mis
628    5261


ROC Analysis in
Index:   RMSEA
Classification:  CvM1
Estimation Method:   WLSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
629           CvM1 RMSEA     WLSMV         50        ALL 0.9077966
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
629   0.8166526    0.9124279 0.01396333   0.8502255   0.8061538 16883
    Num-Mis
629   15246


ROC Analysis in
Index:   RMSEA
Classification:  CvM1
Estimation Method:   WLSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
630           CvM1 RMSEA     WLSMV         50          5 0.8305913
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
630   0.6944256    0.8345038 0.02039191    0.788899   0.7241135  4162
    Num-Mis
630    3851


ROC Analysis in
Index:   RMSEA
Classification:  CvM1
Estimation Method:   WLSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
631           CvM1 RMSEA     WLSMV         50         10 0.9700414
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
631   0.9262605    0.9706179 0.01960808   0.9043359   0.9247855  5746
    Num-Mis
631    5218


ROC Analysis in
Index:   RMSEA
Classification:  CvM1
Estimation Method:   WLSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
632           CvM1 RMSEA     WLSMV         50         30 0.9608898
    partial-AUC Smoothed-AUC   Threshold Specificity Sensitivity Num-C
632   0.8940829    0.9601915 0.008776837   0.8777845   0.9349329  6975
    Num-Mis
632    6177


ROC Analysis in
Index:   RMSEA
Classification:  CvM1
Estimation Method:   WLSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
633           CvM1 RMSEA     WLSMV        100        ALL 0.9875578
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
633   0.9696511    0.9872043 0.01666949   0.9666667   0.9305556 20592
    Num-Mis
633   18372


ROC Analysis in
Index:   RMSEA
Classification:  CvM1
Estimation Method:   WLSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
634           CvM1 RMSEA     WLSMV        100          5 0.9599354
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
634     0.90313    0.9593642 0.02119588   0.8796484   0.9157472  5756
    Num-Mis
634    5241


ROC Analysis in
Index:   RMSEA
Classification:  CvM1
Estimation Method:   WLSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
635           CvM1 RMSEA     WLSMV        100         10 0.9994671
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
635   0.9985199    0.9992942 0.01988183    0.986211   0.9939345  7084
    Num-Mis
635    6279


ROC Analysis in
Index:   RMSEA
Classification:  CvM1
Estimation Method:   WLSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
636           CvM1 RMSEA     WLSMV        100         30 0.9999952
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
636   0.9999867    0.9999902 0.01213483   0.9994544   0.9992682  7752
    Num-Mis
636    6852


ROC Analysis in
Index:   RMSEA
Classification:  CvM1
Estimation Method:   WLSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
637           CvM1 RMSEA     WLSMV        200        ALL 0.9998556
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
637   0.9995988    0.9998186 0.0196625   0.9946543   0.9955276 23402
    Num-Mis
637   20741


ROC Analysis in
Index:   RMSEA
Classification:  CvM1
Estimation Method:   WLSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
638           CvM1 RMSEA     WLSMV        200          5 0.9988722
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
638   0.9968673    0.9987194 0.0196625   0.9844011   0.9854674  7296
    Num-Mis
638    6545


ROC Analysis in
Index:   RMSEA
Classification:  CvM1
Estimation Method:   WLSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in ci.auc.roc(roc, ...): ci.auc() of a ROC curve with AUC == 1 is
always 1-1 and can be misleading.

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS AUC partial-AUC
639           CvM1 RMSEA     WLSMV        200         10   1           1
    Smoothed-AUC  Threshold Specificity Sensitivity Num-C Num-Mis
639           NA 0.01881367           1           1  7940    7036


ROC Analysis in
Index:   RMSEA
Classification:  CvM1
Estimation Method:   WLSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in ci.auc.roc(roc, ...): ci.auc() of a ROC curve with AUC == 1 is
always 1-1 and can be misleading.

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS AUC partial-AUC
640           CvM1 RMSEA     WLSMV        200         30   1           1
    Smoothed-AUC  Threshold Specificity Sensitivity Num-C Num-Mis
640           NA 0.01420712           1           1  8166    7160


ROC Analysis in
Index:   SRMRW
Classification:  CvM1
Estimation Method:   ALL
Level-2 Sample Size:     ALL
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
641           CvM1 SRMRW       ALL        ALL        ALL 0.8725866
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity  Num-C
641    0.869644    0.8879885 0.03824319   0.9745978   0.7240689 241960
    Num-Mis
641  223868


ROC Analysis in
Index:   SRMRW
Classification:  CvM1
Estimation Method:   ALL
Level-2 Sample Size:     ALL
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
642           CvM1 SRMRW       ALL        ALL          5 0.7894316
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
642   0.7707523    0.8098569 0.04684671   0.9203083   0.5881022 68483
    Num-Mis
642   63082


ROC Analysis in
Index:   SRMRW
Classification:  CvM1
Estimation Method:   ALL
Level-2 Sample Size:     ALL
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
643           CvM1 SRMRW       ALL        ALL         10 0.9043661
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
643   0.8808475    0.9122141 0.04157495   0.9362945   0.7703326 81868
    Num-Mis
643   75749


ROC Analysis in
Index:   SRMRW
Classification:  CvM1
Estimation Method:   ALL
Level-2 Sample Size:     ALL
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
644           CvM1 SRMRW       ALL        ALL         30 0.9778388
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
644   0.9770409     0.978348 0.03536799   0.9929956   0.9407072 91609
    Num-Mis
644   85037


ROC Analysis in
Index:   SRMRW
Classification:  CvM1
Estimation Method:   ALL
Level-2 Sample Size:     30
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
645           CvM1 SRMRW       ALL         30        ALL 0.7478239
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
645   0.7259798    0.7775137 0.0443695   0.9208784    0.499876 47053
    Num-Mis
645   43767


ROC Analysis in
Index:   SRMRW
Classification:  CvM1
Estimation Method:   ALL
Level-2 Sample Size:     30
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
646           CvM1 SRMRW       ALL         30          5 0.6902827
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
646   0.6210644    0.6889253 0.07247021   0.8029925   0.4834888 11584
    Num-Mis
646   10825


ROC Analysis in
Index:   SRMRW
Classification:  CvM1
Estimation Method:   ALL
Level-2 Sample Size:     30
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
647           CvM1 SRMRW       ALL         30         10 0.8275618
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
647   0.7570969    0.8223777 0.05610027   0.8019825   0.7216242 15551
    Num-Mis
647   14595


ROC Analysis in
Index:   SRMRW
Classification:  CvM1
Estimation Method:   ALL
Level-2 Sample Size:     30
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
648           CvM1 SRMRW       ALL         30         30 0.9428504
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
648   0.9376569    0.9429342  0.038644   0.9673277   0.8722692 19918
    Num-Mis
648   18347


ROC Analysis in
Index:   SRMRW
Classification:  CvM1
Estimation Method:   ALL
Level-2 Sample Size:     50
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
649           CvM1 SRMRW       ALL         50        ALL 0.8346506
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
649   0.8045808    0.8534621 0.04221965   0.9260513   0.6388515 56136
    Num-Mis
649   52168


ROC Analysis in
Index:   SRMRW
Classification:  CvM1
Estimation Method:   ALL
Level-2 Sample Size:     50
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
650           CvM1 SRMRW       ALL         50          5 0.7905489
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
650   0.6996188    0.7916984 0.06295491   0.7665937   0.6708358 14818
    Num-Mis
650   13894


ROC Analysis in
Index:   SRMRW
Classification:  CvM1
Estimation Method:   ALL
Level-2 Sample Size:     50
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
651           CvM1 SRMRW       ALL         50         10 0.9230671
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
651   0.8834498    0.9161851 0.04674038   0.8967615   0.8343606 18954
    Num-Mis
651   17559


ROC Analysis in
Index:   SRMRW
Classification:  CvM1
Estimation Method:   ALL
Level-2 Sample Size:     50
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS      AUC
652           CvM1 SRMRW       ALL         50         30 0.977276
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
652   0.9724297    0.9783181 0.03522341   0.9910627   0.9270793 22364
    Num-Mis
652   20715


ROC Analysis in
Index:   SRMRW
Classification:  CvM1
Estimation Method:   ALL
Level-2 Sample Size:     100
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
653           CvM1 SRMRW       ALL        100        ALL 0.9476294
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
653   0.9178977    0.9548357 0.03901065   0.9583169   0.8187135 65882
    Num-Mis
653   60844


ROC Analysis in
Index:   SRMRW
Classification:  CvM1
Estimation Method:   ALL
Level-2 Sample Size:     100
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
654           CvM1 SRMRW       ALL        100          5 0.9322345
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
654   0.8754798    0.9303612 0.05015186   0.8628837   0.8594111 18983
    Num-Mis
654   17448


ROC Analysis in
Index:   SRMRW
Classification:  CvM1
Estimation Method:   ALL
Level-2 Sample Size:     100
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
655           CvM1 SRMRW       ALL        100         10 0.9883287
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
655   0.9783579    0.9893719 0.03791551   0.9812834   0.9355578 22711
    Num-Mis
655   20843


ROC Analysis in
Index:   SRMRW
Classification:  CvM1
Estimation Method:   ALL
Level-2 Sample Size:     100
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
656           CvM1 SRMRW       ALL        100         30 0.9972387
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
656   0.9951013    0.9970526 0.03460798   0.9964544   0.9807235 24188
    Num-Mis
656   22553


ROC Analysis in
Index:   SRMRW
Classification:  CvM1
Estimation Method:   ALL
Level-2 Sample Size:     200
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
657           CvM1 SRMRW       ALL        200        ALL 0.9955902
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
657   0.9907704    0.9961891 0.03631168   0.9907084   0.9639636 72889
    Num-Mis
657   67089


ROC Analysis in
Index:   SRMRW
Classification:  CvM1
Estimation Method:   ALL
Level-2 Sample Size:     200
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
658           CvM1 SRMRW       ALL        200          5 0.9935031
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
658   0.9845913    0.9936985 0.03969441   0.9776003   0.9496349 23098
    Num-Mis
658   20915


ROC Analysis in
Index:   SRMRW
Classification:  CvM1
Estimation Method:   ALL
Level-2 Sample Size:     200
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
659           CvM1 SRMRW       ALL        200         10 0.9995816
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
659   0.9988549    0.9995935 0.03430265    0.997151   0.9892636 24652
    Num-Mis
659   22752


ROC Analysis in
Index:   SRMRW
Classification:  CvM1
Estimation Method:   ALL
Level-2 Sample Size:     200
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
660           CvM1 SRMRW       ALL        200         30 0.9999975
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
660   0.9999932    0.9999917 0.03354014   0.9995746   0.9997719 25139
    Num-Mis
660   23422


ROC Analysis in
Index:   SRMRW
Classification:  CvM1
Estimation Method:   MLR
Level-2 Sample Size:     ALL
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
661           CvM1 SRMRW       MLR        ALL        ALL 0.8871775
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
661   0.8923593    0.9020861 0.0363859   0.9786016   0.7756013 89822
    Num-Mis
661   85647


ROC Analysis in
Index:   SRMRW
Classification:  CvM1
Estimation Method:   MLR
Level-2 Sample Size:     ALL
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
662           CvM1 SRMRW       MLR        ALL          5 0.8078806
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
662    0.783544    0.8382935 0.04240668   0.9350993   0.5989939 26722
    Num-Mis
662   24572


ROC Analysis in
Index:   SRMRW
Classification:  CvM1
Estimation Method:   MLR
Level-2 Sample Size:     ALL
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
663           CvM1 SRMRW       MLR        ALL         10 0.9513605
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
663   0.9191745    0.9602993 0.03823155   0.9547235   0.8285431 30195
    Num-Mis
663   28785


ROC Analysis in
Index:   SRMRW
Classification:  CvM1
Estimation Method:   MLR
Level-2 Sample Size:     ALL
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
664           CvM1 SRMRW       MLR        ALL         30 0.9999925
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
664   0.9999792    0.9999897 0.03072066   0.9996142   0.9984522 32905
    Num-Mis
664   32290


ROC Analysis in
Index:   SRMRW
Classification:  CvM1
Estimation Method:   MLR
Level-2 Sample Size:     30
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
665           CvM1 SRMRW       MLR         30        ALL 0.7691124
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
665   0.7550909    0.7975211 0.04167316   0.9065351   0.5674345 18778
    Num-Mis
665   17854


ROC Analysis in
Index:   SRMRW
Classification:  CvM1
Estimation Method:   MLR
Level-2 Sample Size:     30
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
666           CvM1 SRMRW       MLR         30          5 0.8134797
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
666   0.6814219    0.8121017 0.07094323   0.6598985   0.8198146  5086
    Num-Mis
666    4739


ROC Analysis in
Index:   SRMRW
Classification:  CvM1
Estimation Method:   MLR
Level-2 Sample Size:     30
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
667           CvM1 SRMRW       MLR         30         10 0.9545869
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
667   0.8932373    0.9488942 0.04805876    0.902746   0.8695229  6138
    Num-Mis
667    5832


ROC Analysis in
Index:   SRMRW
Classification:  CvM1
Estimation Method:   MLR
Level-2 Sample Size:     30
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
668           CvM1 SRMRW       MLR         30         30 0.9999206
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
668   0.9997795     0.999877 0.03284309   0.9964349   0.9988072  7554
    Num-Mis
668    7283


ROC Analysis in
Index:   SRMRW
Classification:  CvM1
Estimation Method:   MLR
Level-2 Sample Size:     50
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
669           CvM1 SRMRW       MLR         50        ALL 0.8612133
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
669   0.8399044    0.8886555 0.03800634   0.9598775    0.692027 21335
    Num-Mis
669   20417


ROC Analysis in
Index:   SRMRW
Classification:  CvM1
Estimation Method:   MLR
Level-2 Sample Size:     50
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
670           CvM1 SRMRW       MLR         50          5 0.9049826
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
670   0.8016492    0.9068459 0.05631149   0.7961745   0.8548228  6083
    Num-Mis
670    5724


ROC Analysis in
Index:   SRMRW
Classification:  CvM1
Estimation Method:   MLR
Level-2 Sample Size:     50
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
671           CvM1 SRMRW       MLR         50         10 0.9930012
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
671   0.9810103    0.9921228 0.04067596   0.9521674   0.9652283  7170
    Num-Mis
671    6790


ROC Analysis in
Index:   SRMRW
Classification:  CvM1
Estimation Method:   MLR
Level-2 Sample Size:     50
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in ci.auc.roc(roc, ...): ci.auc() of a ROC curve with AUC == 1 is
always 1-1 and can be misleading.

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS AUC partial-AUC
672           CvM1 SRMRW       MLR         50         30   1           1
    Smoothed-AUC  Threshold Specificity Sensitivity Num-C Num-Mis
672           NA 0.02916651           1           1  8082    7903


ROC Analysis in
Index:   SRMRW
Classification:  CvM1
Estimation Method:   MLR
Level-2 Sample Size:     100
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
673           CvM1 SRMRW       MLR        100        ALL 0.9811041
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
673   0.9612017    0.9843424 0.03687401   0.9668212   0.9045421 24013
    Num-Mis
673   22901


ROC Analysis in
Index:   SRMRW
Classification:  CvM1
Estimation Method:   MLR
Level-2 Sample Size:     100
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
674           CvM1 SRMRW       MLR        100          5 0.9873474
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
674   0.9657824    0.9869547 0.04287668   0.9387528    0.956144  7258
    Num-Mis
674    6637


ROC Analysis in
Index:   SRMRW
Classification:  CvM1
Estimation Method:   MLR
Level-2 Sample Size:     100
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
675           CvM1 SRMRW       MLR        100         10 0.9999927
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
675   0.9999798    0.9999789 0.03214973   0.9991475   0.9996377  8237
    Num-Mis
675    7823


ROC Analysis in
Index:   SRMRW
Classification:  CvM1
Estimation Method:   MLR
Level-2 Sample Size:     100
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in ci.auc.roc(roc, ...): ci.auc() of a ROC curve with AUC == 1 is
always 1-1 and can be misleading.

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS AUC partial-AUC
676           CvM1 SRMRW       MLR        100         30   1           1
    Smoothed-AUC  Threshold Specificity Sensitivity Num-C Num-Mis
676           NA 0.02504541           1           1  8518    8441


ROC Analysis in
Index:   SRMRW
Classification:  CvM1
Estimation Method:   MLR
Level-2 Sample Size:     200
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
677           CvM1 SRMRW       MLR        200        ALL 0.9999544
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
677   0.9998732    0.9999314 0.0332518   0.9976915   0.9975539 25696
    Num-Mis
677   24475


ROC Analysis in
Index:   SRMRW
Classification:  CvM1
Estimation Method:   MLR
Level-2 Sample Size:     200
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
678           CvM1 SRMRW       MLR        200          5 0.9998847
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
678   0.9996796    0.9998409 0.03503656   0.9938367    0.999278  8295
    Num-Mis
678    7472


ROC Analysis in
Index:   SRMRW
Classification:  CvM1
Estimation Method:   MLR
Level-2 Sample Size:     200
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in ci.auc.roc(roc, ...): ci.auc() of a ROC curve with AUC == 1 is
always 1-1 and can be misleading.

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS AUC partial-AUC
679           CvM1 SRMRW       MLR        200         10   1           1
    Smoothed-AUC  Threshold Specificity Sensitivity Num-C Num-Mis
679           NA 0.02725551           1           1  8650    8340


ROC Analysis in
Index:   SRMRW
Classification:  CvM1
Estimation Method:   MLR
Level-2 Sample Size:     200
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in ci.auc.roc(roc, ...): ci.auc() of a ROC curve with AUC == 1 is
always 1-1 and can be misleading.

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS AUC partial-AUC
680           CvM1 SRMRW       MLR        200         30   1           1
    Smoothed-AUC  Threshold Specificity Sensitivity Num-C Num-Mis
680           NA 0.02244555           1           1  8751    8663


ROC Analysis in
Index:   SRMRW
Classification:  CvM1
Estimation Method:   ULSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
681           CvM1 SRMRW     ULSMV        ALL        ALL 0.8539734
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
681   0.8638233    0.8708944 0.04520983   0.9759944   0.7229818 77713
    Num-Mis
681   71584


ROC Analysis in
Index:   SRMRW
Classification:  CvM1
Estimation Method:   ULSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
682           CvM1 SRMRW     ULSMV        ALL          5 0.7598259
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
682   0.7603518    0.7871987 0.05039326   0.9510373   0.5421397 21448
    Num-Mis
682   19987


ROC Analysis in
Index:   SRMRW
Classification:  CvM1
Estimation Method:   ULSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
683           CvM1 SRMRW     ULSMV        ALL         10 0.8733545
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
683   0.8639919    0.8868045 0.04676229   0.9599446   0.7288608 26416
    Num-Mis
683   24300


ROC Analysis in
Index:   SRMRW
Classification:  CvM1
Estimation Method:   ULSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
684           CvM1 SRMRW     ULSMV        ALL         30 0.9598636
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
684   0.9607296    0.9624836 0.0423881   0.9949569    0.903857 29849
    Num-Mis
684   27297


ROC Analysis in
Index:   SRMRW
Classification:  CvM1
Estimation Method:   ULSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
685           CvM1 SRMRW     ULSMV         30        ALL 0.7223925
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
685   0.7061635    0.7475078 0.05277202   0.9221951   0.4635086 14727
    Num-Mis
685   13635


ROC Analysis in
Index:   SRMRW
Classification:  CvM1
Estimation Method:   ULSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS      AUC
686           CvM1 SRMRW     ULSMV         30          5 0.661179
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
686   0.5818408    0.6587591 0.09810666   0.6191919   0.6193277  3399
    Num-Mis
686    3200


ROC Analysis in
Index:   SRMRW
Classification:  CvM1
Estimation Method:   ULSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
687           CvM1 SRMRW     ULSMV         30         10 0.7972812
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
687   0.7296397    0.8010994 0.06939408   0.7519325   0.7134537  4926
    Num-Mis
687    4632


ROC Analysis in
Index:   SRMRW
Classification:  CvM1
Estimation Method:   ULSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
688           CvM1 SRMRW     ULSMV         30         30 0.9042897
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
688   0.8874242    0.9114546 0.0465746   0.9626556   0.7664042  6402
    Num-Mis
688    5803


ROC Analysis in
Index:   SRMRW
Classification:  CvM1
Estimation Method:   ULSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS     AUC
689           CvM1 SRMRW     ULSMV         50        ALL 0.81707
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
689   0.7951393    0.8398375 0.04815927   0.9530649   0.6027376 17918
    Num-Mis
689   16505


ROC Analysis in
Index:   SRMRW
Classification:  CvM1
Estimation Method:   ULSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS      AUC
690           CvM1 SRMRW     ULSMV         50          5 0.790244
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
690   0.6880289    0.7932803 0.07614242   0.7300522   0.7034483  4573
    Num-Mis
690    4319


ROC Analysis in
Index:   SRMRW
Classification:  CvM1
Estimation Method:   ULSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
691           CvM1 SRMRW     ULSMV         50         10 0.9055049
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
691    0.867266    0.9127474 0.0541133   0.9217125   0.7700283  6038
    Num-Mis
691    5551


ROC Analysis in
Index:   SRMRW
Classification:  CvM1
Estimation Method:   ULSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
692           CvM1 SRMRW     ULSMV         50         30 0.9682549
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
692   0.9580511      0.97002 0.04363759   0.9832232   0.8942781  7307
    Num-Mis
692    6635


ROC Analysis in
Index:   SRMRW
Classification:  CvM1
Estimation Method:   ULSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
693           CvM1 SRMRW     ULSMV        100        ALL 0.9452556
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
693   0.9241853    0.9535525 0.04609349   0.9610167   0.8391553 21277
    Num-Mis
693   19571


ROC Analysis in
Index:   SRMRW
Classification:  CvM1
Estimation Method:   ULSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
694           CvM1 SRMRW     ULSMV        100          5 0.9311267
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
694   0.8696499     0.934428 0.05742377   0.8652355    0.844894  5969
    Num-Mis
694    5570


ROC Analysis in
Index:   SRMRW
Classification:  CvM1
Estimation Method:   ULSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
695           CvM1 SRMRW     ULSMV        100         10 0.9891953
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
695   0.9778426    0.9893732 0.04551502   0.9720812   0.9423444  7390
    Num-Mis
695    6741


ROC Analysis in
Index:   SRMRW
Classification:  CvM1
Estimation Method:   ULSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
696           CvM1 SRMRW     ULSMV        100         30 0.9984657
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
696   0.9970149    0.9984672 0.04176947   0.9981618   0.9844742  7918
    Num-Mis
696    7260


ROC Analysis in
Index:   SRMRW
Classification:  CvM1
Estimation Method:   ULSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
697           CvM1 SRMRW     ULSMV        200        ALL 0.9974331
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
697   0.9948313    0.9974843 0.04296533   0.9921587   0.9794586 23791
    Num-Mis
697   21873


ROC Analysis in
Index:   SRMRW
Classification:  CvM1
Estimation Method:   ULSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
698           CvM1 SRMRW     ULSMV        200          5 0.9954771
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
698   0.9887137    0.9956556 0.04469891   0.9816559   0.9535415  7507
    Num-Mis
698    6898


ROC Analysis in
Index:   SRMRW
Classification:  CvM1
Estimation Method:   ULSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
699           CvM1 SRMRW     ULSMV        200         10 0.9999481
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
699   0.9998559    0.9999253 0.04131731    0.996342   0.9986077  8062
    Num-Mis
699    7376


ROC Analysis in
Index:   SRMRW
Classification:  CvM1
Estimation Method:   ULSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in ci.auc.roc(roc, ...): ci.auc() of a ROC curve with AUC == 1 is
always 1-1 and can be misleading.

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS AUC partial-AUC
700           CvM1 SRMRW     ULSMV        200         30   1           1
    Smoothed-AUC  Threshold Specificity Sensitivity Num-C Num-Mis
700           NA 0.03876653           1           1  8222    7599


ROC Analysis in
Index:   SRMRW
Classification:  CvM1
Estimation Method:   WLSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
701           CvM1 SRMRW     WLSMV        ALL        ALL 0.9054811
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
701   0.9108359    0.9153222 0.04442599   0.9792572   0.8145823 74425
    Num-Mis
701   66637


ROC Analysis in
Index:   SRMRW
Classification:  CvM1
Estimation Method:   WLSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
702           CvM1 SRMRW     WLSMV        ALL          5 0.8105721
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
702   0.8019763    0.8395929 0.04885211   0.9541443   0.6209113 20313
    Num-Mis
702   18523


ROC Analysis in
Index:   SRMRW
Classification:  CvM1
Estimation Method:   WLSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
703           CvM1 SRMRW     WLSMV        ALL         10 0.9440333
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
703   0.9187393    0.9534607 0.04538322   0.9657567   0.8243597 25257
    Num-Mis
703   22664


ROC Analysis in
Index:   SRMRW
Classification:  CvM1
Estimation Method:   WLSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS      AUC
704           CvM1 SRMRW     WLSMV        ALL         30 0.999985
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
704   0.9999584    0.9999816 0.03903019    0.999114   0.9974452 28855
    Num-Mis
704   25450


ROC Analysis in
Index:   SRMRW
Classification:  CvM1
Estimation Method:   WLSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
705           CvM1 SRMRW     WLSMV         30        ALL 0.8007005
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
705   0.7857364    0.8218018 0.0527773   0.8958457   0.6336207 13548
    Num-Mis
705   12278


ROC Analysis in
Index:   SRMRW
Classification:  CvM1
Estimation Method:   WLSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
706           CvM1 SRMRW     WLSMV         30          5 0.7783223
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
706   0.6405395    0.7746981 0.08500221   0.7175209   0.7136061  3099
    Num-Mis
706    2886


ROC Analysis in
Index:   SRMRW
Classification:  CvM1
Estimation Method:   WLSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
707           CvM1 SRMRW     WLSMV         30         10 0.9488751
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
707   0.8826035    0.9440681 0.06074398   0.8680377   0.8930446  4487
    Num-Mis
707    4131


ROC Analysis in
Index:   SRMRW
Classification:  CvM1
Estimation Method:   WLSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
708           CvM1 SRMRW     WLSMV         30         30 0.9998172
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
708   0.9994921    0.9997138 0.04150601   0.9926686    0.998063  5962
    Num-Mis
708    5261


ROC Analysis in
Index:   SRMRW
Classification:  CvM1
Estimation Method:   WLSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
709           CvM1 SRMRW     WLSMV         50        ALL 0.8774132
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
709   0.8616127    0.8970432 0.04652215    0.962497   0.7311111 16883
    Num-Mis
709   15246


ROC Analysis in
Index:   SRMRW
Classification:  CvM1
Estimation Method:   WLSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
710           CvM1 SRMRW     WLSMV         50          5 0.8916275
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
710   0.7865353    0.8912098 0.06914229   0.7970883   0.8340426  4162
    Num-Mis
710    3851


ROC Analysis in
Index:   SRMRW
Classification:  CvM1
Estimation Method:   WLSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
711           CvM1 SRMRW     WLSMV         50         10 0.9924305
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
711   0.9798111      0.99224 0.04914004   0.9649002   0.9535588  5746
    Num-Mis
711    5218


ROC Analysis in
Index:   SRMRW
Classification:  CvM1
Estimation Method:   WLSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in ci.auc.roc(roc, ...): ci.auc() of a ROC curve with AUC == 1 is
always 1-1 and can be misleading.

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS AUC partial-AUC
712           CvM1 SRMRW     WLSMV         50         30   1           1
    Smoothed-AUC  Threshold Specificity Sensitivity Num-C Num-Mis
712           NA 0.03726937           1           1  6975    6177


ROC Analysis in
Index:   SRMRW
Classification:  CvM1
Estimation Method:   WLSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
713           CvM1 SRMRW     WLSMV        100        ALL 0.9815336
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
713   0.9649875    0.9844871 0.04514939   0.9660643   0.9134722 20592
    Num-Mis
713   18372


ROC Analysis in
Index:   SRMRW
Classification:  CvM1
Estimation Method:   WLSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
714           CvM1 SRMRW     WLSMV        100          5 0.9835182
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
714    0.957352     0.982469 0.05338245   0.9053414   0.9674022  5756
    Num-Mis
714    5241


ROC Analysis in
Index:   SRMRW
Classification:  CvM1
Estimation Method:   WLSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
715           CvM1 SRMRW     WLSMV        100         10 0.9999777
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
715    0.999938    0.9999593 0.03916489   0.9982014   0.9983825  7084
    Num-Mis
715    6279


ROC Analysis in
Index:   SRMRW
Classification:  CvM1
Estimation Method:   WLSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in ci.auc.roc(roc, ...): ci.auc() of a ROC curve with AUC == 1 is
always 1-1 and can be misleading.

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS AUC partial-AUC
716           CvM1 SRMRW     WLSMV        100         30   1           1
    Smoothed-AUC  Threshold Specificity Sensitivity Num-C Num-Mis
716           NA 0.03146296           1           1  7752    6852


ROC Analysis in
Index:   SRMRW
Classification:  CvM1
Estimation Method:   WLSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
717           CvM1 SRMRW     WLSMV        200        ALL 0.9999536
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
717   0.9998711    0.9999286 0.04023907   0.9985745   0.9979451 23402
    Num-Mis
717   20741


ROC Analysis in
Index:   SRMRW
Classification:  CvM1
Estimation Method:   WLSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
718           CvM1 SRMRW     WLSMV        200          5 0.9998534
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
718   0.9995928    0.9997931 0.04100113   0.9955432   0.9960723  7296
    Num-Mis
718    6545


ROC Analysis in
Index:   SRMRW
Classification:  CvM1
Estimation Method:   WLSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in ci.auc.roc(roc, ...): ci.auc() of a ROC curve with AUC == 1 is
always 1-1 and can be misleading.

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS AUC partial-AUC
719           CvM1 SRMRW     WLSMV        200         10   1           1
    Smoothed-AUC  Threshold Specificity Sensitivity Num-C Num-Mis
719           NA 0.03436883           1           1  7940    7036


ROC Analysis in
Index:   SRMRW
Classification:  CvM1
Estimation Method:   WLSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in ci.auc.roc(roc, ...): ci.auc() of a ROC curve with AUC == 1 is
always 1-1 and can be misleading.

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS AUC partial-AUC
720           CvM1 SRMRW     WLSMV        200         30   1           1
    Smoothed-AUC  Threshold Specificity Sensitivity Num-C Num-Mis
720           NA 0.02818581           1           1  8166    7160


ROC Analysis in
Index:   SRMRB
Classification:  CvM1
Estimation Method:   ALL
Level-2 Sample Size:     ALL
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
721           CvM1 SRMRB       ALL        ALL        ALL 0.5373652
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity  Num-C
721   0.5405211    0.5473238 0.05688385   0.8393742   0.2433847 241960
    Num-Mis
721  223868


ROC Analysis in
Index:   SRMRB
Classification:  CvM1
Estimation Method:   ALL
Level-2 Sample Size:     ALL
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS      AUC
722           CvM1 SRMRB       ALL        ALL          5 0.511451
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
722   0.5189814    0.5158289 0.06367922   0.8388975   0.2048393 68483
    Num-Mis
722   63082


ROC Analysis in
Index:   SRMRB
Classification:  CvM1
Estimation Method:   ALL
Level-2 Sample Size:     ALL
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
723           CvM1 SRMRB       ALL        ALL         10 0.5364506
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
723   0.5393037    0.5462023 0.05683478    0.843883   0.2356236 81868
    Num-Mis
723   75749


ROC Analysis in
Index:   SRMRB
Classification:  CvM1
Estimation Method:   ALL
Level-2 Sample Size:     ALL
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
724           CvM1 SRMRB       ALL        ALL         30 0.5644537
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
724    0.558878    0.5772041 0.05689041   0.7916898   0.3296589 91609
    Num-Mis
724   85037


ROC Analysis in
Index:   SRMRB
Classification:  CvM1
Estimation Method:   ALL
Level-2 Sample Size:     30
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
725           CvM1 SRMRB       ALL         30        ALL 0.5169104
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
725   0.5209573    0.5216189 0.1091351   0.8137217   0.2317088 47053
    Num-Mis
725   43767


ROC Analysis in
Index:   SRMRB
Classification:  CvM1
Estimation Method:   ALL
Level-2 Sample Size:     30
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in auc.roc(fit_roc[[key]], partial.auc = c(1, 0.8),
partial.auc.focus = "sp", : Partial AUC correction not defined for ROC
curves below the diagonal.

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS      AUC
726           CvM1 SRMRB       ALL         30          5 0.495215
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
726          NA    0.4941905 0.1091366   0.9292394   0.0743635 11584
    Num-Mis
726   10825


ROC Analysis in
Index:   SRMRB
Classification:  CvM1
Estimation Method:   ALL
Level-2 Sample Size:     30
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS      AUC
727           CvM1 SRMRB       ALL         30         10 0.510919
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
727   0.5119085    0.5114705  0.127003   0.6466113   0.3879131 15551
    Num-Mis
727   14595


ROC Analysis in
Index:   SRMRB
Classification:  CvM1
Estimation Method:   ALL
Level-2 Sample Size:     30
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
728           CvM1 SRMRB       ALL         30         30 0.5371528
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
728   0.5365358    0.5425804 0.1034905   0.7988669   0.2838625 19918
    Num-Mis
728   18347


ROC Analysis in
Index:   SRMRB
Classification:  CvM1
Estimation Method:   ALL
Level-2 Sample Size:     50
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
729           CvM1 SRMRB       ALL         50        ALL 0.5328607
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
729   0.5342461    0.5394629 0.08366725   0.8392136   0.2326436 56136
    Num-Mis
729   52168


ROC Analysis in
Index:   SRMRB
Classification:  CvM1
Estimation Method:   ALL
Level-2 Sample Size:     50
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
730           CvM1 SRMRB       ALL         50          5 0.5027483
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
730   0.5036412    0.5024658 0.1204906   0.5353756   0.4802462 14818
    Num-Mis
730   13894


ROC Analysis in
Index:   SRMRB
Classification:  CvM1
Estimation Method:   ALL
Level-2 Sample Size:     50
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
731           CvM1 SRMRB       ALL         50         10 0.5227738
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
731   0.5241468    0.5236382 0.08570331    0.837684   0.2246533 18954
    Num-Mis
731   17559


ROC Analysis in
Index:   SRMRB
Classification:  CvM1
Estimation Method:   ALL
Level-2 Sample Size:     50
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
732           CvM1 SRMRB       ALL         50         30 0.5704121
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
732   0.5600296    0.5782446 0.07952331   0.8152421   0.3125721 22364
    Num-Mis
732   20715


ROC Analysis in
Index:   SRMRB
Classification:  CvM1
Estimation Method:   ALL
Level-2 Sample Size:     100
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS      AUC
733           CvM1 SRMRB       ALL        100        ALL 0.559458
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
733   0.5556116     0.567846 0.0629752   0.8109009   0.3016313 65882
    Num-Mis
733   60844


ROC Analysis in
Index:   SRMRB
Classification:  CvM1
Estimation Method:   ALL
Level-2 Sample Size:     100
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
734           CvM1 SRMRB       ALL        100          5 0.5091267
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
734      0.5139    0.5092448 0.07404405   0.7671648   0.2702328 18983
    Num-Mis
734   17448


ROC Analysis in
Index:   SRMRB
Classification:  CvM1
Estimation Method:   ALL
Level-2 Sample Size:     100
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
735           CvM1 SRMRB       ALL        100         10 0.5607982
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
735   0.5542571    0.5652553 0.06668203   0.7570187   0.3650025 22711
    Num-Mis
735   20843


ROC Analysis in
Index:   SRMRB
Classification:  CvM1
Estimation Method:   ALL
Level-2 Sample Size:     100
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
736           CvM1 SRMRB       ALL        100         30 0.6207136
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
736   0.5941298    0.6343866 0.05907777   0.8125277   0.3751785 24188
    Num-Mis
736   22553


ROC Analysis in
Index:   SRMRB
Classification:  CvM1
Estimation Method:   ALL
Level-2 Sample Size:     200
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
737           CvM1 SRMRB       ALL        200        ALL 0.6006662
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
737   0.5798555     0.608633 0.04939283   0.7487166   0.4137263 72889
    Num-Mis
737   67089


ROC Analysis in
Index:   SRMRB
Classification:  CvM1
Estimation Method:   ALL
Level-2 Sample Size:     200
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
738           CvM1 SRMRB       ALL        200          5 0.5396148
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
738   0.5342475    0.5396815 0.0608186   0.6325751    0.452279 23098
    Num-Mis
738   20915


ROC Analysis in
Index:   SRMRB
Classification:  CvM1
Estimation Method:   ALL
Level-2 Sample Size:     200
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS      AUC
739           CvM1 SRMRB       ALL        200         10 0.610754
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
739   0.5916787    0.6211413 0.04668508   0.8319088   0.3574513 24652
    Num-Mis
739   22752


ROC Analysis in
Index:   SRMRB
Classification:  CvM1
Estimation Method:   ALL
Level-2 Sample Size:     200
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
740           CvM1 SRMRB       ALL        200         30 0.6822653
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
740   0.6273477    0.7034347 0.04291026   0.8281339   0.4173794 25139
    Num-Mis
740   23422


ROC Analysis in
Index:   SRMRB
Classification:  CvM1
Estimation Method:   MLR
Level-2 Sample Size:     ALL
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
741           CvM1 SRMRB       MLR        ALL        ALL 0.5119369
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
741   0.5386247    0.5263223 0.06064401   0.9205975   0.1574389 89822
    Num-Mis
741   85647


ROC Analysis in
Index:   SRMRB
Classification:  CvM1
Estimation Method:   MLR
Level-2 Sample Size:     ALL
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1

Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
742           CvM1 SRMRB       MLR        ALL          5 0.5020198
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
742   0.5081647     0.491299 0.1469625   0.6750552    0.361431 26722
    Num-Mis
742   24572


ROC Analysis in
Index:   SRMRB
Classification:  CvM1
Estimation Method:   MLR
Level-2 Sample Size:     ALL
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
743           CvM1 SRMRB       MLR        ALL         10 0.4891833
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
743   0.5103983    0.4756123 0.1754708   0.9016129   0.1225966 30195
    Num-Mis
743   28785


ROC Analysis in
Index:   SRMRB
Classification:  CvM1
Estimation Method:   MLR
Level-2 Sample Size:     ALL
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1

Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
744           CvM1 SRMRB       MLR        ALL         30 0.5442763
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
744   0.5634235    0.5620789 0.05867225   0.9128086   0.2109624 32905
    Num-Mis
744   32290


ROC Analysis in
Index:   SRMRB
Classification:  CvM1
Estimation Method:   MLR
Level-2 Sample Size:     30
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1

Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
745           CvM1 SRMRB       MLR         30        ALL 0.5176976
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
745   0.5134211     0.512463 0.1769509   0.6980078   0.3461477 18778
    Num-Mis
745   17854


ROC Analysis in
Index:   SRMRB
Classification:  CvM1
Estimation Method:   MLR
Level-2 Sample Size:     30
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS      AUC
746           CvM1 SRMRB       MLR         30          5 0.520452
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
746   0.5119774    0.5187178 0.2077856   0.6954315   0.3493627  5086
    Num-Mis
746    4739


ROC Analysis in
Index:   SRMRB
Classification:  CvM1
Estimation Method:   MLR
Level-2 Sample Size:     30
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
747           CvM1 SRMRB       MLR         30         10 0.5154774
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
747   0.5156946    0.5129154 0.2018534   0.8346682   0.2137293  6138
    Num-Mis
747    5832


ROC Analysis in
Index:   SRMRB
Classification:  CvM1
Estimation Method:   MLR
Level-2 Sample Size:     30
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS      AUC
748           CvM1 SRMRB       MLR         30         30 0.503517
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
748   0.5133129    0.4972264 0.1636746   0.7606952   0.2858847  7554
    Num-Mis
748    7283


ROC Analysis in
Index:   SRMRB
Classification:  CvM1
Estimation Method:   MLR
Level-2 Sample Size:     50
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
749           CvM1 SRMRB       MLR         50        ALL 0.5025324
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
749   0.5162119    0.4954694 0.1506943   0.8051885   0.2349803 21335
    Num-Mis
749   20417


ROC Analysis in
Index:   SRMRB
Classification:  CvM1
Estimation Method:   MLR
Level-2 Sample Size:     50
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
750           CvM1 SRMRB       MLR         50          5 0.5091839
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
750   0.5146512     0.505339 0.1982304   0.8643156   0.1825787  6083
    Num-Mis
750    5724


ROC Analysis in
Index:   SRMRB
Classification:  CvM1
Estimation Method:   MLR
Level-2 Sample Size:     50
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1

Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
751           CvM1 SRMRB       MLR         50         10 0.4967197
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
751   0.5202269    0.4996083 0.09555262   0.9133034   0.1328027  7170
    Num-Mis
751    6790


ROC Analysis in
Index:   SRMRB
Classification:  CvM1
Estimation Method:   MLR
Level-2 Sample Size:     50
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
752           CvM1 SRMRB       MLR         50         30 0.5264439
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
752    0.542043     0.534538 0.08706109   0.8895487   0.1985213  8082
    Num-Mis
752    7903


ROC Analysis in
Index:   SRMRB
Classification:  CvM1
Estimation Method:   MLR
Level-2 Sample Size:     100
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS      AUC
753           CvM1 SRMRB       MLR        100        ALL 0.530241
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
753   0.5516749    0.5400581 0.06929902   0.9169806    0.186299 24013
    Num-Mis
753   22901


ROC Analysis in
Index:   SRMRB
Classification:  CvM1
Estimation Method:   MLR
Level-2 Sample Size:     100
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
754           CvM1 SRMRB       MLR        100          5 0.5000896
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
754   0.5229357    0.5041242 0.08795011   0.8012249   0.2540339  7258
    Num-Mis
754    6637


ROC Analysis in
Index:   SRMRB
Classification:  CvM1
Estimation Method:   MLR
Level-2 Sample Size:     100
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
755           CvM1 SRMRB       MLR        100         10 0.5366059
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
755   0.5526849    0.5431489 0.07391423     0.85763   0.2652174  8237
    Num-Mis
755    7823


ROC Analysis in
Index:   SRMRB
Classification:  CvM1
Estimation Method:   MLR
Level-2 Sample Size:     100
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS      AUC
756           CvM1 SRMRB       MLR        100         30 0.597332
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
756   0.5952772    0.6137458 0.06636612   0.9188406   0.2714135  8518
    Num-Mis
756    8441


ROC Analysis in
Index:   SRMRB
Classification:  CvM1
Estimation Method:   MLR
Level-2 Sample Size:     200
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
757           CvM1 SRMRB       MLR        200        ALL 0.6002532
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
757   0.6017885    0.6084729 0.05690368   0.8526616   0.3497962 25696
    Num-Mis
757   24475


ROC Analysis in
Index:   SRMRB
Classification:  CvM1
Estimation Method:   MLR
Level-2 Sample Size:     200
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS      AUC
758           CvM1 SRMRB       MLR        200          5 0.540112
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
758   0.5642004    0.5456028 0.06683294   0.7699024   0.3779783  8295
    Num-Mis
758    7472


ROC Analysis in
Index:   SRMRB
Classification:  CvM1
Estimation Method:   MLR
Level-2 Sample Size:     200
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
759           CvM1 SRMRB       MLR        200         10 0.6063594
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
759   0.6134587     0.625466 0.05683368     0.85886   0.3693319  8650
    Num-Mis
759    8340


ROC Analysis in
Index:   SRMRB
Classification:  CvM1
Estimation Method:   MLR
Level-2 Sample Size:     200
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
760           CvM1 SRMRB       MLR        200         30 0.7149506
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
760   0.6647205    0.7323676 0.05434486   0.8541226    0.468216  8751
    Num-Mis
760    8663


ROC Analysis in
Index:   SRMRB
Classification:  CvM1
Estimation Method:   ULSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
761           CvM1 SRMRB     ULSMV        ALL        ALL 0.5417012
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
761   0.5436814    0.5561858 0.04164053   0.9423023   0.1430545 77713
    Num-Mis
761   71584


ROC Analysis in
Index:   SRMRB
Classification:  CvM1
Estimation Method:   ULSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
762           CvM1 SRMRB     ULSMV        ALL          5 0.5171083
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
762   0.5214175    0.5237293 0.05121389   0.8927801   0.1540003 21448
    Num-Mis
762   19987


ROC Analysis in
Index:   SRMRB
Classification:  CvM1
Estimation Method:   ULSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
763           CvM1 SRMRB     ULSMV        ALL         10 0.5416102
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
763   0.5438388    0.5557221 0.04171516   0.9499653   0.1407137 26416
    Num-Mis
763   24300


ROC Analysis in
Index:   SRMRB
Classification:  CvM1
Estimation Method:   ULSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
764           CvM1 SRMRB     ULSMV        ALL         30 0.5622975
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
764   0.5627041    0.5795686 0.03771459   0.9602706    0.163827 29849
    Num-Mis
764   27297


ROC Analysis in
Index:   SRMRB
Classification:  CvM1
Estimation Method:   ULSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
765           CvM1 SRMRB     ULSMV         30        ALL 0.5315213
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
765   0.5313327    0.5406677 0.09320872   0.8339024   0.2333911 14727
    Num-Mis
765   13635


ROC Analysis in
Index:   SRMRB
Classification:  CvM1
Estimation Method:   ULSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
766           CvM1 SRMRB     ULSMV         30          5 0.5082743
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
766   0.5008724    0.5107904 0.1719766   0.2161616    0.810084  3399
    Num-Mis
766    3200


ROC Analysis in
Index:   SRMRB
Classification:  CvM1
Estimation Method:   ULSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
767           CvM1 SRMRB     ULSMV         30         10 0.5247072
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
767   0.5169834    0.5260623 0.1060657   0.6809557   0.3779849  4926
    Num-Mis
767    4632


ROC Analysis in
Index:   SRMRB
Classification:  CvM1
Estimation Method:   ULSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
768           CvM1 SRMRB     ULSMV         30         30 0.5508112
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
768   0.5537925    0.5590625 0.09316442   0.7528156   0.3591426  6402
    Num-Mis
768    5803


ROC Analysis in
Index:   SRMRB
Classification:  CvM1
Estimation Method:   ULSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
769           CvM1 SRMRB     ULSMV         50        ALL 0.5414929
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
769   0.5458679     0.552896 0.07031511   0.8749747    0.217275 17918
    Num-Mis
769   16505


ROC Analysis in
Index:   SRMRB
Classification:  CvM1
Estimation Method:   ULSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
770           CvM1 SRMRB     ULSMV         50          5 0.5041225
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
770   0.5066915    0.5051939 0.07591025    0.935123  0.08526646  4573
    Num-Mis
770    4319


ROC Analysis in
Index:   SRMRB
Classification:  CvM1
Estimation Method:   ULSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
771           CvM1 SRMRB     ULSMV         50         10 0.5315198
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
771    0.529736    0.5342565 0.08573584   0.6544343   0.4231857  6038
    Num-Mis
771    5551


ROC Analysis in
Index:   SRMRB
Classification:  CvM1
Estimation Method:   ULSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
772           CvM1 SRMRB     ULSMV         50         30 0.5781443
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
772   0.5831127    0.5914165 0.06332341   0.8973055   0.2675256  7307
    Num-Mis
772    6635


ROC Analysis in
Index:   SRMRB
Classification:  CvM1
Estimation Method:   ULSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS      AUC
773           CvM1 SRMRB     ULSMV        100        ALL 0.565424
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
773   0.5729042    0.5808203 0.05224289   0.8803022   0.2615221 21277
    Num-Mis
773   19571


ROC Analysis in
Index:   SRMRB
Classification:  CvM1
Estimation Method:   ULSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
774           CvM1 SRMRB     ULSMV        100          5 0.5195858
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
774   0.5215237    0.5215216 0.06386909   0.7978533   0.2615607  5969
    Num-Mis
774    5570


ROC Analysis in
Index:   SRMRB
Classification:  CvM1
Estimation Method:   ULSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
775           CvM1 SRMRB     ULSMV        100         10 0.5734787
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
775   0.5775131    0.5819703 0.05424573   0.8659898   0.3073692  7390
    Num-Mis
775    6741


ROC Analysis in
Index:   SRMRB
Classification:  CvM1
Estimation Method:   ULSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
776           CvM1 SRMRB     ULSMV        100         30 0.6123269
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
776      0.6262    0.6358769 0.04712268   0.9149816   0.3306281  7918
    Num-Mis
776    7260


ROC Analysis in
Index:   SRMRB
Classification:  CvM1
Estimation Method:   ULSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
777           CvM1 SRMRB     ULSMV        200        ALL 0.5946685
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
777     0.60113    0.6135338 0.03796987   0.9243542   0.2717882 23791
    Num-Mis
777   21873


ROC Analysis in
Index:   SRMRB
Classification:  CvM1
Estimation Method:   ULSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
778           CvM1 SRMRB     ULSMV        200          5 0.5485897
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
778   0.5479988     0.552586 0.04604062   0.8537432   0.2631379  7507
    Num-Mis
778    6898


ROC Analysis in
Index:   SRMRB
Classification:  CvM1
Estimation Method:   ULSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
779           CvM1 SRMRB     ULSMV        200         10 0.6151656
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
779   0.6268191    0.6331625 0.04076332   0.8696845   0.3901845  8062
    Num-Mis
779    7376


ROC Analysis in
Index:   SRMRB
Classification:  CvM1
Estimation Method:   ULSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
780           CvM1 SRMRB     ULSMV        200         30 0.6568109
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
780   0.6681678    0.6884561 0.03551744   0.9304348   0.3954841  8222
    Num-Mis
780    7599


ROC Analysis in
Index:   SRMRB
Classification:  CvM1
Estimation Method:   WLSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
781           CvM1 SRMRB     WLSMV        ALL        ALL 0.5508515
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
781   0.5465875     0.563222 0.05181384   0.8769188   0.2162308 74425
    Num-Mis
781   66637


ROC Analysis in
Index:   SRMRB
Classification:  CvM1
Estimation Method:   WLSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS      AUC
782           CvM1 SRMRB     WLSMV        ALL          5 0.527581
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
782   0.5263552    0.5338517 0.05511741   0.9084804   0.1482645 20313
    Num-Mis
782   18523


ROC Analysis in
Index:   SRMRB
Classification:  CvM1
Estimation Method:   WLSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
783           CvM1 SRMRB     WLSMV        ALL         10 0.5515587
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
783   0.5453766    0.5640957 0.04930044   0.9192376   0.1716562 25257
    Num-Mis
783   22664


ROC Analysis in
Index:   SRMRB
Classification:  CvM1
Estimation Method:   WLSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
784           CvM1 SRMRB     WLSMV        ALL         30 0.5658735
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
784   0.5627329    0.5811136 0.04899089   0.8637035    0.257738 28855
    Num-Mis
784   25450


ROC Analysis in
Index:   SRMRB
Classification:  CvM1
Estimation Method:   WLSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
785           CvM1 SRMRB     WLSMV         30        ALL 0.5304839
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
785   0.5261644    0.5364067 0.1091351   0.8418398   0.2181034 13548
    Num-Mis
785   12278


ROC Analysis in
Index:   SRMRB
Classification:  CvM1
Estimation Method:   WLSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1

Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in auc.roc(fit_roc[[key]], partial.auc = c(1, 0.8),
partial.auc.focus = "sp", : Partial AUC correction not defined for ROC
curves below the diagonal.

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
786           CvM1 SRMRB     WLSMV         30          5 0.5026628
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
786          NA     0.503243 0.1427754    0.420739   0.6041865  3099
    Num-Mis
786    2886


ROC Analysis in
Index:   SRMRB
Classification:  CvM1
Estimation Method:   WLSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1

Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
787           CvM1 SRMRB     WLSMV         30         10 0.5226729
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
787   0.5128011    0.5231132  0.126682    0.587832   0.4632546  4487
    Num-Mis
787    4131


ROC Analysis in
Index:   SRMRB
Classification:  CvM1
Estimation Method:   WLSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS      AUC
788           CvM1 SRMRB     WLSMV         30         30 0.545364
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
788    0.541438    0.5511129 0.1093062   0.7375367   0.3564165  5962
    Num-Mis
788    5261


ROC Analysis in
Index:   SRMRB
Classification:  CvM1
Estimation Method:   WLSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
789           CvM1 SRMRB     WLSMV         50        ALL 0.5521633
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
789   0.5451948     0.561734 0.08578122   0.8269642   0.2663248 16883
    Num-Mis
789   15246


ROC Analysis in
Index:   SRMRB
Classification:  CvM1
Estimation Method:   WLSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
790           CvM1 SRMRB     WLSMV         50          5 0.5139314
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
790   0.5067122    0.5141737  0.110752   0.5696087   0.4624113  4162
    Num-Mis
790    3851


ROC Analysis in
Index:   SRMRB
Classification:  CvM1
Estimation Method:   WLSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
791           CvM1 SRMRB     WLSMV         50         10 0.5380099
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
791   0.5278596    0.5415944 0.09575503   0.6724019   0.4088844  5746
    Num-Mis
791    5218


ROC Analysis in
Index:   SRMRB
Classification:  CvM1
Estimation Method:   WLSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
792           CvM1 SRMRB     WLSMV         50         30 0.5921415
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
792   0.5766879    0.6009749 0.08023993   0.8422637   0.3224888  6975
    Num-Mis
792    6177


ROC Analysis in
Index:   SRMRB
Classification:  CvM1
Estimation Method:   WLSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
793           CvM1 SRMRB     WLSMV        100        ALL 0.5820353
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
793   0.5737952    0.5926604 0.06007554   0.8831325   0.2673611 20592
    Num-Mis
793   18372


ROC Analysis in
Index:   SRMRB
Classification:  CvM1
Estimation Method:   WLSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
794           CvM1 SRMRB     WLSMV        100          5 0.5260626
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
794    0.521819    0.5262304 0.07377954   0.7592968   0.3039117  5756
    Num-Mis
794    5241


ROC Analysis in
Index:   SRMRB
Classification:  CvM1
Estimation Method:   WLSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
795           CvM1 SRMRB     WLSMV        100         10 0.5864478
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
795   0.5705647    0.5893715 0.06626404   0.7571942   0.4067934  7084
    Num-Mis
795    6279


ROC Analysis in
Index:   SRMRB
Classification:  CvM1
Estimation Method:   WLSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
796           CvM1 SRMRB     WLSMV        100         30 0.6413418
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
796   0.6229768    0.6573908 0.05827554   0.8521549    0.392243  7752
    Num-Mis
796    6852


ROC Analysis in
Index:   SRMRB
Classification:  CvM1
Estimation Method:   WLSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
797           CvM1 SRMRB     WLSMV        200        ALL 0.6263334
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
797   0.6025843    0.6366826 0.04831393   0.7544547   0.4548531 23402
    Num-Mis
797   20741


ROC Analysis in
Index:   SRMRB
Classification:  CvM1
Estimation Method:   WLSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
798           CvM1 SRMRB     WLSMV        200          5 0.5696473
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
798   0.5563124    0.5741081 0.05184086   0.8512535    0.285546  7296
    Num-Mis
798    6545


ROC Analysis in
Index:   SRMRB
Classification:  CvM1
Estimation Method:   WLSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
799           CvM1 SRMRB     WLSMV        200         10 0.6408588
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
799   0.6181333    0.6511216 0.04748123   0.8139779    0.437834  7940
    Num-Mis
799    7036


ROC Analysis in
Index:   SRMRB
Classification:  CvM1
Estimation Method:   WLSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: the condition has
length > 1 and only the first element will be used

Warning in if (is.na(fit_roc_smooth[[key]]) == T) {: An upcoming version of
pROC will set the 'transpose' argument to FALSE by default. Set transpose
= TRUE explicitly to keep the current behavior, or transpose = FALSE to
adopt the new one and silence this warning. Type help(coords_transpose) for
additional information.

Version	Author	Date
982c8f1	noah-padgett	2019-05-18



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
800           CvM1 SRMRB     WLSMV        200         30 0.7071503
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
800   0.6603924    0.7252682 0.04290828   0.8369906   0.4794521  8166
    Num-Mis
800    7160

kable(roc_summary[401:800, ], format = 'html', digits=3) %>%
  kable_styling(full_width = T)

	Classification	Index	Estimator	Level-2 SS	Level-1 SS	AUC	partial-AUC	Smoothed-AUC	Threshold	Specificity	Sensitivity	Num-C	Num-Mis
401	CvM1	CFI	ALL	ALL	ALL	0.894	0.790	0.901	0.971	0.868	0.883	241960	223868
402	CvM1	CFI	ALL	ALL	5	0.846	0.769	0.846	0.968	0.868	0.766	68483	63082
403	CvM1	CFI	ALL	ALL	10	0.927	0.855	0.922	0.968	0.902	0.889	81868	75749
404	CvM1	CFI	ALL	ALL	30	0.908	0.745	0.912	0.976	0.836	0.976	91609	85037
405	CvM1	CFI	ALL	30	ALL	0.739	0.572	0.770	0.963	0.692	0.761	47053	43767
406	CvM1	CFI	ALL	30	5	0.661	0.561	0.666	0.921	0.619	0.633	11584	10825
407	CvM1	CFI	ALL	30	10	0.773	0.612	0.787	0.952	0.675	0.769	15551	14595
408	CvM1	CFI	ALL	30	30	0.783	0.558	0.742	0.965	0.627	0.969	19918	18347
409	CvM1	CFI	ALL	50	ALL	0.851	0.702	0.867	0.970	0.808	0.826	56136	52168
410	CvM1	CFI	ALL	50	5	0.770	0.636	0.775	0.961	0.763	0.654	14818	13894
411	CvM1	CFI	ALL	50	10	0.905	0.795	0.903	0.959	0.817	0.887	18954	17559
412	CvM1	CFI	ALL	50	30	0.878	0.663	0.874	0.980	0.784	0.981	22364	20715
413	CvM1	CFI	ALL	100	ALL	0.950	0.878	0.950	0.971	0.909	0.933	65882	60844
414	CvM1	CFI	ALL	100	5	0.926	0.842	0.923	0.967	0.867	0.847	18983	17448
415	CvM1	CFI	ALL	100	10	0.982	0.961	0.973	0.971	0.956	0.959	22711	20843
416	CvM1	CFI	ALL	100	30	0.938	0.827	0.930	0.982	0.894	0.995	24188	22553
417	CvM1	CFI	ALL	200	ALL	0.997	0.993	0.994	0.975	0.982	0.979	72889	67089
418	CvM1	CFI	ALL	200	5	0.993	0.985	0.989	0.971	0.970	0.967	23098	20915
419	CvM1	CFI	ALL	200	10	0.999	0.998	0.999	0.970	0.992	0.990	24652	22752
420	CvM1	CFI	ALL	200	30	0.998	0.995	0.998	0.987	0.991	0.992	25139	23422
421	CvM1	CFI	MLR	ALL	ALL	0.932	0.922	0.919	0.956	0.948	0.873	89822	85647
422	CvM1	CFI	MLR	ALL	5	0.832	0.799	0.832	0.963	0.907	0.704	26722	24572
423	CvM1	CFI	MLR	ALL	10	0.947	0.929	0.940	0.956	0.955	0.866	30195	28785
424	CvM1	CFI	MLR	ALL	30	1.000	0.999	1.000	0.956	0.995	0.993	32905	32290
425	CvM1	CFI	MLR	30	ALL	0.829	0.798	0.822	0.944	0.918	0.684	18778	17854
426	CvM1	CFI	MLR	30	5	0.699	0.597	0.698	0.889	0.722	0.582	5086	4739
427	CvM1	CFI	MLR	30	10	0.860	0.751	0.858	0.919	0.751	0.811	6138	5832
428	CvM1	CFI	MLR	30	30	0.998	0.995	0.998	0.949	0.973	0.987	7554	7283
429	CvM1	CFI	MLR	50	ALL	0.926	0.882	0.914	0.955	0.921	0.842	21335	20417
430	CvM1	CFI	MLR	50	5	0.795	0.669	0.795	0.929	0.681	0.762	6083	5724
431	CvM1	CFI	MLR	50	10	0.961	0.910	0.959	0.953	0.907	0.886	7170	6790
432	CvM1	CFI	MLR	50	30	1.000	1.000	1.000	0.966	0.998	1.000	8082	7903
433	CvM1	CFI	MLR	100	ALL	0.990	0.977	0.984	0.962	0.959	0.960	24013	22901
434	CvM1	CFI	MLR	100	5	0.946	0.877	0.946	0.959	0.847	0.903	7258	6637
435	CvM1	CFI	MLR	100	10	0.999	0.998	0.999	0.966	0.988	0.991	8237	7823
436	CvM1	CFI	MLR	100	30	1.000	1.000	NA	0.967	1.000	1.000	8518	8441
437	CvM1	CFI	MLR	200	ALL	1.000	0.999	1.000	0.969	0.994	0.995	25696	24475
438	CvM1	CFI	MLR	200	5	0.998	0.995	0.998	0.971	0.985	0.981	8295	7472
439	CvM1	CFI	MLR	200	10	1.000	1.000	NA	0.968	1.000	1.000	8650	8340
440	CvM1	CFI	MLR	200	30	1.000	1.000	NA	0.969	1.000	1.000	8751	8663
441	CvM1	CFI	ULSMV	ALL	ALL	0.822	0.620	0.871	0.977	0.750	0.874	77713	71584
442	CvM1	CFI	ULSMV	ALL	5	0.819	0.697	0.835	0.967	0.789	0.793	21448	19987
443	CvM1	CFI	ULSMV	ALL	10	0.873	0.723	0.892	0.970	0.810	0.888	26416	24300
444	CvM1	CFI	ULSMV	ALL	30	0.792	0.572	0.793	0.987	0.643	0.975	29849	27297
445	CvM1	CFI	ULSMV	30	ALL	0.649	0.529	0.751	0.979	0.482	0.830	14727	13635
446	CvM1	CFI	ULSMV	30	5	0.627	0.536	0.649	0.967	0.606	0.622	3399	3200
447	CvM1	CFI	ULSMV	30	10	0.691	0.543	0.760	0.978	0.559	0.795	4926	4632
448	CvM1	CFI	ULSMV	30	30	0.631	0.519	0.515	0.987	0.337	0.976	6402	5803
449	CvM1	CFI	ULSMV	50	ALL	0.735	0.562	0.799	0.972	0.613	0.834	17918	16505
450	CvM1	CFI	ULSMV	50	5	0.713	0.586	0.727	0.962	0.667	0.670	4573	4319
451	CvM1	CFI	ULSMV	50	10	0.813	0.644	0.837	0.971	0.748	0.803	6038	5551
452	CvM1	CFI	ULSMV	50	30	0.701	0.535	0.674	0.988	0.482	0.977	7307	6635
453	CvM1	CFI	ULSMV	100	ALL	0.875	0.711	0.901	0.971	0.781	0.905	21277	19571
454	CvM1	CFI	ULSMV	100	5	0.880	0.768	0.884	0.967	0.809	0.813	5969	5570
455	CvM1	CFI	ULSMV	100	10	0.946	0.885	0.943	0.970	0.885	0.908	7390	6741
456	CvM1	CFI	ULSMV	100	30	0.823	0.596	0.789	0.984	0.674	0.982	7918	7260
457	CvM1	CFI	ULSMV	200	ALL	0.989	0.973	0.986	0.975	0.954	0.956	23791	21873
458	CvM1	CFI	ULSMV	200	5	0.983	0.961	0.979	0.968	0.934	0.944	7507	6898
459	CvM1	CFI	ULSMV	200	10	0.996	0.991	0.996	0.970	0.977	0.971	8062	7376
460	CvM1	CFI	ULSMV	200	30	0.993	0.981	0.993	0.987	0.972	0.975	8222	7599
461	CvM1	CFI	WLSMV	ALL	ALL	0.945	0.877	0.947	0.977	0.906	0.923	74425	66637
462	CvM1	CFI	WLSMV	ALL	5	0.910	0.831	0.906	0.970	0.882	0.850	20313	18523
463	CvM1	CFI	WLSMV	ALL	10	0.977	0.943	0.970	0.973	0.934	0.953	25257	22664
464	CvM1	CFI	WLSMV	ALL	30	0.950	0.862	0.939	0.988	0.898	0.992	28855	25450
465	CvM1	CFI	WLSMV	30	ALL	0.804	0.632	0.844	0.983	0.732	0.811	13548	12278
466	CvM1	CFI	WLSMV	30	5	0.734	0.600	0.738	0.930	0.513	0.839	3099	2886
467	CvM1	CFI	WLSMV	30	10	0.886	0.765	0.894	0.981	0.835	0.804	4487	4131
468	CvM1	CFI	WLSMV	30	30	0.801	0.584	0.723	0.986	0.612	0.979	5962	5261
469	CvM1	CFI	WLSMV	50	ALL	0.926	0.851	0.930	0.980	0.884	0.849	16883	15246
470	CvM1	CFI	WLSMV	50	5	0.836	0.698	0.840	0.964	0.767	0.769	4162	3851
471	CvM1	CFI	WLSMV	50	10	0.972	0.930	0.971	0.972	0.914	0.916	5746	5218
472	CvM1	CFI	WLSMV	50	30	0.972	0.922	0.969	0.991	0.920	0.977	6975	6177
473	CvM1	CFI	WLSMV	100	ALL	0.992	0.981	0.988	0.972	0.965	0.965	20592	18372
474	CvM1	CFI	WLSMV	100	5	0.960	0.905	0.960	0.969	0.890	0.904	5756	5241
475	CvM1	CFI	WLSMV	100	10	0.999	0.998	0.999	0.973	0.987	0.994	7084	6279
476	CvM1	CFI	WLSMV	100	30	1.000	1.000	NA	0.984	1.000	1.000	7752	6852
477	CvM1	CFI	WLSMV	200	ALL	1.000	1.000	1.000	0.971	0.994	0.996	23402	20741
478	CvM1	CFI	WLSMV	200	5	0.999	0.997	0.999	0.971	0.983	0.988	7296	6545
479	CvM1	CFI	WLSMV	200	10	1.000	1.000	NA	0.977	1.000	1.000	7940	7036
480	CvM1	CFI	WLSMV	200	30	1.000	1.000	NA	0.979	1.000	1.000	8166	7160
481	CvM1	TLI	ALL	ALL	ALL	0.894	0.790	0.901	0.965	0.868	0.883	241960	223868
482	CvM1	TLI	ALL	ALL	5	0.846	0.769	0.846	0.962	0.868	0.766	68483	63082
483	CvM1	TLI	ALL	ALL	10	0.927	0.855	0.922	0.961	0.902	0.889	81868	75749
484	CvM1	TLI	ALL	ALL	30	0.908	0.745	0.912	0.972	0.836	0.976	91609	85037
485	CvM1	TLI	ALL	30	ALL	0.739	0.572	0.770	0.955	0.692	0.761	47053	43767
486	CvM1	TLI	ALL	30	5	0.661	0.561	0.666	0.906	0.619	0.633	11584	10825
487	CvM1	TLI	ALL	30	10	0.773	0.612	0.787	0.942	0.675	0.769	15551	14595
488	CvM1	TLI	ALL	30	30	0.783	0.558	0.742	0.958	0.627	0.969	19918	18347
489	CvM1	TLI	ALL	50	ALL	0.851	0.702	0.867	0.964	0.808	0.826	56136	52168
490	CvM1	TLI	ALL	50	5	0.770	0.636	0.775	0.953	0.763	0.654	14818	13894
491	CvM1	TLI	ALL	50	10	0.905	0.795	0.903	0.951	0.817	0.887	18954	17559
492	CvM1	TLI	ALL	50	30	0.878	0.663	0.874	0.976	0.784	0.981	22364	20715
493	CvM1	TLI	ALL	100	ALL	0.950	0.878	0.950	0.965	0.909	0.933	65882	60844
494	CvM1	TLI	ALL	100	5	0.926	0.842	0.923	0.960	0.867	0.847	18983	17448
495	CvM1	TLI	ALL	100	10	0.982	0.961	0.973	0.965	0.956	0.959	22711	20843
496	CvM1	TLI	ALL	100	30	0.938	0.827	0.930	0.978	0.894	0.995	24188	22553
497	CvM1	TLI	ALL	200	ALL	0.997	0.993	0.994	0.970	0.982	0.979	72889	67089
498	CvM1	TLI	ALL	200	5	0.993	0.985	0.989	0.965	0.970	0.967	23098	20915
499	CvM1	TLI	ALL	200	10	0.999	0.998	0.999	0.964	0.992	0.990	24652	22752
500	CvM1	TLI	ALL	200	30	0.998	0.995	0.998	0.985	0.991	0.992	25139	23422
501	CvM1	TLI	MLR	ALL	ALL	0.932	0.922	0.919	0.948	0.948	0.873	89822	85647
502	CvM1	TLI	MLR	ALL	5	0.832	0.799	0.832	0.955	0.907	0.704	26722	24572
503	CvM1	TLI	MLR	ALL	10	0.947	0.929	0.940	0.947	0.955	0.866	30195	28785
504	CvM1	TLI	MLR	ALL	30	1.000	0.999	1.000	0.947	0.995	0.993	32905	32290
505	CvM1	TLI	MLR	30	ALL	0.829	0.798	0.822	0.932	0.918	0.684	18778	17854
506	CvM1	TLI	MLR	30	5	0.699	0.597	0.698	0.867	0.722	0.582	5086	4739
507	CvM1	TLI	MLR	30	10	0.860	0.751	0.858	0.903	0.751	0.811	6138	5832
508	CvM1	TLI	MLR	30	30	0.998	0.995	0.998	0.939	0.973	0.987	7554	7283
509	CvM1	TLI	MLR	50	ALL	0.926	0.882	0.914	0.945	0.921	0.842	21335	20417
510	CvM1	TLI	MLR	50	5	0.795	0.669	0.795	0.915	0.681	0.762	6083	5724
511	CvM1	TLI	MLR	50	10	0.961	0.910	0.959	0.944	0.907	0.886	7170	6790
512	CvM1	TLI	MLR	50	30	1.000	1.000	1.000	0.959	0.998	1.000	8082	7903
513	CvM1	TLI	MLR	100	ALL	0.990	0.977	0.984	0.954	0.959	0.960	24013	22901
514	CvM1	TLI	MLR	100	5	0.946	0.877	0.946	0.951	0.847	0.903	7258	6637
515	CvM1	TLI	MLR	100	10	0.999	0.998	0.999	0.959	0.988	0.991	8237	7823
516	CvM1	TLI	MLR	100	30	1.000	1.000	NA	0.961	1.000	1.000	8518	8441
517	CvM1	TLI	MLR	200	ALL	1.000	0.999	1.000	0.962	0.994	0.995	25696	24475
518	CvM1	TLI	MLR	200	5	0.998	0.995	0.998	0.965	0.985	0.981	8295	7472
519	CvM1	TLI	MLR	200	10	1.000	1.000	NA	0.962	1.000	1.000	8650	8340
520	CvM1	TLI	MLR	200	30	1.000	1.000	NA	0.963	1.000	1.000	8751	8663
521	CvM1	TLI	ULSMV	ALL	ALL	0.822	0.620	0.871	0.973	0.750	0.874	77713	71584
522	CvM1	TLI	ULSMV	ALL	5	0.819	0.697	0.835	0.960	0.789	0.793	21448	19987
523	CvM1	TLI	ULSMV	ALL	10	0.873	0.723	0.892	0.964	0.810	0.888	26416	24300
524	CvM1	TLI	ULSMV	ALL	30	0.792	0.572	0.793	0.985	0.643	0.975	29849	27297
525	CvM1	TLI	ULSMV	30	ALL	0.649	0.529	0.751	0.975	0.482	0.830	14727	13635
526	CvM1	TLI	ULSMV	30	5	0.627	0.536	0.649	0.960	0.606	0.622	3399	3200
527	CvM1	TLI	ULSMV	30	10	0.691	0.543	0.760	0.973	0.559	0.795	4926	4632
528	CvM1	TLI	ULSMV	30	30	0.631	0.519	0.515	0.985	0.337	0.976	6402	5803
529	CvM1	TLI	ULSMV	50	ALL	0.735	0.562	0.799	0.966	0.613	0.834	17918	16505
530	CvM1	TLI	ULSMV	50	5	0.713	0.586	0.727	0.954	0.667	0.670	4573	4319
531	CvM1	TLI	ULSMV	50	10	0.813	0.644	0.837	0.965	0.748	0.803	6038	5551
532	CvM1	TLI	ULSMV	50	30	0.701	0.535	0.674	0.986	0.482	0.977	7307	6635
533	CvM1	TLI	ULSMV	100	ALL	0.875	0.711	0.901	0.965	0.781	0.905	21277	19571
534	CvM1	TLI	ULSMV	100	5	0.880	0.768	0.884	0.960	0.809	0.813	5969	5570
535	CvM1	TLI	ULSMV	100	10	0.946	0.885	0.943	0.964	0.885	0.908	7390	6741
536	CvM1	TLI	ULSMV	100	30	0.823	0.596	0.789	0.980	0.674	0.982	7918	7260
537	CvM1	TLI	ULSMV	200	ALL	0.989	0.973	0.986	0.970	0.954	0.956	23791	21873
538	CvM1	TLI	ULSMV	200	5	0.983	0.961	0.979	0.962	0.934	0.944	7507	6898
539	CvM1	TLI	ULSMV	200	10	0.996	0.991	0.996	0.964	0.977	0.971	8062	7376
540	CvM1	TLI	ULSMV	200	30	0.993	0.981	0.993	0.985	0.972	0.975	8222	7599
541	CvM1	TLI	WLSMV	ALL	ALL	0.945	0.877	0.947	0.972	0.906	0.923	74425	66637
542	CvM1	TLI	WLSMV	ALL	5	0.910	0.831	0.906	0.964	0.882	0.850	20313	18523
543	CvM1	TLI	WLSMV	ALL	10	0.977	0.943	0.970	0.968	0.934	0.953	25257	22664
544	CvM1	TLI	WLSMV	ALL	30	0.950	0.862	0.939	0.985	0.898	0.992	28855	25450
545	CvM1	TLI	WLSMV	30	ALL	0.804	0.632	0.844	0.979	0.732	0.811	13548	12278
546	CvM1	TLI	WLSMV	30	5	0.734	0.600	0.738	0.916	0.513	0.839	3099	2886
547	CvM1	TLI	WLSMV	30	10	0.886	0.765	0.894	0.977	0.835	0.804	4487	4131
548	CvM1	TLI	WLSMV	30	30	0.801	0.584	0.723	0.984	0.612	0.979	5962	5261
549	CvM1	TLI	WLSMV	50	ALL	0.926	0.851	0.930	0.976	0.884	0.849	16883	15246
550	CvM1	TLI	WLSMV	50	5	0.836	0.698	0.840	0.956	0.767	0.769	4162	3851
551	CvM1	TLI	WLSMV	50	10	0.972	0.930	0.971	0.966	0.914	0.916	5746	5218
552	CvM1	TLI	WLSMV	50	30	0.972	0.922	0.969	0.989	0.920	0.977	6975	6177
553	CvM1	TLI	WLSMV	100	ALL	0.992	0.981	0.988	0.966	0.965	0.965	20592	18372
554	CvM1	TLI	WLSMV	100	5	0.960	0.905	0.960	0.962	0.890	0.904	5756	5241
555	CvM1	TLI	WLSMV	100	10	0.999	0.998	0.999	0.967	0.987	0.994	7084	6279
556	CvM1	TLI	WLSMV	100	30	1.000	1.000	NA	0.981	1.000	1.000	7752	6852
557	CvM1	TLI	WLSMV	200	ALL	1.000	1.000	1.000	0.966	0.994	0.996	23402	20741
558	CvM1	TLI	WLSMV	200	5	0.999	0.997	0.999	0.966	0.983	0.988	7296	6545
559	CvM1	TLI	WLSMV	200	10	1.000	1.000	NA	0.972	1.000	1.000	7940	7036
560	CvM1	TLI	WLSMV	200	30	1.000	1.000	NA	0.975	1.000	1.000	8166	7160
561	CvM1	RMSEA	ALL	ALL	ALL	0.878	0.753	0.884	0.017	0.811	0.859	241960	223868
562	CvM1	RMSEA	ALL	ALL	5	0.833	0.749	0.833	0.020	0.824	0.767	68483	63082
563	CvM1	RMSEA	ALL	ALL	10	0.910	0.820	0.907	0.017	0.855	0.861	81868	75749
564	CvM1	RMSEA	ALL	ALL	30	0.891	0.714	0.893	0.014	0.771	0.943	91609	85037
565	CvM1	RMSEA	ALL	30	ALL	0.720	0.572	0.745	0.023	0.632	0.768	47053	43767
566	CvM1	RMSEA	ALL	30	5	0.632	0.559	0.635	0.023	0.703	0.501	11584	10825
567	CvM1	RMSEA	ALL	30	10	0.740	0.605	0.745	0.020	0.695	0.682	15551	14595
568	CvM1	RMSEA	ALL	30	30	0.765	0.558	0.714	0.023	0.571	0.963	19918	18347
569	CvM1	RMSEA	ALL	50	ALL	0.832	0.675	0.845	0.020	0.718	0.839	56136	52168
570	CvM1	RMSEA	ALL	50	5	0.746	0.625	0.749	0.023	0.695	0.681	14818	13894
571	CvM1	RMSEA	ALL	50	10	0.882	0.746	0.883	0.020	0.795	0.828	18954	17559
572	CvM1	RMSEA	ALL	50	30	0.860	0.650	0.835	0.015	0.700	0.959	22364	20715
573	CvM1	RMSEA	ALL	100	ALL	0.936	0.838	0.939	0.017	0.855	0.911	65882	60844
574	CvM1	RMSEA	ALL	100	5	0.913	0.813	0.915	0.021	0.805	0.870	18983	17448
575	CvM1	RMSEA	ALL	100	10	0.972	0.923	0.973	0.017	0.898	0.950	22711	20843
576	CvM1	RMSEA	ALL	100	30	0.928	0.801	0.906	0.012	0.850	0.991	24188	22553
577	CvM1	RMSEA	ALL	200	ALL	0.984	0.955	0.986	0.015	0.927	0.960	72889	67089
578	CvM1	RMSEA	ALL	200	5	0.989	0.969	0.990	0.018	0.938	0.958	23098	20915
579	CvM1	RMSEA	ALL	200	10	0.997	0.992	0.998	0.013	0.970	0.976	24652	22752
580	CvM1	RMSEA	ALL	200	30	0.990	0.973	0.991	0.007	0.930	0.976	25139	23422
581	CvM1	RMSEA	MLR	ALL	ALL	0.918	0.913	0.905	0.025	0.950	0.856	89822	85647
582	CvM1	RMSEA	MLR	ALL	5	0.809	0.788	0.812	0.024	0.895	0.695	26722	24572
583	CvM1	RMSEA	MLR	ALL	10	0.932	0.917	0.928	0.025	0.955	0.843	30195	28785
584	CvM1	RMSEA	MLR	ALL	30	0.999	0.999	0.999	0.025	0.994	0.989	32905	32290
585	CvM1	RMSEA	MLR	30	ALL	0.799	0.779	0.797	0.030	0.903	0.663	18778	17854
586	CvM1	RMSEA	MLR	30	5	0.678	0.585	0.680	0.048	0.652	0.605	5086	4739
587	CvM1	RMSEA	MLR	30	10	0.834	0.723	0.836	0.034	0.799	0.704	6138	5832
588	CvM1	RMSEA	MLR	30	30	0.997	0.991	0.996	0.027	0.967	0.981	7554	7283
589	CvM1	RMSEA	MLR	50	ALL	0.908	0.869	0.897	0.026	0.912	0.823	21335	20417
590	CvM1	RMSEA	MLR	50	5	0.771	0.655	0.772	0.034	0.657	0.752	6083	5724
591	CvM1	RMSEA	MLR	50	10	0.949	0.889	0.949	0.027	0.877	0.877	7170	6790
592	CvM1	RMSEA	MLR	50	30	1.000	1.000	1.000	0.022	0.997	1.000	8082	7903
593	CvM1	RMSEA	MLR	100	ALL	0.988	0.973	0.982	0.024	0.953	0.958	24013	22901
594	CvM1	RMSEA	MLR	100	5	0.937	0.861	0.938	0.024	0.857	0.871	7258	6637
595	CvM1	RMSEA	MLR	100	10	0.999	0.998	0.999	0.022	0.982	0.991	8237	7823
596	CvM1	RMSEA	MLR	100	30	1.000	1.000	NA	0.020	1.000	1.000	8518	8441
597	CvM1	RMSEA	MLR	200	ALL	1.000	0.999	1.000	0.020	0.996	0.993	25696	24475
598	CvM1	RMSEA	MLR	200	5	0.998	0.994	0.998	0.020	0.985	0.979	8295	7472
599	CvM1	RMSEA	MLR	200	10	1.000	1.000	NA	0.020	1.000	1.000	8650	8340
600	CvM1	RMSEA	MLR	200	30	1.000	1.000	NA	0.018	1.000	1.000	8751	8663
601	CvM1	RMSEA	ULSMV	ALL	ALL	0.809	0.619	0.841	0.012	0.677	0.857	77713	71584
602	CvM1	RMSEA	ULSMV	ALL	5	0.827	0.685	0.841	0.015	0.770	0.778	21448	19987
603	CvM1	RMSEA	ULSMV	ALL	10	0.867	0.702	0.878	0.013	0.759	0.870	26416	24300
604	CvM1	RMSEA	ULSMV	ALL	30	0.778	0.572	0.745	0.006	0.587	0.924	29849	27297
605	CvM1	RMSEA	ULSMV	30	ALL	0.651	0.529	0.705	0.010	0.460	0.803	14727	13635
606	CvM1	RMSEA	ULSMV	30	5	0.633	0.536	0.647	0.012	0.629	0.596	3399	3200
607	CvM1	RMSEA	ULSMV	30	10	0.688	0.543	0.693	0.014	0.469	0.838	4926	4632
608	CvM1	RMSEA	ULSMV	30	30	0.375	NA	0.564	-Inf	0.000	1.000	6402	5803
609	CvM1	RMSEA	ULSMV	50	ALL	0.729	0.562	0.755	0.012	0.589	0.800	17918	16505
610	CvM1	RMSEA	ULSMV	50	5	0.712	0.586	0.718	0.016	0.650	0.666	4573	4319
611	CvM1	RMSEA	ULSMV	50	10	0.803	0.635	0.799	0.014	0.668	0.803	6038	5551
612	CvM1	RMSEA	ULSMV	50	30	0.693	0.535	0.584	0.009	0.422	0.957	7307	6635
613	CvM1	RMSEA	ULSMV	100	ALL	0.854	0.687	0.858	0.011	0.765	0.827	21277	19571
614	CvM1	RMSEA	ULSMV	100	5	0.876	0.750	0.878	0.017	0.772	0.826	5969	5570
615	CvM1	RMSEA	ULSMV	100	10	0.933	0.838	0.934	0.014	0.818	0.892	7390	6741
616	CvM1	RMSEA	ULSMV	100	30	0.809	0.596	0.738	0.009	0.594	0.984	7918	7260
617	CvM1	RMSEA	ULSMV	200	ALL	0.958	0.895	0.962	0.013	0.821	0.947	23791	21873
618	CvM1	RMSEA	ULSMV	200	5	0.976	0.939	0.978	0.015	0.926	0.907	7507	6898
619	CvM1	RMSEA	ULSMV	200	10	0.993	0.980	0.994	0.011	0.973	0.935	8062	7376
620	CvM1	RMSEA	ULSMV	200	30	0.976	0.937	0.978	0.006	0.903	0.929	8222	7599
621	CvM1	RMSEA	WLSMV	ALL	ALL	0.940	0.861	0.944	0.017	0.879	0.914	74425	66637
622	CvM1	RMSEA	WLSMV	ALL	5	0.912	0.829	0.909	0.021	0.858	0.870	20313	18523
623	CvM1	RMSEA	WLSMV	ALL	10	0.977	0.940	0.973	0.020	0.921	0.961	25257	22664
624	CvM1	RMSEA	WLSMV	ALL	30	0.947	0.853	0.945	0.011	0.873	0.979	28855	25450
625	CvM1	RMSEA	WLSMV	30	ALL	0.796	0.631	0.831	0.014	0.707	0.794	13548	12278
626	CvM1	RMSEA	WLSMV	30	5	0.732	0.601	0.737	0.021	0.700	0.652	3099	2886
627	CvM1	RMSEA	WLSMV	30	10	0.880	0.757	0.887	0.018	0.754	0.863	4487	4131
628	CvM1	RMSEA	WLSMV	30	30	0.793	0.584	0.728	0.008	0.617	0.904	5962	5261
629	CvM1	RMSEA	WLSMV	50	ALL	0.908	0.817	0.912	0.014	0.850	0.806	16883	15246
630	CvM1	RMSEA	WLSMV	50	5	0.831	0.694	0.835	0.020	0.789	0.724	4162	3851
631	CvM1	RMSEA	WLSMV	50	10	0.970	0.926	0.971	0.020	0.904	0.925	5746	5218
632	CvM1	RMSEA	WLSMV	50	30	0.961	0.894	0.960	0.009	0.878	0.935	6975	6177
633	CvM1	RMSEA	WLSMV	100	ALL	0.988	0.970	0.987	0.017	0.967	0.931	20592	18372
634	CvM1	RMSEA	WLSMV	100	5	0.960	0.903	0.959	0.021	0.880	0.916	5756	5241
635	CvM1	RMSEA	WLSMV	100	10	0.999	0.999	0.999	0.020	0.986	0.994	7084	6279
636	CvM1	RMSEA	WLSMV	100	30	1.000	1.000	1.000	0.012	0.999	0.999	7752	6852
637	CvM1	RMSEA	WLSMV	200	ALL	1.000	1.000	1.000	0.020	0.995	0.996	23402	20741
638	CvM1	RMSEA	WLSMV	200	5	0.999	0.997	0.999	0.020	0.984	0.985	7296	6545
639	CvM1	RMSEA	WLSMV	200	10	1.000	1.000	NA	0.019	1.000	1.000	7940	7036
640	CvM1	RMSEA	WLSMV	200	30	1.000	1.000	NA	0.014	1.000	1.000	8166	7160
641	CvM1	SRMRW	ALL	ALL	ALL	0.873	0.870	0.888	0.038	0.975	0.724	241960	223868
642	CvM1	SRMRW	ALL	ALL	5	0.789	0.771	0.810	0.047	0.920	0.588	68483	63082
643	CvM1	SRMRW	ALL	ALL	10	0.904	0.881	0.912	0.042	0.936	0.770	81868	75749
644	CvM1	SRMRW	ALL	ALL	30	0.978	0.977	0.978	0.035	0.993	0.941	91609	85037
645	CvM1	SRMRW	ALL	30	ALL	0.748	0.726	0.778	0.044	0.921	0.500	47053	43767
646	CvM1	SRMRW	ALL	30	5	0.690	0.621	0.689	0.072	0.803	0.483	11584	10825
647	CvM1	SRMRW	ALL	30	10	0.828	0.757	0.822	0.056	0.802	0.722	15551	14595
648	CvM1	SRMRW	ALL	30	30	0.943	0.938	0.943	0.039	0.967	0.872	19918	18347
649	CvM1	SRMRW	ALL	50	ALL	0.835	0.805	0.853	0.042	0.926	0.639	56136	52168
650	CvM1	SRMRW	ALL	50	5	0.791	0.700	0.792	0.063	0.767	0.671	14818	13894
651	CvM1	SRMRW	ALL	50	10	0.923	0.883	0.916	0.047	0.897	0.834	18954	17559
652	CvM1	SRMRW	ALL	50	30	0.977	0.972	0.978	0.035	0.991	0.927	22364	20715
653	CvM1	SRMRW	ALL	100	ALL	0.948	0.918	0.955	0.039	0.958	0.819	65882	60844
654	CvM1	SRMRW	ALL	100	5	0.932	0.875	0.930	0.050	0.863	0.859	18983	17448
655	CvM1	SRMRW	ALL	100	10	0.988	0.978	0.989	0.038	0.981	0.936	22711	20843
656	CvM1	SRMRW	ALL	100	30	0.997	0.995	0.997	0.035	0.996	0.981	24188	22553
657	CvM1	SRMRW	ALL	200	ALL	0.996	0.991	0.996	0.036	0.991	0.964	72889	67089
658	CvM1	SRMRW	ALL	200	5	0.994	0.985	0.994	0.040	0.978	0.950	23098	20915
659	CvM1	SRMRW	ALL	200	10	1.000	0.999	1.000	0.034	0.997	0.989	24652	22752
660	CvM1	SRMRW	ALL	200	30	1.000	1.000	1.000	0.034	1.000	1.000	25139	23422
661	CvM1	SRMRW	MLR	ALL	ALL	0.887	0.892	0.902	0.036	0.979	0.776	89822	85647
662	CvM1	SRMRW	MLR	ALL	5	0.808	0.784	0.838	0.042	0.935	0.599	26722	24572
663	CvM1	SRMRW	MLR	ALL	10	0.951	0.919	0.960	0.038	0.955	0.829	30195	28785
664	CvM1	SRMRW	MLR	ALL	30	1.000	1.000	1.000	0.031	1.000	0.998	32905	32290
665	CvM1	SRMRW	MLR	30	ALL	0.769	0.755	0.798	0.042	0.907	0.567	18778	17854
666	CvM1	SRMRW	MLR	30	5	0.813	0.681	0.812	0.071	0.660	0.820	5086	4739
667	CvM1	SRMRW	MLR	30	10	0.955	0.893	0.949	0.048	0.903	0.870	6138	5832
668	CvM1	SRMRW	MLR	30	30	1.000	1.000	1.000	0.033	0.996	0.999	7554	7283
669	CvM1	SRMRW	MLR	50	ALL	0.861	0.840	0.889	0.038	0.960	0.692	21335	20417
670	CvM1	SRMRW	MLR	50	5	0.905	0.802	0.907	0.056	0.796	0.855	6083	5724
671	CvM1	SRMRW	MLR	50	10	0.993	0.981	0.992	0.041	0.952	0.965	7170	6790
672	CvM1	SRMRW	MLR	50	30	1.000	1.000	NA	0.029	1.000	1.000	8082	7903
673	CvM1	SRMRW	MLR	100	ALL	0.981	0.961	0.984	0.037	0.967	0.905	24013	22901
674	CvM1	SRMRW	MLR	100	5	0.987	0.966	0.987	0.043	0.939	0.956	7258	6637
675	CvM1	SRMRW	MLR	100	10	1.000	1.000	1.000	0.032	0.999	1.000	8237	7823
676	CvM1	SRMRW	MLR	100	30	1.000	1.000	NA	0.025	1.000	1.000	8518	8441
677	CvM1	SRMRW	MLR	200	ALL	1.000	1.000	1.000	0.033	0.998	0.998	25696	24475
678	CvM1	SRMRW	MLR	200	5	1.000	1.000	1.000	0.035	0.994	0.999	8295	7472
679	CvM1	SRMRW	MLR	200	10	1.000	1.000	NA	0.027	1.000	1.000	8650	8340
680	CvM1	SRMRW	MLR	200	30	1.000	1.000	NA	0.022	1.000	1.000	8751	8663
681	CvM1	SRMRW	ULSMV	ALL	ALL	0.854	0.864	0.871	0.045	0.976	0.723	77713	71584
682	CvM1	SRMRW	ULSMV	ALL	5	0.760	0.760	0.787	0.050	0.951	0.542	21448	19987
683	CvM1	SRMRW	ULSMV	ALL	10	0.873	0.864	0.887	0.047	0.960	0.729	26416	24300
684	CvM1	SRMRW	ULSMV	ALL	30	0.960	0.961	0.962	0.042	0.995	0.904	29849	27297
685	CvM1	SRMRW	ULSMV	30	ALL	0.722	0.706	0.748	0.053	0.922	0.464	14727	13635
686	CvM1	SRMRW	ULSMV	30	5	0.661	0.582	0.659	0.098	0.619	0.619	3399	3200
687	CvM1	SRMRW	ULSMV	30	10	0.797	0.730	0.801	0.069	0.752	0.713	4926	4632
688	CvM1	SRMRW	ULSMV	30	30	0.904	0.887	0.911	0.047	0.963	0.766	6402	5803
689	CvM1	SRMRW	ULSMV	50	ALL	0.817	0.795	0.840	0.048	0.953	0.603	17918	16505
690	CvM1	SRMRW	ULSMV	50	5	0.790	0.688	0.793	0.076	0.730	0.703	4573	4319
691	CvM1	SRMRW	ULSMV	50	10	0.906	0.867	0.913	0.054	0.922	0.770	6038	5551
692	CvM1	SRMRW	ULSMV	50	30	0.968	0.958	0.970	0.044	0.983	0.894	7307	6635
693	CvM1	SRMRW	ULSMV	100	ALL	0.945	0.924	0.954	0.046	0.961	0.839	21277	19571
694	CvM1	SRMRW	ULSMV	100	5	0.931	0.870	0.934	0.057	0.865	0.845	5969	5570
695	CvM1	SRMRW	ULSMV	100	10	0.989	0.978	0.989	0.046	0.972	0.942	7390	6741
696	CvM1	SRMRW	ULSMV	100	30	0.998	0.997	0.998	0.042	0.998	0.984	7918	7260
697	CvM1	SRMRW	ULSMV	200	ALL	0.997	0.995	0.997	0.043	0.992	0.979	23791	21873
698	CvM1	SRMRW	ULSMV	200	5	0.995	0.989	0.996	0.045	0.982	0.954	7507	6898
699	CvM1	SRMRW	ULSMV	200	10	1.000	1.000	1.000	0.041	0.996	0.999	8062	7376
700	CvM1	SRMRW	ULSMV	200	30	1.000	1.000	NA	0.039	1.000	1.000	8222	7599
701	CvM1	SRMRW	WLSMV	ALL	ALL	0.905	0.911	0.915	0.044	0.979	0.815	74425	66637
702	CvM1	SRMRW	WLSMV	ALL	5	0.811	0.802	0.840	0.049	0.954	0.621	20313	18523
703	CvM1	SRMRW	WLSMV	ALL	10	0.944	0.919	0.953	0.045	0.966	0.824	25257	22664
704	CvM1	SRMRW	WLSMV	ALL	30	1.000	1.000	1.000	0.039	0.999	0.997	28855	25450
705	CvM1	SRMRW	WLSMV	30	ALL	0.801	0.786	0.822	0.053	0.896	0.634	13548	12278
706	CvM1	SRMRW	WLSMV	30	5	0.778	0.641	0.775	0.085	0.718	0.714	3099	2886
707	CvM1	SRMRW	WLSMV	30	10	0.949	0.883	0.944	0.061	0.868	0.893	4487	4131
708	CvM1	SRMRW	WLSMV	30	30	1.000	0.999	1.000	0.042	0.993	0.998	5962	5261
709	CvM1	SRMRW	WLSMV	50	ALL	0.877	0.862	0.897	0.047	0.962	0.731	16883	15246
710	CvM1	SRMRW	WLSMV	50	5	0.892	0.787	0.891	0.069	0.797	0.834	4162	3851
711	CvM1	SRMRW	WLSMV	50	10	0.992	0.980	0.992	0.049	0.965	0.954	5746	5218
712	CvM1	SRMRW	WLSMV	50	30	1.000	1.000	NA	0.037	1.000	1.000	6975	6177
713	CvM1	SRMRW	WLSMV	100	ALL	0.982	0.965	0.984	0.045	0.966	0.913	20592	18372
714	CvM1	SRMRW	WLSMV	100	5	0.984	0.957	0.982	0.053	0.905	0.967	5756	5241
715	CvM1	SRMRW	WLSMV	100	10	1.000	1.000	1.000	0.039	0.998	0.998	7084	6279
716	CvM1	SRMRW	WLSMV	100	30	1.000	1.000	NA	0.031	1.000	1.000	7752	6852
717	CvM1	SRMRW	WLSMV	200	ALL	1.000	1.000	1.000	0.040	0.999	0.998	23402	20741
718	CvM1	SRMRW	WLSMV	200	5	1.000	1.000	1.000	0.041	0.996	0.996	7296	6545
719	CvM1	SRMRW	WLSMV	200	10	1.000	1.000	NA	0.034	1.000	1.000	7940	7036
720	CvM1	SRMRW	WLSMV	200	30	1.000	1.000	NA	0.028	1.000	1.000	8166	7160
721	CvM1	SRMRB	ALL	ALL	ALL	0.537	0.541	0.547	0.057	0.839	0.243	241960	223868
722	CvM1	SRMRB	ALL	ALL	5	0.511	0.519	0.516	0.064	0.839	0.205	68483	63082
723	CvM1	SRMRB	ALL	ALL	10	0.536	0.539	0.546	0.057	0.844	0.236	81868	75749
724	CvM1	SRMRB	ALL	ALL	30	0.564	0.559	0.577	0.057	0.792	0.330	91609	85037
725	CvM1	SRMRB	ALL	30	ALL	0.517	0.521	0.522	0.109	0.814	0.232	47053	43767
726	CvM1	SRMRB	ALL	30	5	0.495	NA	0.494	0.109	0.929	0.074	11584	10825
727	CvM1	SRMRB	ALL	30	10	0.511	0.512	0.511	0.127	0.647	0.388	15551	14595
728	CvM1	SRMRB	ALL	30	30	0.537	0.537	0.543	0.103	0.799	0.284	19918	18347
729	CvM1	SRMRB	ALL	50	ALL	0.533	0.534	0.539	0.084	0.839	0.233	56136	52168
730	CvM1	SRMRB	ALL	50	5	0.503	0.504	0.502	0.120	0.535	0.480	14818	13894
731	CvM1	SRMRB	ALL	50	10	0.523	0.524	0.524	0.086	0.838	0.225	18954	17559
732	CvM1	SRMRB	ALL	50	30	0.570	0.560	0.578	0.080	0.815	0.313	22364	20715
733	CvM1	SRMRB	ALL	100	ALL	0.559	0.556	0.568	0.063	0.811	0.302	65882	60844
734	CvM1	SRMRB	ALL	100	5	0.509	0.514	0.509	0.074	0.767	0.270	18983	17448
735	CvM1	SRMRB	ALL	100	10	0.561	0.554	0.565	0.067	0.757	0.365	22711	20843
736	CvM1	SRMRB	ALL	100	30	0.621	0.594	0.634	0.059	0.813	0.375	24188	22553
737	CvM1	SRMRB	ALL	200	ALL	0.601	0.580	0.609	0.049	0.749	0.414	72889	67089
738	CvM1	SRMRB	ALL	200	5	0.540	0.534	0.540	0.061	0.633	0.452	23098	20915
739	CvM1	SRMRB	ALL	200	10	0.611	0.592	0.621	0.047	0.832	0.357	24652	22752
740	CvM1	SRMRB	ALL	200	30	0.682	0.627	0.703	0.043	0.828	0.417	25139	23422
741	CvM1	SRMRB	MLR	ALL	ALL	0.512	0.539	0.526	0.061	0.921	0.157	89822	85647
742	CvM1	SRMRB	MLR	ALL	5	0.502	0.508	0.491	0.147	0.675	0.361	26722	24572
743	CvM1	SRMRB	MLR	ALL	10	0.489	0.510	0.476	0.175	0.902	0.123	30195	28785
744	CvM1	SRMRB	MLR	ALL	30	0.544	0.563	0.562	0.059	0.913	0.211	32905	32290
745	CvM1	SRMRB	MLR	30	ALL	0.518	0.513	0.512	0.177	0.698	0.346	18778	17854
746	CvM1	SRMRB	MLR	30	5	0.520	0.512	0.519	0.208	0.695	0.349	5086	4739
747	CvM1	SRMRB	MLR	30	10	0.515	0.516	0.513	0.202	0.835	0.214	6138	5832
748	CvM1	SRMRB	MLR	30	30	0.504	0.513	0.497	0.164	0.761	0.286	7554	7283
749	CvM1	SRMRB	MLR	50	ALL	0.503	0.516	0.495	0.151	0.805	0.235	21335	20417
750	CvM1	SRMRB	MLR	50	5	0.509	0.515	0.505	0.198	0.864	0.183	6083	5724
751	CvM1	SRMRB	MLR	50	10	0.497	0.520	0.500	0.096	0.913	0.133	7170	6790
752	CvM1	SRMRB	MLR	50	30	0.526	0.542	0.535	0.087	0.890	0.199	8082	7903
753	CvM1	SRMRB	MLR	100	ALL	0.530	0.552	0.540	0.069	0.917	0.186	24013	22901
754	CvM1	SRMRB	MLR	100	5	0.500	0.523	0.504	0.088	0.801	0.254	7258	6637
755	CvM1	SRMRB	MLR	100	10	0.537	0.553	0.543	0.074	0.858	0.265	8237	7823
756	CvM1	SRMRB	MLR	100	30	0.597	0.595	0.614	0.066	0.919	0.271	8518	8441
757	CvM1	SRMRB	MLR	200	ALL	0.600	0.602	0.608	0.057	0.853	0.350	25696	24475
758	CvM1	SRMRB	MLR	200	5	0.540	0.564	0.546	0.067	0.770	0.378	8295	7472
759	CvM1	SRMRB	MLR	200	10	0.606	0.613	0.625	0.057	0.859	0.369	8650	8340
760	CvM1	SRMRB	MLR	200	30	0.715	0.665	0.732	0.054	0.854	0.468	8751	8663
761	CvM1	SRMRB	ULSMV	ALL	ALL	0.542	0.544	0.556	0.042	0.942	0.143	77713	71584
762	CvM1	SRMRB	ULSMV	ALL	5	0.517	0.521	0.524	0.051	0.893	0.154	21448	19987
763	CvM1	SRMRB	ULSMV	ALL	10	0.542	0.544	0.556	0.042	0.950	0.141	26416	24300
764	CvM1	SRMRB	ULSMV	ALL	30	0.562	0.563	0.580	0.038	0.960	0.164	29849	27297
765	CvM1	SRMRB	ULSMV	30	ALL	0.532	0.531	0.541	0.093	0.834	0.233	14727	13635
766	CvM1	SRMRB	ULSMV	30	5	0.508	0.501	0.511	0.172	0.216	0.810	3399	3200
767	CvM1	SRMRB	ULSMV	30	10	0.525	0.517	0.526	0.106	0.681	0.378	4926	4632
768	CvM1	SRMRB	ULSMV	30	30	0.551	0.554	0.559	0.093	0.753	0.359	6402	5803
769	CvM1	SRMRB	ULSMV	50	ALL	0.541	0.546	0.553	0.070	0.875	0.217	17918	16505
770	CvM1	SRMRB	ULSMV	50	5	0.504	0.507	0.505	0.076	0.935	0.085	4573	4319
771	CvM1	SRMRB	ULSMV	50	10	0.532	0.530	0.534	0.086	0.654	0.423	6038	5551
772	CvM1	SRMRB	ULSMV	50	30	0.578	0.583	0.591	0.063	0.897	0.268	7307	6635
773	CvM1	SRMRB	ULSMV	100	ALL	0.565	0.573	0.581	0.052	0.880	0.262	21277	19571
774	CvM1	SRMRB	ULSMV	100	5	0.520	0.522	0.522	0.064	0.798	0.262	5969	5570
775	CvM1	SRMRB	ULSMV	100	10	0.573	0.578	0.582	0.054	0.866	0.307	7390	6741
776	CvM1	SRMRB	ULSMV	100	30	0.612	0.626	0.636	0.047	0.915	0.331	7918	7260
777	CvM1	SRMRB	ULSMV	200	ALL	0.595	0.601	0.614	0.038	0.924	0.272	23791	21873
778	CvM1	SRMRB	ULSMV	200	5	0.549	0.548	0.553	0.046	0.854	0.263	7507	6898
779	CvM1	SRMRB	ULSMV	200	10	0.615	0.627	0.633	0.041	0.870	0.390	8062	7376
780	CvM1	SRMRB	ULSMV	200	30	0.657	0.668	0.688	0.036	0.930	0.395	8222	7599
781	CvM1	SRMRB	WLSMV	ALL	ALL	0.551	0.547	0.563	0.052	0.877	0.216	74425	66637
782	CvM1	SRMRB	WLSMV	ALL	5	0.528	0.526	0.534	0.055	0.908	0.148	20313	18523
783	CvM1	SRMRB	WLSMV	ALL	10	0.552	0.545	0.564	0.049	0.919	0.172	25257	22664
784	CvM1	SRMRB	WLSMV	ALL	30	0.566	0.563	0.581	0.049	0.864	0.258	28855	25450
785	CvM1	SRMRB	WLSMV	30	ALL	0.530	0.526	0.536	0.109	0.842	0.218	13548	12278
786	CvM1	SRMRB	WLSMV	30	5	0.503	NA	0.503	0.143	0.421	0.604	3099	2886
787	CvM1	SRMRB	WLSMV	30	10	0.523	0.513	0.523	0.127	0.588	0.463	4487	4131
788	CvM1	SRMRB	WLSMV	30	30	0.545	0.541	0.551	0.109	0.738	0.356	5962	5261
789	CvM1	SRMRB	WLSMV	50	ALL	0.552	0.545	0.562	0.086	0.827	0.266	16883	15246
790	CvM1	SRMRB	WLSMV	50	5	0.514	0.507	0.514	0.111	0.570	0.462	4162	3851
791	CvM1	SRMRB	WLSMV	50	10	0.538	0.528	0.542	0.096	0.672	0.409	5746	5218
792	CvM1	SRMRB	WLSMV	50	30	0.592	0.577	0.601	0.080	0.842	0.322	6975	6177
793	CvM1	SRMRB	WLSMV	100	ALL	0.582	0.574	0.593	0.060	0.883	0.267	20592	18372
794	CvM1	SRMRB	WLSMV	100	5	0.526	0.522	0.526	0.074	0.759	0.304	5756	5241
795	CvM1	SRMRB	WLSMV	100	10	0.586	0.571	0.589	0.066	0.757	0.407	7084	6279
796	CvM1	SRMRB	WLSMV	100	30	0.641	0.623	0.657	0.058	0.852	0.392	7752	6852
797	CvM1	SRMRB	WLSMV	200	ALL	0.626	0.603	0.637	0.048	0.754	0.455	23402	20741
798	CvM1	SRMRB	WLSMV	200	5	0.570	0.556	0.574	0.052	0.851	0.286	7296	6545
799	CvM1	SRMRB	WLSMV	200	10	0.641	0.618	0.651	0.047	0.814	0.438	7940	7036
800	CvM1	SRMRB	WLSMV	200	30	0.707	0.660	0.725	0.043	0.837	0.479	8166	7160

Detecting Misspecification at Level-2

General overview over all conditions

j <- 3 ## Which class?
for(index in INDEX){
    ## Print out which iteration so we know what we am looking at
    cat('\n\nROC Analysis in')
    cat('\nIndex:\t', index)
    cat('\nClassification:\t', CLASS[j])
    ## Set up iteration key
    key <- paste0(index,'.',CLASS[j])
    ## Create formula
    model <- as.formula(paste0(CLASS[j], '~', index))
    ## Fit ROC curve
    fit_roc[[key]] <-  roc(model, data=sim_results,
                           plot =TRUE, ci=TRUE, print.auc=TRUE)
    ## Create a plot of "smoothed" curve for plotting
    fit_roc_smooth[[key]] <-  smooth(roc(model, data=sim_results))
    ## Compute partial AUC for specificity .8-1
    p.auc <- auc(fit_roc[[key]], partial.auc = c(1,.8),
                 partial.auc.focus = 'sp', partial.auc.correct = T)
    ## get summary info
    roc_summary_gen[ig, 2] <- index
    roc_summary_gen[ig, 1] <- CLASS[j]
    roc_summary_gen[ig, 3] <- fit_roc[[key]]$auc ## total AUC
    roc_summary_gen[ig, 4] <- p.auc ## corrected partial AUC (.5 is no discrimination)
    roc_summary_gen[ig, 5] <- fit_roc_smooth[[key]]$auc ## smoothed AUC
    roc_summary_gen[ig, 6:8] <- coords(fit_roc[[key]], "best", 
                                   ret=c("threshold", "specificity", 'sensitivity'))
    ## print summary
    cat('\n\nSummary of ROC:\n')
    print(roc_summary_gen[ig, ])
    ## add to summary iterator
    ig <- ig + 1
} ## End loop round index



ROC Analysis in
Index:   CFI
Classification:  CvM2

Setting levels: control = 0, case = 1

Setting direction: controls < cases

Setting levels: control = 0, case = 1

Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
   Classification Index      AUC partial-AUC Smoothed-AUC
11           CvM2   CFI 0.669498   0.5483643    0.6916255
   Optimal-Threshold Specificity Sensitivity
11         0.9932424   0.5757261    0.694697


ROC Analysis in
Index:   TLI
Classification:  CvM2

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
   Classification Index      AUC partial-AUC Smoothed-AUC
12           CvM2   TLI 0.669498   0.5483643    0.6916017
   Optimal-Threshold Specificity Sensitivity
12         0.9918909   0.5757261    0.694697


ROC Analysis in
Index:   RMSEA
Classification:  CvM2

Setting levels: control = 0, case = 1

Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
   Classification Index       AUC partial-AUC Smoothed-AUC
13           CvM2 RMSEA 0.6644986   0.5483738     0.691537
   Optimal-Threshold Specificity Sensitivity
13       0.008547063    0.587031   0.6791605


ROC Analysis in
Index:   SRMRW
Classification:  CvM2

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
   Classification Index       AUC partial-AUC Smoothed-AUC
14           CvM2 SRMRW 0.5270719   0.5050447    0.5241129
   Optimal-Threshold Specificity Sensitivity
14        0.03089441   0.4519094   0.5970939


ROC Analysis in
Index:   SRMRB
Classification:  CvM2

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.

Version	Author	Date
982c8f1	noah-padgett	2019-05-18



Summary of ROC:
   Classification Index       AUC partial-AUC Smoothed-AUC
15           CvM2 SRMRB 0.6343083   0.6012881     0.641562
   Optimal-Threshold Specificity Sensitivity
15        0.07595733   0.7774833   0.4406392

kable(roc_summary_gen[11:15,], format = 'html', digits=3) %>%
  kable_styling(full_width = T)

	Classification	Index	AUC	partial-AUC	Smoothed-AUC	Optimal-Threshold	Specificity	Sensitivity
11	CvM2	CFI	0.669	0.548	0.692	0.993	0.576	0.695
12	CvM2	TLI	0.669	0.548	0.692	0.992	0.576	0.695
13	CvM2	RMSEA	0.664	0.548	0.692	0.009	0.587	0.679
14	CvM2	SRMRW	0.527	0.505	0.524	0.031	0.452	0.597
15	CvM2	SRMRB	0.634	0.601	0.642	0.076	0.777	0.441

print(xtable(roc_summary_gen[11:15,c(2:3,6:8)], digits = 3), booktabs=T,include.rownames = F)

% latex table generated in R 3.6.0 by xtable 1.8-4 package
% Sat Sep 28 20:54:10 2019
\begin{table}[ht]
\centering
\begin{tabular}{lrrrr}
  \toprule
Index & AUC & Optimal-Threshold & Specificity & Sensitivity \\ 
  \midrule
CFI & 0.669 & 0.993 & 0.576 & 0.695 \\ 
  TLI & 0.669 & 0.992 & 0.576 & 0.695 \\ 
  RMSEA & 0.664 & 0.009 & 0.587 & 0.679 \\ 
  SRMRW & 0.527 & 0.031 & 0.452 & 0.597 \\ 
  SRMRB & 0.634 & 0.076 & 0.777 & 0.441 \\ 
   \bottomrule
\end{tabular}
\end{table}

More fine grained information within/across conditions

j <- 3 ## Which class?
for(index in INDEX){
  for(est in EST){
    for(s2 in SS_L2){
      for(s1 in SS_L1){
    ## Print out which iteration so we know what we are looking at
    cat('\n\nROC Analysis in')
    cat('\nIndex:\t', index)
    cat('\nClassification:\t', CLASS[j])
    cat('\nEstimation Method:\t', est)
    cat('\nLevel-2 Sample Size:\t', s2)
    cat('\nLevel-1 Sample Size:\t', s1)
    ## Set up iteration key
    key <- paste0(index,'.',CLASS[j],'.',est,'.', s2,'.',s1)
    # Subset data as  needed
    if(est == 'ALL' & s2 == 'ALL' & s1 == 'ALL') mydata <- sim_results
    if(est != 'ALL' & s2 == 'ALL' & s1 == 'ALL'){
      mydata <- filter(sim_results, Estimator == est)
    }
    if(est == 'ALL' & s2 != 'ALL' & s1 == 'ALL'){
      mydata <- filter(sim_results, ss_l2 == s2)
    }
    if(est == 'ALL' & s2 == 'ALL' & s1 != 'ALL'){
      mydata <- filter(sim_results, ss_l1 == s1)
    }
    if(est != 'ALL' & s2 != 'ALL' & s1 == 'ALL'){
      mydata <- filter(sim_results, Estimator == est, ss_l2 == s2)
    }
    if(est != 'ALL' & s2 == 'ALL' & s1 != 'ALL'){
      mydata <- filter(sim_results, Estimator == est, ss_l1 == s1)
    }
    if(est == 'ALL' & s2 != 'ALL' & s1 != 'ALL'){
      mydata <- filter(sim_results, ss_l2 == s2, ss_l1 == s1)
    }
    if(est != 'ALL' & s2 != 'ALL' & s1 != 'ALL'){
      mydata <- filter(sim_results, Estimator == est, ss_l2 == s2, ss_l1 == s1)
    }
    ## Create formula
    model <- as.formula(paste0(CLASS[j], '~', index))
    ## Fit ROC curve
    fit_roc[[key]] <-  roc(model, data=mydata,
                           plot =TRUE, ci=TRUE, print.auc=TRUE)
    ## Create a plot of "smoothed" curve for plotting
    fit_roc_smooth[[key]] <-  smooth(roc(model, data=mydata))
    ## Compute partial AUC for specificity .8-1
    p.auc <- auc(fit_roc[[key]], partial.auc = c(1,.8),
                 partial.auc.focus = 'sp', partial.auc.correct = T)
    ## get summary info
    roc_summary[i, 2] <- index
    roc_summary[i, 1] <- CLASS[j]
    roc_summary[i, 3] <- est ##estimator
    roc_summary[i, 4] <- s2 ## level-2 sample size
    roc_summary[i, 5] <- s1 ## level-1 sample size
    roc_summary[i, 6] <- fit_roc[[key]]$auc ## total AUC
    roc_summary[i, 7] <- p.auc ## corrected partial AUC (.5 is no discrimination)
    roc_summary[i, 8] <- fit_roc_smooth[[key]]$auc ## smoothed AUC
    roc_summary[i, 9:11] <- coords(fit_roc[[key]], "best", 
                                   ret=c("threshold", "specificity", 'sensitivity'))
    
    ## add number of C and number of miss models in analysis
    n.C <- nrow(mydata[ mydata[, CLASS[j]] == 1, ])
    n.M <- nrow(mydata[ mydata[, CLASS[j]] == 0, ])
    roc_summary[i, 12] <- n.C
    roc_summary[i, 13] <- n.M
    
    ## print summary
    cat('\n\nSummary of ROC:\n')
    print(roc_summary[i, ])
    ## add to summary iterator
    i <- i + 1
      } ## end loop around ss l1
    } ## End loop around ss l2
  } ## End loop around estimator
} ## End loop round index



ROC Analysis in
Index:   CFI
Classification:  CvM2
Estimation Method:   ALL
Level-2 Sample Size:     ALL
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1

Setting direction: controls < cases

Setting levels: control = 0, case = 1

Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS      AUC
801           CvM2   CFI       ALL        ALL        ALL 0.669498
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity  Num-C
801   0.5483643    0.6916255 0.9932424   0.5757261    0.694697 223583
    Num-Mis
801  223868


ROC Analysis in
Index:   CFI
Classification:  CvM2
Estimation Method:   ALL
Level-2 Sample Size:     ALL
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
802           CvM2   CFI       ALL        ALL          5 0.6537712
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
802   0.5494136     0.672035 0.9808516   0.5930675   0.6750309 62891
    Num-Mis
802   63082


ROC Analysis in
Index:   CFI
Classification:  CvM2
Estimation Method:   ALL
Level-2 Sample Size:     ALL
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
803           CvM2   CFI       ALL        ALL         10 0.6868558
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
803   0.5562659    0.6991266 0.9897646   0.5883244   0.7138198 75570
    Num-Mis
803   75749


ROC Analysis in
Index:   CFI
Classification:  CvM2
Estimation Method:   ALL
Level-2 Sample Size:     ALL
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
804           CvM2   CFI       ALL        ALL         30 0.6859195
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
804   0.5432224    0.6818632 0.9977618   0.5561923    0.752057 85122
    Num-Mis
804   85037


ROC Analysis in
Index:   CFI
Classification:  CvM2
Estimation Method:   ALL
Level-2 Sample Size:     30
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
805           CvM2   CFI       ALL         30        ALL 0.5571691
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
805   0.5121357    0.5703557  0.982239   0.4548321   0.6436012 43725
    Num-Mis
805   43767


ROC Analysis in
Index:   CFI
Classification:  CvM2
Estimation Method:   ALL
Level-2 Sample Size:     30
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
806           CvM2   CFI       ALL         30          5 0.5558622
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
806   0.5163284    0.5593021 0.9714739   0.6697271   0.4202168 10875
    Num-Mis
806   10825


ROC Analysis in
Index:   CFI
Classification:  CvM2
Estimation Method:   ALL
Level-2 Sample Size:     30
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
807           CvM2   CFI       ALL         30         10 0.5687139
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
807   0.5153229    0.5717318 0.9896405   0.5885426   0.5155807 14530
    Num-Mis
807   14595


ROC Analysis in
Index:   CFI
Classification:  CvM2
Estimation Method:   ALL
Level-2 Sample Size:     30
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
808           CvM2   CFI       ALL         30         30 0.5653174
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
808   0.5096071    0.5353504 0.9855688   0.2921805   0.8176522 18320
    Num-Mis
808   18347


ROC Analysis in
Index:   CFI
Classification:  CvM2
Estimation Method:   ALL
Level-2 Sample Size:     50
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
809           CvM2   CFI       ALL         50        ALL 0.6198096
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
809   0.5302955    0.6305037 0.9903164   0.5378983   0.6546301 52104
    Num-Mis
809   52168


ROC Analysis in
Index:   CFI
Classification:  CvM2
Estimation Method:   ALL
Level-2 Sample Size:     50
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
810           CvM2   CFI       ALL         50          5 0.5948814
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
810   0.5268786    0.5997862 0.9705302    0.565846   0.5860631 13813
    Num-Mis
810   13894


ROC Analysis in
Index:   CFI
Classification:  CvM2
Estimation Method:   ALL
Level-2 Sample Size:     50
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS      AUC
811           CvM2   CFI       ALL         50         10 0.649427
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
811   0.5450052    0.6517859 0.9796916   0.5152999   0.7040062 17513
    Num-Mis
811   17559


ROC Analysis in
Index:   CFI
Classification:  CvM2
Estimation Method:   ALL
Level-2 Sample Size:     50
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
812           CvM2   CFI       ALL         50         30 0.6368159
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
812   0.5248322    0.5987275 0.9949972   0.4307494   0.7882866 20778
    Num-Mis
812   20715


ROC Analysis in
Index:   CFI
Classification:  CvM2
Estimation Method:   ALL
Level-2 Sample Size:     100
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
813           CvM2   CFI       ALL        100        ALL 0.7070931
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
813    0.563424    0.7055479 0.9935551   0.5985989   0.7195181 60747
    Num-Mis
813   60844


ROC Analysis in
Index:   CFI
Classification:  CvM2
Estimation Method:   ALL
Level-2 Sample Size:     100
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
814           CvM2   CFI       ALL        100          5 0.6954108
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
814   0.5612294    0.7006346 0.9770019     0.54755   0.7572067 17384
    Num-Mis
814   17448


ROC Analysis in
Index:   CFI
Classification:  CvM2
Estimation Method:   ALL
Level-2 Sample Size:     100
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
815           CvM2   CFI       ALL        100         10 0.7313011
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
815   0.5724079    0.7310743 0.9861017   0.5301418   0.8350739 20799
    Num-Mis
815   20843


ROC Analysis in
Index:   CFI
Classification:  CvM2
Estimation Method:   ALL
Level-2 Sample Size:     100
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS      AUC
816           CvM2   CFI       ALL        100         30 0.730952
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
816   0.5589811    0.7180036 0.9968498   0.5428333    0.828653 22564
    Num-Mis
816   22553


ROC Analysis in
Index:   CFI
Classification:  CvM2
Estimation Method:   ALL
Level-2 Sample Size:     200
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
817           CvM2   CFI       ALL        200        ALL 0.7708037
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
817   0.6046883     0.762266 0.9949944   0.6187998   0.7876187 67007
    Num-Mis
817   67089


ROC Analysis in
Index:   CFI
Classification:  CvM2
Estimation Method:   ALL
Level-2 Sample Size:     200
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
818           CvM2   CFI       ALL        200          5 0.7574869
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
818   0.5765767    0.7561613 0.9851123   0.5671809   0.8589776 20819
    Num-Mis
818   20915


ROC Analysis in
Index:   CFI
Classification:  CvM2
Estimation Method:   ALL
Level-2 Sample Size:     200
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
819           CvM2   CFI       ALL        200         10 0.7782347
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
819   0.6003632    0.7696373 0.9897233    0.534854   0.9108414 22728
    Num-Mis
819   22752


ROC Analysis in
Index:   CFI
Classification:  CvM2
Estimation Method:   ALL
Level-2 Sample Size:     200
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
820           CvM2   CFI       ALL        200         30 0.8157548
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
820   0.6372054     0.811226  0.998338   0.6944222   0.8167408 23460
    Num-Mis
820   23422


ROC Analysis in
Index:   CFI
Classification:  CvM2
Estimation Method:   MLR
Level-2 Sample Size:     ALL
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
821           CvM2   CFI       MLR        ALL        ALL 0.6563579
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
821   0.5668005    0.6660005 0.9910719   0.6517222   0.6117996 85732
    Num-Mis
821   85647


ROC Analysis in
Index:   CFI
Classification:  CvM2
Estimation Method:   MLR
Level-2 Sample Size:     ALL
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
822           CvM2   CFI       MLR        ALL          5 0.6296136
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
822   0.5570349    0.6392389  0.980848   0.6694317   0.5816657 24596
    Num-Mis
822   24572


ROC Analysis in
Index:   CFI
Classification:  CvM2
Estimation Method:   MLR
Level-2 Sample Size:     ALL
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
823           CvM2   CFI       MLR        ALL         10 0.6639266
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
823   0.5627594    0.6737213 0.9884746   0.6621232   0.6266601 28824
    Num-Mis
823   28785


ROC Analysis in
Index:   CFI
Classification:  CvM2
Estimation Method:   MLR
Level-2 Sample Size:     ALL
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
824           CvM2   CFI       MLR        ALL         30 0.6972484
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
824   0.5792197    0.7053938  0.995119   0.6423684   0.6694892 32312
    Num-Mis
824   32290


ROC Analysis in
Index:   CFI
Classification:  CvM2
Estimation Method:   MLR
Level-2 Sample Size:     30
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
825           CvM2   CFI       MLR         30        ALL 0.5737514
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
825   0.5334177    0.5745675 0.9722712   0.6382269   0.4942017 17855
    Num-Mis
825   17854


ROC Analysis in
Index:   CFI
Classification:  CvM2
Estimation Method:   MLR
Level-2 Sample Size:     30
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
826           CvM2   CFI       MLR         30          5 0.5634798
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
826     0.52507    0.5618621 0.8961821    0.572759   0.5509849  4747
    Num-Mis
826    4739


ROC Analysis in
Index:   CFI
Classification:  CvM2
Estimation Method:   MLR
Level-2 Sample Size:     30
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
827           CvM2   CFI       MLR         30         10 0.5866561
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
827   0.5273446    0.5864262 0.9400219    0.476631   0.6567673  5832
    Num-Mis
827    5832


ROC Analysis in
Index:   CFI
Classification:  CvM2
Estimation Method:   MLR
Level-2 Sample Size:     30
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
828           CvM2   CFI       MLR         30         30 0.6376628
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
828   0.5426916    0.6369829 0.9816456   0.5757335   0.6298211  7276
    Num-Mis
828    7283


ROC Analysis in
Index:   CFI
Classification:  CvM2
Estimation Method:   MLR
Level-2 Sample Size:     50
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
829           CvM2   CFI       MLR         50        ALL 0.6240773
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
829   0.5437137     0.629051 0.9857614   0.6232819   0.5860472 20411
    Num-Mis
829   20417


ROC Analysis in
Index:   CFI
Classification:  CvM2
Estimation Method:   MLR
Level-2 Sample Size:     50
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
830           CvM2   CFI       MLR         50          5 0.5982245
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
830    0.529993    0.6005209  0.970561    0.684878   0.4753937  5706
    Num-Mis
830    5724


ROC Analysis in
Index:   CFI
Classification:  CvM2
Estimation Method:   MLR
Level-2 Sample Size:     50
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
831           CvM2   CFI       MLR         50         10 0.6565269
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
831   0.5475926    0.6576445 0.9665633   0.4887029   0.7528278  6787
    Num-Mis
831    6790


ROC Analysis in
Index:   CFI
Classification:  CvM2
Estimation Method:   MLR
Level-2 Sample Size:     50
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
832           CvM2   CFI       MLR         50         30 0.6883666
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
832   0.5505073    0.6846075   0.98762   0.4769517   0.8240296  7918
    Num-Mis
832    7903


ROC Analysis in
Index:   CFI
Classification:  CvM2
Estimation Method:   MLR
Level-2 Sample Size:     100
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
833           CvM2   CFI       MLR        100        ALL 0.7039781
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
833   0.5681606    0.7139309 0.9910455   0.5946827   0.7282256 22941
    Num-Mis
833   22901


ROC Analysis in
Index:   CFI
Classification:  CvM2
Estimation Method:   MLR
Level-2 Sample Size:     100
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
834           CvM2   CFI       MLR        100          5 0.6909142
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
834   0.5612518    0.6985805 0.9769555   0.5799751   0.7269342  6647
    Num-Mis
834    6637


ROC Analysis in
Index:   CFI
Classification:  CvM2
Estimation Method:   MLR
Level-2 Sample Size:     100
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
835           CvM2   CFI       MLR        100         10 0.7323176
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
835   0.5617259    0.7289678 0.9845452   0.5315249   0.8543478  7855
    Num-Mis
835    7823


ROC Analysis in
Index:   CFI
Classification:  CvM2
Estimation Method:   MLR
Level-2 Sample Size:     100
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
836           CvM2   CFI       MLR        100         30 0.7568674
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
836    0.581323    0.7481976 0.9945819    0.550546   0.8565386  8439
    Num-Mis
836    8441


ROC Analysis in
Index:   CFI
Classification:  CvM2
Estimation Method:   MLR
Level-2 Sample Size:     200
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS      AUC
837           CvM2   CFI       MLR        200        ALL 0.763702
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
837   0.5941958    0.7671768 0.9949952   0.6258934   0.7894001 24525
    Num-Mis
837   24475


ROC Analysis in
Index:   CFI
Classification:  CvM2
Estimation Method:   MLR
Level-2 Sample Size:     200
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS      AUC
838           CvM2   CFI       MLR        200          5 0.760585
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
838   0.5761326    0.7591939 0.9862474   0.5924982   0.8451264  7496
    Num-Mis
838    7472


ROC Analysis in
Index:   CFI
Classification:  CvM2
Estimation Method:   MLR
Level-2 Sample Size:     200
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
839           CvM2   CFI       MLR        200         10 0.7780748
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
839   0.5871836    0.7634796 0.9910705   0.5547065   0.9030807  8350
    Num-Mis
839    8340


ROC Analysis in
Index:   CFI
Classification:  CvM2
Estimation Method:   MLR
Level-2 Sample Size:     200
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
840           CvM2   CFI       MLR        200         30 0.8149302
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
840   0.6212558    0.8037239 0.9971726   0.6202749   0.8978127  8679
    Num-Mis
840    8663


ROC Analysis in
Index:   CFI
Classification:  CvM2
Estimation Method:   ULSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
841           CvM2   CFI     ULSMV        ALL        ALL 0.6910939
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
841   0.5457522    0.7194653 0.9780187    0.434941   0.8713091 71635
    Num-Mis
841   71584


ROC Analysis in
Index:   CFI
Classification:  CvM2
Estimation Method:   ULSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
842           CvM2   CFI     ULSMV        ALL          5 0.6770039
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
842   0.5469117    0.7056246 0.9714056   0.5347287     0.76733 19976
    Num-Mis
842   19987


ROC Analysis in
Index:   CFI
Classification:  CvM2
Estimation Method:   ULSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
843           CvM2   CFI     ULSMV        ALL         10 0.7105684
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
843   0.5553749    0.7222579 0.9703367   0.4545746   0.8877934 24264
    Num-Mis
843   24300


ROC Analysis in
Index:   CFI
Classification:  CvM2
Estimation Method:   ULSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
844           CvM2   CFI     ULSMV        ALL         30 0.6960056
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
844   0.5399437     0.596045 0.9988338   0.5221088   0.8116458 27395
    Num-Mis
844   27297


ROC Analysis in
Index:   CFI
Classification:  CvM2
Estimation Method:   ULSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
845           CvM2   CFI     ULSMV         30        ALL 0.5698707
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
845   0.5119867     0.621666 0.9989036    0.408514   0.7198151 13710
    Num-Mis
845   13635


ROC Analysis in
Index:   CFI
Classification:  CvM2
Estimation Method:   ULSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
846           CvM2   CFI     ULSMV         30          5 0.5645743
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
846   0.5135569    0.5835209 0.9715114   0.5206684   0.6042017  3256
    Num-Mis
846    3200


ROC Analysis in
Index:   CFI
Classification:  CvM2
Estimation Method:   ULSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
847           CvM2   CFI     ULSMV         30         10 0.5893104
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
847   0.5164941     0.611295 0.9990669   0.4874782   0.6639487  4632
    Num-Mis
847    4632


ROC Analysis in
Index:   CFI
Classification:  CvM2
Estimation Method:   ULSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
848           CvM2   CFI     ULSMV         30         30 0.5705933
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
848   0.5095164    0.4219801 0.9988069   0.2496692   0.8814523  5822
    Num-Mis
848    5803


ROC Analysis in
Index:   CFI
Classification:  CvM2
Estimation Method:   ULSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
849           CvM2   CFI     ULSMV         50        ALL 0.6413399
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
849   0.5307251    0.6649297 0.9960443   0.5451946   0.6847074 16566
    Num-Mis
849   16505


ROC Analysis in
Index:   CFI
Classification:  CvM2
Estimation Method:   ULSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
850           CvM2   CFI     ULSMV         50          5 0.6061247
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
850   0.5274046    0.6149412 0.9907846   0.6444444   0.5210031  4339
    Num-Mis
850    4319


ROC Analysis in
Index:   CFI
Classification:  CvM2
Estimation Method:   ULSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
851           CvM2   CFI     ULSMV         50         10 0.6752522
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
851   0.5479215    0.6821228 0.9713875   0.4661336   0.7997172  5547
    Num-Mis
851    5551


ROC Analysis in
Index:   CFI
Classification:  CvM2
Estimation Method:   ULSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
852           CvM2   CFI     ULSMV         50         30 0.6470825
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
852   0.5246667    0.5368471 0.9978689   0.3989977   0.8609322  6680
    Num-Mis
852    6635


ROC Analysis in
Index:   CFI
Classification:  CvM2
Estimation Method:   ULSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
853           CvM2   CFI     ULSMV        100        ALL 0.7287674
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
853   0.5651194    0.7122927 0.9703406   0.4467522   0.9064949 19541
    Num-Mis
853   19571


ROC Analysis in
Index:   CFI
Classification:  CvM2
Estimation Method:   ULSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
854           CvM2   CFI     ULSMV        100          5 0.7154631
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
854   0.5654469    0.7183605 0.9745036   0.5842857   0.7524085  5546
    Num-Mis
854    5570


ROC Analysis in
Index:   CFI
Classification:  CvM2
Estimation Method:   ULSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
855           CvM2   CFI     ULSMV        100         10 0.7500658
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
855    0.581952    0.7365093 0.9643035   0.4697885   0.9316533  6712
    Num-Mis
855    6741


ROC Analysis in
Index:   CFI
Classification:  CvM2
Estimation Method:   ULSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
856           CvM2   CFI     ULSMV        100         30 0.7326408
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
856   0.5559721    0.6578111 0.9987656   0.5859125   0.8055752  7283
    Num-Mis
856    7260


ROC Analysis in
Index:   CFI
Classification:  CvM2
Estimation Method:   ULSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
857           CvM2   CFI     ULSMV        200        ALL 0.7841778
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
857   0.6095449    0.7502173 0.9778986   0.5111478   0.9460936 21818
    Num-Mis
857   21873


ROC Analysis in
Index:   CFI
Classification:  CvM2
Estimation Method:   ULSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
858           CvM2   CFI     ULSMV        200          5 0.7625315
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
858   0.5758343    0.7452186 0.9750077   0.5358869   0.9143184  6835
    Num-Mis
858    6898


ROC Analysis in
Index:   CFI
Classification:  CvM2
Estimation Method:   ULSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS      AUC
859           CvM2   CFI     ULSMV        200         10 0.779721
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
859   0.6037176    0.7440758 0.9684118   0.5045202   0.9738949  7373
    Num-Mis
859    7376


ROC Analysis in
Index:   CFI
Classification:  CvM2
Estimation Method:   ULSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS      AUC
860           CvM2   CFI     ULSMV        200         30 0.821772
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
860   0.6507471    0.7808965 0.9988433   0.7664835    0.742388  7610
    Num-Mis
860    7599


ROC Analysis in
Index:   CFI
Classification:  CvM2
Estimation Method:   WLSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
861           CvM2   CFI     WLSMV        ALL        ALL 0.6649316
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
861   0.5417451     0.699861 0.9965207   0.5626895   0.7070446 66216
    Num-Mis
861   66637


ROC Analysis in
Index:   CFI
Classification:  CvM2
Estimation Method:   WLSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
862           CvM2   CFI     WLSMV        ALL          5 0.6629151
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
862   0.5470545    0.6832071  0.986618   0.5852068   0.6916155 18319
    Num-Mis
862   18523


ROC Analysis in
Index:   CFI
Classification:  CvM2
Estimation Method:   WLSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
863           CvM2   CFI     WLSMV        ALL         10 0.6942403
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
863   0.5530581    0.7103399 0.9915102   0.5354673   0.7807627 22482
    Num-Mis
863   22664


ROC Analysis in
Index:   CFI
Classification:  CvM2
Estimation Method:   WLSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
864           CvM2   CFI     WLSMV        ALL         30 0.6753114
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
864   0.5341159    0.6580383 0.9984593   0.4582844   0.8471062 25415
    Num-Mis
864   25450


ROC Analysis in
Index:   CFI
Classification:  CvM2
Estimation Method:   WLSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
865           CvM2   CFI     WLSMV         30        ALL 0.5410447
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
865   0.5074232    0.5593931 0.9996324   0.4411517   0.6353448 12160
    Num-Mis
865   12278


ROC Analysis in
Index:   CFI
Classification:  CvM2
Estimation Method:   WLSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
866           CvM2   CFI     WLSMV         30          5 0.5531451
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
866   0.5138365    0.5573847 0.9679801   0.4746717   0.6146527  2872
    Num-Mis
866    2886


ROC Analysis in
Index:   CFI
Classification:  CvM2
Estimation Method:   WLSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
867           CvM2   CFI     WLSMV         30         10 0.5610682
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
867   0.5119129    0.5769417 0.9910153   0.4238035   0.6883202  4066
    Num-Mis
867    4131


ROC Analysis in
Index:   CFI
Classification:  CvM2
Estimation Method:   WLSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
868           CvM2   CFI     WLSMV         30         30 0.5305092
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
868   0.5040842    0.5345607 0.9998055   0.2347909   0.8251816  5222
    Num-Mis
868    5261


ROC Analysis in
Index:   CFI
Classification:  CvM2
Estimation Method:   WLSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
869           CvM2   CFI     WLSMV         50        ALL 0.6008016
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
869   0.5233961    0.6215385 0.9993251    0.589546   0.5864957 15127
    Num-Mis
869   15246


ROC Analysis in
Index:   CFI
Classification:  CvM2
Estimation Method:   WLSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
870           CvM2   CFI     WLSMV         50          5 0.5900847
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
870   0.5229755    0.5944339 0.9704936   0.4340255   0.7078014  3768
    Num-Mis
870    3851


ROC Analysis in
Index:   CFI
Classification:  CvM2
Estimation Method:   WLSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
871           CvM2   CFI     WLSMV         50         10 0.6346992
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
871    0.539221    0.6388301 0.9931149   0.6133663   0.5916204  5179
    Num-Mis
871    5218


ROC Analysis in
Index:   CFI
Classification:  CvM2
Estimation Method:   WLSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
872           CvM2   CFI     WLSMV         50         30 0.6066038
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
872   0.5170463    0.5562701 0.9994466   0.3933225   0.7974786  6180
    Num-Mis
872    6177


ROC Analysis in
Index:   CFI
Classification:  CvM2
Estimation Method:   WLSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
873           CvM2   CFI     WLSMV        100        ALL 0.6988463
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
873    0.558847    0.7123726 0.9960752   0.6032572   0.7077778 18265
    Num-Mis
873   18372


ROC Analysis in
Index:   CFI
Classification:  CvM2
Estimation Method:   WLSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
874           CvM2   CFI     WLSMV        100          5 0.6891992
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
874   0.5584747    0.6937177 0.9849592    0.582681   0.7106319  5191
    Num-Mis
874    5241


ROC Analysis in
Index:   CFI
Classification:  CvM2
Estimation Method:   WLSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
875           CvM2   CFI     WLSMV        100         10 0.7332334
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
875   0.5732233    0.7224614 0.9900752   0.5353175   0.8362313  6232
    Num-Mis
875    6279


ROC Analysis in
Index:   CFI
Classification:  CvM2
Estimation Method:   WLSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
876           CvM2   CFI     WLSMV        100         30 0.7227988
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
876   0.5516772    0.6912255 0.9977057   0.4917973   0.8810831  6842
    Num-Mis
876    6852


ROC Analysis in
Index:   CFI
Classification:  CvM2
Estimation Method:   WLSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
877           CvM2   CFI     WLSMV        200        ALL 0.7756177
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
877   0.6106023    0.7671411 0.9967114   0.6439521   0.7711834 20664
    Num-Mis
877   20741


ROC Analysis in
Index:   CFI
Classification:  CvM2
Estimation Method:   WLSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
878           CvM2   CFI     WLSMV        200          5 0.7598017
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
878   0.5784405    0.7492788 0.9863408   0.5474453   0.8825609  6488
    Num-Mis
878    6545


ROC Analysis in
Index:   CFI
Classification:  CvM2
Estimation Method:   WLSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
879           CvM2   CFI     WLSMV        200         10 0.7944228
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
879    0.610778    0.7755693 0.9931882   0.5806906   0.8902743  7005
    Num-Mis
879    7036


ROC Analysis in
Index:   CFI
Classification:  CvM2
Estimation Method:   WLSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
880           CvM2   CFI     WLSMV        200         30 0.8265385
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
880   0.6384748    0.8120607 0.9985929   0.6627707   0.8726027  7171
    Num-Mis
880    7160


ROC Analysis in
Index:   TLI
Classification:  CvM2
Estimation Method:   ALL
Level-2 Sample Size:     ALL
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS      AUC
881           CvM2   TLI       ALL        ALL        ALL 0.669498
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity  Num-C
881   0.5483643    0.6916017 0.9918909   0.5757261    0.694697 223583
    Num-Mis
881  223868


ROC Analysis in
Index:   TLI
Classification:  CvM2
Estimation Method:   ALL
Level-2 Sample Size:     ALL
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
882           CvM2   TLI       ALL        ALL          5 0.6537712
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
882   0.5494136    0.6720351 0.9770219   0.5930675   0.6750309 62891
    Num-Mis
882   63082


ROC Analysis in
Index:   TLI
Classification:  CvM2
Estimation Method:   ALL
Level-2 Sample Size:     ALL
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
883           CvM2   TLI       ALL        ALL         10 0.6868558
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
883   0.5562659    0.6991284 0.9877175   0.5883244   0.7138198 75570
    Num-Mis
883   75749


ROC Analysis in
Index:   TLI
Classification:  CvM2
Estimation Method:   ALL
Level-2 Sample Size:     ALL
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
884           CvM2   TLI       ALL        ALL         30 0.6859196
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
884   0.5432224    0.6819013 0.9973141   0.5561923    0.752057 85122
    Num-Mis
884   85037


ROC Analysis in
Index:   TLI
Classification:  CvM2
Estimation Method:   ALL
Level-2 Sample Size:     30
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
885           CvM2   TLI       ALL         30        ALL 0.5571692
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
885   0.5121357    0.5703566 0.9786868   0.4548321   0.6436012 43725
    Num-Mis
885   43767


ROC Analysis in
Index:   TLI
Classification:  CvM2
Estimation Method:   ALL
Level-2 Sample Size:     30
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
886           CvM2   TLI       ALL         30          5 0.5558623
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
886   0.5163284    0.5593022 0.9657686   0.6697271   0.4202168 10875
    Num-Mis
886   10825


ROC Analysis in
Index:   TLI
Classification:  CvM2
Estimation Method:   ALL
Level-2 Sample Size:     30
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
887           CvM2   TLI       ALL         30         10 0.5687139
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
887   0.5153229    0.5717318 0.9875687   0.5885426   0.5155807 14530
    Num-Mis
887   14595


ROC Analysis in
Index:   TLI
Classification:  CvM2
Estimation Method:   ALL
Level-2 Sample Size:     30
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
888           CvM2   TLI       ALL         30         30 0.5653174
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
888   0.5096071    0.5353443 0.9826826   0.2921805   0.8176522 18320
    Num-Mis
888   18347


ROC Analysis in
Index:   TLI
Classification:  CvM2
Estimation Method:   ALL
Level-2 Sample Size:     50
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
889           CvM2   TLI       ALL         50        ALL 0.6198096
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
889   0.5302955     0.630503 0.9883797   0.5378983   0.6546301 52104
    Num-Mis
889   52168


ROC Analysis in
Index:   TLI
Classification:  CvM2
Estimation Method:   ALL
Level-2 Sample Size:     50
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
890           CvM2   TLI       ALL         50          5 0.5948814
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
890   0.5268786     0.599784 0.9646363    0.565846   0.5860631 13813
    Num-Mis
890   13894


ROC Analysis in
Index:   TLI
Classification:  CvM2
Estimation Method:   ALL
Level-2 Sample Size:     50
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS      AUC
891           CvM2   TLI       ALL         50         10 0.649427
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
891   0.5450052    0.6517874 0.9756299   0.5152999   0.7040062 17513
    Num-Mis
891   17559


ROC Analysis in
Index:   TLI
Classification:  CvM2
Estimation Method:   ALL
Level-2 Sample Size:     50
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
892           CvM2   TLI       ALL         50         30 0.6368159
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
892   0.5248322    0.5987377 0.9939966   0.4307494   0.7882866 20778
    Num-Mis
892   20715


ROC Analysis in
Index:   TLI
Classification:  CvM2
Estimation Method:   ALL
Level-2 Sample Size:     100
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
893           CvM2   TLI       ALL        100        ALL 0.7070931
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
893    0.563424     0.705554 0.9922661   0.5985989   0.7195181 60747
    Num-Mis
893   60844


ROC Analysis in
Index:   TLI
Classification:  CvM2
Estimation Method:   ALL
Level-2 Sample Size:     100
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
894           CvM2   TLI       ALL        100          5 0.6954108
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
894   0.5612294    0.7006356 0.9724023     0.54755   0.7572067 17384
    Num-Mis
894   17448


ROC Analysis in
Index:   TLI
Classification:  CvM2
Estimation Method:   ALL
Level-2 Sample Size:     100
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
895           CvM2   TLI       ALL        100         10 0.7313011
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
895   0.5724079    0.7310745  0.983322   0.5301418   0.8350739 20799
    Num-Mis
895   20843


ROC Analysis in
Index:   TLI
Classification:  CvM2
Estimation Method:   ALL
Level-2 Sample Size:     100
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS      AUC
896           CvM2   TLI       ALL        100         30 0.730952
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
896   0.5589811    0.7180022 0.9962198   0.5428333    0.828653 22564
    Num-Mis
896   22553


ROC Analysis in
Index:   TLI
Classification:  CvM2
Estimation Method:   ALL
Level-2 Sample Size:     200
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
897           CvM2   TLI       ALL        200        ALL 0.7708037
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
897   0.6046883    0.7622799 0.9939932   0.6187998   0.7876187 67007
    Num-Mis
897   67089


ROC Analysis in
Index:   TLI
Classification:  CvM2
Estimation Method:   ALL
Level-2 Sample Size:     200
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
898           CvM2   TLI       ALL        200          5 0.7574869
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
898   0.5765767    0.7561631 0.9821347   0.5671809   0.8589776 20819
    Num-Mis
898   20915


ROC Analysis in
Index:   TLI
Classification:  CvM2
Estimation Method:   ALL
Level-2 Sample Size:     200
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
899           CvM2   TLI       ALL        200         10 0.7782347
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
899   0.6003632     0.769641  0.987668    0.534854   0.9108414 22728
    Num-Mis
899   22752


ROC Analysis in
Index:   TLI
Classification:  CvM2
Estimation Method:   ALL
Level-2 Sample Size:     200
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
900           CvM2   TLI       ALL        200         30 0.8157549
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
900   0.6372055    0.8112288 0.9980056   0.6944222   0.8167408 23460
    Num-Mis
900   23422


ROC Analysis in
Index:   TLI
Classification:  CvM2
Estimation Method:   MLR
Level-2 Sample Size:     ALL
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
901           CvM2   TLI       MLR        ALL        ALL 0.6563579
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
901   0.5668005    0.6660014 0.9892863   0.6517222   0.6117996 85732
    Num-Mis
901   85647


ROC Analysis in
Index:   TLI
Classification:  CvM2
Estimation Method:   MLR
Level-2 Sample Size:     ALL
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
902           CvM2   TLI       MLR        ALL          5 0.6296136
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
902   0.5570349     0.639241 0.9770177   0.6694317   0.5816657 24596
    Num-Mis
902   24572


ROC Analysis in
Index:   TLI
Classification:  CvM2
Estimation Method:   MLR
Level-2 Sample Size:     ALL
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
903           CvM2   TLI       MLR        ALL         10 0.6639266
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
903   0.5627594    0.6737236 0.9861695   0.6621232   0.6266601 28824
    Num-Mis
903   28785


ROC Analysis in
Index:   TLI
Classification:  CvM2
Estimation Method:   MLR
Level-2 Sample Size:     ALL
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
904           CvM2   TLI       MLR        ALL         30 0.6972484
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
904   0.5792197    0.7053934 0.9941428   0.6423684   0.6694892 32312
    Num-Mis
904   32290


ROC Analysis in
Index:   TLI
Classification:  CvM2
Estimation Method:   MLR
Level-2 Sample Size:     30
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
905           CvM2   TLI       MLR         30        ALL 0.5737514
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
905   0.5334178    0.5745668 0.9667255   0.6382269   0.4942017 17855
    Num-Mis
905   17854


ROC Analysis in
Index:   TLI
Classification:  CvM2
Estimation Method:   MLR
Level-2 Sample Size:     30
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
906           CvM2   TLI       MLR         30          5 0.5634798
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
906     0.52507    0.5618621 0.8754185    0.572759   0.5509849  4747
    Num-Mis
906    4739


ROC Analysis in
Index:   TLI
Classification:  CvM2
Estimation Method:   MLR
Level-2 Sample Size:     30
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
907           CvM2   TLI       MLR         30         10 0.5866561
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
907   0.5273446    0.5864252 0.9280263    0.476631   0.6567673  5832
    Num-Mis
907    5832


ROC Analysis in
Index:   TLI
Classification:  CvM2
Estimation Method:   MLR
Level-2 Sample Size:     30
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
908           CvM2   TLI       MLR         30         30 0.6376629
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
908   0.5426916    0.6369812 0.9779748   0.5757335   0.6298211  7276
    Num-Mis
908    7283


ROC Analysis in
Index:   TLI
Classification:  CvM2
Estimation Method:   MLR
Level-2 Sample Size:     50
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
909           CvM2   TLI       MLR         50        ALL 0.6240773
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
909   0.5437137    0.6290537 0.9829136   0.6232819   0.5860472 20411
    Num-Mis
909   20417


ROC Analysis in
Index:   TLI
Classification:  CvM2
Estimation Method:   MLR
Level-2 Sample Size:     50
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
910           CvM2   TLI       MLR         50          5 0.5982244
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
910    0.529993    0.6005203 0.9646732    0.684878   0.4753937  5706
    Num-Mis
910    5724


ROC Analysis in
Index:   TLI
Classification:  CvM2
Estimation Method:   MLR
Level-2 Sample Size:     50
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
911           CvM2   TLI       MLR         50         10 0.6565269
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
911   0.5475926    0.6576393  0.959876   0.4887029   0.7528278  6787
    Num-Mis
911    6790


ROC Analysis in
Index:   TLI
Classification:  CvM2
Estimation Method:   MLR
Level-2 Sample Size:     50
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
912           CvM2   TLI       MLR         50         30 0.6883666
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
912   0.5505073    0.6846075  0.985144   0.4769517   0.8240296  7918
    Num-Mis
912    7903


ROC Analysis in
Index:   TLI
Classification:  CvM2
Estimation Method:   MLR
Level-2 Sample Size:     100
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
913           CvM2   TLI       MLR        100        ALL 0.7039781
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
913   0.5681607    0.7139426 0.9892547   0.5946827   0.7282256 22941
    Num-Mis
913   22901


ROC Analysis in
Index:   TLI
Classification:  CvM2
Estimation Method:   MLR
Level-2 Sample Size:     100
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
914           CvM2   TLI       MLR        100          5 0.6909142
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
914   0.5612518    0.6985805 0.9723466   0.5799751   0.7269342  6647
    Num-Mis
914    6637


ROC Analysis in
Index:   TLI
Classification:  CvM2
Estimation Method:   MLR
Level-2 Sample Size:     100
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
915           CvM2   TLI       MLR        100         10 0.7323176
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
915   0.5617259    0.7289678 0.9814543   0.5315249   0.8543478  7855
    Num-Mis
915    7823


ROC Analysis in
Index:   TLI
Classification:  CvM2
Estimation Method:   MLR
Level-2 Sample Size:     100
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
916           CvM2   TLI       MLR        100         30 0.7568674
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
916   0.5813232    0.7481851 0.9934982    0.550546   0.8565386  8439
    Num-Mis
916    8441


ROC Analysis in
Index:   TLI
Classification:  CvM2
Estimation Method:   MLR
Level-2 Sample Size:     200
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS      AUC
917           CvM2   TLI       MLR        200        ALL 0.763702
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
917   0.5941958    0.7671765 0.9939943   0.6258934   0.7894001 24525
    Num-Mis
917   24475


ROC Analysis in
Index:   TLI
Classification:  CvM2
Estimation Method:   MLR
Level-2 Sample Size:     200
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
918           CvM2   TLI       MLR        200          5 0.7605849
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
918   0.5761326    0.7591901 0.9834968   0.5924982   0.8451264  7496
    Num-Mis
918    7472


ROC Analysis in
Index:   TLI
Classification:  CvM2
Estimation Method:   MLR
Level-2 Sample Size:     200
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
919           CvM2   TLI       MLR        200         10 0.7780747
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
919   0.5871836    0.7634831 0.9892846   0.5547065   0.9030807  8350
    Num-Mis
919    8340


ROC Analysis in
Index:   TLI
Classification:  CvM2
Estimation Method:   MLR
Level-2 Sample Size:     200
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
920           CvM2   TLI       MLR        200         30 0.8149302
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
920    0.621256    0.8037375 0.9966072   0.6202749   0.8978127  8679
    Num-Mis
920    8663


ROC Analysis in
Index:   TLI
Classification:  CvM2
Estimation Method:   ULSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
921           CvM2   TLI     ULSMV        ALL        ALL 0.6910939
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
921   0.5457522    0.7194622 0.9736224    0.434941   0.8713091 71635
    Num-Mis
921   71584


ROC Analysis in
Index:   TLI
Classification:  CvM2
Estimation Method:   ULSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
922           CvM2   TLI     ULSMV        ALL          5 0.6770039
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
922   0.5469117    0.7056334 0.9656867   0.5347287     0.76733 19976
    Num-Mis
922   19987


ROC Analysis in
Index:   TLI
Classification:  CvM2
Estimation Method:   ULSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
923           CvM2   TLI     ULSMV        ALL         10 0.7105685
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
923   0.5553749    0.7222741  0.964404   0.4545746   0.8877934 24264
    Num-Mis
923   24300


ROC Analysis in
Index:   TLI
Classification:  CvM2
Estimation Method:   ULSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
924           CvM2   TLI     ULSMV        ALL         30 0.6960056
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
924   0.5399437    0.5960799 0.9986005   0.5221088   0.8116458 27395
    Num-Mis
924   27297


ROC Analysis in
Index:   TLI
Classification:  CvM2
Estimation Method:   ULSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
925           CvM2   TLI     ULSMV         30        ALL 0.5698707
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
925   0.5119867     0.621666 0.9986843    0.408514   0.7198151 13710
    Num-Mis
925   13635


ROC Analysis in
Index:   TLI
Classification:  CvM2
Estimation Method:   ULSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
926           CvM2   TLI     ULSMV         30          5 0.5645743
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
926   0.5135569    0.5835209 0.9658137   0.5206684   0.6042017  3256
    Num-Mis
926    3200


ROC Analysis in
Index:   TLI
Classification:  CvM2
Estimation Method:   ULSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
927           CvM2   TLI     ULSMV         30         10 0.5893104
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
927   0.5164941     0.611295 0.9988803   0.4874782   0.6639487  4632
    Num-Mis
927    4632


ROC Analysis in
Index:   TLI
Classification:  CvM2
Estimation Method:   ULSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
928           CvM2   TLI     ULSMV         30         30 0.5705933
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
928   0.5095164    0.4219801 0.9985683   0.2496692   0.8814523  5822
    Num-Mis
928    5803


ROC Analysis in
Index:   TLI
Classification:  CvM2
Estimation Method:   ULSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
929           CvM2   TLI     ULSMV         50        ALL 0.6413399
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
929   0.5307251    0.6649314 0.9952531   0.5451946   0.6847074 16566
    Num-Mis
929   16505


ROC Analysis in
Index:   TLI
Classification:  CvM2
Estimation Method:   ULSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
930           CvM2   TLI     ULSMV         50          5 0.6061247
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
930   0.5274046    0.6149412 0.9889415   0.6444444   0.5210031  4339
    Num-Mis
930    4319


ROC Analysis in
Index:   TLI
Classification:  CvM2
Estimation Method:   ULSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
931           CvM2   TLI     ULSMV         50         10 0.6752522
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
931   0.5479215    0.6821228  0.965665   0.4661336   0.7997172  5547
    Num-Mis
931    5551


ROC Analysis in
Index:   TLI
Classification:  CvM2
Estimation Method:   ULSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
932           CvM2   TLI     ULSMV         50         30 0.6470825
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
932   0.5246667    0.5368471 0.9974426   0.3989977   0.8609322  6680
    Num-Mis
932    6635


ROC Analysis in
Index:   TLI
Classification:  CvM2
Estimation Method:   ULSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
933           CvM2   TLI     ULSMV        100        ALL 0.7287674
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
933   0.5651194     0.712292 0.9644088   0.4467522   0.9064949 19541
    Num-Mis
933   19571


ROC Analysis in
Index:   TLI
Classification:  CvM2
Estimation Method:   ULSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
934           CvM2   TLI     ULSMV        100          5 0.7154631
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
934   0.5654469    0.7183605 0.9694044   0.5842857   0.7524085  5546
    Num-Mis
934    5570


ROC Analysis in
Index:   TLI
Classification:  CvM2
Estimation Method:   ULSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
935           CvM2   TLI     ULSMV        100         10 0.7500658
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
935    0.581952    0.7365093 0.9571642   0.4697885   0.9316533  6712
    Num-Mis
935    6741


ROC Analysis in
Index:   TLI
Classification:  CvM2
Estimation Method:   ULSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
936           CvM2   TLI     ULSMV        100         30 0.7326408
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
936   0.5559721    0.6578111 0.9985187   0.5859125   0.8055752  7283
    Num-Mis
936    7260


ROC Analysis in
Index:   TLI
Classification:  CvM2
Estimation Method:   ULSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
937           CvM2   TLI     ULSMV        200        ALL 0.7841778
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
937   0.6095449    0.7502023 0.9734783   0.5111478   0.9460936 21818
    Num-Mis
937   21873


ROC Analysis in
Index:   TLI
Classification:  CvM2
Estimation Method:   ULSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
938           CvM2   TLI     ULSMV        200          5 0.7625316
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
938   0.5758343    0.7452154 0.9700092   0.5358869   0.9143184  6835
    Num-Mis
938    6898


ROC Analysis in
Index:   TLI
Classification:  CvM2
Estimation Method:   ULSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS      AUC
939           CvM2   TLI     ULSMV        200         10 0.779721
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
939   0.6037176    0.7440429 0.9620941   0.5045202   0.9738949  7373
    Num-Mis
939    7376


ROC Analysis in
Index:   TLI
Classification:  CvM2
Estimation Method:   ULSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
940           CvM2   TLI     ULSMV        200         30 0.8217721
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
940   0.6507471    0.7809086  0.998612   0.7664835    0.742388  7610
    Num-Mis
940    7599


ROC Analysis in
Index:   TLI
Classification:  CvM2
Estimation Method:   WLSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
941           CvM2   TLI     WLSMV        ALL        ALL 0.6649316
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
941   0.5417451    0.6998447 0.9958248   0.5626895   0.7070446 66216
    Num-Mis
941   66637


ROC Analysis in
Index:   TLI
Classification:  CvM2
Estimation Method:   WLSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
942           CvM2   TLI     WLSMV        ALL          5 0.6629151
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
942   0.5470545    0.6831942 0.9839416   0.5852068   0.6916155 18319
    Num-Mis
942   18523


ROC Analysis in
Index:   TLI
Classification:  CvM2
Estimation Method:   WLSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
943           CvM2   TLI     WLSMV        ALL         10 0.6942403
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
943   0.5530581    0.7103146 0.9898123   0.5354673   0.7807627 22482
    Num-Mis
943   22664


ROC Analysis in
Index:   TLI
Classification:  CvM2
Estimation Method:   WLSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
944           CvM2   TLI     WLSMV        ALL         30 0.6753114
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
944   0.5341159    0.6580439 0.9981512   0.4582844   0.8471062 25415
    Num-Mis
944   25450


ROC Analysis in
Index:   TLI
Classification:  CvM2
Estimation Method:   WLSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
945           CvM2   TLI     WLSMV         30        ALL 0.5410447
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
945   0.5074232    0.5593931 0.9995589   0.4411517   0.6353448 12160
    Num-Mis
945   12278


ROC Analysis in
Index:   TLI
Classification:  CvM2
Estimation Method:   WLSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
946           CvM2   TLI     WLSMV         30          5 0.5531451
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
946   0.5138365    0.5573847 0.9615762   0.4746717   0.6146527  2872
    Num-Mis
946    2886


ROC Analysis in
Index:   TLI
Classification:  CvM2
Estimation Method:   WLSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
947           CvM2   TLI     WLSMV         30         10 0.5610682
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
947   0.5119129    0.5769417 0.9892183   0.4238035   0.6883202  4066
    Num-Mis
947    4131


ROC Analysis in
Index:   TLI
Classification:  CvM2
Estimation Method:   WLSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
948           CvM2   TLI     WLSMV         30         30 0.5305092
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
948   0.5040842    0.5345607 0.9997666   0.2347909   0.8251816  5222
    Num-Mis
948    5261


ROC Analysis in
Index:   TLI
Classification:  CvM2
Estimation Method:   WLSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
949           CvM2   TLI     WLSMV         50        ALL 0.6008016
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
949   0.5233961    0.6215412 0.9991902    0.589546   0.5864957 15127
    Num-Mis
949   15246


ROC Analysis in
Index:   TLI
Classification:  CvM2
Estimation Method:   WLSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
950           CvM2   TLI     WLSMV         50          5 0.5900847
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
950   0.5229755    0.5944339 0.9645923   0.4340255   0.7078014  3768
    Num-Mis
950    3851


ROC Analysis in
Index:   TLI
Classification:  CvM2
Estimation Method:   WLSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
951           CvM2   TLI     WLSMV         50         10 0.6346991
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
951    0.539221    0.6388316 0.9917378   0.6133663   0.5916204  5179
    Num-Mis
951    5218


ROC Analysis in
Index:   TLI
Classification:  CvM2
Estimation Method:   WLSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
952           CvM2   TLI     WLSMV         50         30 0.6066038
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
952   0.5170463    0.5562711 0.9993359   0.3933225   0.7974786  6180
    Num-Mis
952    6177


ROC Analysis in
Index:   TLI
Classification:  CvM2
Estimation Method:   WLSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
953           CvM2   TLI     WLSMV        100        ALL 0.6988463
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
953    0.558847    0.7123729 0.9952903   0.6032572   0.7077778 18265
    Num-Mis
953   18372


ROC Analysis in
Index:   TLI
Classification:  CvM2
Estimation Method:   WLSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
954           CvM2   TLI     WLSMV        100          5 0.6891992
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
954   0.5584747    0.6937169  0.981951    0.582681   0.7106319  5191
    Num-Mis
954    5241


ROC Analysis in
Index:   TLI
Classification:  CvM2
Estimation Method:   WLSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
955           CvM2   TLI     WLSMV        100         10 0.7332334
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
955   0.5732233    0.7224591 0.9880902   0.5353175   0.8362313  6232
    Num-Mis
955    6279


ROC Analysis in
Index:   TLI
Classification:  CvM2
Estimation Method:   WLSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
956           CvM2   TLI     WLSMV        100         30 0.7227988
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
956   0.5516772    0.6912255 0.9972468   0.4917973   0.8810831  6842
    Num-Mis
956    6852


ROC Analysis in
Index:   TLI
Classification:  CvM2
Estimation Method:   WLSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
957           CvM2   TLI     WLSMV        200        ALL 0.7756177
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
957   0.6106023    0.7671296 0.9960537   0.6439521   0.7711834 20664
    Num-Mis
957   20741


ROC Analysis in
Index:   TLI
Classification:  CvM2
Estimation Method:   WLSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
958           CvM2   TLI     WLSMV        200          5 0.7598017
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
958   0.5784405    0.7492788  0.983609   0.5474453   0.8825609  6488
    Num-Mis
958    6545


ROC Analysis in
Index:   TLI
Classification:  CvM2
Estimation Method:   WLSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
959           CvM2   TLI     WLSMV        200         10 0.7944229
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
959   0.6107782    0.7755709 0.9918259   0.5806906   0.8902743  7005
    Num-Mis
959    7036


ROC Analysis in
Index:   TLI
Classification:  CvM2
Estimation Method:   WLSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
960           CvM2   TLI     WLSMV        200         30 0.8265385
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
960   0.6384749    0.8120601 0.9983115   0.6627707   0.8726027  7171
    Num-Mis
960    7160


ROC Analysis in
Index:   RMSEA
Classification:  CvM2
Estimation Method:   ALL
Level-2 Sample Size:     ALL
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1

Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
961           CvM2 RMSEA       ALL        ALL        ALL 0.6644986
    partial-AUC Smoothed-AUC   Threshold Specificity Sensitivity  Num-C
961   0.5483738     0.691537 0.008547063    0.587031   0.6791605 223583
    Num-Mis
961  223868


ROC Analysis in
Index:   RMSEA
Classification:  CvM2
Estimation Method:   ALL
Level-2 Sample Size:     ALL
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS     AUC
962           CvM2 RMSEA       ALL        ALL          5 0.65363
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
962   0.5494565    0.6710572 0.01585464   0.5782123   0.6867239 62891
    Num-Mis
962   63082


ROC Analysis in
Index:   RMSEA
Classification:  CvM2
Estimation Method:   ALL
Level-2 Sample Size:     ALL
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
963           CvM2 RMSEA       ALL        ALL         10 0.6857763
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
963   0.5562659    0.7043255  0.011702   0.5727523   0.7335673 75570
    Num-Mis
963   75749


ROC Analysis in
Index:   RMSEA
Classification:  CvM2
Estimation Method:   ALL
Level-2 Sample Size:     ALL
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
964           CvM2 RMSEA       ALL        ALL         30 0.6806013
    partial-AUC Smoothed-AUC   Threshold Specificity Sensitivity Num-C
964   0.5432224    0.7093997 0.005470247   0.5437541   0.7633629 85122
    Num-Mis
964   85037


ROC Analysis in
Index:   RMSEA
Classification:  CvM2
Estimation Method:   ALL
Level-2 Sample Size:     30
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
965           CvM2 RMSEA       ALL         30        ALL 0.5509011
    partial-AUC Smoothed-AUC   Threshold Specificity Sensitivity Num-C
965   0.5121577    0.5582005 0.003869499    0.606752   0.4786706 43725
    Num-Mis
965   43767


ROC Analysis in
Index:   RMSEA
Classification:  CvM2
Estimation Method:   ALL
Level-2 Sample Size:     30
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
966           CvM2 RMSEA       ALL         30          5 0.5487943
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
966   0.5164921    0.5493668 0.01923467   0.6358072   0.4436602 10875
    Num-Mis
966   10825


ROC Analysis in
Index:   RMSEA
Classification:  CvM2
Estimation Method:   ALL
Level-2 Sample Size:     30
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS      AUC
967           CvM2 RMSEA       ALL         30         10 0.558209
    partial-AUC Smoothed-AUC   Threshold Specificity Sensitivity Num-C
967   0.5153229    0.5551916 0.008093642   0.6189588   0.4770538 14530
    Num-Mis
967   14595


ROC Analysis in
Index:   RMSEA
Classification:  CvM2
Estimation Method:   ALL
Level-2 Sample Size:     30
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1

Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in auc.roc(fit_roc[[key]], partial.auc = c(1, 0.8),
partial.auc.focus = "sp", : Partial AUC correction not defined for ROC
curves below the diagonal.

Warning in auc.roc(fit_roc[[key]], partial.auc = c(1, 0.8),
partial.auc.focus = "sp", : An upcoming version of pROC will set the
'transpose' argument to FALSE by default. Set transpose = TRUE explicitly
to keep the current behavior, or transpose = FALSE to adopt the new one
and silence this warning. Type help(coords_transpose) for additional
information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
968           CvM2 RMSEA       ALL         30         30 0.4430961
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
968          NA     0.472944      -Inf           0           1 18320
    Num-Mis
968   18347


ROC Analysis in
Index:   RMSEA
Classification:  CvM2
Estimation Method:   ALL
Level-2 Sample Size:     50
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1

Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
969           CvM2 RMSEA       ALL         50        ALL 0.6123434
    partial-AUC Smoothed-AUC   Threshold Specificity Sensitivity Num-C
969   0.5302955    0.6252843 0.008678979    0.581881   0.5977237 52104
    Num-Mis
969   52168


ROC Analysis in
Index:   RMSEA
Classification:  CvM2
Estimation Method:   ALL
Level-2 Sample Size:     50
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
970           CvM2 RMSEA       ALL         50          5 0.5896882
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
970   0.5268786    0.5903733 0.02149005    0.489449   0.6470121 13813
    Num-Mis
970   13894


ROC Analysis in
Index:   RMSEA
Classification:  CvM2
Estimation Method:   ALL
Level-2 Sample Size:     50
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
971           CvM2 RMSEA       ALL         50         10 0.6419132
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
971   0.5450052    0.6401259 0.01379486   0.5589045    0.648228 17513
    Num-Mis
971   17559


ROC Analysis in
Index:   RMSEA
Classification:  CvM2
Estimation Method:   ALL
Level-2 Sample Size:     50
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
972           CvM2 RMSEA       ALL         50         30 0.6294281
    partial-AUC Smoothed-AUC   Threshold Specificity Sensitivity Num-C
972   0.5248322    0.5911372 0.004548511   0.5222222   0.6788415 20778
    Num-Mis
972   20715


ROC Analysis in
Index:   RMSEA
Classification:  CvM2
Estimation Method:   ALL
Level-2 Sample Size:     100
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
973           CvM2 RMSEA       ALL        100        ALL 0.7045812
    partial-AUC Smoothed-AUC   Threshold Specificity Sensitivity Num-C
973    0.563424    0.7102897 0.008991137   0.5957531   0.7202656 60747
    Num-Mis
973   60844


ROC Analysis in
Index:   RMSEA
Classification:  CvM2
Estimation Method:   ALL
Level-2 Sample Size:     100
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
974           CvM2 RMSEA       ALL        100          5 0.6958923
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
974   0.5612332    0.6957933 0.01567153   0.5847962   0.7134269 17384
    Num-Mis
974   17448


ROC Analysis in
Index:   RMSEA
Classification:  CvM2
Estimation Method:   ALL
Level-2 Sample Size:     100
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
975           CvM2 RMSEA       ALL        100         10 0.7356396
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
975   0.5723871    0.7215674 0.01392941   0.4975937   0.8683138 20799
    Num-Mis
975   20843


ROC Analysis in
Index:   RMSEA
Classification:  CvM2
Estimation Method:   ALL
Level-2 Sample Size:     100
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
976           CvM2 RMSEA       ALL        100         30 0.7307521
    partial-AUC Smoothed-AUC   Threshold Specificity Sensitivity Num-C
976   0.5589811    0.7000308 0.006423158   0.5347313   0.8374584 22564
    Num-Mis
976   22553


ROC Analysis in
Index:   RMSEA
Classification:  CvM2
Estimation Method:   ALL
Level-2 Sample Size:     200
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
977           CvM2 RMSEA       ALL        200        ALL 0.7713377
    partial-AUC Smoothed-AUC   Threshold Specificity Sensitivity Num-C
977   0.6047376    0.7601646 0.006347401   0.6788897   0.7290744 67007
    Num-Mis
977   67089


ROC Analysis in
Index:   RMSEA
Classification:  CvM2
Estimation Method:   ALL
Level-2 Sample Size:     200
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
978           CvM2 RMSEA       ALL        200          5 0.7590622
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
978    0.576572    0.7424493 0.01485023   0.5359542   0.8922186 20819
    Num-Mis
978   20915


ROC Analysis in
Index:   RMSEA
Classification:  CvM2
Estimation Method:   ALL
Level-2 Sample Size:     200
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
979           CvM2 RMSEA       ALL        200         10 0.7874867
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
979    0.600501    0.7662125 0.01025851   0.5815198   0.8833003 22728
    Num-Mis
979   22752


ROC Analysis in
Index:   RMSEA
Classification:  CvM2
Estimation Method:   ALL
Level-2 Sample Size:     200
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
980           CvM2 RMSEA       ALL        200         30 0.8222461
    partial-AUC Smoothed-AUC   Threshold Specificity Sensitivity Num-C
980    0.637244    0.8039214 0.005261147   0.6360096   0.8819706 23460
    Num-Mis
980   23422


ROC Analysis in
Index:   RMSEA
Classification:  CvM2
Estimation Method:   MLR
Level-2 Sample Size:     ALL
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
981           CvM2 RMSEA       MLR        ALL        ALL 0.6534093
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
981   0.5662283    0.6618017 0.01035133   0.6719006   0.5763542 85732
    Num-Mis
981   85647


ROC Analysis in
Index:   RMSEA
Classification:  CvM2
Estimation Method:   MLR
Level-2 Sample Size:     ALL
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
982           CvM2 RMSEA       MLR        ALL          5 0.6298685
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
982   0.5558453    0.6373972 0.01584379   0.6861338   0.5476803 24596
    Num-Mis
982   24572


ROC Analysis in
Index:   RMSEA
Classification:  CvM2
Estimation Method:   MLR
Level-2 Sample Size:     ALL
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS      AUC
983           CvM2 RMSEA       MLR        ALL         10 0.661849
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
983   0.5618591    0.6693237 0.01345272   0.6367526   0.6339941 28824
    Num-Mis
983   28785


ROC Analysis in
Index:   RMSEA
Classification:  CvM2
Estimation Method:   MLR
Level-2 Sample Size:     ALL
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
984           CvM2 RMSEA       MLR        ALL         30 0.6940938
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
984   0.5787045    0.7005411  0.007577   0.6670924   0.6353455 32312
    Num-Mis
984   32290


ROC Analysis in
Index:   RMSEA
Classification:  CvM2
Estimation Method:   MLR
Level-2 Sample Size:     30
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
985           CvM2 RMSEA       MLR         30        ALL 0.5700731
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
985   0.5318355    0.5716016 0.01943743   0.6647601    0.456394 17855
    Num-Mis
985   17854


ROC Analysis in
Index:   RMSEA
Classification:  CvM2
Estimation Method:   MLR
Level-2 Sample Size:     30
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
986           CvM2 RMSEA       MLR         30          5 0.5656084
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
986   0.5228088     0.566123 0.04529535   0.5506403   0.5463499  4747
    Num-Mis
986    4739


ROC Analysis in
Index:   RMSEA
Classification:  CvM2
Estimation Method:   MLR
Level-2 Sample Size:     30
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
987           CvM2 RMSEA       MLR         30         10 0.5826056
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
987   0.5251233    0.5841969 0.03496651   0.3787731   0.7439143  5832
    Num-Mis
987    5832


ROC Analysis in
Index:   RMSEA
Classification:  CvM2
Estimation Method:   MLR
Level-2 Sample Size:     30
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
988           CvM2 RMSEA       MLR         30         30 0.6314016
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
988    0.541107    0.6322466 0.02033794   0.3885805   0.8095427  7276
    Num-Mis
988    7283


ROC Analysis in
Index:   RMSEA
Classification:  CvM2
Estimation Method:   MLR
Level-2 Sample Size:     50
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
989           CvM2 RMSEA       MLR         50        ALL 0.6204771
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
989   0.5433476    0.6236611 0.0134883   0.6464236   0.5487086 20411
    Num-Mis
989   20417


ROC Analysis in
Index:   RMSEA
Classification:  CvM2
Estimation Method:   MLR
Level-2 Sample Size:     50
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
990           CvM2 RMSEA       MLR         50          5 0.5987315
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
990   0.5293669    0.5980533 0.03530303   0.3721951   0.7731299  5706
    Num-Mis
990    5724


ROC Analysis in
Index:   RMSEA
Classification:  CvM2
Estimation Method:   MLR
Level-2 Sample Size:     50
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
991           CvM2 RMSEA       MLR         50         10 0.6517702
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
991   0.5464512    0.6503037 0.02563385   0.3820084   0.8408044  6787
    Num-Mis
991    6790


ROC Analysis in
Index:   RMSEA
Classification:  CvM2
Estimation Method:   MLR
Level-2 Sample Size:     50
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
992           CvM2 RMSEA       MLR         50         30 0.6841815
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
992   0.5503298    0.6769475 0.01336943   0.4650558   0.8269871  7918
    Num-Mis
992    7903


ROC Analysis in
Index:   RMSEA
Classification:  CvM2
Estimation Method:   MLR
Level-2 Sample Size:     100
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
993           CvM2 RMSEA       MLR        100        ALL 0.6993113
    partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
993   0.5679083    0.7034053 0.0103506   0.6211437    0.684552 22941
    Num-Mis
993   22901


ROC Analysis in
Index:   RMSEA
Classification:  CvM2
Estimation Method:   MLR
Level-2 Sample Size:     100
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS      AUC
994           CvM2 RMSEA       MLR        100          5 0.690739
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
994   0.5602774    0.6896376 0.02018605    0.529705   0.7563095  6647
    Num-Mis
994    6637


ROC Analysis in
Index:   RMSEA
Classification:  CvM2
Estimation Method:   MLR
Level-2 Sample Size:     100
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
995           CvM2 RMSEA       MLR        100         10 0.7243385
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
995   0.5615244    0.7140835 0.01574155   0.4915689   0.8768116  7855
    Num-Mis
995    7823


ROC Analysis in
Index:   RMSEA
Classification:  CvM2
Estimation Method:   MLR
Level-2 Sample Size:     100
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
996           CvM2 RMSEA       MLR        100         30 0.7523155
    partial-AUC Smoothed-AUC   Threshold Specificity Sensitivity Num-C
996   0.5811111     0.742874 0.009099335   0.5170835   0.8794501  8439
    Num-Mis
996    8441


ROC Analysis in
Index:   RMSEA
Classification:  CvM2
Estimation Method:   MLR
Level-2 Sample Size:     200
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS     AUC
997           CvM2 RMSEA       MLR        200        ALL 0.75752
    partial-AUC Smoothed-AUC   Threshold Specificity Sensitivity Num-C
997   0.5937881    0.7525731 0.007755049   0.6401875   0.7546884 24525
    Num-Mis
997   24475


ROC Analysis in
Index:   RMSEA
Classification:  CvM2
Estimation Method:   MLR
Level-2 Sample Size:     200
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
998           CvM2 RMSEA       MLR        200          5 0.7524064
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
998   0.5757796    0.7384392 0.01586141   0.5233066   0.8927798  7496
    Num-Mis
998    7472


ROC Analysis in
Index:   RMSEA
Classification:  CvM2
Estimation Method:   MLR
Level-2 Sample Size:     200
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
    Classification Index Estimator Level-2 SS Level-1 SS       AUC
999           CvM2 RMSEA       MLR        200         10 0.7710872
    partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
999   0.5869273    0.7536194 0.01150662   0.5303925   0.9110419  8350
    Num-Mis
999    8340


ROC Analysis in
Index:   RMSEA
Classification:  CvM2
Estimation Method:   MLR
Level-2 Sample Size:     200
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1000           CvM2 RMSEA       MLR        200         30 0.8118705
     partial-AUC Smoothed-AUC   Threshold Specificity Sensitivity Num-C
1000   0.6206459    0.7983628 0.006322809   0.6061856   0.9032809  8679
     Num-Mis
1000    8663


ROC Analysis in
Index:   RMSEA
Classification:  CvM2
Estimation Method:   ULSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1001           CvM2 RMSEA     ULSMV        ALL        ALL 0.6866864
     partial-AUC Smoothed-AUC   Threshold Specificity Sensitivity Num-C
1001   0.5457522    0.7107496 0.006539233   0.5671718   0.7304364 71635
     Num-Mis
1001   71584


ROC Analysis in
Index:   RMSEA
Classification:  CvM2
Estimation Method:   ULSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1002           CvM2 RMSEA     ULSMV        ALL          5 0.6844504
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1002   0.5469117     0.702845 0.01581024   0.5119317   0.7884333 19976
     Num-Mis
1002   19987


ROC Analysis in
Index:   RMSEA
Classification:  CvM2
Estimation Method:   ULSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1003           CvM2 RMSEA     ULSMV        ALL         10 0.7147878
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1003   0.5553749    0.7097142 0.01167086    0.514786   0.8333512 24264
     Num-Mis
1003   24300


ROC Analysis in
Index:   RMSEA
Classification:  CvM2
Estimation Method:   ULSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1004           CvM2 RMSEA     ULSMV        ALL         30 0.6954463
     partial-AUC Smoothed-AUC   Threshold Specificity Sensitivity Num-C
1004   0.5399437    0.6523048 0.003976704   0.4933862   0.8438495 27395
     Num-Mis
1004   27297


ROC Analysis in
Index:   RMSEA
Classification:  CvM2
Estimation Method:   ULSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1

Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in auc.roc(fit_roc[[key]], partial.auc = c(1, 0.8),
partial.auc.focus = "sp", : Partial AUC correction not defined for ROC
curves below the diagonal.

Warning in auc.roc(fit_roc[[key]], partial.auc = c(1, 0.8),
partial.auc.focus = "sp", : An upcoming version of pROC will set the
'transpose' argument to FALSE by default. Set transpose = TRUE explicitly
to keep the current behavior, or transpose = FALSE to adopt the new one
and silence this warning. Type help(coords_transpose) for additional
information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1005           CvM2 RMSEA     ULSMV         30        ALL 0.4330288
     partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
1005          NA    0.3950355      -Inf           0           1 13710
     Num-Mis
1005   13635


ROC Analysis in
Index:   RMSEA
Classification:  CvM2
Estimation Method:   ULSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1

Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1006           CvM2 RMSEA     ULSMV         30          5 0.5663999
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1006   0.5135569    0.5813053 0.01104999   0.5408971   0.5857143  3256
     Num-Mis
1006    3200


ROC Analysis in
Index:   RMSEA
Classification:  CvM2
Estimation Method:   ULSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1

Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in auc.roc(fit_roc[[key]], partial.auc = c(1, 0.8),
partial.auc.focus = "sp", : Partial AUC correction not defined for ROC
curves below the diagonal.

Warning in auc.roc(fit_roc[[key]], partial.auc = c(1, 0.8),
partial.auc.focus = "sp", : An upcoming version of pROC will set the
'transpose' argument to FALSE by default. Set transpose = TRUE explicitly
to keep the current behavior, or transpose = FALSE to adopt the new one
and silence this warning. Type help(coords_transpose) for additional
information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1007           CvM2 RMSEA     ULSMV         30         10 0.4123351
     partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
1007          NA    0.4023458      -Inf           0           1  4632
     Num-Mis
1007    4632


ROC Analysis in
Index:   RMSEA
Classification:  CvM2
Estimation Method:   ULSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in auc.roc(fit_roc[[key]], partial.auc = c(1, 0.8),
partial.auc.focus = "sp", : Partial AUC correction not defined for ROC
curves below the diagonal.

Warning in auc.roc(fit_roc[[key]], partial.auc = c(1, 0.8),
partial.auc.focus = "sp", : An upcoming version of pROC will set the
'transpose' argument to FALSE by default. Set transpose = TRUE explicitly
to keep the current behavior, or transpose = FALSE to adopt the new one
and silence this warning. Type help(coords_transpose) for additional
information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1008           CvM2 RMSEA     ULSMV         30         30 0.4331963
     partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
1008          NA    0.4439082      -Inf           0           1  5822
     Num-Mis
1008    5803


ROC Analysis in
Index:   RMSEA
Classification:  CvM2
Estimation Method:   ULSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1

Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1009           CvM2 RMSEA     ULSMV         50        ALL 0.6349143
     partial-AUC Smoothed-AUC   Threshold Specificity Sensitivity Num-C
1009   0.5307251    0.6587414 0.004617941   0.5639396   0.6614223 16566
     Num-Mis
1009   16505


ROC Analysis in
Index:   RMSEA
Classification:  CvM2
Estimation Method:   ULSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1010           CvM2 RMSEA     ULSMV         50          5 0.6086445
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1010   0.5274046    0.6105281 0.01588627   0.5028571    0.662069  4339
     Num-Mis
1010    4319


ROC Analysis in
Index:   RMSEA
Classification:  CvM2
Estimation Method:   ULSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1011           CvM2 RMSEA     ULSMV         50         10 0.6764119
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1011   0.5479215    0.6657029 0.01144342   0.5451552   0.7219604  5547
     Num-Mis
1011    5551


ROC Analysis in
Index:   RMSEA
Classification:  CvM2
Estimation Method:   ULSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1

Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in auc.roc(fit_roc[[key]], partial.auc = c(1, 0.8),
partial.auc.focus = "sp", : Partial AUC correction not defined for ROC
curves below the diagonal.

Warning in auc.roc(fit_roc[[key]], partial.auc = c(1, 0.8),
partial.auc.focus = "sp", : An upcoming version of pROC will set the
'transpose' argument to FALSE by default. Set transpose = TRUE explicitly
to keep the current behavior, or transpose = FALSE to adopt the new one
and silence this warning. Type help(coords_transpose) for additional
information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1012           CvM2 RMSEA     ULSMV         50         30 0.3565765
     partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
1012          NA    0.4508481      -Inf           0           1  6680
     Num-Mis
1012    6635


ROC Analysis in
Index:   RMSEA
Classification:  CvM2
Estimation Method:   ULSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1

Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1013           CvM2 RMSEA     ULSMV        100        ALL 0.7235371
     partial-AUC Smoothed-AUC   Threshold Specificity Sensitivity Num-C
1013   0.5651194    0.7143194 0.006948076   0.6268025    0.719219 19541
     Num-Mis
1013   19571


ROC Analysis in
Index:   RMSEA
Classification:  CvM2
Estimation Method:   ULSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1014           CvM2 RMSEA     ULSMV        100          5 0.7194853
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1014   0.5654469     0.712695 0.01675939   0.5233333   0.8208092  5546
     Num-Mis
1014    5570


ROC Analysis in
Index:   RMSEA
Classification:  CvM2
Estimation Method:   ULSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS      AUC
1015           CvM2 RMSEA     ULSMV        100         10 0.756005
     partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
1015   0.5819518    0.7363212 0.0116813   0.5755287   0.8300878  6712
     Num-Mis
1015    6741


ROC Analysis in
Index:   RMSEA
Classification:  CvM2
Estimation Method:   ULSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1016           CvM2 RMSEA     ULSMV        100         30 0.7352972
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1016   0.5559721    0.6764206 0.00535709    0.512629   0.8828511  7283
     Num-Mis
1016    7260


ROC Analysis in
Index:   RMSEA
Classification:  CvM2
Estimation Method:   ULSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1017           CvM2 RMSEA     ULSMV        200        ALL 0.7816284
     partial-AUC Smoothed-AUC   Threshold Specificity Sensitivity Num-C
1017    0.609709    0.7602245 0.006341722   0.6628524   0.7584897 21818
     Num-Mis
1017   21873


ROC Analysis in
Index:   RMSEA
Classification:  CvM2
Estimation Method:   ULSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1018           CvM2 RMSEA     ULSMV        200          5 0.7661029
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1018   0.5758343    0.7435276 0.01459106   0.5414652    0.907083  6835
     Num-Mis
1018    6898


ROC Analysis in
Index:   RMSEA
Classification:  CvM2
Estimation Method:   ULSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1019           CvM2 RMSEA     ULSMV        200         10 0.7939776
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1019   0.6043384    0.7686893 0.01170311   0.5375522    0.943961  7373
     Num-Mis
1019    7376


ROC Analysis in
Index:   RMSEA
Classification:  CvM2
Estimation Method:   ULSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1020           CvM2 RMSEA     ULSMV        200         30 0.8332011
     partial-AUC Smoothed-AUC   Threshold Specificity Sensitivity Num-C
1020   0.6512445    0.8232311 0.003864172   0.7160027    0.817311  7610
     Num-Mis
1020    7599


ROC Analysis in
Index:   RMSEA
Classification:  CvM2
Estimation Method:   WLSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1021           CvM2 RMSEA     WLSMV        ALL        ALL 0.6639766
     partial-AUC Smoothed-AUC   Threshold Specificity Sensitivity Num-C
1021   0.5417451    0.6969142 0.006321586   0.5825879   0.6853985 66216
     Num-Mis
1021   66637


ROC Analysis in
Index:   RMSEA
Classification:  CvM2
Estimation Method:   WLSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1022           CvM2 RMSEA     WLSMV        ALL          5 0.6649912
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1022   0.5470545     0.683713 0.01347843   0.5847905   0.6917583 18319
     Num-Mis
1022   18523


ROC Analysis in
Index:   RMSEA
Classification:  CvM2
Estimation Method:   WLSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1023           CvM2 RMSEA     WLSMV        ALL         10 0.6961337
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1023   0.5530581    0.7111108 0.01065188   0.5474013   0.7666477 22482
     Num-Mis
1023   22664


ROC Analysis in
Index:   RMSEA
Classification:  CvM2
Estimation Method:   WLSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1024           CvM2 RMSEA     WLSMV        ALL         30 0.6744429
     partial-AUC Smoothed-AUC   Threshold Specificity Sensitivity Num-C
1024   0.5341159    0.6793592 0.004346847   0.4682726   0.8358062 25415
     Num-Mis
1024   25450


ROC Analysis in
Index:   RMSEA
Classification:  CvM2
Estimation Method:   WLSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1

Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in auc.roc(fit_roc[[key]], partial.auc = c(1, 0.8),
partial.auc.focus = "sp", : Partial AUC correction not defined for ROC
curves below the diagonal.

Warning in auc.roc(fit_roc[[key]], partial.auc = c(1, 0.8),
partial.auc.focus = "sp", : An upcoming version of pROC will set the
'transpose' argument to FALSE by default. Set transpose = TRUE explicitly
to keep the current behavior, or transpose = FALSE to adopt the new one
and silence this warning. Type help(coords_transpose) for additional
information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1025           CvM2 RMSEA     WLSMV         30        ALL 0.4594218
     partial-AUC Smoothed-AUC  Threshold Specificity  Sensitivity Num-C
1025          NA    0.4437043 0.05434185   0.9995797 0.0006465517 12160
     Num-Mis
1025   12278


ROC Analysis in
Index:   RMSEA
Classification:  CvM2
Estimation Method:   WLSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1

Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1026           CvM2 RMSEA     WLSMV         30          5 0.5532424
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1026   0.5138365    0.5567957 0.01669023   0.5422139   0.5451951  2872
     Num-Mis
1026    2886


ROC Analysis in
Index:   RMSEA
Classification:  CvM2
Estimation Method:   WLSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS     AUC
1027           CvM2 RMSEA     WLSMV         30         10 0.56021
     partial-AUC Smoothed-AUC   Threshold Specificity Sensitivity Num-C
1027   0.5119129    0.5747234 0.008477248   0.4779597   0.6292651  4066
     Num-Mis
1027    4131


ROC Analysis in
Index:   RMSEA
Classification:  CvM2
Estimation Method:   WLSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1

Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in auc.roc(fit_roc[[key]], partial.auc = c(1, 0.8),
partial.auc.focus = "sp", : Partial AUC correction not defined for ROC
curves below the diagonal.

Warning in auc.roc(fit_roc[[key]], partial.auc = c(1, 0.8),
partial.auc.focus = "sp", : An upcoming version of pROC will set the
'transpose' argument to FALSE by default. Set transpose = TRUE explicitly
to keep the current behavior, or transpose = FALSE to adopt the new one
and silence this warning. Type help(coords_transpose) for additional
information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS      AUC
1028           CvM2 RMSEA     WLSMV         30         30 0.470107
     partial-AUC Smoothed-AUC  Threshold Specificity  Sensitivity Num-C
1028          NA    0.4552334 0.02109946           1 0.0004842615  5222
     Num-Mis
1028    5261


ROC Analysis in
Index:   RMSEA
Classification:  CvM2
Estimation Method:   WLSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1

Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1029           CvM2 RMSEA     WLSMV         50        ALL 0.5985787
     partial-AUC Smoothed-AUC   Threshold Specificity Sensitivity Num-C
1029   0.5233961    0.6172421 0.003028119   0.5887083   0.5868376 15127
     Num-Mis
1029   15246


ROC Analysis in
Index:   RMSEA
Classification:  CvM2
Estimation Method:   WLSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1030           CvM2 RMSEA     WLSMV         50          5 0.5887712
     partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
1030   0.5229755    0.5927496 0.0129992   0.6048225   0.5326241  3768
     Num-Mis
1030    3851


ROC Analysis in
Index:   RMSEA
Classification:  CvM2
Estimation Method:   WLSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1031           CvM2 RMSEA     WLSMV         50         10 0.6343467
     partial-AUC Smoothed-AUC   Threshold Specificity Sensitivity Num-C
1031    0.539221    0.6383471 0.008610152   0.6366337   0.5689046  5179
     Num-Mis
1031    5218


ROC Analysis in
Index:   RMSEA
Classification:  CvM2
Estimation Method:   WLSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1

Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in auc.roc(fit_roc[[key]], partial.auc = c(1, 0.8),
partial.auc.focus = "sp", : Partial AUC correction not defined for ROC
curves below the diagonal.

Warning in auc.roc(fit_roc[[key]], partial.auc = c(1, 0.8),
partial.auc.focus = "sp", : An upcoming version of pROC will set the
'transpose' argument to FALSE by default. Set transpose = TRUE explicitly
to keep the current behavior, or transpose = FALSE to adopt the new one
and silence this warning. Type help(coords_transpose) for additional
information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1032           CvM2 RMSEA     WLSMV         50         30 0.3959452
     partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
1032          NA    0.4198745      -Inf           0           1  6180
     Num-Mis
1032    6177


ROC Analysis in
Index:   RMSEA
Classification:  CvM2
Estimation Method:   WLSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1

Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1033           CvM2 RMSEA     WLSMV        100        ALL 0.6962231
     partial-AUC Smoothed-AUC   Threshold Specificity Sensitivity Num-C
1033    0.558847    0.7078596 0.007187654   0.6127002   0.6976389 18265
     Num-Mis
1033   18372


ROC Analysis in
Index:   RMSEA
Classification:  CvM2
Estimation Method:   WLSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1034           CvM2 RMSEA     WLSMV        100          5 0.6894993
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1034   0.5584747    0.6911666 0.01525541   0.5494129   0.7452357  5191
     Num-Mis
1034    5241


ROC Analysis in
Index:   RMSEA
Classification:  CvM2
Estimation Method:   WLSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1035           CvM2 RMSEA     WLSMV        100         10 0.7334591
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1035   0.5732246    0.7220799 0.01186078   0.5404762   0.8261221  6232
     Num-Mis
1035    6279


ROC Analysis in
Index:   RMSEA
Classification:  CvM2
Estimation Method:   WLSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1036           CvM2 RMSEA     WLSMV        100         30 0.7217372
     partial-AUC Smoothed-AUC   Threshold Specificity Sensitivity Num-C
1036   0.5516772    0.6888041 0.005433153    0.501276    0.866813  6842
     Num-Mis
1036    6852


ROC Analysis in
Index:   RMSEA
Classification:  CvM2
Estimation Method:   WLSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1037           CvM2 RMSEA     WLSMV        200        ALL 0.7753964
     partial-AUC Smoothed-AUC   Threshold Specificity Sensitivity Num-C
1037   0.6106059    0.7679174 0.006328131   0.6752096   0.7426568 20664
     Num-Mis
1037   20741


ROC Analysis in
Index:   RMSEA
Classification:  CvM2
Estimation Method:   WLSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1038           CvM2 RMSEA     WLSMV        200          5 0.7596518
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1038   0.5784311    0.7457836 0.01412278   0.5412985   0.8915947  6488
     Num-Mis
1038    6545


ROC Analysis in
Index:   RMSEA
Classification:  CvM2
Estimation Method:   WLSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1039           CvM2 RMSEA     WLSMV        200         10 0.7974237
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1039   0.6110922    0.7778021 0.00993403   0.6021846   0.8792305  7005
     Num-Mis
1039    7036


ROC Analysis in
Index:   RMSEA
Classification:  CvM2
Estimation Method:   WLSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1040           CvM2 RMSEA     WLSMV        200         30 0.8269546
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1040   0.6384934    0.8100616 0.00482315   0.6442076   0.8907534  7171
     Num-Mis
1040    7160


ROC Analysis in
Index:   SRMRW
Classification:  CvM2
Estimation Method:   ALL
Level-2 Sample Size:     ALL
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1041           CvM2 SRMRW       ALL        ALL        ALL 0.5270719
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity  Num-C
1041   0.5050447    0.5241129 0.03089441   0.4519094   0.5970939 223583
     Num-Mis
1041  223868


ROC Analysis in
Index:   SRMRW
Classification:  CvM2
Estimation Method:   ALL
Level-2 Sample Size:     ALL
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1042           CvM2 SRMRW       ALL        ALL          5 0.5238228
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1042   0.5115793    0.5229826 0.03700896   0.6386067   0.4048137 62891
     Num-Mis
1042   63082


ROC Analysis in
Index:   SRMRW
Classification:  CvM2
Estimation Method:   ALL
Level-2 Sample Size:     ALL
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1043           CvM2 SRMRW       ALL        ALL         10 0.5297345
     partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
1043   0.5079391    0.5281905 0.0311391   0.4978157   0.5534993 75570
     Num-Mis
1043   75749


ROC Analysis in
Index:   SRMRW
Classification:  CvM2
Estimation Method:   ALL
Level-2 Sample Size:     ALL
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1044           CvM2 SRMRW       ALL        ALL         30 0.5346527
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1044   0.5053971    0.5364883 0.02541707   0.2681299    0.795553 85122
     Num-Mis
1044   85037


ROC Analysis in
Index:   SRMRW
Classification:  CvM2
Estimation Method:   ALL
Level-2 Sample Size:     30
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1045           CvM2 SRMRW       ALL         30        ALL 0.5048396
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1045   0.5001435    0.5045758 0.04060926   0.5717554   0.4399802 43725
     Num-Mis
1045   43767


ROC Analysis in
Index:   SRMRW
Classification:  CvM2
Estimation Method:   ALL
Level-2 Sample Size:     30
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS      AUC
1046           CvM2 SRMRW       ALL         30          5 0.505245
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1046   0.5029866    0.5061326 0.06716653   0.6707473   0.3435846 10875
     Num-Mis
1046   10825


ROC Analysis in
Index:   SRMRW
Classification:  CvM2
Estimation Method:   ALL
Level-2 Sample Size:     30
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1047           CvM2 SRMRW       ALL         30         10 0.5041733
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1047   0.5001755    0.5057371 0.06937477   0.1117746   0.9038716 14530
     Num-Mis
1047   14595


ROC Analysis in
Index:   SRMRW
Classification:  CvM2
Estimation Method:   ALL
Level-2 Sample Size:     30
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1048           CvM2 SRMRW       ALL         30         30 0.5067717
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1048   0.5000111    0.5093702 0.04098274   0.1353547   0.8913487 18320
     Num-Mis
1048   18347


ROC Analysis in
Index:   SRMRW
Classification:  CvM2
Estimation Method:   ALL
Level-2 Sample Size:     50
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1049           CvM2 SRMRW       ALL         50        ALL 0.5172977
     partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
1049   0.5017553    0.5166333 0.0373886   0.4833454   0.5470253 52104
     Num-Mis
1049   52168


ROC Analysis in
Index:   SRMRW
Classification:  CvM2
Estimation Method:   ALL
Level-2 Sample Size:     50
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1050           CvM2 SRMRW       ALL         50          5 0.5133727
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1050   0.5053022    0.5150019 0.05412668   0.6178195   0.4052015 13813
     Num-Mis
1050   13894


ROC Analysis in
Index:   SRMRW
Classification:  CvM2
Estimation Method:   ALL
Level-2 Sample Size:     50
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1051           CvM2 SRMRW       ALL         50         10 0.5160583
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1051   0.5035622    0.5192413 0.04674925   0.1984394   0.8343606 17513
     Num-Mis
1051   17559


ROC Analysis in
Index:   SRMRW
Classification:  CvM2
Estimation Method:   ALL
Level-2 Sample Size:     50
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1052           CvM2 SRMRW       ALL         50         30 0.5162471
     partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
1052   0.5006917    0.5224578 0.0334954   0.1368217   0.9158016 20778
     Num-Mis
1052   20715


ROC Analysis in
Index:   SRMRW
Classification:  CvM2
Estimation Method:   ALL
Level-2 Sample Size:     100
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1053           CvM2 SRMRW       ALL        100        ALL 0.5323261
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1053    0.503121    0.5339022 0.03309533   0.3588441   0.6972255 60747
     Num-Mis
1053   60844


ROC Analysis in
Index:   SRMRW
Classification:  CvM2
Estimation Method:   ALL
Level-2 Sample Size:     100
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1054           CvM2 SRMRW       ALL        100          5 0.5383135
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1054   0.5103637    0.5429858 0.04426877   0.3651351   0.6895329 17384
     Num-Mis
1054   17448


ROC Analysis in
Index:   SRMRW
Classification:  CvM2
Estimation Method:   ALL
Level-2 Sample Size:     100
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1055           CvM2 SRMRW       ALL        100         10 0.5364562
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1055   0.5057399    0.5448917 0.03753676   0.1379179   0.9317371 20799
     Num-Mis
1055   20843


ROC Analysis in
Index:   SRMRW
Classification:  CvM2
Estimation Method:   ALL
Level-2 Sample Size:     100
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1056           CvM2 SRMRW       ALL        100         30 0.5277134
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1056   0.5028458    0.5401833 0.02550302   0.1529846   0.9350309 22564
     Num-Mis
1056   22553


ROC Analysis in
Index:   SRMRW
Classification:  CvM2
Estimation Method:   ALL
Level-2 Sample Size:     200
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1057           CvM2 SRMRW       ALL        200        ALL 0.5540429
     partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
1057   0.5059398    0.5608425 0.0282581    0.275412   0.8219541 67007
     Num-Mis
1057   67089


ROC Analysis in
Index:   SRMRW
Classification:  CvM2
Estimation Method:   ALL
Level-2 Sample Size:     200
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS      AUC
1058           CvM2 SRMRW       ALL        200          5 0.569767
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1058   0.5160193     0.579363 0.03191232   0.3834287   0.7208512 20819
     Num-Mis
1058   20915


ROC Analysis in
Index:   SRMRW
Classification:  CvM2
Estimation Method:   ALL
Level-2 Sample Size:     200
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1059           CvM2 SRMRW       ALL        200         10 0.5583185
     partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
1059   0.5097865    0.5733501  0.027724   0.1611777   0.9476018 22728
     Num-Mis
1059   22752


ROC Analysis in
Index:   SRMRW
Classification:  CvM2
Estimation Method:   ALL
Level-2 Sample Size:     200
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1060           CvM2 SRMRW       ALL        200         30 0.5449993
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1060   0.5064033    0.5663936 0.02286715   0.1531325   0.9737712 23460
     Num-Mis
1060   23422


ROC Analysis in
Index:   SRMRW
Classification:  CvM2
Estimation Method:   MLR
Level-2 Sample Size:     ALL
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1061           CvM2 SRMRW       MLR        ALL        ALL 0.5073959
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1061   0.5013981    0.5076245 0.02750466   0.3862961   0.6304884 85732
     Num-Mis
1061   85647


ROC Analysis in
Index:   SRMRW
Classification:  CvM2
Estimation Method:   MLR
Level-2 Sample Size:     ALL
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1062           CvM2 SRMRW       MLR        ALL          5 0.5220088
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1062   0.5159046    0.5241911 0.02863086   0.7521578   0.2961431 24596
     Num-Mis
1062   24572


ROC Analysis in
Index:   SRMRW
Classification:  CvM2
Estimation Method:   MLR
Level-2 Sample Size:     ALL
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1063           CvM2 SRMRW       MLR        ALL         10 0.5092418
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1063   0.5079653    0.5103482 0.01795352   0.7820117   0.2395441 28824
     Num-Mis
1063   28785


ROC Analysis in
Index:   SRMRW
Classification:  CvM2
Estimation Method:   MLR
Level-2 Sample Size:     ALL
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1064           CvM2 SRMRW       MLR        ALL         30 0.5036608
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1064   0.5040362    0.5043137 0.00866248   0.8843171   0.1269234 32312
     Num-Mis
1064   32290


ROC Analysis in
Index:   SRMRW
Classification:  CvM2
Estimation Method:   MLR
Level-2 Sample Size:     30
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1

Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in auc.roc(fit_roc[[key]], partial.auc = c(1, 0.8),
partial.auc.focus = "sp", : Partial AUC correction not defined for ROC
curves below the diagonal.

Warning in auc.roc(fit_roc[[key]], partial.auc = c(1, 0.8),
partial.auc.focus = "sp", : An upcoming version of pROC will set the
'transpose' argument to FALSE by default. Set transpose = TRUE explicitly
to keep the current behavior, or transpose = FALSE to adopt the new one
and silence this warning. Type help(coords_transpose) for additional
information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS      AUC
1065           CvM2 SRMRW       MLR         30        ALL 0.499033
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1065          NA     0.497029 0.02387039   0.2700985   0.7359809 17855
     Num-Mis
1065   17854


ROC Analysis in
Index:   SRMRW
Classification:  CvM2
Estimation Method:   MLR
Level-2 Sample Size:     30
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1

Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1066           CvM2 SRMRW       MLR         30          5 0.5212308
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1066   0.5054946    0.5230809 0.07146467   0.2043073   0.8360371  4747
     Num-Mis
1066    4739


ROC Analysis in
Index:   SRMRW
Classification:  CvM2
Estimation Method:   MLR
Level-2 Sample Size:     30
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in auc.roc(fit_roc[[key]], partial.auc = c(1, 0.8),
partial.auc.focus = "sp", : Partial AUC correction not defined for ROC
curves below the diagonal.

Warning in auc.roc(fit_roc[[key]], partial.auc = c(1, 0.8),
partial.auc.focus = "sp", : An upcoming version of pROC will set the
'transpose' argument to FALSE by default. Set transpose = TRUE explicitly
to keep the current behavior, or transpose = FALSE to adopt the new one
and silence this warning. Type help(coords_transpose) for additional
information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1067           CvM2 SRMRW       MLR         30         10 0.5036637
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1067          NA    0.5026763 0.04417389   0.3291139   0.6927945  5832
     Num-Mis
1067    5832


ROC Analysis in
Index:   SRMRW
Classification:  CvM2
Estimation Method:   MLR
Level-2 Sample Size:     30
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1

Setting direction: controls < cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls < cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1068           CvM2 SRMRW       MLR         30         30 0.5018503
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1068   0.5007879    0.5018587 0.02387039   0.6736717   0.3395626  7276
     Num-Mis
1068    7283


ROC Analysis in
Index:   SRMRW
Classification:  CvM2
Estimation Method:   MLR
Level-2 Sample Size:     50
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1

Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1069           CvM2 SRMRW       MLR         50        ALL 0.5062681
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1069   0.5009448    0.5079127 0.04770818   0.1812062   0.8336609 20411
     Num-Mis
1069   20417


ROC Analysis in
Index:   SRMRW
Classification:  CvM2
Estimation Method:   MLR
Level-2 Sample Size:     50
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1070           CvM2 SRMRW       MLR         50          5 0.5255121
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1070   0.5062096    0.5266457 0.04770818   0.6297561   0.4173228  5706
     Num-Mis
1070    5724


ROC Analysis in
Index:   SRMRW
Classification:  CvM2
Estimation Method:   MLR
Level-2 Sample Size:     50
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1071           CvM2 SRMRW       MLR         50         10 0.5158353
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1071   0.5034597    0.5152968 0.03114374   0.6238494   0.4084625  6787
     Num-Mis
1071    6790


ROC Analysis in
Index:   SRMRW
Classification:  CvM2
Estimation Method:   MLR
Level-2 Sample Size:     50
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1072           CvM2 SRMRW       MLR         50         30 0.5036688
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1072   0.5009639    0.5038101 0.01894172   0.2780669   0.7360444  7918
     Num-Mis
1072    7903


ROC Analysis in
Index:   SRMRW
Classification:  CvM2
Estimation Method:   MLR
Level-2 Sample Size:     100
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1073           CvM2 SRMRW       MLR        100        ALL 0.5089977
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1073   0.5015937    0.5134646 0.03637123   0.1343115   0.8923135 22941
     Num-Mis
1073   22901


ROC Analysis in
Index:   SRMRW
Classification:  CvM2
Estimation Method:   MLR
Level-2 Sample Size:     100
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS      AUC
1074           CvM2 SRMRW       MLR        100          5 0.555849
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1074   0.5118994     0.557376 0.03755281   0.3539676   0.7339677  6647
     Num-Mis
1074    6637


ROC Analysis in
Index:   SRMRW
Classification:  CvM2
Estimation Method:   MLR
Level-2 Sample Size:     100
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS      AUC
1075           CvM2 SRMRW       MLR        100         10 0.523329
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1075   0.5066601    0.5243129 0.02397024    0.398827    0.640942  7855
     Num-Mis
1075    7823


ROC Analysis in
Index:   SRMRW
Classification:  CvM2
Estimation Method:   MLR
Level-2 Sample Size:     100
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1076           CvM2 SRMRW       MLR        100         30 0.5100183
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1076   0.5024547    0.5101426 0.01335109   0.3022191   0.7204794  8439
     Num-Mis
1076    8441


ROC Analysis in
Index:   SRMRW
Classification:  CvM2
Estimation Method:   MLR
Level-2 Sample Size:     200
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1077           CvM2 SRMRW       MLR        200        ALL 0.5182916
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1077   0.5044682    0.5274508 0.02862269  0.08119508   0.9641235 24525
     Num-Mis
1077   24475


ROC Analysis in
Index:   SRMRW
Classification:  CvM2
Estimation Method:   MLR
Level-2 Sample Size:     200
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1078           CvM2 SRMRW       MLR        200          5 0.5920043
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1078    0.519971    0.5985563 0.02862269   0.2523671   0.8888087  7496
     Num-Mis
1078    7472


ROC Analysis in
Index:   SRMRW
Classification:  CvM2
Estimation Method:   MLR
Level-2 Sample Size:     200
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1079           CvM2 SRMRW       MLR        200         10 0.5477963
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1079   0.5107419    0.5481387 0.01612987   0.5550538   0.5181724  8350
     Num-Mis
1079    8340


ROC Analysis in
Index:   SRMRW
Classification:  CvM2
Estimation Method:   MLR
Level-2 Sample Size:     200
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1080           CvM2 SRMRW       MLR        200         30 0.5230472
     partial-AUC Smoothed-AUC   Threshold Specificity Sensitivity Num-C
1080   0.5066977     0.523142 0.008661681   0.5687285   0.4716336  8679
     Num-Mis
1080    8663


ROC Analysis in
Index:   SRMRW
Classification:  CvM2
Estimation Method:   ULSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1081           CvM2 SRMRW     ULSMV        ALL        ALL 0.5615768
     partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
1081   0.5208701    0.5532337 0.0317971    0.608116   0.5027636 71635
     Num-Mis
1081   71584


ROC Analysis in
Index:   SRMRW
Classification:  CvM2
Estimation Method:   ULSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1082           CvM2 SRMRW     ULSMV        ALL          5 0.5329052
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1082   0.5107292    0.5277895 0.04193575   0.6735102    0.406705 19976
     Num-Mis
1082   19987


ROC Analysis in
Index:   SRMRW
Classification:  CvM2
Estimation Method:   ULSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1083           CvM2 SRMRW     ULSMV        ALL         10 0.5631699
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1083   0.5156979    0.5550135 0.03241579   0.6595495   0.4689744 24264
     Num-Mis
1083   24300


ROC Analysis in
Index:   SRMRW
Classification:  CvM2
Estimation Method:   ULSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1084           CvM2 SRMRW     ULSMV        ALL         30 0.5996252
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1084   0.5114764    0.5875014 0.02541695   0.5651927   0.6177682 27395
     Num-Mis
1084   27297


ROC Analysis in
Index:   SRMRW
Classification:  CvM2
Estimation Method:   ULSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1085           CvM2 SRMRW     ULSMV         30        ALL 0.5132096
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1085   0.5020438    0.5117116 0.04125998   0.7223199   0.3125361 13710
     Num-Mis
1085   13635


ROC Analysis in
Index:   SRMRW
Classification:  CvM2
Estimation Method:   ULSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in auc.roc(fit_roc[[key]], partial.auc = c(1, 0.8),
partial.auc.focus = "sp", : Partial AUC correction not defined for ROC
curves below the diagonal.

Warning in auc.roc(fit_roc[[key]], partial.auc = c(1, 0.8),
partial.auc.focus = "sp", : An upcoming version of pROC will set the
'transpose' argument to FALSE by default. Set transpose = TRUE explicitly
to keep the current behavior, or transpose = FALSE to adopt the new one
and silence this warning. Type help(coords_transpose) for additional
information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1086           CvM2 SRMRW     ULSMV         30          5 0.5034707
     partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
1086          NA    0.5026181 0.1023064   0.3201407   0.7042017  3256
     Num-Mis
1086    3200


ROC Analysis in
Index:   SRMRW
Classification:  CvM2
Estimation Method:   ULSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in auc.roc(fit_roc[[key]], partial.auc = c(1, 0.8),
partial.auc.focus = "sp", : Partial AUC correction not defined for ROC
curves below the diagonal.

Warning in auc.roc(fit_roc[[key]], partial.auc = c(1, 0.8),
partial.auc.focus = "sp", : An upcoming version of pROC will set the
'transpose' argument to FALSE by default. Set transpose = TRUE explicitly
to keep the current behavior, or transpose = FALSE to adopt the new one
and silence this warning. Type help(coords_transpose) for additional
information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1087           CvM2 SRMRW     ULSMV         30         10 0.5159719
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1087          NA    0.5171963 0.06937189   0.3383809   0.7122889  4632
     Num-Mis
1087    4632


ROC Analysis in
Index:   SRMRW
Classification:  CvM2
Estimation Method:   ULSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1088           CvM2 SRMRW     ULSMV         30         30 0.5330342
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1088   0.5006544    0.5317993 0.04098274   0.4067049   0.6767279  5822
     Num-Mis
1088    5803


ROC Analysis in
Index:   SRMRW
Classification:  CvM2
Estimation Method:   ULSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1089           CvM2 SRMRW     ULSMV         50        ALL 0.5393851
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1089     0.50758    0.5358893 0.03449777   0.7156473   0.3528949 16566
     Num-Mis
1089   16505


ROC Analysis in
Index:   SRMRW
Classification:  CvM2
Estimation Method:   ULSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1090           CvM2 SRMRW     ULSMV         50          5 0.5222881
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1090   0.5016138    0.5237076 0.08435191    0.184127    0.862069  4339
     Num-Mis
1090    4319


ROC Analysis in
Index:   SRMRW
Classification:  CvM2
Estimation Method:   ULSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1091           CvM2 SRMRW     ULSMV         50         10 0.5436739
     partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
1091   0.5011232    0.5429035 0.0560052   0.2779868   0.8081998  5547
     Num-Mis
1091    5551


ROC Analysis in
Index:   SRMRW
Classification:  CvM2
Estimation Method:   ULSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1092           CvM2 SRMRW     ULSMV         50         30 0.5698775
     partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
1092   0.5013243    0.5689798 0.0334954   0.4078643    0.751421  6680
     Num-Mis
1092    6635


ROC Analysis in
Index:   SRMRW
Classification:  CvM2
Estimation Method:   ULSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1093           CvM2 SRMRW     ULSMV        100        ALL 0.5798846
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1093   0.5113477     0.575699 0.03458921   0.5196455   0.6132288 19541
     Num-Mis
1093   19571


ROC Analysis in
Index:   SRMRW
Classification:  CvM2
Estimation Method:   ULSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1094           CvM2 SRMRW     ULSMV        100          5 0.5630452
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1094   0.5032413    0.5663203 0.05504775        0.33   0.7923892  5546
     Num-Mis
1094    5570


ROC Analysis in
Index:   SRMRW
Classification:  CvM2
Estimation Method:   ULSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS     AUC
1095           CvM2 SRMRW     ULSMV        100         10 0.59469
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1095   0.5050185    0.5973418 0.03753676   0.4018127   0.8010691  6712
     Num-Mis
1095    6741


ROC Analysis in
Index:   SRMRW
Classification:  CvM2
Estimation Method:   ULSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1096           CvM2 SRMRW     ULSMV        100         30 0.6046102
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1096   0.5025907    0.6106894 0.02550302   0.4567769   0.8073394  7283
     Num-Mis
1096    7260


ROC Analysis in
Index:   SRMRW
Classification:  CvM2
Estimation Method:   ULSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1097           CvM2 SRMRW     ULSMV        200        ALL 0.6301327
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1097   0.5143687    0.6279872 0.02767604   0.5267194   0.6946094 21818
     Num-Mis
1097   21873


ROC Analysis in
Index:   SRMRW
Classification:  CvM2
Estimation Method:   ULSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1098           CvM2 SRMRW     ULSMV        200          5 0.6062107
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1098   0.5041372    0.6159728 0.04087162   0.3332094   0.8891851  6835
     Num-Mis
1098    6898


ROC Analysis in
Index:   SRMRW
Classification:  CvM2
Estimation Method:   ULSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1099           CvM2 SRMRW     ULSMV        200         10 0.6304168
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1099   0.5040486    0.6444342 0.02945181   0.4273296   0.8934911  7373
     Num-Mis
1099    7376


ROC Analysis in
Index:   SRMRW
Classification:  CvM2
Estimation Method:   ULSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1100           CvM2 SRMRW     ULSMV        200         30 0.6356855
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1100   0.5044023    0.6607956 0.02286715   0.4591346   0.9213137  7610
     Num-Mis
1100    7599


ROC Analysis in
Index:   SRMRW
Classification:  CvM2
Estimation Method:   WLSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1101           CvM2 SRMRW     WLSMV        ALL        ALL 0.5153255
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1101   0.5067277    0.5159654 0.02072695   0.7000455   0.3253091 66216
     Num-Mis
1101   66637


ROC Analysis in
Index:   SRMRW
Classification:  CvM2
Estimation Method:   WLSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1102           CvM2 SRMRW     WLSMV        ALL          5 0.5201233
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1102   0.5147645    0.5200321 0.03191146   0.7941993   0.2566776 18319
     Num-Mis
1102   18523


ROC Analysis in
Index:   SRMRW
Classification:  CvM2
Estimation Method:   WLSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1103           CvM2 SRMRW     WLSMV        ALL         10 0.5215005
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1103   0.5184123    0.5225226 0.02071165   0.8341512   0.2196927 22482
     Num-Mis
1103   22664


ROC Analysis in
Index:   SRMRW
Classification:  CvM2
Estimation Method:   WLSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1104           CvM2 SRMRW     WLSMV        ALL         30 0.5177626
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1104   0.5165627    0.5205406 0.01153281   0.8381316   0.2065442 25415
     Num-Mis
1104   25450


ROC Analysis in
Index:   SRMRW
Classification:  CvM2
Estimation Method:   WLSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1105           CvM2 SRMRW     WLSMV         30        ALL 0.5037785
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1105   0.5029971    0.5044854 0.02641473   0.8854561   0.1260776 12160
     Num-Mis
1105   12278


ROC Analysis in
Index:   SRMRW
Classification:  CvM2
Estimation Method:   WLSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1106           CvM2 SRMRW     WLSMV         30          5 0.5089774
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1106   0.5003359    0.5087586 0.08107929   0.4287054   0.5927688  2872
     Num-Mis
1106    2886


ROC Analysis in
Index:   SRMRW
Classification:  CvM2
Estimation Method:   WLSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in auc.roc(fit_roc[[key]], partial.auc = c(1, 0.8),
partial.auc.focus = "sp", : Partial AUC correction not defined for ROC
curves below the diagonal.

Warning in auc.roc(fit_roc[[key]], partial.auc = c(1, 0.8),
partial.auc.focus = "sp", : An upcoming version of pROC will set the
'transpose' argument to FALSE by default. Set transpose = TRUE explicitly
to keep the current behavior, or transpose = FALSE to adopt the new one
and silence this warning. Type help(coords_transpose) for additional
information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1107           CvM2 SRMRW     WLSMV         30         10 0.5043374
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1107          NA    0.5027957 0.05286928   0.4401763   0.5793963  4066
     Num-Mis
1107    4131


ROC Analysis in
Index:   SRMRW
Classification:  CvM2
Estimation Method:   WLSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1108           CvM2 SRMRW     WLSMV         30         30 0.5123144
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1108    0.502564    0.5127057 0.02641473   0.7409696    0.283293  5222
     Num-Mis
1108    5261


ROC Analysis in
Index:   SRMRW
Classification:  CvM2
Estimation Method:   WLSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1109           CvM2 SRMRW     WLSMV         50        ALL 0.5131531
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1109   0.5063751    0.5139251 0.02317166   0.7448484   0.2777778 15127
     Num-Mis
1109   15246


ROC Analysis in
Index:   SRMRW
Classification:  CvM2
Estimation Method:   WLSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1110           CvM2 SRMRW     WLSMV         50          5 0.5156152
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1110     0.50187    0.5139943 0.06185946   0.4701942   0.5673759  3768
     Num-Mis
1110    3851


ROC Analysis in
Index:   SRMRW
Classification:  CvM2
Estimation Method:   WLSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1111           CvM2 SRMRW     WLSMV         50         10 0.5172688
     partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
1111   0.5053642    0.5173744 0.0388987   0.5717822   0.4583544  5179
     Num-Mis
1111    5218


ROC Analysis in
Index:   SRMRW
Classification:  CvM2
Estimation Method:   WLSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1112           CvM2 SRMRW     WLSMV         50         30 0.5236413
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1112   0.5068895    0.5229692 0.02316883   0.3802932   0.6604311  6180
     Num-Mis
1112    6177


ROC Analysis in
Index:   SRMRW
Classification:  CvM2
Estimation Method:   WLSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1113           CvM2 SRMRW     WLSMV        100        ALL 0.5182637
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1113   0.5101616    0.5213352 0.02843019   0.4596962   0.5708333 18265
     Num-Mis
1113   18372


ROC Analysis in
Index:   SRMRW
Classification:  CvM2
Estimation Method:   WLSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1114           CvM2 SRMRW     WLSMV        100          5 0.5416309
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1114   0.5078704    0.5399476 0.04427504   0.4383562   0.6399198  5191
     Num-Mis
1114    5241


ROC Analysis in
Index:   SRMRW
Classification:  CvM2
Estimation Method:   WLSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1115           CvM2 SRMRW     WLSMV        100         10 0.5476465
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1115    0.511822    0.5467561 0.02843019   0.5238095   0.5531743  6232
     Num-Mis
1115    6279


ROC Analysis in
Index:   SRMRW
Classification:  CvM2
Estimation Method:   WLSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1116           CvM2 SRMRW     WLSMV        100         30 0.5444128
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1116   0.5134703    0.5438171 0.01663169   0.3426905   0.7285035  6842
     Num-Mis
1116    6852


ROC Analysis in
Index:   SRMRW
Classification:  CvM2
Estimation Method:   WLSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1117           CvM2 SRMRW     WLSMV        200        ALL 0.5352617
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1117   0.5188827    0.5423448 0.02072453   0.4746108   0.5840687 20664
     Num-Mis
1117   20741


ROC Analysis in
Index:   SRMRW
Classification:  CvM2
Estimation Method:   WLSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1118           CvM2 SRMRW     WLSMV        200          5 0.5803118
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1118   0.5128035    0.5810886 0.03191146   0.4471763   0.6865672  6488
     Num-Mis
1118    6545


ROC Analysis in
Index:   SRMRW
Classification:  CvM2
Estimation Method:   WLSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1119           CvM2 SRMRW     WLSMV        200         10 0.5991563
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1119   0.5190327    0.5988343 0.02072453   0.4859056   0.6740292  7005
     Num-Mis
1119    7036


ROC Analysis in
Index:   SRMRW
Classification:  CvM2
Estimation Method:   WLSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1120           CvM2 SRMRW     WLSMV        200         30 0.5893667
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1120   0.5234448    0.5877698 0.01145816    0.462702   0.6842466  7171
     Num-Mis
1120    7160


ROC Analysis in
Index:   SRMRB
Classification:  CvM2
Estimation Method:   ALL
Level-2 Sample Size:     ALL
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1121           CvM2 SRMRB       ALL        ALL        ALL 0.6343083
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity  Num-C
1121   0.6012881     0.641562 0.07595733   0.7774833   0.4406392 223583
     Num-Mis
1121  223868


ROC Analysis in
Index:   SRMRB
Classification:  CvM2
Estimation Method:   ALL
Level-2 Sample Size:     ALL
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1122           CvM2 SRMRB       ALL        ALL          5 0.5957964
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1122   0.5756658    0.6021792 0.07795057   0.8208481   0.3440874 62891
     Num-Mis
1122   63082


ROC Analysis in
Index:   SRMRB
Classification:  CvM2
Estimation Method:   ALL
Level-2 Sample Size:     ALL
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1123           CvM2 SRMRB       ALL        ALL         10 0.6306228
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1123   0.5986856     0.638614 0.07762711   0.7686725   0.4405446 75570
     Num-Mis
1123   75749


ROC Analysis in
Index:   SRMRB
Classification:  CvM2
Estimation Method:   ALL
Level-2 Sample Size:     ALL
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1124           CvM2 SRMRB       ALL        ALL         30 0.6747211
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1124   0.6245436    0.6833699 0.07136395   0.7732783   0.4967653 85122
     Num-Mis
1124   85037


ROC Analysis in
Index:   SRMRB
Classification:  CvM2
Estimation Method:   ALL
Level-2 Sample Size:     30
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS      AUC
1125           CvM2 SRMRB       ALL         30        ALL 0.584183
     partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
1125   0.5514069    0.5849464 0.1269397   0.7180486    0.406994 43725
     Num-Mis
1125   43767


ROC Analysis in
Index:   SRMRB
Classification:  CvM2
Estimation Method:   ALL
Level-2 Sample Size:     30
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS      AUC
1126           CvM2 SRMRB       ALL         30          5 0.560059
     partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
1126   0.5260481    0.5567136 0.1618427   0.5779138   0.5218049 10875
     Num-Mis
1126   10825


ROC Analysis in
Index:   SRMRB
Classification:  CvM2
Estimation Method:   ALL
Level-2 Sample Size:     30
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1127           CvM2 SRMRB       ALL         30         10 0.5745183
     partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
1127   0.5416049    0.5717778 0.1271804   0.7292405   0.3898017 14530
     Num-Mis
1127   14595


ROC Analysis in
Index:   SRMRB
Classification:  CvM2
Estimation Method:   ALL
Level-2 Sample Size:     30
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1128           CvM2 SRMRB       ALL         30         30 0.6229135
     partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
1128   0.5761248    0.6265972 0.1153295   0.7619324   0.4163997 18320
     Num-Mis
1128   18347


ROC Analysis in
Index:   SRMRB
Classification:  CvM2
Estimation Method:   ALL
Level-2 Sample Size:     50
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1129           CvM2 SRMRB       ALL         50        ALL 0.6297121
     partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
1129   0.5835437    0.6325244 0.1021625    0.726668   0.4658562 52104
     Num-Mis
1129   52168


ROC Analysis in
Index:   SRMRB
Classification:  CvM2
Estimation Method:   ALL
Level-2 Sample Size:     50
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1130           CvM2 SRMRB       ALL         50          5 0.5883687
     partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
1130   0.5422203    0.5840512 0.1233646   0.6264166   0.5130038 13813
     Num-Mis
1130   13894


ROC Analysis in
Index:   SRMRB
Classification:  CvM2
Estimation Method:   ALL
Level-2 Sample Size:     50
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1131           CvM2 SRMRB       ALL         50         10 0.6188552
     partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
1131   0.5775939    0.6176792 0.1024826    0.752601   0.4399076 17513
     Num-Mis
1131   17559


ROC Analysis in
Index:   SRMRB
Classification:  CvM2
Estimation Method:   ALL
Level-2 Sample Size:     50
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS      AUC
1132           CvM2 SRMRB       ALL         50         30 0.687456
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1132   0.6206236    0.6994396 0.09223316   0.7732558   0.4875048 20778
     Num-Mis
1132   20715


ROC Analysis in
Index:   SRMRB
Classification:  CvM2
Estimation Method:   ALL
Level-2 Sample Size:     100
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1133           CvM2 SRMRB       ALL        100        ALL 0.7013011
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1133    0.637066    0.7025249 0.08016667   0.7060858   0.5998769 60747
     Num-Mis
1133   60844


ROC Analysis in
Index:   SRMRB
Classification:  CvM2
Estimation Method:   ALL
Level-2 Sample Size:     100
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1134           CvM2 SRMRB       ALL        100          5 0.6407424
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1134   0.5913299    0.6358175 0.08803708   0.7308808   0.4948358 17384
     Num-Mis
1134   17448


ROC Analysis in
Index:   SRMRB
Classification:  CvM2
Estimation Method:   ALL
Level-2 Sample Size:     100
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1135           CvM2 SRMRB       ALL        100         10 0.6975729
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1135   0.6431798     0.701074 0.07755476   0.7622847   0.5475038 20799
     Num-Mis
1135   20843


ROC Analysis in
Index:   SRMRB
Classification:  CvM2
Estimation Method:   ALL
Level-2 Sample Size:     100
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1136           CvM2 SRMRB       ALL        100         30 0.7813121
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1136   0.6830266    0.8076335 0.07590907   0.6663887   0.7166825 22564
     Num-Mis
1136   22553


ROC Analysis in
Index:   SRMRB
Classification:  CvM2
Estimation Method:   ALL
Level-2 Sample Size:     200
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1137           CvM2 SRMRB       ALL        200        ALL 0.7657439
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1137   0.6811315    0.7645108 0.06132497   0.7288463   0.6769383 67007
     Num-Mis
1137   67089


ROC Analysis in
Index:   SRMRB
Classification:  CvM2
Estimation Method:   ALL
Level-2 Sample Size:     200
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1138           CvM2 SRMRB       ALL        200          5 0.7014304
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1138   0.6421317    0.6978339 0.06859066   0.7422244   0.5930496 20819
     Num-Mis
1138   20915


ROC Analysis in
Index:   SRMRB
Classification:  CvM2
Estimation Method:   ALL
Level-2 Sample Size:     200
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1139           CvM2 SRMRB       ALL        200         10 0.7757223
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1139      0.6961    0.7905704 0.05949249   0.7634121   0.6433656 22728
     Num-Mis
1139   22752


ROC Analysis in
Index:   SRMRB
Classification:  CvM2
Estimation Method:   ALL
Level-2 Sample Size:     200
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1140           CvM2 SRMRB       ALL        200         30 0.8482061
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1140   0.7286083    0.8756482 0.05827201   0.6741496   0.8252936 23460
     Num-Mis
1140   23422


ROC Analysis in
Index:   SRMRB
Classification:  CvM2
Estimation Method:   MLR
Level-2 Sample Size:     ALL
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1141           CvM2 SRMRB       MLR        ALL        ALL 0.6543155
     partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
1141     0.62121    0.6585972 0.0921247   0.8139512   0.4410687 85732
     Num-Mis
1141   85647


ROC Analysis in
Index:   SRMRB
Classification:  CvM2
Estimation Method:   MLR
Level-2 Sample Size:     ALL
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1142           CvM2 SRMRB       MLR        ALL          5 0.6167274
     partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
1142   0.5992008    0.6261995 0.0968636   0.8477749   0.3564002 24596
     Num-Mis
1142   24572


ROC Analysis in
Index:   SRMRB
Classification:  CvM2
Estimation Method:   MLR
Level-2 Sample Size:     ALL
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1143           CvM2 SRMRB       MLR        ALL         10 0.6528229
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1143   0.6207604    0.6607516 0.09198944   0.8156402   0.4320119 28824
     Num-Mis
1143   28785


ROC Analysis in
Index:   SRMRB
Classification:  CvM2
Estimation Method:   MLR
Level-2 Sample Size:     ALL
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1144           CvM2 SRMRB       MLR        ALL         30 0.6985647
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1144   0.6424585    0.7025888 0.08715711   0.8083204   0.5048712 32312
     Num-Mis
1144   32290


ROC Analysis in
Index:   SRMRB
Classification:  CvM2
Estimation Method:   MLR
Level-2 Sample Size:     30
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1145           CvM2 SRMRB       MLR         30        ALL 0.6129457
     partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
1145   0.5931906    0.6194343 0.1432923   0.8706705   0.3243844 17855
     Num-Mis
1145   17854


ROC Analysis in
Index:   SRMRB
Classification:  CvM2
Estimation Method:   MLR
Level-2 Sample Size:     30
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1146           CvM2 SRMRB       MLR         30          5 0.5749976
     partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
1146   0.5521396    0.5755448 0.1719339   0.7706636   0.3760139  4747
     Num-Mis
1146    4739


ROC Analysis in
Index:   SRMRB
Classification:  CvM2
Estimation Method:   MLR
Level-2 Sample Size:     30
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1147           CvM2 SRMRB       MLR         30         10 0.6067431
     partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
1147   0.5826539    0.6095071 0.1505877   0.8174294   0.3753651  5832
     Num-Mis
1147    5832


ROC Analysis in
Index:   SRMRB
Classification:  CvM2
Estimation Method:   MLR
Level-2 Sample Size:     30
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS     AUC
1148           CvM2 SRMRB       MLR         30         30 0.67643
     partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
1148   0.6443408    0.6868709 0.1431613   0.8053132   0.4858847  7276
     Num-Mis
1148    7283


ROC Analysis in
Index:   SRMRB
Classification:  CvM2
Estimation Method:   MLR
Level-2 Sample Size:     50
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1149           CvM2 SRMRB       MLR         50        ALL 0.6775714
     partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
1149     0.63961    0.6832563 0.1215622   0.8244039   0.4658899 20411
     Num-Mis
1149   20417


ROC Analysis in
Index:   SRMRB
Classification:  CvM2
Estimation Method:   MLR
Level-2 Sample Size:     50
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1150           CvM2 SRMRB       MLR         50          5 0.6294495
     partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
1150   0.5902871    0.6276711 0.1432437   0.7185366   0.5113189  5706
     Num-Mis
1150    5724


ROC Analysis in
Index:   SRMRB
Classification:  CvM2
Estimation Method:   MLR
Level-2 Sample Size:     50
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1151           CvM2 SRMRB       MLR         50         10 0.6753729
     partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
1151   0.6477233    0.6798613 0.1252916   0.8029289   0.5073314  6787
     Num-Mis
1151    6790


ROC Analysis in
Index:   SRMRB
Classification:  CvM2
Estimation Method:   MLR
Level-2 Sample Size:     50
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS      AUC
1152           CvM2 SRMRB       MLR         50         30 0.755793
     partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
1152   0.6935536    0.7680506 0.1192679   0.7583643   0.6243993  7918
     Num-Mis
1152    7903


ROC Analysis in
Index:   SRMRB
Classification:  CvM2
Estimation Method:   MLR
Level-2 Sample Size:     100
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1153           CvM2 SRMRB       MLR        100        ALL 0.7562357
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1153   0.6988917    0.7541539 0.09833865   0.7437923   0.6715747 22941
     Num-Mis
1153   22901


ROC Analysis in
Index:   SRMRB
Classification:  CvM2
Estimation Method:   MLR
Level-2 Sample Size:     100
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1154           CvM2 SRMRB       MLR        100          5 0.7028414
     partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
1154    0.673499    0.7041822 0.1041601   0.8221853   0.5490277  6647
     Num-Mis
1154    6637


ROC Analysis in
Index:   SRMRB
Classification:  CvM2
Estimation Method:   MLR
Level-2 Sample Size:     100
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1155           CvM2 SRMRB       MLR        100         10 0.7572508
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1155    0.720596    0.7619678 0.09370854   0.8321114   0.6057971  7855
     Num-Mis
1155    7823


ROC Analysis in
Index:   SRMRB
Classification:  CvM2
Estimation Method:   MLR
Level-2 Sample Size:     100
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1156           CvM2 SRMRB       MLR        100         30 0.8459032
     partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
1156    0.745575    0.8656437 0.0871621   0.8090877   0.6922806  8439
     Num-Mis
1156    8441


ROC Analysis in
Index:   SRMRB
Classification:  CvM2
Estimation Method:   MLR
Level-2 Sample Size:     200
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1157           CvM2 SRMRB       MLR        200        ALL 0.8082239
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1157   0.7289592    0.8002831 0.07205595   0.7953134   0.7101922 24525
     Num-Mis
1157   24475


ROC Analysis in
Index:   SRMRB
Classification:  CvM2
Estimation Method:   MLR
Level-2 Sample Size:     200
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1158           CvM2 SRMRB       MLR        200          5 0.7567302
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1158   0.7226279    0.7596851 0.07946139   0.8284778   0.6234657  7496
     Num-Mis
1158    7472


ROC Analysis in
Index:   SRMRB
Classification:  CvM2
Estimation Method:   MLR
Level-2 Sample Size:     200
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1159           CvM2 SRMRB       MLR        200         10 0.8213621
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1159   0.7513692     0.837613 0.07056176   0.8284126     0.67982  8350
     Num-Mis
1159    8340


ROC Analysis in
Index:   SRMRB
Classification:  CvM2
Estimation Method:   MLR
Level-2 Sample Size:     200
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1160           CvM2 SRMRB       MLR        200         30 0.8891467
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1160   0.7667859    0.9081671 0.07108312   0.6848797   0.9063568  8679
     Num-Mis
1160    8663


ROC Analysis in
Index:   SRMRB
Classification:  CvM2
Estimation Method:   ULSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1161           CvM2 SRMRB     ULSMV        ALL        ALL 0.6293084
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1161   0.6118976    0.6424639 0.06278156   0.8131284   0.4092727 71635
     Num-Mis
1161   71584


ROC Analysis in
Index:   SRMRB
Classification:  CvM2
Estimation Method:   ULSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1162           CvM2 SRMRB     ULSMV        ALL          5 0.5889908
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1162   0.5809251    0.6011212 0.06468718   0.8549527   0.3106718 19976
     Num-Mis
1162   19987


ROC Analysis in
Index:   SRMRB
Classification:  CvM2
Estimation Method:   ULSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1163           CvM2 SRMRB     ULSMV        ALL         10 0.6257959
     partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
1163   0.6097188    0.6408953 0.0583881   0.8690082     0.34873 24264
     Num-Mis
1163   24300


ROC Analysis in
Index:   SRMRB
Classification:  CvM2
Estimation Method:   ULSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1164           CvM2 SRMRB     ULSMV        ALL         30 0.6697489
     partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
1164   0.6384668    0.6816377 0.0592739   0.8116969   0.4635836 27395
     Num-Mis
1164   27297


ROC Analysis in
Index:   SRMRB
Classification:  CvM2
Estimation Method:   ULSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1165           CvM2 SRMRB     ULSMV         30        ALL 0.5752969
     partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
1165   0.5554653    0.5816137 0.1058865   0.7467292   0.3830156 13710
     Num-Mis
1165   13635


ROC Analysis in
Index:   SRMRB
Classification:  CvM2
Estimation Method:   ULSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1166           CvM2 SRMRB     ULSMV         30          5 0.5594067
     partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
1166   0.5294508    0.5585283  0.115527   0.8109059   0.2890756  3256
     Num-Mis
1166    3200


ROC Analysis in
Index:   SRMRB
Classification:  CvM2
Estimation Method:   ULSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1167           CvM2 SRMRB     ULSMV         30         10 0.5705774
     partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
1167   0.5416745    0.5689392 0.1077688   0.7350029   0.4030285  4632
     Num-Mis
1167    4632


ROC Analysis in
Index:   SRMRB
Classification:  CvM2
Estimation Method:   ULSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1168           CvM2 SRMRB     ULSMV         30         30 0.6049319
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1168   0.5866642    0.6146111 0.09580469   0.7962064   0.3954506  5822
     Num-Mis
1168    5803


ROC Analysis in
Index:   SRMRB
Classification:  CvM2
Estimation Method:   ULSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1169           CvM2 SRMRB     ULSMV         50        ALL 0.6217688
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1169    0.596928    0.6310956 0.08156372   0.8327244   0.3708307 16566
     Num-Mis
1169   16505


ROC Analysis in
Index:   SRMRB
Classification:  CvM2
Estimation Method:   ULSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1170           CvM2 SRMRB     ULSMV         50          5 0.5820304
     partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
1170   0.5491223    0.5795077 0.1038426        0.64   0.5040752  4339
     Num-Mis
1170    4319


ROC Analysis in
Index:   SRMRB
Classification:  CvM2
Estimation Method:   ULSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1171           CvM2 SRMRB     ULSMV         50         10 0.6169214
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1171   0.5909829    0.6201618 0.08778242   0.7577611   0.4575872  5547
     Num-Mis
1171    5551


ROC Analysis in
Index:   SRMRB
Classification:  CvM2
Estimation Method:   ULSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1172           CvM2 SRMRB     ULSMV         50         30 0.6735592
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1172    0.646789    0.6929561 0.07646525   0.8350039   0.4604017  6680
     Num-Mis
1172    6635


ROC Analysis in
Index:   SRMRB
Classification:  CvM2
Estimation Method:   ULSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1173           CvM2 SRMRB     ULSMV        100        ALL 0.7006194
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1173   0.6635893    0.7069506 0.06467943   0.8256383   0.4982069 19541
     Num-Mis
1173   19571


ROC Analysis in
Index:   SRMRB
Classification:  CvM2
Estimation Method:   ULSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1174           CvM2 SRMRB     ULSMV        100          5 0.6405202
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1174   0.6105262    0.6384943 0.07458292   0.7714286   0.4807322  5546
     Num-Mis
1174    5570


ROC Analysis in
Index:   SRMRB
Classification:  CvM2
Estimation Method:   ULSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS     AUC
1175           CvM2 SRMRB     ULSMV        100         10 0.69795
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1175   0.6812979    0.7060047 0.06299484   0.8787764   0.4894998  6712
     Num-Mis
1175    6741


ROC Analysis in
Index:   SRMRB
Classification:  CvM2
Estimation Method:   ULSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1176           CvM2 SRMRB     ULSMV        100         30 0.7771907
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1176    0.716372    0.8088081 0.05542041   0.9302739   0.4795342  7283
     Num-Mis
1176    7260


ROC Analysis in
Index:   SRMRB
Classification:  CvM2
Estimation Method:   ULSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1177           CvM2 SRMRB     ULSMV        200        ALL 0.7691662
     partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
1177   0.7151309    0.7722984 0.0493117   0.8469978   0.5710045 21818
     Num-Mis
1177   21873


ROC Analysis in
Index:   SRMRB
Classification:  CvM2
Estimation Method:   ULSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1178           CvM2 SRMRB     ULSMV        200          5 0.7025228
     partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
1178   0.6732784    0.7030371 0.0585027   0.7828189   0.5666413  6835
     Num-Mis
1178    6898


ROC Analysis in
Index:   SRMRB
Classification:  CvM2
Estimation Method:   ULSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1179           CvM2 SRMRB     ULSMV        200         10 0.7780741
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1179   0.7385369    0.7984617 0.04878916    0.876217   0.5743126  7373
     Num-Mis
1179    7376


ROC Analysis in
Index:   SRMRB
Classification:  CvM2
Estimation Method:   ULSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1180           CvM2 SRMRB     ULSMV        200         30 0.8576739
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1180   0.7699511    0.8836514 0.04320725   0.9182692   0.5853575  7610
     Num-Mis
1180    7599


ROC Analysis in
Index:   SRMRB
Classification:  CvM2
Estimation Method:   WLSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1181           CvM2 SRMRB     WLSMV        ALL        ALL 0.6411952
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1181   0.6141866    0.6475006 0.06859458   0.8269027   0.4143974 66216
     Num-Mis
1181   66637


ROC Analysis in
Index:   SRMRB
Classification:  CvM2
Estimation Method:   WLSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1182           CvM2 SRMRB     WLSMV        ALL          5 0.5975262
     partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
1182   0.5896041    0.6087904 0.0740422   0.8288926   0.3580917 18319
     Num-Mis
1182   18523


ROC Analysis in
Index:   SRMRB
Classification:  CvM2
Estimation Method:   WLSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1183           CvM2 SRMRB     WLSMV        ALL         10 0.6338268
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1183    0.609577    0.6416969 0.07063182   0.8059335    0.418782 22482
     Num-Mis
1183   22664


ROC Analysis in
Index:   SRMRB
Classification:  CvM2
Estimation Method:   WLSMV
Level-2 Sample Size:     ALL
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS      AUC
1184           CvM2 SRMRB     WLSMV        ALL         30 0.685282
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1184   0.6367668    0.6909374 0.06658966   0.8150215   0.4847204 25415
     Num-Mis
1184   25450


ROC Analysis in
Index:   SRMRB
Classification:  CvM2
Estimation Method:   WLSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1185           CvM2 SRMRB     WLSMV         30        ALL 0.5977312
     partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
1185   0.5700121     0.601954 0.1199285    0.784153   0.3737069 12160
     Num-Mis
1185   12278


ROC Analysis in
Index:   SRMRB
Classification:  CvM2
Estimation Method:   WLSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1186           CvM2 SRMRB     WLSMV         30          5 0.5717855
     partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
1186   0.5327886    0.5683575  0.152358   0.5778612   0.5547098  2872
     Num-Mis
1186    2886


ROC Analysis in
Index:   SRMRB
Classification:  CvM2
Estimation Method:   WLSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS      AUC
1187           CvM2 SRMRB     WLSMV         30         10 0.595653
     partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
1187   0.5616387    0.5922842 0.1224293   0.7808564   0.3910761  4066
     Num-Mis
1187    4131


ROC Analysis in
Index:   SRMRB
Classification:  CvM2
Estimation Method:   WLSMV
Level-2 Sample Size:     30
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1188           CvM2 SRMRB     WLSMV         30         30 0.6422862
     partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
1188    0.604821    0.6487618 0.1128495   0.8151141   0.4135593  5222
     Num-Mis
1188    5261


ROC Analysis in
Index:   SRMRB
Classification:  CvM2
Estimation Method:   WLSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1189           CvM2 SRMRB     WLSMV         50        ALL 0.6562574
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1189   0.6153377    0.6599925 0.09574142   0.8011392   0.4490598 15127
     Num-Mis
1189   15246


ROC Analysis in
Index:   SRMRB
Classification:  CvM2
Estimation Method:   WLSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1190           CvM2 SRMRB     WLSMV         50          5 0.6092351
     partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
1190   0.5585432    0.6036332 0.1118678   0.6965841   0.4829787  3768
     Num-Mis
1190    3851


ROC Analysis in
Index:   SRMRB
Classification:  CvM2
Estimation Method:   WLSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS      AUC
1191           CvM2 SRMRB     WLSMV         50         10 0.637114
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1191   0.6083267    0.6352686 0.09647139    0.830198   0.4215043  5179
     Num-Mis
1191    5218


ROC Analysis in
Index:   SRMRB
Classification:  CvM2
Estimation Method:   WLSMV
Level-2 Sample Size:     50
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1192           CvM2 SRMRB     WLSMV         50         30 0.7316863
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1192   0.6681658    0.7435035 0.09024532   0.8192182   0.5274502  6180
     Num-Mis
1192    6177


ROC Analysis in
Index:   SRMRB
Classification:  CvM2
Estimation Method:   WLSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1193           CvM2 SRMRB     WLSMV        100        ALL 0.7355552
     partial-AUC Smoothed-AUC Threshold Specificity Sensitivity Num-C
1193   0.6776965    0.7315307 0.0759994    0.745997   0.6347222 18265
     Num-Mis
1193   18372


ROC Analysis in
Index:   SRMRB
Classification:  CvM2
Estimation Method:   WLSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1194           CvM2 SRMRB     WLSMV        100          5 0.6622134
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1194   0.6310124    0.6579972 0.08173744   0.7945205   0.5005015  5191
     Num-Mis
1194    5241


ROC Analysis in
Index:   SRMRB
Classification:  CvM2
Estimation Method:   WLSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1195           CvM2 SRMRB     WLSMV        100         10 0.7236487
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1195   0.6844184    0.7241339 0.07403815   0.8039683   0.5822887  6232
     Num-Mis
1195    6279


ROC Analysis in
Index:   SRMRB
Classification:  CvM2
Estimation Method:   WLSMV
Level-2 Sample Size:     100
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1196           CvM2 SRMRB     WLSMV        100         30 0.8398731
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1196   0.7379365    0.8613429 0.06890352   0.7918338   0.6992316  6842
     Num-Mis
1196    6852


ROC Analysis in
Index:   SRMRB
Classification:  CvM2
Estimation Method:   WLSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     ALL

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1197           CvM2 SRMRB     WLSMV        200        ALL 0.7946389
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1197   0.7129171     0.784547 0.05798319   0.7573653   0.7228333 20664
     Num-Mis
1197   20741


ROC Analysis in
Index:   SRMRB
Classification:  CvM2
Estimation Method:   WLSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     5

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1198           CvM2 SRMRB     WLSMV        200          5 0.7217014
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1198   0.6829853    0.7171084 0.06216532    0.814829   0.5805185  6488
     Num-Mis
1198    6545


ROC Analysis in
Index:   SRMRB
Classification:  CvM2
Estimation Method:   WLSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     10

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1199           CvM2 SRMRB     WLSMV        200         10 0.8071528
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1199   0.7348535    0.8163607 0.05507709    0.824172   0.6654792  7005
     Num-Mis
1199    7036


ROC Analysis in
Index:   SRMRB
Classification:  CvM2
Estimation Method:   WLSMV
Level-2 Sample Size:     200
Level-1 Sample Size:     30

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

Setting levels: control = 0, case = 1
Setting direction: controls > cases

Warning in coords.roc(fit_roc[[key]], "best", ret = c("threshold",
"specificity", : An upcoming version of pROC will set the 'transpose'
argument to FALSE by default. Set transpose = TRUE explicitly to keep the
current behavior, or transpose = FALSE to adopt the new one and silence
this warning. Type help(coords_transpose) for additional information.

Version	Author	Date
982c8f1	noah-padgett	2019-05-18



Summary of ROC:
     Classification Index Estimator Level-2 SS Level-1 SS       AUC
1200           CvM2 SRMRB     WLSMV        200         30 0.8890717
     partial-AUC Smoothed-AUC  Threshold Specificity Sensitivity Num-C
1200   0.7677189    0.9092172 0.05562095   0.6837401   0.9054795  7171
     Num-Mis
1200    7160

kable(roc_summary[801:1200, ], format = 'html', digits=3) %>%
  kable_styling(full_width = T)

	Classification	Index	Estimator	Level-2 SS	Level-1 SS	AUC	partial-AUC	Smoothed-AUC	Threshold	Specificity	Sensitivity	Num-C	Num-Mis
801	CvM2	CFI	ALL	ALL	ALL	0.669	0.548	0.692	0.993	0.576	0.695	223583	223868
802	CvM2	CFI	ALL	ALL	5	0.654	0.549	0.672	0.981	0.593	0.675	62891	63082
803	CvM2	CFI	ALL	ALL	10	0.687	0.556	0.699	0.990	0.588	0.714	75570	75749
804	CvM2	CFI	ALL	ALL	30	0.686	0.543	0.682	0.998	0.556	0.752	85122	85037
805	CvM2	CFI	ALL	30	ALL	0.557	0.512	0.570	0.982	0.455	0.644	43725	43767
806	CvM2	CFI	ALL	30	5	0.556	0.516	0.559	0.971	0.670	0.420	10875	10825
807	CvM2	CFI	ALL	30	10	0.569	0.515	0.572	0.990	0.589	0.516	14530	14595
808	CvM2	CFI	ALL	30	30	0.565	0.510	0.535	0.986	0.292	0.818	18320	18347
809	CvM2	CFI	ALL	50	ALL	0.620	0.530	0.631	0.990	0.538	0.655	52104	52168
810	CvM2	CFI	ALL	50	5	0.595	0.527	0.600	0.971	0.566	0.586	13813	13894
811	CvM2	CFI	ALL	50	10	0.649	0.545	0.652	0.980	0.515	0.704	17513	17559
812	CvM2	CFI	ALL	50	30	0.637	0.525	0.599	0.995	0.431	0.788	20778	20715
813	CvM2	CFI	ALL	100	ALL	0.707	0.563	0.706	0.994	0.599	0.720	60747	60844
814	CvM2	CFI	ALL	100	5	0.695	0.561	0.701	0.977	0.548	0.757	17384	17448
815	CvM2	CFI	ALL	100	10	0.731	0.572	0.731	0.986	0.530	0.835	20799	20843
816	CvM2	CFI	ALL	100	30	0.731	0.559	0.718	0.997	0.543	0.829	22564	22553
817	CvM2	CFI	ALL	200	ALL	0.771	0.605	0.762	0.995	0.619	0.788	67007	67089
818	CvM2	CFI	ALL	200	5	0.757	0.577	0.756	0.985	0.567	0.859	20819	20915
819	CvM2	CFI	ALL	200	10	0.778	0.600	0.770	0.990	0.535	0.911	22728	22752
820	CvM2	CFI	ALL	200	30	0.816	0.637	0.811	0.998	0.694	0.817	23460	23422
821	CvM2	CFI	MLR	ALL	ALL	0.656	0.567	0.666	0.991	0.652	0.612	85732	85647
822	CvM2	CFI	MLR	ALL	5	0.630	0.557	0.639	0.981	0.669	0.582	24596	24572
823	CvM2	CFI	MLR	ALL	10	0.664	0.563	0.674	0.988	0.662	0.627	28824	28785
824	CvM2	CFI	MLR	ALL	30	0.697	0.579	0.705	0.995	0.642	0.669	32312	32290
825	CvM2	CFI	MLR	30	ALL	0.574	0.533	0.575	0.972	0.638	0.494	17855	17854
826	CvM2	CFI	MLR	30	5	0.563	0.525	0.562	0.896	0.573	0.551	4747	4739
827	CvM2	CFI	MLR	30	10	0.587	0.527	0.586	0.940	0.477	0.657	5832	5832
828	CvM2	CFI	MLR	30	30	0.638	0.543	0.637	0.982	0.576	0.630	7276	7283
829	CvM2	CFI	MLR	50	ALL	0.624	0.544	0.629	0.986	0.623	0.586	20411	20417
830	CvM2	CFI	MLR	50	5	0.598	0.530	0.601	0.971	0.685	0.475	5706	5724
831	CvM2	CFI	MLR	50	10	0.657	0.548	0.658	0.967	0.489	0.753	6787	6790
832	CvM2	CFI	MLR	50	30	0.688	0.551	0.685	0.988	0.477	0.824	7918	7903
833	CvM2	CFI	MLR	100	ALL	0.704	0.568	0.714	0.991	0.595	0.728	22941	22901
834	CvM2	CFI	MLR	100	5	0.691	0.561	0.699	0.977	0.580	0.727	6647	6637
835	CvM2	CFI	MLR	100	10	0.732	0.562	0.729	0.985	0.532	0.854	7855	7823
836	CvM2	CFI	MLR	100	30	0.757	0.581	0.748	0.995	0.551	0.857	8439	8441
837	CvM2	CFI	MLR	200	ALL	0.764	0.594	0.767	0.995	0.626	0.789	24525	24475
838	CvM2	CFI	MLR	200	5	0.761	0.576	0.759	0.986	0.592	0.845	7496	7472
839	CvM2	CFI	MLR	200	10	0.778	0.587	0.763	0.991	0.555	0.903	8350	8340
840	CvM2	CFI	MLR	200	30	0.815	0.621	0.804	0.997	0.620	0.898	8679	8663
841	CvM2	CFI	ULSMV	ALL	ALL	0.691	0.546	0.719	0.978	0.435	0.871	71635	71584
842	CvM2	CFI	ULSMV	ALL	5	0.677	0.547	0.706	0.971	0.535	0.767	19976	19987
843	CvM2	CFI	ULSMV	ALL	10	0.711	0.555	0.722	0.970	0.455	0.888	24264	24300
844	CvM2	CFI	ULSMV	ALL	30	0.696	0.540	0.596	0.999	0.522	0.812	27395	27297
845	CvM2	CFI	ULSMV	30	ALL	0.570	0.512	0.622	0.999	0.409	0.720	13710	13635
846	CvM2	CFI	ULSMV	30	5	0.565	0.514	0.584	0.972	0.521	0.604	3256	3200
847	CvM2	CFI	ULSMV	30	10	0.589	0.516	0.611	0.999	0.487	0.664	4632	4632
848	CvM2	CFI	ULSMV	30	30	0.571	0.510	0.422	0.999	0.250	0.881	5822	5803
849	CvM2	CFI	ULSMV	50	ALL	0.641	0.531	0.665	0.996	0.545	0.685	16566	16505
850	CvM2	CFI	ULSMV	50	5	0.606	0.527	0.615	0.991	0.644	0.521	4339	4319
851	CvM2	CFI	ULSMV	50	10	0.675	0.548	0.682	0.971	0.466	0.800	5547	5551
852	CvM2	CFI	ULSMV	50	30	0.647	0.525	0.537	0.998	0.399	0.861	6680	6635
853	CvM2	CFI	ULSMV	100	ALL	0.729	0.565	0.712	0.970	0.447	0.906	19541	19571
854	CvM2	CFI	ULSMV	100	5	0.715	0.565	0.718	0.975	0.584	0.752	5546	5570
855	CvM2	CFI	ULSMV	100	10	0.750	0.582	0.737	0.964	0.470	0.932	6712	6741
856	CvM2	CFI	ULSMV	100	30	0.733	0.556	0.658	0.999	0.586	0.806	7283	7260
857	CvM2	CFI	ULSMV	200	ALL	0.784	0.610	0.750	0.978	0.511	0.946	21818	21873
858	CvM2	CFI	ULSMV	200	5	0.763	0.576	0.745	0.975	0.536	0.914	6835	6898
859	CvM2	CFI	ULSMV	200	10	0.780	0.604	0.744	0.968	0.505	0.974	7373	7376
860	CvM2	CFI	ULSMV	200	30	0.822	0.651	0.781	0.999	0.766	0.742	7610	7599
861	CvM2	CFI	WLSMV	ALL	ALL	0.665	0.542	0.700	0.997	0.563	0.707	66216	66637
862	CvM2	CFI	WLSMV	ALL	5	0.663	0.547	0.683	0.987	0.585	0.692	18319	18523
863	CvM2	CFI	WLSMV	ALL	10	0.694	0.553	0.710	0.992	0.535	0.781	22482	22664
864	CvM2	CFI	WLSMV	ALL	30	0.675	0.534	0.658	0.998	0.458	0.847	25415	25450
865	CvM2	CFI	WLSMV	30	ALL	0.541	0.507	0.559	1.000	0.441	0.635	12160	12278
866	CvM2	CFI	WLSMV	30	5	0.553	0.514	0.557	0.968	0.475	0.615	2872	2886
867	CvM2	CFI	WLSMV	30	10	0.561	0.512	0.577	0.991	0.424	0.688	4066	4131
868	CvM2	CFI	WLSMV	30	30	0.531	0.504	0.535	1.000	0.235	0.825	5222	5261
869	CvM2	CFI	WLSMV	50	ALL	0.601	0.523	0.622	0.999	0.590	0.586	15127	15246
870	CvM2	CFI	WLSMV	50	5	0.590	0.523	0.594	0.970	0.434	0.708	3768	3851
871	CvM2	CFI	WLSMV	50	10	0.635	0.539	0.639	0.993	0.613	0.592	5179	5218
872	CvM2	CFI	WLSMV	50	30	0.607	0.517	0.556	0.999	0.393	0.797	6180	6177
873	CvM2	CFI	WLSMV	100	ALL	0.699	0.559	0.712	0.996	0.603	0.708	18265	18372
874	CvM2	CFI	WLSMV	100	5	0.689	0.558	0.694	0.985	0.583	0.711	5191	5241
875	CvM2	CFI	WLSMV	100	10	0.733	0.573	0.722	0.990	0.535	0.836	6232	6279
876	CvM2	CFI	WLSMV	100	30	0.723	0.552	0.691	0.998	0.492	0.881	6842	6852
877	CvM2	CFI	WLSMV	200	ALL	0.776	0.611	0.767	0.997	0.644	0.771	20664	20741
878	CvM2	CFI	WLSMV	200	5	0.760	0.578	0.749	0.986	0.547	0.883	6488	6545
879	CvM2	CFI	WLSMV	200	10	0.794	0.611	0.776	0.993	0.581	0.890	7005	7036
880	CvM2	CFI	WLSMV	200	30	0.827	0.638	0.812	0.999	0.663	0.873	7171	7160
881	CvM2	TLI	ALL	ALL	ALL	0.669	0.548	0.692	0.992	0.576	0.695	223583	223868
882	CvM2	TLI	ALL	ALL	5	0.654	0.549	0.672	0.977	0.593	0.675	62891	63082
883	CvM2	TLI	ALL	ALL	10	0.687	0.556	0.699	0.988	0.588	0.714	75570	75749
884	CvM2	TLI	ALL	ALL	30	0.686	0.543	0.682	0.997	0.556	0.752	85122	85037
885	CvM2	TLI	ALL	30	ALL	0.557	0.512	0.570	0.979	0.455	0.644	43725	43767
886	CvM2	TLI	ALL	30	5	0.556	0.516	0.559	0.966	0.670	0.420	10875	10825
887	CvM2	TLI	ALL	30	10	0.569	0.515	0.572	0.988	0.589	0.516	14530	14595
888	CvM2	TLI	ALL	30	30	0.565	0.510	0.535	0.983	0.292	0.818	18320	18347
889	CvM2	TLI	ALL	50	ALL	0.620	0.530	0.631	0.988	0.538	0.655	52104	52168
890	CvM2	TLI	ALL	50	5	0.595	0.527	0.600	0.965	0.566	0.586	13813	13894
891	CvM2	TLI	ALL	50	10	0.649	0.545	0.652	0.976	0.515	0.704	17513	17559
892	CvM2	TLI	ALL	50	30	0.637	0.525	0.599	0.994	0.431	0.788	20778	20715
893	CvM2	TLI	ALL	100	ALL	0.707	0.563	0.706	0.992	0.599	0.720	60747	60844
894	CvM2	TLI	ALL	100	5	0.695	0.561	0.701	0.972	0.548	0.757	17384	17448
895	CvM2	TLI	ALL	100	10	0.731	0.572	0.731	0.983	0.530	0.835	20799	20843
896	CvM2	TLI	ALL	100	30	0.731	0.559	0.718	0.996	0.543	0.829	22564	22553
897	CvM2	TLI	ALL	200	ALL	0.771	0.605	0.762	0.994	0.619	0.788	67007	67089
898	CvM2	TLI	ALL	200	5	0.757	0.577	0.756	0.982	0.567	0.859	20819	20915
899	CvM2	TLI	ALL	200	10	0.778	0.600	0.770	0.988	0.535	0.911	22728	22752
900	CvM2	TLI	ALL	200	30	0.816	0.637	0.811	0.998	0.694	0.817	23460	23422
901	CvM2	TLI	MLR	ALL	ALL	0.656	0.567	0.666	0.989	0.652	0.612	85732	85647
902	CvM2	TLI	MLR	ALL	5	0.630	0.557	0.639	0.977	0.669	0.582	24596	24572
903	CvM2	TLI	MLR	ALL	10	0.664	0.563	0.674	0.986	0.662	0.627	28824	28785
904	CvM2	TLI	MLR	ALL	30	0.697	0.579	0.705	0.994	0.642	0.669	32312	32290
905	CvM2	TLI	MLR	30	ALL	0.574	0.533	0.575	0.967	0.638	0.494	17855	17854
906	CvM2	TLI	MLR	30	5	0.563	0.525	0.562	0.875	0.573	0.551	4747	4739
907	CvM2	TLI	MLR	30	10	0.587	0.527	0.586	0.928	0.477	0.657	5832	5832
908	CvM2	TLI	MLR	30	30	0.638	0.543	0.637	0.978	0.576	0.630	7276	7283
909	CvM2	TLI	MLR	50	ALL	0.624	0.544	0.629	0.983	0.623	0.586	20411	20417
910	CvM2	TLI	MLR	50	5	0.598	0.530	0.601	0.965	0.685	0.475	5706	5724
911	CvM2	TLI	MLR	50	10	0.657	0.548	0.658	0.960	0.489	0.753	6787	6790
912	CvM2	TLI	MLR	50	30	0.688	0.551	0.685	0.985	0.477	0.824	7918	7903
913	CvM2	TLI	MLR	100	ALL	0.704	0.568	0.714	0.989	0.595	0.728	22941	22901
914	CvM2	TLI	MLR	100	5	0.691	0.561	0.699	0.972	0.580	0.727	6647	6637
915	CvM2	TLI	MLR	100	10	0.732	0.562	0.729	0.981	0.532	0.854	7855	7823
916	CvM2	TLI	MLR	100	30	0.757	0.581	0.748	0.993	0.551	0.857	8439	8441
917	CvM2	TLI	MLR	200	ALL	0.764	0.594	0.767	0.994	0.626	0.789	24525	24475
918	CvM2	TLI	MLR	200	5	0.761	0.576	0.759	0.983	0.592	0.845	7496	7472
919	CvM2	TLI	MLR	200	10	0.778	0.587	0.763	0.989	0.555	0.903	8350	8340
920	CvM2	TLI	MLR	200	30	0.815	0.621	0.804	0.997	0.620	0.898	8679	8663
921	CvM2	TLI	ULSMV	ALL	ALL	0.691	0.546	0.719	0.974	0.435	0.871	71635	71584
922	CvM2	TLI	ULSMV	ALL	5	0.677	0.547	0.706	0.966	0.535	0.767	19976	19987
923	CvM2	TLI	ULSMV	ALL	10	0.711	0.555	0.722	0.964	0.455	0.888	24264	24300
924	CvM2	TLI	ULSMV	ALL	30	0.696	0.540	0.596	0.999	0.522	0.812	27395	27297
925	CvM2	TLI	ULSMV	30	ALL	0.570	0.512	0.622	0.999	0.409	0.720	13710	13635
926	CvM2	TLI	ULSMV	30	5	0.565	0.514	0.584	0.966	0.521	0.604	3256	3200
927	CvM2	TLI	ULSMV	30	10	0.589	0.516	0.611	0.999	0.487	0.664	4632	4632
928	CvM2	TLI	ULSMV	30	30	0.571	0.510	0.422	0.999	0.250	0.881	5822	5803
929	CvM2	TLI	ULSMV	50	ALL	0.641	0.531	0.665	0.995	0.545	0.685	16566	16505
930	CvM2	TLI	ULSMV	50	5	0.606	0.527	0.615	0.989	0.644	0.521	4339	4319
931	CvM2	TLI	ULSMV	50	10	0.675	0.548	0.682	0.966	0.466	0.800	5547	5551
932	CvM2	TLI	ULSMV	50	30	0.647	0.525	0.537	0.997	0.399	0.861	6680	6635
933	CvM2	TLI	ULSMV	100	ALL	0.729	0.565	0.712	0.964	0.447	0.906	19541	19571
934	CvM2	TLI	ULSMV	100	5	0.715	0.565	0.718	0.969	0.584	0.752	5546	5570
935	CvM2	TLI	ULSMV	100	10	0.750	0.582	0.737	0.957	0.470	0.932	6712	6741
936	CvM2	TLI	ULSMV	100	30	0.733	0.556	0.658	0.999	0.586	0.806	7283	7260
937	CvM2	TLI	ULSMV	200	ALL	0.784	0.610	0.750	0.973	0.511	0.946	21818	21873
938	CvM2	TLI	ULSMV	200	5	0.763	0.576	0.745	0.970	0.536	0.914	6835	6898
939	CvM2	TLI	ULSMV	200	10	0.780	0.604	0.744	0.962	0.505	0.974	7373	7376
940	CvM2	TLI	ULSMV	200	30	0.822	0.651	0.781	0.999	0.766	0.742	7610	7599
941	CvM2	TLI	WLSMV	ALL	ALL	0.665	0.542	0.700	0.996	0.563	0.707	66216	66637
942	CvM2	TLI	WLSMV	ALL	5	0.663	0.547	0.683	0.984	0.585	0.692	18319	18523
943	CvM2	TLI	WLSMV	ALL	10	0.694	0.553	0.710	0.990	0.535	0.781	22482	22664
944	CvM2	TLI	WLSMV	ALL	30	0.675	0.534	0.658	0.998	0.458	0.847	25415	25450
945	CvM2	TLI	WLSMV	30	ALL	0.541	0.507	0.559	1.000	0.441	0.635	12160	12278
946	CvM2	TLI	WLSMV	30	5	0.553	0.514	0.557	0.962	0.475	0.615	2872	2886
947	CvM2	TLI	WLSMV	30	10	0.561	0.512	0.577	0.989	0.424	0.688	4066	4131
948	CvM2	TLI	WLSMV	30	30	0.531	0.504	0.535	1.000	0.235	0.825	5222	5261
949	CvM2	TLI	WLSMV	50	ALL	0.601	0.523	0.622	0.999	0.590	0.586	15127	15246
950	CvM2	TLI	WLSMV	50	5	0.590	0.523	0.594	0.965	0.434	0.708	3768	3851
951	CvM2	TLI	WLSMV	50	10	0.635	0.539	0.639	0.992	0.613	0.592	5179	5218
952	CvM2	TLI	WLSMV	50	30	0.607	0.517	0.556	0.999	0.393	0.797	6180	6177
953	CvM2	TLI	WLSMV	100	ALL	0.699	0.559	0.712	0.995	0.603	0.708	18265	18372
954	CvM2	TLI	WLSMV	100	5	0.689	0.558	0.694	0.982	0.583	0.711	5191	5241
955	CvM2	TLI	WLSMV	100	10	0.733	0.573	0.722	0.988	0.535	0.836	6232	6279
956	CvM2	TLI	WLSMV	100	30	0.723	0.552	0.691	0.997	0.492	0.881	6842	6852
957	CvM2	TLI	WLSMV	200	ALL	0.776	0.611	0.767	0.996	0.644	0.771	20664	20741
958	CvM2	TLI	WLSMV	200	5	0.760	0.578	0.749	0.984	0.547	0.883	6488	6545
959	CvM2	TLI	WLSMV	200	10	0.794	0.611	0.776	0.992	0.581	0.890	7005	7036
960	CvM2	TLI	WLSMV	200	30	0.827	0.638	0.812	0.998	0.663	0.873	7171	7160
961	CvM2	RMSEA	ALL	ALL	ALL	0.664	0.548	0.692	0.009	0.587	0.679	223583	223868
962	CvM2	RMSEA	ALL	ALL	5	0.654	0.549	0.671	0.016	0.578	0.687	62891	63082
963	CvM2	RMSEA	ALL	ALL	10	0.686	0.556	0.704	0.012	0.573	0.734	75570	75749
964	CvM2	RMSEA	ALL	ALL	30	0.681	0.543	0.709	0.005	0.544	0.763	85122	85037
965	CvM2	RMSEA	ALL	30	ALL	0.551	0.512	0.558	0.004	0.607	0.479	43725	43767
966	CvM2	RMSEA	ALL	30	5	0.549	0.516	0.549	0.019	0.636	0.444	10875	10825
967	CvM2	RMSEA	ALL	30	10	0.558	0.515	0.555	0.008	0.619	0.477	14530	14595
968	CvM2	RMSEA	ALL	30	30	0.443	NA	0.473	-Inf	0.000	1.000	18320	18347
969	CvM2	RMSEA	ALL	50	ALL	0.612	0.530	0.625	0.009	0.582	0.598	52104	52168
970	CvM2	RMSEA	ALL	50	5	0.590	0.527	0.590	0.021	0.489	0.647	13813	13894
971	CvM2	RMSEA	ALL	50	10	0.642	0.545	0.640	0.014	0.559	0.648	17513	17559
972	CvM2	RMSEA	ALL	50	30	0.629	0.525	0.591	0.005	0.522	0.679	20778	20715
973	CvM2	RMSEA	ALL	100	ALL	0.705	0.563	0.710	0.009	0.596	0.720	60747	60844
974	CvM2	RMSEA	ALL	100	5	0.696	0.561	0.696	0.016	0.585	0.713	17384	17448
975	CvM2	RMSEA	ALL	100	10	0.736	0.572	0.722	0.014	0.498	0.868	20799	20843
976	CvM2	RMSEA	ALL	100	30	0.731	0.559	0.700	0.006	0.535	0.837	22564	22553
977	CvM2	RMSEA	ALL	200	ALL	0.771	0.605	0.760	0.006	0.679	0.729	67007	67089
978	CvM2	RMSEA	ALL	200	5	0.759	0.577	0.742	0.015	0.536	0.892	20819	20915
979	CvM2	RMSEA	ALL	200	10	0.787	0.601	0.766	0.010	0.582	0.883	22728	22752
980	CvM2	RMSEA	ALL	200	30	0.822	0.637	0.804	0.005	0.636	0.882	23460	23422
981	CvM2	RMSEA	MLR	ALL	ALL	0.653	0.566	0.662	0.010	0.672	0.576	85732	85647
982	CvM2	RMSEA	MLR	ALL	5	0.630	0.556	0.637	0.016	0.686	0.548	24596	24572
983	CvM2	RMSEA	MLR	ALL	10	0.662	0.562	0.669	0.013	0.637	0.634	28824	28785
984	CvM2	RMSEA	MLR	ALL	30	0.694	0.579	0.701	0.008	0.667	0.635	32312	32290
985	CvM2	RMSEA	MLR	30	ALL	0.570	0.532	0.572	0.019	0.665	0.456	17855	17854
986	CvM2	RMSEA	MLR	30	5	0.566	0.523	0.566	0.045	0.551	0.546	4747	4739
987	CvM2	RMSEA	MLR	30	10	0.583	0.525	0.584	0.035	0.379	0.744	5832	5832
988	CvM2	RMSEA	MLR	30	30	0.631	0.541	0.632	0.020	0.389	0.810	7276	7283
989	CvM2	RMSEA	MLR	50	ALL	0.620	0.543	0.624	0.013	0.646	0.549	20411	20417
990	CvM2	RMSEA	MLR	50	5	0.599	0.529	0.598	0.035	0.372	0.773	5706	5724
991	CvM2	RMSEA	MLR	50	10	0.652	0.546	0.650	0.026	0.382	0.841	6787	6790
992	CvM2	RMSEA	MLR	50	30	0.684	0.550	0.677	0.013	0.465	0.827	7918	7903
993	CvM2	RMSEA	MLR	100	ALL	0.699	0.568	0.703	0.010	0.621	0.685	22941	22901
994	CvM2	RMSEA	MLR	100	5	0.691	0.560	0.690	0.020	0.530	0.756	6647	6637
995	CvM2	RMSEA	MLR	100	10	0.724	0.562	0.714	0.016	0.492	0.877	7855	7823
996	CvM2	RMSEA	MLR	100	30	0.752	0.581	0.743	0.009	0.517	0.879	8439	8441
997	CvM2	RMSEA	MLR	200	ALL	0.758	0.594	0.753	0.008	0.640	0.755	24525	24475
998	CvM2	RMSEA	MLR	200	5	0.752	0.576	0.738	0.016	0.523	0.893	7496	7472
999	CvM2	RMSEA	MLR	200	10	0.771	0.587	0.754	0.012	0.530	0.911	8350	8340
1000	CvM2	RMSEA	MLR	200	30	0.812	0.621	0.798	0.006	0.606	0.903	8679	8663
1001	CvM2	RMSEA	ULSMV	ALL	ALL	0.687	0.546	0.711	0.007	0.567	0.730	71635	71584
1002	CvM2	RMSEA	ULSMV	ALL	5	0.684	0.547	0.703	0.016	0.512	0.788	19976	19987
1003	CvM2	RMSEA	ULSMV	ALL	10	0.715	0.555	0.710	0.012	0.515	0.833	24264	24300
1004	CvM2	RMSEA	ULSMV	ALL	30	0.695	0.540	0.652	0.004	0.493	0.844	27395	27297
1005	CvM2	RMSEA	ULSMV	30	ALL	0.433	NA	0.395	-Inf	0.000	1.000	13710	13635
1006	CvM2	RMSEA	ULSMV	30	5	0.566	0.514	0.581	0.011	0.541	0.586	3256	3200
1007	CvM2	RMSEA	ULSMV	30	10	0.412	NA	0.402	-Inf	0.000	1.000	4632	4632
1008	CvM2	RMSEA	ULSMV	30	30	0.433	NA	0.444	-Inf	0.000	1.000	5822	5803
1009	CvM2	RMSEA	ULSMV	50	ALL	0.635	0.531	0.659	0.005	0.564	0.661	16566	16505
1010	CvM2	RMSEA	ULSMV	50	5	0.609	0.527	0.611	0.016	0.503	0.662	4339	4319
1011	CvM2	RMSEA	ULSMV	50	10	0.676	0.548	0.666	0.011	0.545	0.722	5547	5551
1012	CvM2	RMSEA	ULSMV	50	30	0.357	NA	0.451	-Inf	0.000	1.000	6680	6635
1013	CvM2	RMSEA	ULSMV	100	ALL	0.724	0.565	0.714	0.007	0.627	0.719	19541	19571
1014	CvM2	RMSEA	ULSMV	100	5	0.719	0.565	0.713	0.017	0.523	0.821	5546	5570
1015	CvM2	RMSEA	ULSMV	100	10	0.756	0.582	0.736	0.012	0.576	0.830	6712	6741
1016	CvM2	RMSEA	ULSMV	100	30	0.735	0.556	0.676	0.005	0.513	0.883	7283	7260
1017	CvM2	RMSEA	ULSMV	200	ALL	0.782	0.610	0.760	0.006	0.663	0.758	21818	21873
1018	CvM2	RMSEA	ULSMV	200	5	0.766	0.576	0.744	0.015	0.541	0.907	6835	6898
1019	CvM2	RMSEA	ULSMV	200	10	0.794	0.604	0.769	0.012	0.538	0.944	7373	7376
1020	CvM2	RMSEA	ULSMV	200	30	0.833	0.651	0.823	0.004	0.716	0.817	7610	7599
1021	CvM2	RMSEA	WLSMV	ALL	ALL	0.664	0.542	0.697	0.006	0.583	0.685	66216	66637
1022	CvM2	RMSEA	WLSMV	ALL	5	0.665	0.547	0.684	0.013	0.585	0.692	18319	18523
1023	CvM2	RMSEA	WLSMV	ALL	10	0.696	0.553	0.711	0.011	0.547	0.767	22482	22664
1024	CvM2	RMSEA	WLSMV	ALL	30	0.674	0.534	0.679	0.004	0.468	0.836	25415	25450
1025	CvM2	RMSEA	WLSMV	30	ALL	0.459	NA	0.444	0.054	1.000	0.001	12160	12278
1026	CvM2	RMSEA	WLSMV	30	5	0.553	0.514	0.557	0.017	0.542	0.545	2872	2886
1027	CvM2	RMSEA	WLSMV	30	10	0.560	0.512	0.575	0.008	0.478	0.629	4066	4131
1028	CvM2	RMSEA	WLSMV	30	30	0.470	NA	0.455	0.021	1.000	0.000	5222	5261
1029	CvM2	RMSEA	WLSMV	50	ALL	0.599	0.523	0.617	0.003	0.589	0.587	15127	15246
1030	CvM2	RMSEA	WLSMV	50	5	0.589	0.523	0.593	0.013	0.605	0.533	3768	3851
1031	CvM2	RMSEA	WLSMV	50	10	0.634	0.539	0.638	0.009	0.637	0.569	5179	5218
1032	CvM2	RMSEA	WLSMV	50	30	0.396	NA	0.420	-Inf	0.000	1.000	6180	6177
1033	CvM2	RMSEA	WLSMV	100	ALL	0.696	0.559	0.708	0.007	0.613	0.698	18265	18372
1034	CvM2	RMSEA	WLSMV	100	5	0.689	0.558	0.691	0.015	0.549	0.745	5191	5241
1035	CvM2	RMSEA	WLSMV	100	10	0.733	0.573	0.722	0.012	0.540	0.826	6232	6279
1036	CvM2	RMSEA	WLSMV	100	30	0.722	0.552	0.689	0.005	0.501	0.867	6842	6852
1037	CvM2	RMSEA	WLSMV	200	ALL	0.775	0.611	0.768	0.006	0.675	0.743	20664	20741
1038	CvM2	RMSEA	WLSMV	200	5	0.760	0.578	0.746	0.014	0.541	0.892	6488	6545
1039	CvM2	RMSEA	WLSMV	200	10	0.797	0.611	0.778	0.010	0.602	0.879	7005	7036
1040	CvM2	RMSEA	WLSMV	200	30	0.827	0.638	0.810	0.005	0.644	0.891	7171	7160
1041	CvM2	SRMRW	ALL	ALL	ALL	0.527	0.505	0.524	0.031	0.452	0.597	223583	223868
1042	CvM2	SRMRW	ALL	ALL	5	0.524	0.512	0.523	0.037	0.639	0.405	62891	63082
1043	CvM2	SRMRW	ALL	ALL	10	0.530	0.508	0.528	0.031	0.498	0.553	75570	75749
1044	CvM2	SRMRW	ALL	ALL	30	0.535	0.505	0.536	0.025	0.268	0.796	85122	85037
1045	CvM2	SRMRW	ALL	30	ALL	0.505	0.500	0.505	0.041	0.572	0.440	43725	43767
1046	CvM2	SRMRW	ALL	30	5	0.505	0.503	0.506	0.067	0.671	0.344	10875	10825
1047	CvM2	SRMRW	ALL	30	10	0.504	0.500	0.506	0.069	0.112	0.904	14530	14595
1048	CvM2	SRMRW	ALL	30	30	0.507	0.500	0.509	0.041	0.135	0.891	18320	18347
1049	CvM2	SRMRW	ALL	50	ALL	0.517	0.502	0.517	0.037	0.483	0.547	52104	52168
1050	CvM2	SRMRW	ALL	50	5	0.513	0.505	0.515	0.054	0.618	0.405	13813	13894
1051	CvM2	SRMRW	ALL	50	10	0.516	0.504	0.519	0.047	0.198	0.834	17513	17559
1052	CvM2	SRMRW	ALL	50	30	0.516	0.501	0.522	0.033	0.137	0.916	20778	20715
1053	CvM2	SRMRW	ALL	100	ALL	0.532	0.503	0.534	0.033	0.359	0.697	60747	60844
1054	CvM2	SRMRW	ALL	100	5	0.538	0.510	0.543	0.044	0.365	0.690	17384	17448
1055	CvM2	SRMRW	ALL	100	10	0.536	0.506	0.545	0.038	0.138	0.932	20799	20843
1056	CvM2	SRMRW	ALL	100	30	0.528	0.503	0.540	0.026	0.153	0.935	22564	22553
1057	CvM2	SRMRW	ALL	200	ALL	0.554	0.506	0.561	0.028	0.275	0.822	67007	67089
1058	CvM2	SRMRW	ALL	200	5	0.570	0.516	0.579	0.032	0.383	0.721	20819	20915
1059	CvM2	SRMRW	ALL	200	10	0.558	0.510	0.573	0.028	0.161	0.948	22728	22752
1060	CvM2	SRMRW	ALL	200	30	0.545	0.506	0.566	0.023	0.153	0.974	23460	23422
1061	CvM2	SRMRW	MLR	ALL	ALL	0.507	0.501	0.508	0.028	0.386	0.630	85732	85647
1062	CvM2	SRMRW	MLR	ALL	5	0.522	0.516	0.524	0.029	0.752	0.296	24596	24572
1063	CvM2	SRMRW	MLR	ALL	10	0.509	0.508	0.510	0.018	0.782	0.240	28824	28785
1064	CvM2	SRMRW	MLR	ALL	30	0.504	0.504	0.504	0.009	0.884	0.127	32312	32290
1065	CvM2	SRMRW	MLR	30	ALL	0.499	NA	0.497	0.024	0.270	0.736	17855	17854
1066	CvM2	SRMRW	MLR	30	5	0.521	0.505	0.523	0.071	0.204	0.836	4747	4739
1067	CvM2	SRMRW	MLR	30	10	0.504	NA	0.503	0.044	0.329	0.693	5832	5832
1068	CvM2	SRMRW	MLR	30	30	0.502	0.501	0.502	0.024	0.674	0.340	7276	7283
1069	CvM2	SRMRW	MLR	50	ALL	0.506	0.501	0.508	0.048	0.181	0.834	20411	20417
1070	CvM2	SRMRW	MLR	50	5	0.526	0.506	0.527	0.048	0.630	0.417	5706	5724
1071	CvM2	SRMRW	MLR	50	10	0.516	0.503	0.515	0.031	0.624	0.408	6787	6790
1072	CvM2	SRMRW	MLR	50	30	0.504	0.501	0.504	0.019	0.278	0.736	7918	7903
1073	CvM2	SRMRW	MLR	100	ALL	0.509	0.502	0.513	0.036	0.134	0.892	22941	22901
1074	CvM2	SRMRW	MLR	100	5	0.556	0.512	0.557	0.038	0.354	0.734	6647	6637
1075	CvM2	SRMRW	MLR	100	10	0.523	0.507	0.524	0.024	0.399	0.641	7855	7823
1076	CvM2	SRMRW	MLR	100	30	0.510	0.502	0.510	0.013	0.302	0.720	8439	8441
1077	CvM2	SRMRW	MLR	200	ALL	0.518	0.504	0.527	0.029	0.081	0.964	24525	24475
1078	CvM2	SRMRW	MLR	200	5	0.592	0.520	0.599	0.029	0.252	0.889	7496	7472
1079	CvM2	SRMRW	MLR	200	10	0.548	0.511	0.548	0.016	0.555	0.518	8350	8340
1080	CvM2	SRMRW	MLR	200	30	0.523	0.507	0.523	0.009	0.569	0.472	8679	8663
1081	CvM2	SRMRW	ULSMV	ALL	ALL	0.562	0.521	0.553	0.032	0.608	0.503	71635	71584
1082	CvM2	SRMRW	ULSMV	ALL	5	0.533	0.511	0.528	0.042	0.674	0.407	19976	19987
1083	CvM2	SRMRW	ULSMV	ALL	10	0.563	0.516	0.555	0.032	0.660	0.469	24264	24300
1084	CvM2	SRMRW	ULSMV	ALL	30	0.600	0.511	0.588	0.025	0.565	0.618	27395	27297
1085	CvM2	SRMRW	ULSMV	30	ALL	0.513	0.502	0.512	0.041	0.722	0.313	13710	13635
1086	CvM2	SRMRW	ULSMV	30	5	0.503	NA	0.503	0.102	0.320	0.704	3256	3200
1087	CvM2	SRMRW	ULSMV	30	10	0.516	NA	0.517	0.069	0.338	0.712	4632	4632
1088	CvM2	SRMRW	ULSMV	30	30	0.533	0.501	0.532	0.041	0.407	0.677	5822	5803
1089	CvM2	SRMRW	ULSMV	50	ALL	0.539	0.508	0.536	0.034	0.716	0.353	16566	16505
1090	CvM2	SRMRW	ULSMV	50	5	0.522	0.502	0.524	0.084	0.184	0.862	4339	4319
1091	CvM2	SRMRW	ULSMV	50	10	0.544	0.501	0.543	0.056	0.278	0.808	5547	5551
1092	CvM2	SRMRW	ULSMV	50	30	0.570	0.501	0.569	0.033	0.408	0.751	6680	6635
1093	CvM2	SRMRW	ULSMV	100	ALL	0.580	0.511	0.576	0.035	0.520	0.613	19541	19571
1094	CvM2	SRMRW	ULSMV	100	5	0.563	0.503	0.566	0.055	0.330	0.792	5546	5570
1095	CvM2	SRMRW	ULSMV	100	10	0.595	0.505	0.597	0.038	0.402	0.801	6712	6741
1096	CvM2	SRMRW	ULSMV	100	30	0.605	0.503	0.611	0.026	0.457	0.807	7283	7260
1097	CvM2	SRMRW	ULSMV	200	ALL	0.630	0.514	0.628	0.028	0.527	0.695	21818	21873
1098	CvM2	SRMRW	ULSMV	200	5	0.606	0.504	0.616	0.041	0.333	0.889	6835	6898
1099	CvM2	SRMRW	ULSMV	200	10	0.630	0.504	0.644	0.029	0.427	0.893	7373	7376
1100	CvM2	SRMRW	ULSMV	200	30	0.636	0.504	0.661	0.023	0.459	0.921	7610	7599
1101	CvM2	SRMRW	WLSMV	ALL	ALL	0.515	0.507	0.516	0.021	0.700	0.325	66216	66637
1102	CvM2	SRMRW	WLSMV	ALL	5	0.520	0.515	0.520	0.032	0.794	0.257	18319	18523
1103	CvM2	SRMRW	WLSMV	ALL	10	0.522	0.518	0.523	0.021	0.834	0.220	22482	22664
1104	CvM2	SRMRW	WLSMV	ALL	30	0.518	0.517	0.521	0.012	0.838	0.207	25415	25450
1105	CvM2	SRMRW	WLSMV	30	ALL	0.504	0.503	0.504	0.026	0.885	0.126	12160	12278
1106	CvM2	SRMRW	WLSMV	30	5	0.509	0.500	0.509	0.081	0.429	0.593	2872	2886
1107	CvM2	SRMRW	WLSMV	30	10	0.504	NA	0.503	0.053	0.440	0.579	4066	4131
1108	CvM2	SRMRW	WLSMV	30	30	0.512	0.503	0.513	0.026	0.741	0.283	5222	5261
1109	CvM2	SRMRW	WLSMV	50	ALL	0.513	0.506	0.514	0.023	0.745	0.278	15127	15246
1110	CvM2	SRMRW	WLSMV	50	5	0.516	0.502	0.514	0.062	0.470	0.567	3768	3851
1111	CvM2	SRMRW	WLSMV	50	10	0.517	0.505	0.517	0.039	0.572	0.458	5179	5218
1112	CvM2	SRMRW	WLSMV	50	30	0.524	0.507	0.523	0.023	0.380	0.660	6180	6177
1113	CvM2	SRMRW	WLSMV	100	ALL	0.518	0.510	0.521	0.028	0.460	0.571	18265	18372
1114	CvM2	SRMRW	WLSMV	100	5	0.542	0.508	0.540	0.044	0.438	0.640	5191	5241
1115	CvM2	SRMRW	WLSMV	100	10	0.548	0.512	0.547	0.028	0.524	0.553	6232	6279
1116	CvM2	SRMRW	WLSMV	100	30	0.544	0.513	0.544	0.017	0.343	0.729	6842	6852
1117	CvM2	SRMRW	WLSMV	200	ALL	0.535	0.519	0.542	0.021	0.475	0.584	20664	20741
1118	CvM2	SRMRW	WLSMV	200	5	0.580	0.513	0.581	0.032	0.447	0.687	6488	6545
1119	CvM2	SRMRW	WLSMV	200	10	0.599	0.519	0.599	0.021	0.486	0.674	7005	7036
1120	CvM2	SRMRW	WLSMV	200	30	0.589	0.523	0.588	0.011	0.463	0.684	7171	7160
1121	CvM2	SRMRB	ALL	ALL	ALL	0.634	0.601	0.642	0.076	0.777	0.441	223583	223868
1122	CvM2	SRMRB	ALL	ALL	5	0.596	0.576	0.602	0.078	0.821	0.344	62891	63082
1123	CvM2	SRMRB	ALL	ALL	10	0.631	0.599	0.639	0.078	0.769	0.441	75570	75749
1124	CvM2	SRMRB	ALL	ALL	30	0.675	0.625	0.683	0.071	0.773	0.497	85122	85037
1125	CvM2	SRMRB	ALL	30	ALL	0.584	0.551	0.585	0.127	0.718	0.407	43725	43767
1126	CvM2	SRMRB	ALL	30	5	0.560	0.526	0.557	0.162	0.578	0.522	10875	10825
1127	CvM2	SRMRB	ALL	30	10	0.575	0.542	0.572	0.127	0.729	0.390	14530	14595
1128	CvM2	SRMRB	ALL	30	30	0.623	0.576	0.627	0.115	0.762	0.416	18320	18347
1129	CvM2	SRMRB	ALL	50	ALL	0.630	0.584	0.633	0.102	0.727	0.466	52104	52168
1130	CvM2	SRMRB	ALL	50	5	0.588	0.542	0.584	0.123	0.626	0.513	13813	13894
1131	CvM2	SRMRB	ALL	50	10	0.619	0.578	0.618	0.102	0.753	0.440	17513	17559
1132	CvM2	SRMRB	ALL	50	30	0.687	0.621	0.699	0.092	0.773	0.488	20778	20715
1133	CvM2	SRMRB	ALL	100	ALL	0.701	0.637	0.703	0.080	0.706	0.600	60747	60844
1134	CvM2	SRMRB	ALL	100	5	0.641	0.591	0.636	0.088	0.731	0.495	17384	17448
1135	CvM2	SRMRB	ALL	100	10	0.698	0.643	0.701	0.078	0.762	0.548	20799	20843
1136	CvM2	SRMRB	ALL	100	30	0.781	0.683	0.808	0.076	0.666	0.717	22564	22553
1137	CvM2	SRMRB	ALL	200	ALL	0.766	0.681	0.765	0.061	0.729	0.677	67007	67089
1138	CvM2	SRMRB	ALL	200	5	0.701	0.642	0.698	0.069	0.742	0.593	20819	20915
1139	CvM2	SRMRB	ALL	200	10	0.776	0.696	0.791	0.059	0.763	0.643	22728	22752
1140	CvM2	SRMRB	ALL	200	30	0.848	0.729	0.876	0.058	0.674	0.825	23460	23422
1141	CvM2	SRMRB	MLR	ALL	ALL	0.654	0.621	0.659	0.092	0.814	0.441	85732	85647
1142	CvM2	SRMRB	MLR	ALL	5	0.617	0.599	0.626	0.097	0.848	0.356	24596	24572
1143	CvM2	SRMRB	MLR	ALL	10	0.653	0.621	0.661	0.092	0.816	0.432	28824	28785
1144	CvM2	SRMRB	MLR	ALL	30	0.699	0.642	0.703	0.087	0.808	0.505	32312	32290
1145	CvM2	SRMRB	MLR	30	ALL	0.613	0.593	0.619	0.143	0.871	0.324	17855	17854
1146	CvM2	SRMRB	MLR	30	5	0.575	0.552	0.576	0.172	0.771	0.376	4747	4739
1147	CvM2	SRMRB	MLR	30	10	0.607	0.583	0.610	0.151	0.817	0.375	5832	5832
1148	CvM2	SRMRB	MLR	30	30	0.676	0.644	0.687	0.143	0.805	0.486	7276	7283
1149	CvM2	SRMRB	MLR	50	ALL	0.678	0.640	0.683	0.122	0.824	0.466	20411	20417
1150	CvM2	SRMRB	MLR	50	5	0.629	0.590	0.628	0.143	0.719	0.511	5706	5724
1151	CvM2	SRMRB	MLR	50	10	0.675	0.648	0.680	0.125	0.803	0.507	6787	6790
1152	CvM2	SRMRB	MLR	50	30	0.756	0.694	0.768	0.119	0.758	0.624	7918	7903
1153	CvM2	SRMRB	MLR	100	ALL	0.756	0.699	0.754	0.098	0.744	0.672	22941	22901
1154	CvM2	SRMRB	MLR	100	5	0.703	0.673	0.704	0.104	0.822	0.549	6647	6637
1155	CvM2	SRMRB	MLR	100	10	0.757	0.721	0.762	0.094	0.832	0.606	7855	7823
1156	CvM2	SRMRB	MLR	100	30	0.846	0.746	0.866	0.087	0.809	0.692	8439	8441
1157	CvM2	SRMRB	MLR	200	ALL	0.808	0.729	0.800	0.072	0.795	0.710	24525	24475
1158	CvM2	SRMRB	MLR	200	5	0.757	0.723	0.760	0.079	0.828	0.623	7496	7472
1159	CvM2	SRMRB	MLR	200	10	0.821	0.751	0.838	0.071	0.828	0.680	8350	8340
1160	CvM2	SRMRB	MLR	200	30	0.889	0.767	0.908	0.071	0.685	0.906	8679	8663
1161	CvM2	SRMRB	ULSMV	ALL	ALL	0.629	0.612	0.642	0.063	0.813	0.409	71635	71584
1162	CvM2	SRMRB	ULSMV	ALL	5	0.589	0.581	0.601	0.065	0.855	0.311	19976	19987
1163	CvM2	SRMRB	ULSMV	ALL	10	0.626	0.610	0.641	0.058	0.869	0.349	24264	24300
1164	CvM2	SRMRB	ULSMV	ALL	30	0.670	0.638	0.682	0.059	0.812	0.464	27395	27297
1165	CvM2	SRMRB	ULSMV	30	ALL	0.575	0.555	0.582	0.106	0.747	0.383	13710	13635
1166	CvM2	SRMRB	ULSMV	30	5	0.559	0.529	0.559	0.116	0.811	0.289	3256	3200
1167	CvM2	SRMRB	ULSMV	30	10	0.571	0.542	0.569	0.108	0.735	0.403	4632	4632
1168	CvM2	SRMRB	ULSMV	30	30	0.605	0.587	0.615	0.096	0.796	0.395	5822	5803
1169	CvM2	SRMRB	ULSMV	50	ALL	0.622	0.597	0.631	0.082	0.833	0.371	16566	16505
1170	CvM2	SRMRB	ULSMV	50	5	0.582	0.549	0.580	0.104	0.640	0.504	4339	4319
1171	CvM2	SRMRB	ULSMV	50	10	0.617	0.591	0.620	0.088	0.758	0.458	5547	5551
1172	CvM2	SRMRB	ULSMV	50	30	0.674	0.647	0.693	0.076	0.835	0.460	6680	6635
1173	CvM2	SRMRB	ULSMV	100	ALL	0.701	0.664	0.707	0.065	0.826	0.498	19541	19571
1174	CvM2	SRMRB	ULSMV	100	5	0.641	0.611	0.638	0.075	0.771	0.481	5546	5570
1175	CvM2	SRMRB	ULSMV	100	10	0.698	0.681	0.706	0.063	0.879	0.489	6712	6741
1176	CvM2	SRMRB	ULSMV	100	30	0.777	0.716	0.809	0.055	0.930	0.480	7283	7260
1177	CvM2	SRMRB	ULSMV	200	ALL	0.769	0.715	0.772	0.049	0.847	0.571	21818	21873
1178	CvM2	SRMRB	ULSMV	200	5	0.703	0.673	0.703	0.059	0.783	0.567	6835	6898
1179	CvM2	SRMRB	ULSMV	200	10	0.778	0.739	0.798	0.049	0.876	0.574	7373	7376
1180	CvM2	SRMRB	ULSMV	200	30	0.858	0.770	0.884	0.043	0.918	0.585	7610	7599
1181	CvM2	SRMRB	WLSMV	ALL	ALL	0.641	0.614	0.648	0.069	0.827	0.414	66216	66637
1182	CvM2	SRMRB	WLSMV	ALL	5	0.598	0.590	0.609	0.074	0.829	0.358	18319	18523
1183	CvM2	SRMRB	WLSMV	ALL	10	0.634	0.610	0.642	0.071	0.806	0.419	22482	22664
1184	CvM2	SRMRB	WLSMV	ALL	30	0.685	0.637	0.691	0.067	0.815	0.485	25415	25450
1185	CvM2	SRMRB	WLSMV	30	ALL	0.598	0.570	0.602	0.120	0.784	0.374	12160	12278
1186	CvM2	SRMRB	WLSMV	30	5	0.572	0.533	0.568	0.152	0.578	0.555	2872	2886
1187	CvM2	SRMRB	WLSMV	30	10	0.596	0.562	0.592	0.122	0.781	0.391	4066	4131
1188	CvM2	SRMRB	WLSMV	30	30	0.642	0.605	0.649	0.113	0.815	0.414	5222	5261
1189	CvM2	SRMRB	WLSMV	50	ALL	0.656	0.615	0.660	0.096	0.801	0.449	15127	15246
1190	CvM2	SRMRB	WLSMV	50	5	0.609	0.559	0.604	0.112	0.697	0.483	3768	3851
1191	CvM2	SRMRB	WLSMV	50	10	0.637	0.608	0.635	0.096	0.830	0.422	5179	5218
1192	CvM2	SRMRB	WLSMV	50	30	0.732	0.668	0.744	0.090	0.819	0.527	6180	6177
1193	CvM2	SRMRB	WLSMV	100	ALL	0.736	0.678	0.732	0.076	0.746	0.635	18265	18372
1194	CvM2	SRMRB	WLSMV	100	5	0.662	0.631	0.658	0.082	0.795	0.501	5191	5241
1195	CvM2	SRMRB	WLSMV	100	10	0.724	0.684	0.724	0.074	0.804	0.582	6232	6279
1196	CvM2	SRMRB	WLSMV	100	30	0.840	0.738	0.861	0.069	0.792	0.699	6842	6852
1197	CvM2	SRMRB	WLSMV	200	ALL	0.795	0.713	0.785	0.058	0.757	0.723	20664	20741
1198	CvM2	SRMRB	WLSMV	200	5	0.722	0.683	0.717	0.062	0.815	0.581	6488	6545
1199	CvM2	SRMRB	WLSMV	200	10	0.807	0.735	0.816	0.055	0.824	0.665	7005	7036
1200	CvM2	SRMRB	WLSMV	200	30	0.889	0.768	0.909	0.056	0.684	0.905	7171	7160

Summarizing the Results

So, I need to parse down 1200 rows of information into somethingthat can fit into a single page table. The above (and very large tables) are condensed to only include the AUC and optimal threshold. The remaining information is left here for reference.

Detecting Any Misspecification

c <- filter(roc_summary, Classification == "C", `Level-2 SS` != 'ALL', `Level-1 SS` == 'ALL')
# Next make the columns the estimator factor
c1 <- cbind(c[ c$Estimator == 'MLR', c(2,4,6,9)],
            c[ c$Estimator == 'ULSMV', c(6,9)],
            c[ c$Estimator == 'WLSMV', c(6,9)])
kable(c1, format = 'html',digits=3, row.names = F) %>%
  kable_styling(full_width = T) %>%
  add_header_above(c(' '=2, 'MLR'=2, 'USLMV'=2, 'WLSMV'=2))

		MLR		USLMV		WLSMV
Index	Level-2 SS	AUC	Threshold	AUC	Threshold	AUC	Threshold
CFI	30	0.747	0.940	0.628	0.982	0.705	0.983
CFI	50	0.828	0.956	0.712	0.973	0.802	0.980
CFI	100	0.893	0.974	0.827	0.971	0.876	0.979
CFI	200	0.917	0.981	0.911	0.975	0.910	0.986
TLI	30	0.746	0.928	0.628	0.979	0.705	0.979
TLI	50	0.827	0.947	0.712	0.968	0.801	0.976
TLI	100	0.893	0.968	0.827	0.965	0.876	0.975
TLI	200	0.917	0.977	0.911	0.970	0.910	0.984
RMSEA	30	0.725	0.029	0.628	0.008	0.700	0.013
RMSEA	50	0.814	0.026	0.706	0.009	0.790	0.011
RMSEA	100	0.889	0.020	0.813	0.012	0.873	0.015
RMSEA	200	0.915	0.017	0.893	0.013	0.910	0.014
SRMRW	30	0.674	0.042	0.642	0.052	0.682	0.050
SRMRW	50	0.737	0.037	0.709	0.048	0.733	0.047
SRMRW	100	0.816	0.037	0.803	0.046	0.797	0.045
SRMRW	200	0.832	0.032	0.855	0.042	0.813	0.038
SRMRB	30	0.565	0.143	0.567	0.106	0.576	0.117
SRMRB	50	0.610	0.119	0.596	0.081	0.613	0.096
SRMRB	100	0.673	0.093	0.649	0.060	0.670	0.071
SRMRB	200	0.732	0.069	0.701	0.047	0.723	0.054

print(xtable(c1, digits = 3), booktabs=T,include.rownames = F)

% latex table generated in R 3.6.0 by xtable 1.8-4 package
% Sat Sep 28 23:01:03 2019
\begin{table}[ht]
\centering
\begin{tabular}{llrrrrrr}
  \toprule
Index & Level-2 SS & AUC & Threshold & AUC & Threshold & AUC & Threshold \\ 
  \midrule
CFI & 30 & 0.747 & 0.940 & 0.628 & 0.982 & 0.705 & 0.983 \\ 
  CFI & 50 & 0.828 & 0.956 & 0.712 & 0.973 & 0.802 & 0.980 \\ 
  CFI & 100 & 0.893 & 0.974 & 0.827 & 0.971 & 0.876 & 0.979 \\ 
  CFI & 200 & 0.917 & 0.981 & 0.911 & 0.975 & 0.910 & 0.986 \\ 
  TLI & 30 & 0.746 & 0.928 & 0.628 & 0.979 & 0.705 & 0.979 \\ 
  TLI & 50 & 0.827 & 0.947 & 0.712 & 0.968 & 0.801 & 0.976 \\ 
  TLI & 100 & 0.893 & 0.968 & 0.827 & 0.965 & 0.876 & 0.975 \\ 
  TLI & 200 & 0.917 & 0.977 & 0.911 & 0.970 & 0.910 & 0.984 \\ 
  RMSEA & 30 & 0.725 & 0.029 & 0.628 & 0.008 & 0.700 & 0.013 \\ 
  RMSEA & 50 & 0.814 & 0.026 & 0.706 & 0.009 & 0.790 & 0.011 \\ 
  RMSEA & 100 & 0.889 & 0.020 & 0.813 & 0.012 & 0.873 & 0.015 \\ 
  RMSEA & 200 & 0.915 & 0.017 & 0.893 & 0.013 & 0.910 & 0.014 \\ 
  SRMRW & 30 & 0.674 & 0.042 & 0.642 & 0.052 & 0.682 & 0.050 \\ 
  SRMRW & 50 & 0.737 & 0.037 & 0.709 & 0.048 & 0.733 & 0.047 \\ 
  SRMRW & 100 & 0.816 & 0.037 & 0.803 & 0.046 & 0.797 & 0.045 \\ 
  SRMRW & 200 & 0.832 & 0.032 & 0.855 & 0.042 & 0.813 & 0.038 \\ 
  SRMRB & 30 & 0.565 & 0.143 & 0.567 & 0.106 & 0.576 & 0.117 \\ 
  SRMRB & 50 & 0.610 & 0.119 & 0.596 & 0.081 & 0.613 & 0.096 \\ 
  SRMRB & 100 & 0.673 & 0.093 & 0.649 & 0.060 & 0.670 & 0.071 \\ 
  SRMRB & 200 & 0.732 & 0.069 & 0.701 & 0.047 & 0.723 & 0.054 \\ 
   \bottomrule
\end{tabular}
\end{table}

Detecting Misspecification at level-1

c <- filter(roc_summary, Classification == "CvM1", `Level-2 SS` != 'ALL', `Level-1 SS` == 'ALL')
# Next make the columns the estimator factor
c1 <- cbind(c[ c$Estimator == 'MLR', c(2,4,6,9)],
            c[ c$Estimator == 'ULSMV', c(6,9)],
            c[ c$Estimator == 'WLSMV', c(6,9)])
kable(c1, format = 'html',digits=3, row.names = F) %>%
  kable_styling(full_width = T) %>%
  add_header_above(c(' '=2, 'MLR'=2, 'USLMV'=2, 'WLSMV'=2))

		MLR		USLMV		WLSMV
Index	Level-2 SS	AUC	Threshold	AUC	Threshold	AUC	Threshold
CFI	30	0.829	0.944	0.649	0.979	0.804	0.983
CFI	50	0.926	0.955	0.735	0.972	0.926	0.980
CFI	100	0.990	0.962	0.875	0.971	0.992	0.972
CFI	200	1.000	0.969	0.989	0.975	1.000	0.971
TLI	30	0.829	0.932	0.649	0.975	0.804	0.979
TLI	50	0.926	0.945	0.735	0.966	0.926	0.976
TLI	100	0.990	0.954	0.875	0.965	0.992	0.966
TLI	200	1.000	0.962	0.989	0.970	1.000	0.966
RMSEA	30	0.799	0.030	0.651	0.010	0.796	0.014
RMSEA	50	0.908	0.026	0.729	0.012	0.908	0.014
RMSEA	100	0.988	0.024	0.854	0.011	0.988	0.017
RMSEA	200	1.000	0.020	0.958	0.013	1.000	0.020
SRMRW	30	0.769	0.042	0.722	0.053	0.801	0.053
SRMRW	50	0.861	0.038	0.817	0.048	0.877	0.047
SRMRW	100	0.981	0.037	0.945	0.046	0.982	0.045
SRMRW	200	1.000	0.033	0.997	0.043	1.000	0.040
SRMRB	30	0.518	0.177	0.532	0.093	0.530	0.109
SRMRB	50	0.503	0.151	0.541	0.070	0.552	0.086
SRMRB	100	0.530	0.069	0.565	0.052	0.582	0.060
SRMRB	200	0.600	0.057	0.595	0.038	0.626	0.048

print(xtable(c1, digits = 3), booktabs=T,include.rownames = F)

% latex table generated in R 3.6.0 by xtable 1.8-4 package
% Sat Sep 28 23:01:03 2019
\begin{table}[ht]
\centering
\begin{tabular}{llrrrrrr}
  \toprule
Index & Level-2 SS & AUC & Threshold & AUC & Threshold & AUC & Threshold \\ 
  \midrule
CFI & 30 & 0.829 & 0.944 & 0.649 & 0.979 & 0.804 & 0.983 \\ 
  CFI & 50 & 0.926 & 0.955 & 0.735 & 0.972 & 0.926 & 0.980 \\ 
  CFI & 100 & 0.990 & 0.962 & 0.875 & 0.971 & 0.992 & 0.972 \\ 
  CFI & 200 & 1.000 & 0.969 & 0.989 & 0.975 & 1.000 & 0.971 \\ 
  TLI & 30 & 0.829 & 0.932 & 0.649 & 0.975 & 0.804 & 0.979 \\ 
  TLI & 50 & 0.926 & 0.945 & 0.735 & 0.966 & 0.926 & 0.976 \\ 
  TLI & 100 & 0.990 & 0.954 & 0.875 & 0.965 & 0.992 & 0.966 \\ 
  TLI & 200 & 1.000 & 0.962 & 0.989 & 0.970 & 1.000 & 0.966 \\ 
  RMSEA & 30 & 0.799 & 0.030 & 0.651 & 0.010 & 0.796 & 0.014 \\ 
  RMSEA & 50 & 0.908 & 0.026 & 0.729 & 0.012 & 0.908 & 0.014 \\ 
  RMSEA & 100 & 0.988 & 0.024 & 0.854 & 0.011 & 0.988 & 0.017 \\ 
  RMSEA & 200 & 1.000 & 0.020 & 0.958 & 0.013 & 1.000 & 0.020 \\ 
  SRMRW & 30 & 0.769 & 0.042 & 0.722 & 0.053 & 0.801 & 0.053 \\ 
  SRMRW & 50 & 0.861 & 0.038 & 0.817 & 0.048 & 0.877 & 0.047 \\ 
  SRMRW & 100 & 0.981 & 0.037 & 0.945 & 0.046 & 0.982 & 0.045 \\ 
  SRMRW & 200 & 1.000 & 0.033 & 0.997 & 0.043 & 1.000 & 0.040 \\ 
  SRMRB & 30 & 0.518 & 0.177 & 0.532 & 0.093 & 0.530 & 0.109 \\ 
  SRMRB & 50 & 0.503 & 0.151 & 0.541 & 0.070 & 0.552 & 0.086 \\ 
  SRMRB & 100 & 0.530 & 0.069 & 0.565 & 0.052 & 0.582 & 0.060 \\ 
  SRMRB & 200 & 0.600 & 0.057 & 0.595 & 0.038 & 0.626 & 0.048 \\ 
   \bottomrule
\end{tabular}
\end{table}

Detecting Misspecification at level-2

c <- filter(roc_summary, Classification == "CvM2", `Level-2 SS` != 'ALL', `Level-1 SS` == 'ALL')
# Next make the columns the estimator factor
c1 <- cbind(c[ c$Estimator == 'MLR', c(2,4,6,9)],
            c[ c$Estimator == 'ULSMV', c(6,9)],
            c[ c$Estimator == 'WLSMV', c(6,9)])
kable(c1, format = 'html',digits=3, row.names = F) %>%
  kable_styling(full_width = T) %>%
  add_header_above(c(' '=2, 'MLR'=2, 'USLMV'=2, 'WLSMV'=2))

		MLR		USLMV		WLSMV
Index	Level-2 SS	AUC	Threshold	AUC	Threshold	AUC	Threshold
CFI	30	0.574	0.972	0.570	0.999	0.541	1.000
CFI	50	0.624	0.986	0.641	0.996	0.601	0.999
CFI	100	0.704	0.991	0.729	0.970	0.699	0.996
CFI	200	0.764	0.995	0.784	0.978	0.776	0.997
TLI	30	0.574	0.967	0.570	0.999	0.541	1.000
TLI	50	0.624	0.983	0.641	0.995	0.601	0.999
TLI	100	0.704	0.989	0.729	0.964	0.699	0.995
TLI	200	0.764	0.994	0.784	0.973	0.776	0.996
RMSEA	30	0.570	0.019	0.433	-Inf	0.459	0.054
RMSEA	50	0.620	0.013	0.635	0.005	0.599	0.003
RMSEA	100	0.699	0.010	0.724	0.007	0.696	0.007
RMSEA	200	0.758	0.008	0.782	0.006	0.775	0.006
SRMRW	30	0.499	0.024	0.513	0.041	0.504	0.026
SRMRW	50	0.506	0.048	0.539	0.034	0.513	0.023
SRMRW	100	0.509	0.036	0.580	0.035	0.518	0.028
SRMRW	200	0.518	0.029	0.630	0.028	0.535	0.021
SRMRB	30	0.613	0.143	0.575	0.106	0.598	0.120
SRMRB	50	0.678	0.122	0.622	0.082	0.656	0.096
SRMRB	100	0.756	0.098	0.701	0.065	0.736	0.076
SRMRB	200	0.808	0.072	0.769	0.049	0.795	0.058

print(xtable(c1, digits = 3), booktabs=T,include.rownames = F)

% latex table generated in R 3.6.0 by xtable 1.8-4 package
% Sat Sep 28 23:01:03 2019
\begin{table}[ht]
\centering
\begin{tabular}{llrrrrrr}
  \toprule
Index & Level-2 SS & AUC & Threshold & AUC & Threshold & AUC & Threshold \\ 
  \midrule
CFI & 30 & 0.574 & 0.972 & 0.570 & 0.999 & 0.541 & 1.000 \\ 
  CFI & 50 & 0.624 & 0.986 & 0.641 & 0.996 & 0.601 & 0.999 \\ 
  CFI & 100 & 0.704 & 0.991 & 0.729 & 0.970 & 0.699 & 0.996 \\ 
  CFI & 200 & 0.764 & 0.995 & 0.784 & 0.978 & 0.776 & 0.997 \\ 
  TLI & 30 & 0.574 & 0.967 & 0.570 & 0.999 & 0.541 & 1.000 \\ 
  TLI & 50 & 0.624 & 0.983 & 0.641 & 0.995 & 0.601 & 0.999 \\ 
  TLI & 100 & 0.704 & 0.989 & 0.729 & 0.964 & 0.699 & 0.995 \\ 
  TLI & 200 & 0.764 & 0.994 & 0.784 & 0.973 & 0.776 & 0.996 \\ 
  RMSEA & 30 & 0.570 & 0.019 & 0.433 & -Inf & 0.459 & 0.054 \\ 
  RMSEA & 50 & 0.620 & 0.013 & 0.635 & 0.005 & 0.599 & 0.003 \\ 
  RMSEA & 100 & 0.699 & 0.010 & 0.724 & 0.007 & 0.696 & 0.007 \\ 
  RMSEA & 200 & 0.758 & 0.008 & 0.782 & 0.006 & 0.775 & 0.006 \\ 
  SRMRW & 30 & 0.499 & 0.024 & 0.513 & 0.041 & 0.504 & 0.026 \\ 
  SRMRW & 50 & 0.506 & 0.048 & 0.539 & 0.034 & 0.513 & 0.023 \\ 
  SRMRW & 100 & 0.509 & 0.036 & 0.580 & 0.035 & 0.518 & 0.028 \\ 
  SRMRW & 200 & 0.518 & 0.029 & 0.630 & 0.028 & 0.535 & 0.021 \\ 
  SRMRB & 30 & 0.613 & 0.143 & 0.575 & 0.106 & 0.598 & 0.120 \\ 
  SRMRB & 50 & 0.678 & 0.122 & 0.622 & 0.082 & 0.656 & 0.096 \\ 
  SRMRB & 100 & 0.756 & 0.098 & 0.701 & 0.065 & 0.736 & 0.076 \\ 
  SRMRB & 200 & 0.808 & 0.072 & 0.769 & 0.049 & 0.795 & 0.058 \\ 
   \bottomrule
\end{tabular}
\end{table}

ROC Curves

First extract the data

roc_smooth_data <- as.data.frame(matrix(0,ncol=7, nrow=514*(3*4*5*5)))
colnames(roc_smooth_data) <- c('Index', 'Classification', 'Estimator','Level-2 SS', 'AUC', 'Sensitivity', 'Specificity')
i <- 1
j <- 514
for(index in INDEX){
  for(est in EST){
    for(class in CLASS){
    for(s2 in SS_L2){
    ## Set up iteration key
    key <- paste0(index,'.',class,'.',est,'.', s2,'.ALL')
      ## update extracted data
      roc_smooth_data[i:j, 1] <- index
      roc_smooth_data[i:j, 2] <- class
      roc_smooth_data[i:j, 3] <- est
      roc_smooth_data[i:j, 4] <- s2
      ## extract smooth fit object
      fit <- fit_roc_smooth[[key]]
      if(is.null(fit) == T){
        ## update sen,spec, and auc
        roc_smooth_data[i:j, 5] <- NA
        roc_smooth_data[i:j, 6] <- NA
        roc_smooth_data[i:j, 7] <- NA
      } else {
        ## update sen,spec, and auc
        roc_smooth_data[i:j, 5] <- fit$auc
        roc_smooth_data[i:j, 6] <- fit$sensitivities
        roc_smooth_data[i:j, 7] <- fit$specificities
      }
      
      ## update iterators
      i <- i + 514
      j <- j + 514
    }
  }
}}
## Forcing factor orders
roc_smooth_data$Index <- factor(
  roc_smooth_data$Index, ordered = T,
  levels=c('CFI', 'TLI', 'RMSEA', 'SRMRW', 'SRMRB'))
roc_smooth_data$Classification <- factor(
  roc_smooth_data$Classification,
  levels=c('C','CvM1','CvM2'),
  labels=c('Any Mis.', 'Level-1 Mis.', 'Level-2 Mis.'),
  ordered = T
)
roc_smooth_data$Estimator <- as.factor(roc_smooth_data$Estimator)
roc_smooth_data$`Level-2 SS` <- factor(roc_smooth_data$`Level-2 SS`,
                                       levels=c('ALL','30','50', '100', '200'),
                                       ordered = T)

Plot by Misspecification and Estimation Method

subdata <- filter(roc_smooth_data, `Level-2 SS`=='ALL', Estimator!='ALL')
p <- ggplot(subdata, aes(x = Specificity, y=Sensitivity, group = Index)) +
  geom_line(aes(linetype=Index, color=Index))+
  facet_grid(Estimator~Classification) +
  scale_x_reverse() +
  scale_color_brewer(palette="Set1") +
  guides(color=guide_legend(title="Fit Statistics"),
         linetype=guide_legend(title="Fit Statistics"))
p

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

if(save.fig == T) ggsave('roc_plot_mis_est.pdf', plot = p, height = 6,width = 9,units = 'in')

Plot by Misspecification and Level-2 Sample Size

subdata <- filter(roc_smooth_data, `Level-2 SS`!='ALL', Estimator=='ALL')
p <- ggplot(subdata, aes(x = Specificity, y=Sensitivity, group = Index)) +
  geom_line(aes(linetype=Index, color=Index))+
  facet_grid(`Level-2 SS` ~Classification) +
  scale_x_reverse() +
  scale_color_brewer(palette="Set1") +
  guides(color=guide_legend(title="Fit Statistics"),
         linetype=guide_legend(title="Fit Statistics"))
p

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

if(save.fig == T) ggsave('roc_plot_mis_n2.pdf', plot = p, height = 6,width = 9, units = 'in')

Figures of Subconditions and Smaller Plots for Exporting

Figures by Classification Outcome

subdata <- filter(roc_smooth_data, `Level-2 SS`=='ALL',  Estimator == "ALL")
p <- ggplot(subdata, aes(x = Specificity, y=Sensitivity, group = Index)) +
  geom_line(aes(linetype=Index, color=Index))+
  facet_grid(.~Classification) +
  scale_x_reverse() +
  scale_color_brewer(palette="Set1") +
  guides(color=guide_legend(title="Fit Statistics"),
         linetype=guide_legend(title="Fit Statistics"))
p

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

if(save.fig == T) ggsave('roc_class_all.pdf', plot = p, height = 4, width = 9, units = 'in')
subdata <- filter(roc_smooth_data, `Level-2 SS`=='ALL', Estimator == "MLR")
p <- ggplot(subdata, aes(x = Specificity, y=Sensitivity, group = Index)) +
  geom_line(aes(linetype=Index, color=Index))+
  facet_grid(.~Classification) +
  scale_x_reverse() +
  scale_color_brewer(palette="Set1") +
  guides(color=guide_legend(title="Fit Statistics"),
         linetype=guide_legend(title="Fit Statistics"))
p

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

if(save.fig == T) ggsave('roc_class_mlr.pdf', plot = p, height = 4,width = 9,units = 'in')
subdata <- filter(roc_smooth_data, `Level-2 SS`=='ALL', Estimator == "ULSMV")
p <- ggplot(subdata, aes(x = Specificity, y=Sensitivity, group = Index)) +
  geom_line(aes(linetype=Index, color=Index))+
  facet_grid(.~Classification) +
  scale_x_reverse() +
  scale_color_brewer(palette="Set1") +
  guides(color=guide_legend(title="Fit Statistics"),
         linetype=guide_legend(title="Fit Statistics"))
p

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

if(save.fig == T) ggsave('roc_class_ulsmv.pdf', plot = p, height = 4,width = 9,units = 'in')
subdata <- filter(roc_smooth_data,`Level-2 SS`=='ALL',  Estimator == "WLSMV")
p <- ggplot(subdata, aes(x = Specificity, y=Sensitivity, group = Index)) +
  geom_line(aes(linetype=Index, color=Index))+
  facet_grid(.~Classification) +
  scale_x_reverse() +
  scale_color_brewer(palette="Set1") +
  guides(color=guide_legend(title="Fit Statistics"),
         linetype=guide_legend(title="Fit Statistics"))
p

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

if(save.fig == T) ggsave('roc_class_wlsmv.pdf', plot = p, height = 4,width = 9,units = 'in')

Figures by Estimation Method

subdata <- filter(roc_smooth_data, `Level-2 SS`=='ALL', Estimator!='ALL',
                  Classification == "Any Mis.")
p <- ggplot(subdata, aes(x = Specificity, y=Sensitivity, group = Index)) +
  geom_line(aes(linetype=Index, color=Index))+
  facet_grid(.~Estimator) +
  scale_x_reverse() +
  scale_color_brewer(palette="Set1") +
  guides(color=guide_legend(title="Fit Statistics"),
         linetype=guide_legend(title="Fit Statistics"))
p

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

if(save.fig == T) ggsave('roc_est_c.pdf', plot = p, height = 4,width = 9,units = 'in')
subdata <- filter(roc_smooth_data, `Level-2 SS`=='ALL', Estimator!='ALL',
                  Classification == "Level-1 Mis.")
p <- ggplot(subdata, aes(x = Specificity, y=Sensitivity, group = Index)) +
  geom_line(aes(linetype=Index, color=Index))+
  facet_grid(.~Estimator) +
  scale_x_reverse() +
  scale_color_brewer(palette="Set1") +
  guides(color=guide_legend(title="Fit Statistics"),
         linetype=guide_legend(title="Fit Statistics"))
p

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

if(save.fig == T) ggsave('roc_est_cl1.pdf', plot = p, height = 4,width = 9,units = 'in')
subdata <- filter(roc_smooth_data,`Level-2 SS`=='ALL', Estimator!='ALL',
                  Classification == "Level-2 Mis.")
p <- ggplot(subdata, aes(x = Specificity, y=Sensitivity, group = Index)) +
  geom_line(aes(linetype=Index, color=Index))+
  facet_grid(.~Estimator) +
  scale_x_reverse() +
  scale_color_brewer(palette="Set1") +
  guides(color=guide_legend(title="Fit Statistics"),
         linetype=guide_legend(title="Fit Statistics"))
p

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

if(save.fig == T) ggsave('roc_est_cl2.pdf', plot = p, height = 4,width = 9,units = 'in')

Figures by Level-2 Sample Size

subdata <- filter(roc_smooth_data, Estimator=='ALL', Classification == "Any Mis.",
                  `Level-2 SS`!='ALL')
p <- ggplot(subdata, aes(x = Specificity, y=Sensitivity, group = Index)) +
  geom_line(aes(linetype=Index, color=Index))+
  facet_grid(.~`Level-2 SS`) +
  scale_x_reverse() +
  scale_color_brewer(palette="Set1") +
  guides(color=guide_legend(title="Fit Statistics"),
         linetype=guide_legend(title="Fit Statistics"))
p

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

if(save.fig == T) ggsave('roc_n2_c.pdf', plot = p, height = 4,width = 9,units = 'in')
subdata <- filter(roc_smooth_data, `Level-2 SS`!='ALL', Estimator=='ALL',
                  Classification == "Level-1 Mis.")
p <- ggplot(subdata, aes(x = Specificity, y=Sensitivity, group = Index)) +
  geom_line(aes(linetype=Index, color=Index))+
  facet_grid(.~`Level-2 SS`) +
  scale_x_reverse() +
  scale_color_brewer(palette="Set1") +
  guides(color=guide_legend(title="Fit Statistics"),
         linetype=guide_legend(title="Fit Statistics"))
p

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

if(save.fig == T) ggsave('roc_n2_cl1.pdf', plot = p, height = 4,width = 9,units = 'in')
subdata <- filter(roc_smooth_data,`Level-2 SS`!='ALL', Estimator=='ALL',  Classification == "Level-2 Mis.")
p <- ggplot(subdata, aes(x = Specificity, y=Sensitivity, group = Index)) +
  geom_line(aes(linetype=Index, color=Index))+
  facet_grid(.~`Level-2 SS`) +
  scale_x_reverse() +
  scale_color_brewer(palette="Set1") +
  guides(color=guide_legend(title="Fit Statistics"),
         linetype=guide_legend(title="Fit Statistics"))
p

Version	Author	Date
982c8f1	noah-padgett	2019-05-18

if(save.fig == T) ggsave('roc_n2_cl2.pdf', plot = p, height = 4,width = 9,units = 'in')

sessionInfo()

R version 3.6.0 (2019-04-26)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 18362)

Matrix products: default

locale:
[1] LC_COLLATE=English_United States.1252 
[2] LC_CTYPE=English_United States.1252   
[3] LC_MONETARY=English_United States.1252
[4] LC_NUMERIC=C                          
[5] LC_TIME=English_United States.1252    

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
 [1] pROC_1.15.0      xtable_1.8-4     kableExtra_1.1.0 psych_1.8.12    
 [5] car_3.0-3        carData_3.0-2    forcats_0.4.0    stringr_1.4.0   
 [9] dplyr_0.8.1      purrr_0.3.2      readr_1.3.1      tidyr_0.8.3     
[13] tibble_2.1.1     ggplot2_3.2.0    tidyverse_1.2.1 

loaded via a namespace (and not attached):
 [1] Rcpp_1.0.1         lubridate_1.7.4    lattice_0.20-38   
 [4] assertthat_0.2.1   rprojroot_1.3-2    digest_0.6.19     
 [7] R6_2.4.0           cellranger_1.1.0   plyr_1.8.4        
[10] backports_1.1.4    evaluate_0.14      highr_0.8         
[13] httr_1.4.0         pillar_1.4.1       rlang_0.3.4       
[16] lazyeval_0.2.2     curl_3.3           readxl_1.3.1      
[19] rstudioapi_0.10    data.table_1.12.2  whisker_0.3-2     
[22] rmarkdown_1.13     labeling_0.3       webshot_0.5.1     
[25] foreign_0.8-71     munsell_0.5.0      broom_0.5.2       
[28] compiler_3.6.0     modelr_0.1.4       xfun_0.7          
[31] pkgconfig_2.0.2    mnormt_1.5-5       htmltools_0.3.6   
[34] tidyselect_0.2.5   workflowr_1.4.0    rio_0.5.16        
[37] viridisLite_0.3.0  crayon_1.3.4       withr_2.1.2       
[40] grid_3.6.0         nlme_3.1-139       jsonlite_1.6      
[43] gtable_0.3.0       git2r_0.26.1       magrittr_1.5      
[46] scales_1.0.0       zip_2.0.2          cli_1.1.0         
[49] stringi_1.4.3      reshape2_1.4.3     fs_1.3.1          
[52] xml2_1.2.0         generics_0.0.2     openxlsx_4.1.0    
[55] RColorBrewer_1.1-2 tools_3.6.0        glue_1.3.1        
[58] hms_0.4.2          abind_1.4-5        parallel_3.6.0    
[61] yaml_2.2.0         colorspace_1.4-1   rvest_0.3.4       
[64] knitr_1.23         haven_2.1.0

ROC Analyses

2019-05-09

Packages and Set-Up

Data Management

Adding Labels to Conditions

ROC Analysis Labels

Introduction to ROC Analysis

Detecting any type of misspecification

General overview over all conditions

More fine grained information within/across conditions

Detecting Misspecification at Level-1

General overview over all conditions

More fine grained information within/across conditions

Detecting Misspecification at Level-2

General overview over all conditions

More fine grained information within/across conditions

Summarizing the Results

Detecting Any Misspecification

Detecting Misspecification at level-1

Detecting Misspecification at level-2

ROC Curves

First extract the data

Plot by Misspecification and Estimation Method

Plot by Misspecification and Level-2 Sample Size

Figures of Subconditions and Smaller Plots for Exporting

Figures by Classification Outcome

Figures by Estimation Method

Figures by Level-2 Sample Size