Last updated: 2019-08-17

Checks: 7 0

Knit directory: 2019-feature-selection/

This reproducible R Markdown analysis was created with workflowr (version 1.4.0). The Checks tab describes the reproducibility checks that were applied when the results were created. The Past versions tab lists the development history.


Great! Since the R Markdown file has been committed to the Git repository, you know the exact version of the code that produced these results.

Great job! The global environment was empty. Objects defined in the global environment can affect the analysis in your R Markdown file in unknown ways. For reproduciblity it’s best to always run the code in an empty environment.

The command set.seed(20190522) was run prior to running the code in the R Markdown file. Setting a seed ensures that any results that rely on randomness, e.g. subsampling or permutations, are reproducible.

Great job! Recording the operating system, R version, and package versions is critical for reproducibility.

Nice! There were no cached chunks for this analysis, so you can be confident that you successfully produced the results during this run.

Great job! Using relative paths to the files within your workflowr project makes it easier to run your code on other machines.

Great! You are using Git for version control. Tracking code development and connecting the code version to the results is critical for reproducibility. The version displayed above was the version of the Git repository at the time these results were generated.

Note that you need to be careful to ensure that all relevant files for the analysis have been committed to Git prior to generating the results (you can use wflow_publish or wflow_git_commit). workflowr only checks the R Markdown file, but you know if there are other scripts or data files that it depends on. Below is the status of the Git repository when the results were generated:


Ignored files:
    Ignored:    .Rhistory
    Ignored:    .Rproj.user/
    Ignored:    .Ruserdata/
    Ignored:    .drake/
    Ignored:    analysis/rosm.cache/
    Ignored:    data/
    Ignored:    inst/study-area-map/study-area.qgs~
    Ignored:    log/
    Ignored:    packrat/lib-R/
    Ignored:    packrat/lib-ext/
    Ignored:    packrat/lib/
    Ignored:    reviews/
    Ignored:    rosm.cache/
    Ignored:    tests/

Untracked files:
    Untracked:  .drake_history/
    Untracked:  inst/Benchmark for Filter Methods for Feature Selection in High-Dimensional  Classification Data.pdf
    Untracked:  test-hyperspec.rda
    Untracked:  test-out.gri

Unstaged changes:
    Modified:   _drake.R
    Modified:   code/98-paper/journal/defoliation-distribution-plot-1.pdf

Note that any generated files, e.g. HTML, png, CSS, etc., are not included in this status report because it is ok for generated content to have uncommitted changes.


These are the previous versions of the R Markdown and HTML files. If you’ve configured a remote Git repository (see ?wflow_git_remote), click on the hyperlinks in the table below to view them.

File Version Author Date Message
html df85aba pat-s 2019-07-12 Build site.
html 3a44a95 pat-s 2019-07-10 Build site.
html c238ce4 pat-s 2019-07-10 Build site.
Rmd e98cb01 pat-s 2019-07-10 wflow_publish(knitr_in(“analysis/eval-performance.Rmd”), view = FALSE,
Rmd 24e318f pat-s 2019-07-01 update reports
Rmd ca5c5bc pat-s 2019-06-28 add eval-performance report

Aggregated performances

(Table) All leaner/filter/task combinations ordered by performance.

Warning: Expected 2 pieces. Additional pieces discarded in 18 rows [6, 15,
24, 33, 42, 51, 60, 69, 78, 87, 96, 105, 114, 123, 132, 141, 150, 159].
Warning: Expected 2 pieces. Missing pieces filled with `NA` in 18 rows [1,
10, 19, 28, 37, 46, 55, 64, 73, 82, 91, 100, 109, 118, 127, 136, 145, 154].
learner_group task.id filter best
SVM VI PCA 1.822990
SVM defoliation-all-plots-HR-VI Car 1.827170
SVM VI CMIM 1.828968
SVM VI Info 1.832112
SVM defoliation-all-plots-HR-VI Borda 1.833585
SVM defoliation-all-plots-HR-VI CMIM 1.834629
SVM HR Pearson 1.835110
SVM defoliation-all-plots-HR-VI MRMR 1.836532
SVM VI No Filter 1.837101
SVM defoliation-all-plots-HR-NRI No Filter 1.837101
SVM defoliation-all-plots-HR-NRI-VI No Filter 1.837101
SVM defoliation-all-plots-HR-VI No Filter 1.837101
SVM HR No Filter 1.837101
SVM defoliation-all-plots-HR-VI Info 1.837494
SVM defoliation-all-plots-HR-NRI Relief 1.837753
SVM NRI CMIM 1.837772
SVM VI Relief 1.838980
SVM defoliation-all-plots-HR-NRI-VI MRMR 1.839602
SVM NRI PCA 1.839763
SVM defoliation-all-plots-HR-NRI-VI PCA 1.839781
SVM HR CMIM 1.839929
SVM HR MRMR 1.839938
SVM defoliation-all-plots-HR-NRI-VI Car 1.840088
SVM defoliation-all-plots-HR-NRI-VI CMIM 1.840088
SVM HR Car 1.840133
SVM HR Borda 1.840423
SVM defoliation-all-plots-HR-VI Relief 1.840567
SVM defoliation-all-plots-HR-NRI PCA 1.840637
SVM defoliation-all-plots-HR-NRI Car 1.841355
SVM VI Pearson 1.841647
SVM HR Info 1.845974
SVM defoliation-all-plots-HR-VI Pearson 1.854415
SVM NRI No Filter 1.862812
SVM defoliation-all-plots-HR-VI PCA 1.863684
SVM NRI Car 1.864166
SVM defoliation-all-plots-HR-NRI CMIM 1.866752
SVM NRI MRMR 1.870250
SVM HR PCA 1.870688
SVM defoliation-all-plots-HR-NRI MRMR 1.870723
SVM defoliation-all-plots-HR-NRI Borda 1.873015
SVM defoliation-all-plots-HR-NRI-VI Info 1.873985
SVM NRI Relief 1.875979
SVM VI Borda 1.876451
SVM NRI Pearson 1.879307
SVM defoliation-all-plots-HR-NRI-VI Pearson 1.882458
SVM defoliation-all-plots-HR-NRI Pearson 1.887703
SVM defoliation-all-plots-HR-NRI Info 1.890634
SVM HR Relief 1.894857
SVM defoliation-all-plots-HR-NRI-VI Relief 1.900559
SVM NRI Info 1.903714
SVM defoliation-all-plots-HR-NRI-VI Borda 1.906305
SVM VI Car 1.920964
SVM VI MRMR 1.934234
SVM NRI Borda 1.976663
XG HR Info 2.141743
XG HR CMIM 2.159212
XG HR MRMR 2.163146
XG defoliation-all-plots-HR-VI PCA 2.191695
XG HR Car 2.195092
XG NRI MRMR 2.223025
XG defoliation-all-plots-HR-NRI-VI MRMR 2.250640
RF defoliation-all-plots-HR-NRI Borda 2.255191
XG defoliation-all-plots-HR-NRI MRMR 2.267891
RF defoliation-all-plots-HR-NRI-VI Borda 2.280211
RF NRI Car 2.288766
RF NRI Borda 2.290181
RF defoliation-all-plots-HR-NRI Car 2.291882
RF defoliation-all-plots-HR-NRI-VI Car 2.293350
XG NRI Car 2.305419
XG defoliation-all-plots-HR-NRI Info 2.313288
XG defoliation-all-plots-HR-NRI-VI Relief 2.330368
XG defoliation-all-plots-HR-NRI-VI Car 2.331484
XG defoliation-all-plots-HR-NRI-VI CMIM 2.331644
XG defoliation-all-plots-HR-NRI Car 2.332452
XG NRI Info 2.346927
XG NRI Relief 2.352454
XG NRI No Filter 2.353493
XG defoliation-all-plots-HR-NRI Borda 2.359013
XG defoliation-all-plots-HR-NRI CMIM 2.372463
RF defoliation-all-plots-HR-NRI Info 2.382571
RF NRI Info 2.383427
XG HR PCA 2.387424
RF defoliation-all-plots-HR-NRI-VI Info 2.389203
XG defoliation-all-plots-HR-NRI-VI Info 2.393840
RF NRI Relief 2.393892
RF defoliation-all-plots-HR-NRI Relief 2.403064
XG defoliation-all-plots-HR-VI Relief 2.417937
RF HR Info 2.423537
XG defoliation-all-plots-HR-NRI-VI No Filter 2.430075
XG defoliation-all-plots-HR-NRI No Filter 2.433078
XG VI PCA 2.435016
RF VI Borda 2.438750
RF defoliation-all-plots-HR-NRI Pearson 2.446430
XG VI Borda 2.450155
RF defoliation-all-plots-HR-NRI-VI Relief 2.450712
RF HR Borda 2.453136
RF defoliation-all-plots-HR-VI Info 2.454620
RF NRI Pearson 2.454894
RF defoliation-all-plots-HR-NRI-VI PCA 2.456446
RF defoliation-all-plots-HR-NRI PCA 2.463518
RF defoliation-all-plots-HR-NRI-VI Pearson 2.464900
XG defoliation-all-plots-HR-NRI Pearson 2.471025
RF defoliation-all-plots-HR-VI PCA 2.474679
RF NRI MRMR 2.476739
RF defoliation-all-plots-HR-NRI MRMR 2.479038
XG NRI PCA 2.479044
XG defoliation-all-plots-HR-NRI-VI Pearson 2.482297
XG defoliation-all-plots-HR-NRI-VI Borda 2.483488
XG HR Borda 2.483780
RF NRI PCA 2.486899
XG defoliation-all-plots-HR-VI Info 2.486914
RF VI MRMR 2.496644
XG NRI Pearson 2.498566
RF defoliation-all-plots-HR-VI Borda 2.501562
RF HR PCA 2.508722
XG defoliation-all-plots-HR-NRI PCA 2.511731
XG defoliation-all-plots-HR-NRI Relief 2.511844
RF VI CMIM 2.518239
XG defoliation-all-plots-HR-NRI-VI PCA 2.522766
RF defoliation-all-plots-HR-VI Relief 2.523667
RF VI PCA 2.526413
RF defoliation-all-plots-HR-VI CMIM 2.527329
RF defoliation-all-plots-HR-NRI-VI MRMR 2.530956
RF VI Info 2.531361
XG VI Info 2.538958
RF VI Relief 2.546695
RF HR Relief 2.550136
RF defoliation-all-plots-HR-VI MRMR 2.551026
RF HR Pearson 2.555868
RF defoliation-all-plots-HR-NRI-VI CMIM 2.556866
RF NRI CMIM 2.562908
RF defoliation-all-plots-HR-NRI-VI No Filter 2.570548
XG VI No Filter 2.574851
RF NRI No Filter 2.575447
RF VI No Filter 2.583432
RF defoliation-all-plots-HR-NRI No Filter 2.584828
RF HR Car 2.588946
RF defoliation-all-plots-HR-NRI CMIM 2.589443
XG defoliation-all-plots-HR-VI MRMR 2.593910
XG VI CMIM 2.594678
XG HR Pearson 2.614220
RF HR CMIM 2.616294
XG defoliation-all-plots-HR-VI CMIM 2.619256
RF defoliation-all-plots-HR-VI Pearson 2.621757
RF defoliation-all-plots-HR-VI Car 2.624929
XG VI Relief 2.634002
RF defoliation-all-plots-HR-VI No Filter 2.638489
XG HR Relief 2.644360
RF VI Pearson 2.656051
RF HR MRMR 2.664423
XG defoliation-all-plots-HR-VI No Filter 2.665235
XG defoliation-all-plots-HR-VI Car 2.677492
XG VI Car 2.681144
XG HR No Filter 2.682939
RF VI Car 2.698937
XG VI Pearson 2.710132
XG VI MRMR 2.761796
XG NRI Borda 2.767447
XG defoliation-all-plots-HR-VI Pearson 2.768983
RF HR No Filter 2.815297
XG defoliation-all-plots-HR-VI Borda 2.834501
XG NRI CMIM 3.706941

(Table) Best learner/filter/task combination

Warning: Expected 2 pieces. Additional pieces discarded in 18 rows [6, 15,
24, 33, 42, 51, 60, 69, 78, 87, 96, 105, 114, 123, 132, 141, 150, 159].
Warning: Expected 2 pieces. Missing pieces filled with `NA` in 18 rows [1,
10, 19, 28, 37, 46, 55, 64, 73, 82, 91, 100, 109, 118, 127, 136, 145, 154].
task.id learner.id learner_group filter rmse.test.rmse
defoliation-all-plots-VI SVM PCA SVM PCA 1.822990
defoliation-all-plots-HR XG Info Gain XG Info 2.141743
defoliation-all-plots-HR-NRI RF Borda RF Borda 2.255191

(Plot) Best learner/filter combs for all tasks

Warning: Expected 1 pieces. Additional pieces discarded in 144 rows [2, 3,
4, 5, 6, 7, 8, 9, 11, 12, 13, 14, 15, 16, 17, 18, 20, 21, 22, 23, ...].

Version Author Date
c238ce4 pat-s 2019-07-10

R version 3.5.2 (2018-12-20)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: CentOS Linux 7 (Core)

Matrix products: default
BLAS: /opt/R/3.5.2/lib64/R/lib/libRblas.so
LAPACK: /usr/lib64/libopenblaso-r0.3.3.so

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C              
 [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8    
 [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                 
 [9] LC_ADDRESS=C               LC_TELEPHONE=C            
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] ggpubr_0.1.6      ggsci_2.9         ggrepel_0.8.0     ggplot2_3.1.0    
[5] dplyr_0.8.0.1     magrittr_1.5      mlr_2.14.0.9000   ParamHelpers_1.12
[9] tidyselect_0.2.5 

loaded via a namespace (and not attached):
 [1] storr_1.2.1       xfun_0.5          purrr_0.3.0      
 [4] splines_3.5.2     lattice_0.20-38   colorspace_1.4-0 
 [7] htmltools_0.3.6   yaml_2.2.0        survival_2.43-3  
[10] rlang_0.3.4       R.oo_1.22.0       pillar_1.3.1     
[13] txtq_0.1.4        glue_1.3.0        withr_2.1.2      
[16] R.utils_2.8.0     plyr_1.8.4        stringr_1.4.0    
[19] munsell_0.5.0     gtable_0.2.0      workflowr_1.4.0  
[22] R.methodsS3_1.7.1 evaluate_0.13     labeling_0.3     
[25] knitr_1.23        parallelMap_1.4   parallel_3.5.2   
[28] highr_0.7         Rcpp_1.0.0        backports_1.1.3  
[31] scales_1.0.0      filelock_1.0.2    checkmate_1.9.1  
[34] fs_1.2.6          fastmatch_1.1-0   digest_0.6.18    
[37] stringi_1.3.1     BBmisc_1.11       grid_3.5.2       
[40] rprojroot_1.3-2   cli_1.1.0         tools_3.5.2      
[43] base64url_1.4     lazyeval_0.2.1    tibble_2.0.1     
[46] tidyr_0.8.2       crayon_1.3.4      whisker_0.3-2    
[49] pkgconfig_2.0.2   Matrix_1.2-15     data.table_1.12.0
[52] drake_7.5.2       assertthat_0.2.0  rmarkdown_1.13   
[55] R6_2.4.0          igraph_1.2.4      git2r_0.24.0     
[58] compiler_3.5.2