Last updated: 2020-02-20
Checks: 7 0
Knit directory: MHWNWA/
This reproducible R Markdown analysis was created with workflowr (version 1.6.0). The Checks tab describes the reproducibility checks that were applied when the results were created. The Past versions tab lists the development history.
Great! Since the R Markdown file has been committed to the Git repository, you know the exact version of the code that produced these results.
Great job! The global environment was empty. Objects defined in the global environment can affect the analysis in your R Markdown file in unknown ways. For reproduciblity it’s best to always run the code in an empty environment.
The command set.seed(20190513) was run prior to running the code in the R Markdown file. Setting a seed ensures that any results that rely on randomness, e.g. subsampling or permutations, are reproducible.
Great job! Recording the operating system, R version, and package versions is critical for reproducibility.
Nice! There were no cached chunks for this analysis, so you can be confident that you successfully produced the results during this run.
Great job! Using relative paths to the files within your workflowr project makes it easier to run your code on other machines.
Great! You are using Git for version control. Tracking code development and connecting the code version to the results is critical for reproducibility. The version displayed above was the version of the Git repository at the time these results were generated.
Note that you need to be careful to ensure that all relevant files for the analysis have been committed to Git prior to generating the results (you can use wflow_publish or wflow_git_commit). workflowr only checks the R Markdown file, but you know if there are other scripts or data files that it depends on. Below is the status of the Git repository when the results were generated:
Ignored files:
Ignored: .Rhistory
Ignored: .Rproj.user/
Ignored: LaTeX/SOM_node_summary.docx
Ignored: LaTeX/SOM_node_summary.pdf
Ignored: data/NAPA/NAPA_clim_U.Rda
Ignored: data/NAPA/NAPA_clim_V.Rda
Ignored: data/NAPA/NAPA_clim_W.Rda
Ignored: data/NAPA/NAPA_clim_emp_ice.Rda
Ignored: data/NAPA/NAPA_clim_emp_oce.Rda
Ignored: data/NAPA/NAPA_clim_fmmflx.Rda
Ignored: data/NAPA/NAPA_clim_mldkz5.Rda
Ignored: data/NAPA/NAPA_clim_mldr10_1.Rda
Ignored: data/NAPA/NAPA_clim_qemp_oce.Rda
Ignored: data/NAPA/NAPA_clim_qla_oce.Rda
Ignored: data/NAPA/NAPA_clim_qns.Rda
Ignored: data/NAPA/NAPA_clim_qsb_oce.Rda
Ignored: data/NAPA/NAPA_clim_qt.Rda
Ignored: data/NAPA/NAPA_clim_runoffs.Rda
Ignored: data/NAPA/NAPA_clim_ssh.Rda
Ignored: data/NAPA/NAPA_clim_sss.Rda
Ignored: data/NAPA/NAPA_clim_sst.Rda
Ignored: data/NAPA/NAPA_clim_taum.Rda
Ignored: data/NAPA/NAPA_clim_vars.Rda
Ignored: data/NAPA/NAPA_clim_vecs.Rda
Ignored: data/NAPA/synoptic_vec_states.Rda
Ignored: data/SOM/packet.Rda
Ignored: data/SOM/synoptic_states.Rda
Ignored: data/SOM/synoptic_states_other.Rda
Ignored: data/anom/ALL_anom.Rda
Ignored: data/anom/ALL_other.Rda
Ignored: data/anom/ERA5_mslp_anom.Rda
Ignored: data/anom/ERA5_qnet_anom.Rda
Ignored: data/anom/ERA5_t2m_anom.Rda
Ignored: data/anom/ERA5_u_anom.Rda
Ignored: data/anom/ERA5_v_anom.Rda
Ignored: data/anom/GLORYS_mld_anom.Rda
Ignored: data/anom/GLORYS_u_anom.Rda
Ignored: data/anom/GLORYS_v_anom.Rda
Ignored: data/anom/OISST_sst_anom.Rda
Ignored: data/base/ERA5_lhf.Rda
Ignored: data/base/ERA5_lwr.Rda
Ignored: data/base/ERA5_mslp.Rda
Ignored: data/base/ERA5_qnet.Rda
Ignored: data/base/ERA5_shf.Rda
Ignored: data/base/ERA5_swr.Rda
Ignored: data/base/ERA5_t2m.Rda
Ignored: data/base/ERA5_u.Rda
Ignored: data/base/ERA5_v.Rda
Ignored: data/base/GLORYS_mld.Rda
Ignored: data/base/GLORYS_u.Rda
Ignored: data/base/GLORYS_v.Rda
Ignored: data/base/OAFlux.Rda
Ignored: data/base/OISST_sst.Rda
Ignored: data/clim/ALL_clim.Rda
Ignored: data/clim/ERA5_mslp_clim.Rda
Ignored: data/clim/ERA5_qnet_clim.Rda
Ignored: data/clim/ERA5_t2m_clim.Rda
Ignored: data/clim/ERA5_u_clim.Rda
Ignored: data/clim/ERA5_v_clim.Rda
Ignored: data/clim/GLORYS_mld_clim.Rda
Ignored: data/clim/GLORYS_u_clim.Rda
Ignored: data/clim/GLORYS_v_clim.Rda
Ignored: data/clim/OISST_sst_clim.Rda
Ignored: data/comparison/node_mean_all_anom.Rda
Ignored: data/comparison/packet_all.Rda
Ignored: data/comparison/packet_all_anom.Rda
Ignored: data/comparison/packet_nolab.Rda
Ignored: data/comparison/packet_nolab14.Rda
Ignored: data/comparison/packet_nolabgsl.Rda
Ignored: data/comparison/packet_nolabmod.Rda
Ignored: data/comparison/som_all.Rda
Ignored: data/comparison/som_all_anom.Rda
Ignored: data/comparison/som_nolab.Rda
Ignored: data/comparison/som_nolab14.Rda
Ignored: data/comparison/som_nolab_16.Rda
Ignored: data/comparison/som_nolab_9.Rda
Ignored: data/comparison/som_nolabgsl.Rda
Ignored: data/comparison/som_nolabmod.Rda
Ignored: data/eddy_trajectory_2.0exp_19930101_20180118.nc
Ignored: talk/IMBeR_2019.html
Ignored: talk/MHWNWA.html
Unstaged changes:
Modified: code/functions.R
Modified: output/SOM/air_u_v_mslp_anom.pdf
Modified: output/SOM/air_u_v_mslp_real.pdf
Modified: output/SOM/cum_int_season.pdf
Modified: output/SOM/cum_int_season.png
Modified: output/SOM/duration_rate_onset.pdf
Modified: output/SOM/max_int_region.pdf
Modified: output/SOM/max_int_region.png
Modified: output/SOM/mld_anom_sd.pdf
Modified: output/SOM/msl_anom_sd.pdf
Modified: output/SOM/msl_sd.pdf
Modified: output/SOM/node_10_panels.pdf
Modified: output/SOM/node_10_panels.png
Modified: output/SOM/node_11_panels.pdf
Modified: output/SOM/node_11_panels.png
Modified: output/SOM/node_12_panels.pdf
Modified: output/SOM/node_12_panels.png
Modified: output/SOM/node_1_panels.pdf
Modified: output/SOM/node_1_panels.png
Modified: output/SOM/node_2_panels.pdf
Modified: output/SOM/node_2_panels.png
Modified: output/SOM/node_3_panels.pdf
Modified: output/SOM/node_3_panels.png
Modified: output/SOM/node_4_panels.pdf
Modified: output/SOM/node_4_panels.png
Modified: output/SOM/node_5_panels.pdf
Modified: output/SOM/node_5_panels.png
Modified: output/SOM/node_6_panels.pdf
Modified: output/SOM/node_6_panels.png
Modified: output/SOM/node_7_panels.pdf
Modified: output/SOM/node_7_panels.png
Modified: output/SOM/node_8_panels.pdf
Modified: output/SOM/node_8_panels.png
Modified: output/SOM/node_9_panels.pdf
Modified: output/SOM/node_9_panels.png
Modified: output/SOM/qnet_anom_sd.pdf
Modified: output/SOM/qnet_mld_anom.pdf
Modified: output/SOM/region_season.pdf
Modified: output/SOM/sst_anom_sd.pdf
Modified: output/SOM/sst_sd.pdf
Modified: output/SOM/sst_u_v_anom.pdf
Modified: output/SOM/sst_u_v_real.pdf
Modified: output/SOM/t2m_anom_sd.pdf
Modified: output/SOM/t2m_sd.pdf
Modified: output/SOM/u10_anom_sd.pdf
Modified: output/SOM/u10_sd.pdf
Modified: output/SOM/u_anom_sd.pdf
Modified: output/SOM/u_sd.pdf
Modified: output/SOM/v10_anom_sd.pdf
Modified: output/SOM/v10_sd.pdf
Modified: output/SOM/v_anom_sd.pdf
Modified: output/SOM/v_sd.pdf
Note that any generated files, e.g. HTML, png, CSS, etc., are not included in this status report because it is ok for generated content to have uncommitted changes.
These are the previous versions of the R Markdown and HTML files. If you’ve configured a remote Git repository (see ?wflow_git_remote), click on the hyperlinks in the table below to view them.
| File | Version | Author | Date | Message |
|---|---|---|---|---|
| Rmd | a3b88a8 | Robert William Schlegel | 2020-02-12 | Working through a full edit of the workflowr site. This will seque into a write up of the draft manuscript. |
| html | a7fbd91 | robwschlegel | 2019-08-28 | Build site. |
| Rmd | d980be6 | robwschlegel | 2019-08-26 | Working towards the node summary figure pipeline update |
| Rmd | 27b124b | robwschlegel | 2019-08-23 | Working on code to smoothly introduce the other data needed for creating summary figures but that aren’t used in the SOM calculation |
| Rmd | 9c4a1d8 | robwschlegel | 2019-08-21 | An additional thought |
| Rmd | ac54ed0 | robwschlegel | 2019-08-21 | Another round of figure creation |
| html | 826c73d | robwschlegel | 2019-08-15 | Build site. |
| html | 143b1a6 | robwschlegel | 2019-08-15 | Build site. |
| Rmd | 78f4977 | robwschlegel | 2019-08-15 | Re-publish entire site. |
| Rmd | 07fe2a2 | robwschlegel | 2019-08-14 | Nearly through the node summaries for the three SOM experiments |
| Rmd | a61b420 | robwschlegel | 2019-08-13 | Working on SOM write-up |
| html | 20ae166 | robwschlegel | 2019-08-11 | Build site. |
| html | 19bea26 | robwschlegel | 2019-08-11 | Build site. |
| html | 2652a3a | robwschlegel | 2019-08-11 | Build site. |
| Rmd | adc762b | robwschlegel | 2019-08-08 | Re-worked the GLORYS data and propogated update through to SOM analysis figures for all experiments |
| html | f0d2efb | robwschlegel | 2019-08-07 | Build site. |
| Rmd | ed626bf | robwschlegel | 2019-08-07 | Ran a bunch of figures and had a meeting with Eric. More changes coming to GLORYS data tomorrow before settling on one of the experimental SOMs |
| html | f66aa38 | robwschlegel | 2019-08-01 | Build site. |
| Rmd | 5e12d9e | robwschlegel | 2019-08-01 | Re-publish entire site. |
| Rmd | 9a9fa7d | robwschlegel | 2019-08-01 | A more in depth dive into the potential criteria to meet for the SOM model |
| Rmd | 240a7a0 | robwschlegel | 2019-07-31 | Ran the base SOM results |
| html | aa82e6e | robwschlegel | 2019-07-31 | Build site. |
| Rmd | 498909b | robwschlegel | 2019-07-31 | Re-publish entire site. |
| html | 35987b4 | robwschlegel | 2019-07-09 | Build site. |
| Rmd | 34efa43 | robwschlegel | 2019-07-09 | Added some thinking to the SOM vignette. |
| html | e2f6f42 | robwschlegel | 2019-07-09 | Build site. |
| Rmd | 609cca8 | robwschlegel | 2019-07-09 | Added some thinking to the SOM vignette. |
| html | 81e961d | robwschlegel | 2019-07-09 | Build site. |
| Rmd | 7ff9b8b | robwschlegel | 2019-06-17 | More work on the talk |
| Rmd | b25762e | robwschlegel | 2019-06-12 | More work on figures |
| Rmd | 413bb8b | robwschlegel | 2019-06-12 | Working on pixel interpolation |
| html | c23c50b | robwschlegel | 2019-06-10 | Build site. |
| html | 028d3cc | robwschlegel | 2019-06-10 | Build site. |
| Rmd | c6b3c7b | robwschlegel | 2019-06-10 | Re-publish entire site. |
| Rmd | 1b53eeb | robwschlegel | 2019-06-10 | SOM packet pipeline testing |
| Rmd | 4504e12 | robwschlegel | 2019-06-07 | Working on joining in vector data |
| html | c61a15f | robwschlegel | 2019-06-06 | Build site. |
| Rmd | 44ac335 | robwschlegel | 2019-06-06 | Working on inclusion of vectors into SOM pipeline |
| html | 6dd6da8 | robwschlegel | 2019-06-06 | Build site. |
| Rmd | 07137d9 | robwschlegel | 2019-06-06 | Site wide update, including newly functioning SOM pipeline. |
| Rmd | 990693a | robwschlegel | 2019-06-05 | First SOM result visuals |
| Rmd | 25e7e9a | robwschlegel | 2019-06-05 | SOM pipeline nearly finished |
| Rmd | 4838cc8 | robwschlegel | 2019-06-04 | Working on SOM functions |
| Rmd | 94ce8f6 | robwschlegel | 2019-06-04 | Functions for creating data packets are up and running |
| Rmd | 65301ed | robwschlegel | 2019-05-30 | Push before getting rid of some testing structure |
| html | c09b4f7 | robwschlegel | 2019-05-24 | Build site. |
| Rmd | 5dc8bd9 | robwschlegel | 2019-05-24 | Finished initial creation of SST prep vignette. |
| html | a29be6b | robwschlegel | 2019-05-13 | Build site. |
| html | ea61999 | robwschlegel | 2019-05-13 | Build site. |
| Rmd | f8f28b1 | robwschlegel | 2019-05-13 | Skeleton files |
This vignette contains the code used to perform the self-organising map (SOM) analysis on the mean synoptic states created in the Variable preparation vignette. We’ll start by creating custom packets that meet certain experimental criteria before feeding them into a SOM. The full summary of the results may be seen in the Node summary vignette.
# Load functions and objects to be used below
source("code/functions.R")In this last stage before running our SOM analysis we will create a data packet that can be fed directly into the SOM algorithm. This means that it must be converted into a super-wide matrix format. In the first run of this analysis on the NAPA model data it was found that the inclusion of the Labrador Sea complicated the results quite a bit. It was also unclear whether or not the Gulf of St Lawrence (gsl) region should be included in the analysis. So in the second run of this analysis multiple different SOM variations were employed and it was decided that the gsl region should be included.
Up first we must create the synoptic state packets from all of the variables we prepared in the previous vignette.
## Create one big anomaly packet
system.time(synoptic_states <- plyr::ddply(OISST_MHW_event, c("region", "event_no"),
data_packet_func, .parallel = T)) # 204 seconds
# Save
saveRDS(synoptic_states, "data/SOM/synoptic_states.Rda")
## Create other synoptic states per MHW per variable
doMC::registerDoMC(cores = 10) # NB: Be careful here...
system.time(synoptic_states_other <- plyr::ddply(OISST_MHW_event, c("region", "event_no"),
data_packet_func, .parallel = T, df = ALL_other)) # 122 seconds
# Save
saveRDS(synoptic_states_other, "data/SOM/synoptic_states_other.Rda")With all of our data ready we may now prepare and save them for the SOM.
## Create wide data packet that is fed to SOM
system.time(packet <- readRDS("data/SOM/synoptic_states.Rda") %>%
select(region, event_no, synoptic) %>%
unnest() %>%
wide_packet_func()) # 122 seconds
# Save
saveRDS(packet, "data/SOM/packet.Rda")Now we feed the SOM with a function that ingests the data packet and produces results for us. The function below has been greatly expanded on from the previous version of this project and now performs all of the SOM related work in one go. This allowed me to remove a couple hundreds lines of code and text from this vignette.
# The SOM on the entire study area
packet <- readRDS("data/SOM/packet.Rda")
system.time(som <- som_model_PCI(packet)) # 83 seconds
saveRDS(som, file = "data/SOM/som.Rda")And there we have our SOM results. Up next in the Node summary vignette we will show the results with a range of visuals.
sessionInfo()R version 3.6.2 (2019-12-12)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 16.04.6 LTS
Matrix products: default
BLAS: /usr/lib/openblas-base/libblas.so.3
LAPACK: /usr/lib/libopenblasp-r0.2.18.so
locale:
[1] LC_CTYPE=en_CA.UTF-8 LC_NUMERIC=C
[3] LC_TIME=en_CA.UTF-8 LC_COLLATE=en_CA.UTF-8
[5] LC_MONETARY=en_CA.UTF-8 LC_MESSAGES=en_CA.UTF-8
[7] LC_PAPER=en_CA.UTF-8 LC_NAME=C
[9] LC_ADDRESS=C LC_TELEPHONE=C
[11] LC_MEASUREMENT=en_CA.UTF-8 LC_IDENTIFICATION=C
attached base packages:
[1] stats graphics grDevices utils datasets methods base
other attached packages:
[1] ggforce_0.3.1.9000 yasomi_0.3 proxy_0.4-23
[4] e1071_1.7-3 tidync_0.2.3 ncdf4_1.17
[7] heatwaveR_0.4.2.9001 data.table_1.12.8 lubridate_1.7.4
[10] forcats_0.4.0 stringr_1.4.0 dplyr_0.8.4
[13] purrr_0.3.3 readr_1.3.1 tidyr_1.0.2
[16] tibble_2.1.3 ggplot2_3.2.1 tidyverse_1.3.0
loaded via a namespace (and not attached):
[1] httr_1.4.1 foreach_1.4.4 jsonlite_1.6.1 viridisLite_0.3.0
[5] modelr_0.1.5 assertthat_0.2.1 cellranger_1.1.0 yaml_2.2.1
[9] pillar_1.4.3 backports_1.1.5 lattice_0.20-35 glue_1.3.1
[13] digest_0.6.23 promises_1.1.0 polyclip_1.10-0 rvest_0.3.5
[17] colorspace_1.4-1 htmltools_0.4.0 httpuv_1.5.2 pkgconfig_2.0.3
[21] broom_0.5.3 haven_2.2.0 scales_1.1.0 tweenr_1.0.1
[25] whisker_0.4 later_1.0.0 git2r_0.26.1 farver_2.0.3
[29] generics_0.0.2 withr_2.1.2 lazyeval_0.2.2 cli_2.0.1
[33] magrittr_1.5 crayon_1.3.4 readxl_1.3.1 evaluate_0.14
[37] fs_1.3.1 fansi_0.4.1 doParallel_1.0.15 nlme_3.1-137
[41] MASS_7.3-50 xml2_1.2.2 class_7.3-14 tools_3.6.2
[45] hms_0.5.3 lifecycle_0.1.0 plotly_4.9.1 munsell_0.5.0
[49] reprex_0.3.0 compiler_3.6.2 RNetCDF_2.1-1 rlang_0.4.4
[53] grid_3.6.2 iterators_1.0.10 rstudioapi_0.10 htmlwidgets_1.5.1
[57] rmarkdown_2.0 codetools_0.2-15 gtable_0.3.0 DBI_1.0.0
[61] R6_2.4.1 ncmeta_0.2.0 knitr_1.27 workflowr_1.6.0
[65] rprojroot_1.3-2 stringi_1.4.5 parallel_3.6.2 Rcpp_1.0.3
[69] vctrs_0.2.2 dbplyr_1.4.2 tidyselect_1.0.0 xfun_0.12