Last updated: 2021-01-23
Checks: 7 0
Knit directory: MHWflux/
This reproducible R Markdown analysis was created with workflowr (version 1.6.2). The Checks tab describes the reproducibility checks that were applied when the results were created. The Past versions tab lists the development history.
Great! Since the R Markdown file has been committed to the Git repository, you know the exact version of the code that produced these results.
Great job! The global environment was empty. Objects defined in the global environment can affect the analysis in your R Markdown file in unknown ways. For reproduciblity it’s best to always run the code in an empty environment.
The command set.seed(666) was run prior to running the code in the R Markdown file. Setting a seed ensures that any results that rely on randomness, e.g. subsampling or permutations, are reproducible.
Great job! Recording the operating system, R version, and package versions is critical for reproducibility.
Nice! There were no cached chunks for this analysis, so you can be confident that you successfully produced the results during this run.
Great job! Using relative paths to the files within your workflowr project makes it easier to run your code on other machines.
Great! You are using Git for version control. Tracking code development and connecting the code version to the results is critical for reproducibility.
The results in this page were generated with repository version 3f6b550. See the Past versions tab to see a history of the changes made to the R Markdown and HTML files.
Note that you need to be careful to ensure that all relevant files for the analysis have been committed to Git prior to generating the results (you can use wflow_publish or wflow_git_commit). workflowr only checks the R Markdown file, but you know if there are other scripts or data files that it depends on. Below is the status of the Git repository when the results were generated:
Ignored files:
Ignored: .Rhistory
Ignored: .Rproj.user/
Ignored: data/ALL_anom.Rda
Ignored: data/ALL_other.Rda
Ignored: data/ALL_ts_anom.Rda
Ignored: data/ERA5_down.Rda
Ignored: data/ERA5_down_anom.Rda
Ignored: data/ERA5_evp_anom.Rda
Ignored: data/ERA5_lhf_anom.Rda
Ignored: data/ERA5_lwr_anom.Rda
Ignored: data/ERA5_mslp_anom.Rda
Ignored: data/ERA5_pcp_anom.Rda
Ignored: data/ERA5_qnet_anom.Rda
Ignored: data/ERA5_shf_anom.Rda
Ignored: data/ERA5_swr_MLD.Rda
Ignored: data/ERA5_swr_anom.Rda
Ignored: data/ERA5_t2m_anom.Rda
Ignored: data/ERA5_tcc_anom.Rda
Ignored: data/ERA5_u_anom.Rda
Ignored: data/ERA5_v_anom.Rda
Ignored: data/GLORYS_all_anom.Rda
Ignored: data/OISST_all_anom.Rda
Ignored: data/packet.Rda
Ignored: data/som.Rda
Ignored: data/synoptic_states.Rda
Ignored: data/synoptic_states_other.Rda
Unstaged changes:
Modified: output/magnitude_schematic.pdf
Note that any generated files, e.g. HTML, png, CSS, etc., are not included in this status report because it is ok for generated content to have uncommitted changes.
These are the previous versions of the repository in which changes were made to the R Markdown (analysis/som.Rmd) and HTML (docs/som.html) files. If you’ve configured a remote Git repository (see ?wflow_git_remote), click on the hyperlinks in the table below to view the files as they were in that past version.
| File | Version | Author | Date | Message |
|---|---|---|---|---|
| Rmd | 3f6b550 | robwschlegel | 2021-01-23 | Re-built site. |
| Rmd | 0d10736 | robwschlegel | 2021-01-23 | Preparing text to explain the needed additional analyses from reviewers |
| html | 76b55b6 | robwschlegel | 2020-12-21 | Build site. |
| html | 4a00400 | robwschlegel | 2020-12-21 | Build site. |
| html | 65f38bf | robwschlegel | 2020-12-21 | Build site. |
| html | 33f4595 | robwschlegel | 2020-11-10 | Build site. |
| html | a438235 | robwschlegel | 2020-11-10 | Build site. |
| Rmd | cbc5b74 | robwschlegel | 2020-11-10 | Re-built site. |
| Rmd | e513e01 | robwschlegel | 2020-10-09 | Added Qsw* to the SOM data and removed the SOM correlation figure |
| Rmd | 54144a7 | robwschlegel | 2020-09-28 | Working on non-numeric node labels |
| Rmd | 5bfa2a7 | robwschlegel | 2020-09-03 | Push before beginning large structural additions to app |
| html | 8d65758 | robwschlegel | 2020-09-03 | Build site. |
| Rmd | d1c9bad | robwschlegel | 2020-09-03 | Re-built site. |
| Rmd | 66f3736 | robwschlegel | 2020-08-26 | More edits to the figures |
| Rmd | 4b04d7a | robwschlegel | 2020-08-14 | Renamed some files in preparation for the file runs on the SOM sized data |
| Rmd | c0c599d | robwschlegel | 2020-08-12 | Combining the MHWNWA and MHWflux code bases |
This vignette contains the code used to perform the self-organising map (SOM) analysis on the mean synoptic states created in the data preparation vignette. We’ll start by creating custom packets that meet certain experimental criteria before feeding them into a SOM.
# Load functions and objects to be used below
source("code/functions.R")In this step we will create a data packet that can be fed directly into the SOM algorithm. This means that it must be converted into a super-wide matrix format. In the first run of this analysis on the NAPA model data it was found that the inclusion of the Labrador Sea complicated the results quite a bit. It was also unclear whether or not the Gulf of St Lawrence (GSL) region should be included in the analysis. So in the second run of this analysis multiple different SOM variations were employed and it was decided that the GSL region should be included.
Up first we must create the synoptic state packets.
# Set number of cores
# NB: 50 cores uses too much RAM
registerDoParallel(cores = 20)
# Load needed data
ALL_anom <- readRDS("data/ALL_anom.Rda")
ALL_other <- readRDS("data/ALL_other.Rda")
# Create one big anomaly packet from OISST data
system.time(synoptic_states <- plyr::ddply(OISST_MHW_event, c("region", "event_no"),
data_packet_func, .parallel = T, df = ALL_anom)) # 129 seconds
# Save
saveRDS(synoptic_states, "data/synoptic_states.Rda")
# Create other synoptic states per MHW per variable
doParallel::registerDoParallel(cores = 10) # NB: Be careful here...
system.time(synoptic_states_other <- plyr::ddply(OISST_MHW_event, c("region", "event_no"),
data_packet_func, .parallel = T, df = ALL_other)) # 212 seconds
# Save
saveRDS(synoptic_states_other, "data/synoptic_states_other.Rda")With all of our data ready we may now prepare and save them for the SOM.
## Create wide data packet that is fed to SOM
system.time(packet <- synoptic_states %>%
select(region, event_no, synoptic) %>%
unnest(cols = "synoptic") %>%
wide_packet_func()) # 79 seconds
# Save
saveRDS(packet, "data/packet.Rda")Now we feed the SOM with a function that ingests the data packet and produces results for us. The function below has been greatly expanded on from the previous version of this project and now performs all of the SOM related work in one go. This allowed me to remove a couple hundreds lines of code and text from this vignette.
# # OISST SOM analysis
packet <- readRDS("data/packet.Rda")
synoptic_states_other <- readRDS("data/synoptic_states_other.Rda")
system.time(som <- som_model_PCI(packet, synoptic_states_other)) # 176 seconds
saveRDS(som, file = "data/som.Rda")
# saveRDS(som, file = "shiny/som.Rda")A reviewer of the manuscript noted that the MHWs appear to be clustered closely together both within and across regions. This is by design in the methodology, but just how exactly these events cluster together in the SOM nodes warrants further investigation. The answer of how closely events cluster together within each region was answered in the MHWs vs. heat flux vignette. In this section we will first create an index of MHWs that can be said to be occurring across multiple regions at once. We then look to see how often these events are clustered into the same, or different nodes.
# Based on start and end dates of events, find overlap across regionsAnother minor point raised by a reviewer was that the atmospheric pattern in Node I appears to precede that of Node L, and they wanted to know if the MHWs clustered into Node I actually were coming before those events in Node L, and that these two nodes were really just a continuation of a single atmospheric pattern. To determine this we find the gaps in occurrence of the start, peak, and end dates of the MHWs between these nodes.
sessionInfo()R version 4.0.3 (2020-10-10)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 20.04.1 LTS
Matrix products: default
BLAS: /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3
LAPACK: /usr/lib/x86_64-linux-gnu/openblas-pthread/liblapack.so.3
locale:
[1] LC_CTYPE=en_CA.UTF-8 LC_NUMERIC=C
[3] LC_TIME=en_CA.UTF-8 LC_COLLATE=en_CA.UTF-8
[5] LC_MONETARY=en_CA.UTF-8 LC_MESSAGES=en_CA.UTF-8
[7] LC_PAPER=en_CA.UTF-8 LC_NAME=C
[9] LC_ADDRESS=C LC_TELEPHONE=C
[11] LC_MEASUREMENT=en_CA.UTF-8 LC_IDENTIFICATION=C
attached base packages:
[1] parallel stats graphics grDevices utils datasets methods
[8] base
other attached packages:
[1] doParallel_1.0.16 iterators_1.0.13 foreach_1.5.1 Metrics_0.1.4
[5] yasomi_0.3 proxy_0.4-24 e1071_1.7-4 ggraph_2.0.4
[9] correlation_0.5.0 tidync_0.2.4 heatwaveR_0.4.5 lubridate_1.7.9.2
[13] data.table_1.13.6 forcats_0.5.0 stringr_1.4.0 dplyr_1.0.2
[17] purrr_0.3.4 readr_1.4.0 tidyr_1.1.2 tibble_3.0.4
[21] ggplot2_3.3.3 tidyverse_1.3.0
loaded via a namespace (and not attached):
[1] fs_1.5.0 insight_0.12.0 httr_1.4.2 rprojroot_2.0.2
[5] tools_4.0.3 backports_1.2.1 R6_2.5.0 DBI_1.1.1
[9] colorspace_2.0-0 withr_2.3.0 tidyselect_1.1.0 gridExtra_2.3
[13] compiler_4.0.3 git2r_0.28.0 cli_2.2.0 rvest_0.3.6
[17] RNetCDF_2.4-2 xml2_1.3.2 bayestestR_0.8.0 scales_1.1.1
[21] digest_0.6.27 rmarkdown_2.6 pkgconfig_2.0.3 htmltools_0.5.1
[25] dbplyr_2.0.0 rlang_0.4.10 readxl_1.3.1 rstudioapi_0.13
[29] farver_2.0.3 generics_0.1.0 jsonlite_1.7.2 magrittr_2.0.1
[33] ncmeta_0.3.0 parameters_0.10.1 Rcpp_1.0.5 munsell_0.5.0
[37] fansi_0.4.1 viridis_0.5.1 lifecycle_0.2.0 stringi_1.5.3
[41] whisker_0.4 yaml_2.2.1 MASS_7.3-53 grid_4.0.3
[45] promises_1.1.1 ggrepel_0.9.0 crayon_1.3.4 graphlayouts_0.7.1
[49] haven_2.3.1 hms_1.0.0 knitr_1.30 pillar_1.4.7
[53] igraph_1.2.6 effectsize_0.4.1 codetools_0.2-18 reprex_0.3.0
[57] glue_1.4.2 evaluate_0.14 modelr_0.1.8 vctrs_0.3.6
[61] tweenr_1.0.1 httpuv_1.5.5 cellranger_1.1.0 gtable_0.3.0
[65] polyclip_1.10-0 assertthat_0.2.1 xfun_0.20 ggforce_0.3.2
[69] broom_0.7.3 tidygraph_1.2.0 later_1.1.0.1 class_7.3-17
[73] ncdf4_1.17 viridisLite_0.3.0 workflowr_1.6.2 ellipsis_0.3.1