Last updated: 2021-10-26
Checks: 7 0
Knit directory: bgc_argo_r_argodata/
This reproducible R Markdown analysis was created with workflowr (version 1.6.2). The Checks tab describes the reproducibility checks that were applied when the results were created. The Past versions tab lists the development history.
Great! Since the R Markdown file has been committed to the Git repository, you know the exact version of the code that produced these results.
Great job! The global environment was empty. Objects defined in the global environment can affect the analysis in your R Markdown file in unknown ways. For reproduciblity it’s best to always run the code in an empty environment.
The command set.seed(20211008) was run prior to running the code in the R Markdown file. Setting a seed ensures that any results that rely on randomness, e.g. subsampling or permutations, are reproducible.
Great job! Recording the operating system, R version, and package versions is critical for reproducibility.
Nice! There were no cached chunks for this analysis, so you can be confident that you successfully produced the results during this run.
Great job! Using relative paths to the files within your workflowr project makes it easier to run your code on other machines.
Great! You are using Git for version control. Tracking code development and connecting the code version to the results is critical for reproducibility.
The results in this page were generated with repository version 062b272. See the Past versions tab to see a history of the changes made to the R Markdown and HTML files.
Note that you need to be careful to ensure that all relevant files for the analysis have been committed to Git prior to generating the results (you can use wflow_publish or wflow_git_commit). workflowr only checks the R Markdown file, but you know if there are other scripts or data files that it depends on. Below is the status of the Git repository when the results were generated:
Ignored files:
Ignored: .Rhistory
Ignored: .Rproj.user/
Ignored: output/figures/
Untracked files:
Untracked: code/creating_dataframe.R
Untracked: code/creating_map.R
Unstaged changes:
Modified: code/Workflowr_project_managment.R
Deleted: output/README.md
Note that any generated files, e.g. HTML, png, CSS, etc., are not included in this status report because it is ok for generated content to have uncommitted changes.
These are the previous versions of the repository in which changes were made to the R Markdown (analysis/loading_data.Rmd) and HTML (docs/loading_data.html) files. If you’ve configured a remote Git repository (see ?wflow_git_remote), click on the hyperlinks in the table below to view the files as they were in that past version.
| File | Version | Author | Date | Message |
|---|---|---|---|---|
| html | b57291a | pasqualina-vonlanthendinenna | 2021-10-26 | Build site. |
| html | bba33bf | pasqualina-vonlanthendinenna | 2021-10-26 | Build site. |
| Rmd | 4bc1859 | pasqualina-vonlanthendinenna | 2021-10-26 | run with full data |
| html | aa7280d | jens-daniel-mueller | 2021-10-22 | Build site. |
| Rmd | ca7ba6b | jens-daniel-mueller | 2021-10-22 | adding revised code |
| html | d84c904 | pasqualina-vonlanthendinenna | 2021-10-22 | Build site. |
| html | 8ecdb43 | pasqualina-vonlanthendinenna | 2021-10-22 | Build site. |
| Rmd | 2b22099 | pasqualina-vonlanthendinenna | 2021-10-22 | edited text |
| html | c81f21c | pasqualina-vonlanthendinenna | 2021-10-21 | Build site. |
| html | b55928d | pasqualina-vonlanthendinenna | 2021-10-21 | Build site. |
| Rmd | 58c24b6 | pasqualina-vonlanthendinenna | 2021-10-21 | timeseries attempt 2, added description |
| html | 62d8519 | pasqualina-vonlanthendinenna | 2021-10-20 | Build site. |
| html | b8feac2 | pasqualina-vonlanthendinenna | 2021-10-20 | Build site. |
| html | 701fffa | pasqualina-vonlanthendinenna | 2021-10-20 | Build site. |
| Rmd | b88a839 | pasqualina-vonlanthendinenna | 2021-10-20 | adding revised code |
| html | d3a081c | jens-daniel-mueller | 2021-10-19 | Build site. |
| Rmd | f460b9a | jens-daniel-mueller | 2021-10-19 | code review jens |
| html | c5e1577 | pasqualina-vonlanthendinenna | 2021-10-18 | Build site. |
| Rmd | 8cc0106 | pasqualina-vonlanthendinenna | 2021-10-18 | added description in loading_data |
| html | 305873e | pasqualina-vonlanthendinenna | 2021-10-18 | Build site. |
| Rmd | 6b484e7 | pasqualina-vonlanthendinenna | 2021-10-18 | added description in loading_data |
| html | 83724a0 | pasqualina-vonlanthendinenna | 2021-10-14 | Build site. |
| html | 4840e49 | pasqualina-vonlanthendinenna | 2021-10-12 | Build site. |
| Rmd | 2fb35f7 | pasqualina-vonlanthendinenna | 2021-10-12 | added reading data in page |
| html | ff925ab | pasqualina-vonlanthendinenna | 2021-10-11 | Build site. |
| Rmd | 7e0cf34 | pasqualina-vonlanthendinenna | 2021-10-11 | added reading data in page |
Using the argodata package to load in bgc argo data from the server and store it in a dataframe with the corresponding metadata
The cache directory stores previously downloaded files to access them more quickly. Cached files are used indefinitely by default because of the considerable time it takes to refresh them. If you use a persistent cache, you should update the index files regularly by using argo_update_global() (data files are also updated occasionally; update these using argo_update_data())
# set cache directory
argo_set_cache_dir(cache_dir = path_argo)
# check cache directory
argo_cache_dir()[1] "/nfs/kryo/work/updata/bgc_argo_r_argodata"# check argo mirror
argo_mirror()[1] "https://data-argo.ifremer.fr"argo_update_global(max_global_cache_age = Inf) # age argument: age of the cached files to update in hours (Inf means always use the cached file, and -Inf means always download from the server)
# ex: max_global_cache_age = 5 updates files that have been in the cache for more than 5 hours, max_global_cache_age = 0.5 updates files that have been in the cache for more than 30 minutes, etc.
argo_update_data(max_data_cache_age = Inf)Load in the synthetic (merged core and bgc) index files (uses the data stored on the ifremer server by default), keeping only delayed-mode data.
A core-Argo profile contains the CTD sensor parameters (pressure, temperature, salinity) that are measured with the same vertical sampling scheme and at the same location and time. Additional parameters from other sensors are stored in the b-Argo profile files.
A b-Argo profile contains all the parameters from a float, except the core-Argo parameters temperature, pressure, and salinity. A float that performs only CTD measurements does not have a b-Argo file. The vertical level PRES is the simple and unambiguous link between the parameters in the core-Argo and b-Argo files. The same PRES is recorded in the core-Argo and b-Argo files. PRES is the only parameter duplicated in core-Argo and b-Argo profile files.
To facilitate the use of BGC-Argo data, the regional data centers merge each b-Argo file with its corresponding core-Argo file into one synthetic (s-Argo) file. The goal of a simplified s-Argo file is to co-locate as many BGC observations as possible while preserving the character of the sampling pattern, i.e., sample interval, number of samples, and approximate pressure locations. Data come from the single c- and b-Argo files. The synthetic pressure axis is constructed from the BGC sampling levels from each cycle. This means that there is no fixed vertical grid for all floats and all cycles.
The co-location takes different vertical attachments of BGC sensors into account by displacing the pressure location, which is not the case in the c- and b-files. The single-cycle s–file profiles contain all the c-file parameter observations in their original location and resolution.
The adjusted pressure parameter (pres_adjusted) is only available in the core- and s-Argo profile files. The variables profile_pres_qc, pres_adjusted, and pres_adjusted_error, are not duplicated in the b-Argo files.
Delayed-mode data are denoted by data_mode = 'D', and are quality-checked by PIs, who apply any necessary adjustments. For the core CTD data, delayed-mode data is generally available 12 months after the transmission of raw data, because the raw data is usually of good quality. Their delayed-mode assessment involves evaluation of the long-term sensor stability, which typically requires a float record of 12 months. Incorrect QC flag attribution and erroneous raw data not flagged during real-time procedures are corrected in delayed-mode.
Delayed-mode BGC data may be available as early as 5-6 cycles after initial data transmission as the raw data are typically unfit for scientific usage. Adjustments significantly increase the accuracy of these data. In b- and s-Argo profile files, the variable parameter_data_mode indicates the mode of each parameter. Biogeochemical parameters in the same file may receive their delayed-mode adjustments at different times.
Synthetic files info: https://archimer.ifremer.fr/doc/00445/55637/80863.pdf
Argo User Manual: https://archimer.ifremer.fr/doc/00187/29825/86414.pdf
bgc_subset <- argo_global_synthetic_prof() %>%
argo_filter_data_mode(data_mode = 'delayed') %>% # load in delayed-mode data
argo_filter_date(date_min = '2013-01-01',
date_max = Sys.time()) Loading argo_global_synthetic_prof()# check the dates
# max(bgc_subset$date, na.rm = TRUE)
# min(bgc_subset$date, na.rm = TRUE)
# checking alternative functions
# argo_global_meta(download = NULL, quiet = FALSE)
# argo_global_prof(download = NULL, quiet = FALSE)
# laods in core-argo files (CTD data)
# argo_global_tech(download = NULL, quiet = FALSE)
# An argo technical file contains technical information from an Argo float. This information is registered for each cycle performed by the float.
# argo_global_traj(download = NULL, quiet = FALSE)
# argo_global_bio_traj(download = NULL, quiet = FALSE)
# load the trajectory files, which contain all received Argos and GPS locations of Argo floats. A trajectory file often contains core and BGC measurements performed at various intermediate times during the cycle and outside the vertical profiles. The full profiles collected upon ascent are not included and are stored in the profile files.
# argo_global_bio_prof(download = NULL, quiet = FALSE)
# A B-Argo profile file contains all the parameters from a float, except the core-Argo parameters temperature, pressure, and salinity. A float that performs only CTD measurements does not have a B-Argo file
# argo_global_synthetic_prof(download = NULL, quiet = FALSE)Read in the adjusted bgc and core variables corresponding to the index files downloaded above, with their quality control flags. (can take a while)
bgc_data <- argo_prof_levels(
path = bgc_subset,
vars =
c(
'PRES_ADJUSTED',
'PRES_ADJUSTED_QC',
'PRES_ADJUSTED_ERROR',
'PSAL_ADJUSTED',
'PSAL_ADJUSTED_QC',
'PSAL_ADJUSTED_ERROR',
'TEMP_ADJUSTED',
'TEMP_ADJUSTED_QC',
'TEMP_ADJUSTED_ERROR',
'DOXY_ADJUSTED',
'DOXY_ADJUSTED_QC',
'DOXY_ADJUSTED_ERROR',
'NITRATE_ADJUSTED',
'NITRATE_ADJUSTED_QC',
'NITRATE_ADJUSTED_ERROR',
'PH_IN_SITU_TOTAL_ADJUSTED',
'PH_IN_SITU_TOTAL_ADJUSTED_QC',
'PH_IN_SITU_TOTAL_ADJUSTED_ERROR'
),
quiet = TRUE
)
# read in the profiles (takes a while)The data is read in from the cached files stored in the path specified in set_argo_cache_dir() (in this case, /nfs/kryo/work/updata/bgc_argo_r_argodata). To download data directly from the files stored on the ifremer server, set max_global_cache_age and max_data_cache_age to -Inf, which will force a new download.
Read in the corresponding metadata:
bgc_metadata <- argo_prof_prof(path = bgc_subset)Extracting from 135159 filesJoin the metadata and data together into one dataset
bgc_merge <-
full_join(bgc_data, bgc_metadata)Joining, by = c("file", "n_prof")bgc_metadata <- bgc_metadata %>%
rename(lon = longitude,
lat = latitude) %>%
mutate(lon = if_else(lon < 20, lon + 360, lon)) %>%
mutate(
lat = cut(lat, seq(-90, 90, 1), seq(-89.5, 89.5, 1)),
lat = as.numeric(as.character(lat)),
lon = cut(lon, seq(20, 380, 1), seq(20.5, 379.5, 1)),
lon = as.numeric(as.character(lon))
) %>%
select(
-c(profile_chla_qc:profile_cdom_qc),
-c(profile_cndc_qc:profile_up_radiance555_qc)
)
bgc_merge <- bgc_merge %>%
rename(lon = longitude,
lat = latitude) %>%
mutate(lon = if_else(lon < 20, lon + 360, lon)) %>%
mutate(
lat = cut(lat, seq(-90, 90, 1), seq(-89.5, 89.5, 1)),
lat = as.numeric(as.character(lat)),
lon = cut(lon, seq(20, 380, 1), seq(20.5, 379.5, 1)),
lon = as.numeric(as.character(lon))
) %>%
select(
-c(profile_chla_qc:profile_cdom_qc),
-c(profile_cndc_qc:profile_up_radiance555_qc)
)path_argo_preprocessed <- paste0(path_argo, "/preprocessed_bgc_data")
bgc_subset %>%
write_rds(file = paste0(path_argo_preprocessed, "/bgc_subset.rds"))
bgc_metadata %>%
write_rds(file = paste0(path_argo_preprocessed, "/bgc_metadata.rds"))
bgc_data %>%
write_rds(file = paste0(path_argo_preprocessed, "/bgc_data.rds"))
bgc_merge %>%
write_rds(file = paste0(path_argo_preprocessed, "/bgc_merge.rds"))The resulting dataframe contains:
the file name (file column)
the sampling level (n_level column)
the number of profiles per file (n_prof column; Each single-cycle synthetic profile has the dimension n_prof = 1).
adjusted values for pressure (PRES, in dbar), salinity (PSAL, in psu), temperature (TEMP, in degrees C), dissolved oxygen (DOXY, in µmol kg-1), pH (PH_IN_SITU_TOTAL), and nitrate (NITRATE, in µmol kg-1) (parameter_adjusted columns). This column is mandatory, so if no adjustment was performed (i.e. parameter_adjusted does not exist), FillValue is inserted (e.g., temp_adjusted:FillValue = 99999.f).
a quality control flag associated with these adjusted values (parameter_adjusted_qc columns). If an adjusted value does not exist (e.g., temp_adjusted = 99999.f), then FillValue is inserted (e.g., temp_adjusted_qc = " ").
an error estimate on the adjustment of the measurement (parameter_adjusted_error columns). If no adjusted value exists (e.g., temp_adjusted = 99999.f), then FillValue is inserted (e.g., temp_adjusted_error = 99999.f)
WMO float identifier (platform_number column)
name of the project in charge of the float (project_name column)
name of principal investigator in charge of the float (pi_name column)
float cycle number (cycle_number column; cycle 0 is the launch cycle during which technical data or configuration information is transmitted; cycle 1 is the first complete cycle)
descending (D) or ascending (A) profile (direction column). Profile measurements are taken on ascent, occasionally during descent (rarely both).
code for the data centre in charge of the float data management (data_centre column)
the type of float (platform_type column)
firmware version of the float (firmware_version column)
instrument type from the WMO code table 1770 (wmo_inst_type column)
the date and time at which the measurement was taken, in UTC (date column)
a quality control flag for the date and time value (date_qc column)
the date and time of the profile location (date_location column)
latitude in degrees N (latitude column)
longitude in degrees E (longitude column)
quality control flag on the position (position_qc column)
name of the system in charge of positioning the float locations (positioning_system column)
unique number of the mission to which this float belongs (config_mission_number column,
a global quality control flag on the profile of the parameter (profile_parameter_qc column; FillValue = " ")
QC flags for values (‘parameter_adjusted_qc’ columns) are between 1 and 8, where:
1 is ‘good’ data,
2 is ‘probably good’ data,
3 is ‘probably bad’ data,
4 is ‘bad’ data,
5 is ‘value changed’,
8 is ‘estimated value’,
9 is ‘missing value’ (data parameter will record FillValue)
(6 and 7 are not used).
Profile QC flags (‘profile_parameter_qc’ columns) are QC codes attributed to the entire profile, and indicate the number of depth levels (in %) where the value is considered to be good data (QC flags of 1, 2, 5, and 8; QC flags of 9 or " " are not used in the computation):
‘A’ means 100% of profile levels contain good data,
‘B’ means 75-<100% of profile levels contain good data,
‘C’ means 50-75% of profile levels contain good data,
‘D’ means 25-50% of profile levels contain good data,
‘E’ means >0-50% of profile levels contain good data,
‘F’ means 0% of profile levels contain good data.
During delayed-mode processing, the QC flags assigned by the real-time process can be reassigned if needed.
sessionInfo()R version 4.0.3 (2020-10-10)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: openSUSE Leap 15.2
Matrix products: default
BLAS: /usr/local/R-4.0.3/lib64/R/lib/libRblas.so
LAPACK: /usr/local/R-4.0.3/lib64/R/lib/libRlapack.so
locale:
[1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C
[3] LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8
[5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8
[7] LC_PAPER=en_US.UTF-8 LC_NAME=C
[9] LC_ADDRESS=C LC_TELEPHONE=C
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
attached base packages:
[1] stats graphics grDevices utils datasets methods base
other attached packages:
[1] lubridate_1.7.9 argodata_0.0.0.9000 forcats_0.5.0
[4] stringr_1.4.0 dplyr_1.0.5 purrr_0.3.4
[7] readr_1.4.0 tidyr_1.1.3 tibble_3.1.3
[10] ggplot2_3.3.5 tidyverse_1.3.0 workflowr_1.6.2
loaded via a namespace (and not attached):
[1] Rcpp_1.0.7 prettyunits_1.1.1 assertthat_0.2.1 rprojroot_2.0.2
[5] digest_0.6.27 utf8_1.2.2 R6_2.5.1 cellranger_1.1.0
[9] backports_1.1.10 reprex_0.3.0 evaluate_0.14 httr_1.4.2
[13] pillar_1.6.2 progress_1.2.2 rlang_0.4.11 readxl_1.3.1
[17] rstudioapi_0.13 whisker_0.4 jquerylib_0.1.4 blob_1.2.1
[21] rmarkdown_2.10 bit_4.0.4 munsell_0.5.0 broom_0.7.9
[25] compiler_4.0.3 httpuv_1.6.2 modelr_0.1.8 xfun_0.25
[29] pkgconfig_2.0.3 htmltools_0.5.1.1 tidyselect_1.1.0 fansi_0.5.0
[33] tzdb_0.1.2 crayon_1.4.1 dbplyr_1.4.4 withr_2.4.2
[37] later_1.3.0 grid_4.0.3 jsonlite_1.7.2 gtable_0.3.0
[41] lifecycle_1.0.0 DBI_1.1.1 git2r_0.27.1 magrittr_2.0.1
[45] scales_1.1.1 vroom_1.5.5 cli_3.0.1 stringi_1.5.3
[49] fs_1.5.0 promises_1.2.0.1 xml2_1.3.2 bslib_0.2.5.1
[53] ellipsis_0.3.2 generics_0.1.0 vctrs_0.3.8 tools_4.0.3
[57] bit64_4.0.5 glue_1.4.2 RNetCDF_2.4-2 hms_0.5.3
[61] parallel_4.0.3 yaml_2.2.1 colorspace_2.0-2 rvest_0.3.6
[65] knitr_1.33 haven_2.3.1 sass_0.4.0