Last updated: 2020-10-21
Checks: 7 0
Knit directory: r4ds_book/
This reproducible R Markdown analysis was created with workflowr (version 1.6.2). The Checks tab describes the reproducibility checks that were applied when the results were created. The Past versions tab lists the development history.
Great! Since the R Markdown file has been committed to the Git repository, you know the exact version of the code that produced these results.
Great job! The global environment was empty. Objects defined in the global environment can affect the analysis in your R Markdown file in unknown ways. For reproduciblity it’s best to always run the code in an empty environment.
The command set.seed(20200814) was run prior to running the code in the R Markdown file. Setting a seed ensures that any results that rely on randomness, e.g. subsampling or permutations, are reproducible.
Great job! Recording the operating system, R version, and package versions is critical for reproducibility.
Nice! There were no cached chunks for this analysis, so you can be confident that you successfully produced the results during this run.
Great job! Using relative paths to the files within your workflowr project makes it easier to run your code on other machines.
Great! You are using Git for version control. Tracking code development and connecting the code version to the results is critical for reproducibility.
The results in this page were generated with repository version 8e445e8. See the Past versions tab to see a history of the changes made to the R Markdown and HTML files.
Note that you need to be careful to ensure that all relevant files for the analysis have been committed to Git prior to generating the results (you can use wflow_publish or wflow_git_commit). workflowr only checks the R Markdown file, but you know if there are other scripts or data files that it depends on. Below is the status of the Git repository when the results were generated:
Ignored files:
Ignored: .Rproj.user/
Untracked files:
Untracked: VideoDecodeStats/
Untracked: analysis/images/
Untracked: code_snipp.txt
Note that any generated files, e.g. HTML, png, CSS, etc., are not included in this status report because it is ok for generated content to have uncommitted changes.
These are the previous versions of the repository in which changes were made to the R Markdown (analysis/ch8_import_data.Rmd) and HTML (docs/ch8_import_data.html) files. If you’ve configured a remote Git repository (see ?wflow_git_remote), click on the hyperlinks in the table below to view the files as they were in that past version.
| File | Version | Author | Date | Message |
|---|---|---|---|---|
| Rmd | 8e445e8 | sciencificity | 2020-10-21 | updated the workflow-projects section |
| html | 4497db4 | sciencificity | 2020-10-18 | Build site. |
| Rmd | f81b11a | sciencificity | 2020-10-18 | added Chapter 14 and some of Chapter 8 |
options(scipen=10000)
library(tidyverse)
library(flair)
library(nycflights13)
library(palmerpenguins)
library(gt)
library(skimr)
library(emo)
library(tidyquant)
library(lubridate)
library(magrittr)
theme_set(theme_tq())Inline csv file
read_csv("a,b,c
1,2,3
4,5,6")# A tibble: 2 x 3
a b c
<dbl> <dbl> <dbl>
1 1 2 3
2 4 5 6Skip some columns
read_csv("The first line of metadata
The second line of metadata
x,y,z
1,2,3", skip = 2)# A tibble: 1 x 3
x y z
<dbl> <dbl> <dbl>
1 1 2 3read_csv("# A comment I want to skip
x,y,z
1,2,3", comment = "#")# A tibble: 1 x 3
x y z
<dbl> <dbl> <dbl>
1 1 2 3No column names in data
read_csv("1,2,3\n4,5,6", # \n adds a new line
col_names = FALSE) # cols will be labelled seq from X1 .. Xn# A tibble: 2 x 3
X1 X2 X3
<dbl> <dbl> <dbl>
1 1 2 3
2 4 5 6read_csv("1,2,3\n4,5,6",
col_names = c("x", "y", "z")) # cols named as you provided here# A tibble: 2 x 3
x y z
<dbl> <dbl> <dbl>
1 1 2 3
2 4 5 6NA values
read_csv("a,b,c,d\nnull,1,2,.",
na = c(".",
"null"))# A tibble: 1 x 4
a b c d
<lgl> <dbl> <dbl> <lgl>
1 NA 1 2 NA # here we specify that the . and null
# must be considered to be missing valuesWhat function would you use to read a file where fields were separated with
“|”?
read_delim()
# from the ?read_delim help page
read_delim("a|b\n1.0|2.0", delim = "|")# A tibble: 1 x 2
a b
<dbl> <dbl>
1 1 2Apart from file, skip, and comment, what other arguments do read_csv() and read_tsv() have in common?
All columns are common across the functions.
What are the most important arguments to read_fwf()?
Sometimes strings in a CSV file contain commas. To prevent them from causing problems they need to be surrounded by a quoting character, like " or '. By default, read_csv() assumes that the quoting character will be ". What argument to read_csv() do you need to specify to read the following text into a data frame?
"x,y\n1,'a,b'"Specify the quote argument.
read_csv("x,y\n1,'a,b'", quote = "'")# A tibble: 1 x 2
x y
<dbl> <chr>
1 1 a,b Identify what is wrong with each of the following inline CSV files. What happens when you run the code?
read_csv("a,b\n1,2,3\n4,5,6") # only 2 cols specified but
read_csv("a,b,c\n1,2\n1,2,3,4")
read_csv("a,b\n\"1")
read_csv("a,b\n1,2\na,b")
read_csv("a;b\n1;3")read_csv(“a,b1,2,34,5,6”)
only 2 cols specified but 3 values provided
read_csv(“a,b,c1,21,2,3,4”)
3 col names provided, but either too few, or too many column values provided
read_csv(“a,b"1”)
2 col names provided, but only one value provided.
closing " missing
read_csv(“a,b1,2,b”) Nothing syntactically a problem, but the rows are filled
with the column headings?
read_csv(“a;b1;3”) the read_csv2 which reads ; as delimiters should have been used
They all run, but most have warnings, and some are not imported as expected.
read_csv("a,b\n1,2,3\n4,5,6") # only 2 cols specified but Warning: 2 parsing failures.
row col expected actual file
1 -- 2 columns 3 columns literal data
2 -- 2 columns 3 columns literal data# A tibble: 2 x 2
a b
<dbl> <dbl>
1 1 2
2 4 5read_csv("a,b,c\n1,2\n1,2,3,4")Warning: 2 parsing failures.
row col expected actual file
1 -- 3 columns 2 columns literal data
2 -- 3 columns 4 columns literal data# A tibble: 2 x 3
a b c
<dbl> <dbl> <dbl>
1 1 2 NA
2 1 2 3read_csv("a,b\n\"1")Warning: 2 parsing failures.
row col expected actual file
1 a closing quote at end of file literal data
1 -- 2 columns 1 columns literal data# A tibble: 1 x 2
a b
<dbl> <chr>
1 1 <NA> read_csv("a,b\n1,2\na,b")# A tibble: 2 x 2
a b
<chr> <chr>
1 1 2
2 a b read_csv("a;b\n1;3")# A tibble: 1 x 1
`a;b`
<chr>
1 1;3 str(parse_logical(c("TRUE", "FALSE", "NA")))
logi [1:3] TRUE FALSE NA
str(parse_integer(c("1", "2", "3")))
int [1:3] 1 2 3
str(parse_date(c("2010-01-01", "1979-10-14")))
Date[1:2], format: "2010-01-01" "1979-10-14"
All parse_xxx() variants provide a uniform specification to use.
parse_x(character_vector_to_parse, na = c(x, y))
parse_integer(c("1", "231", ".", "456"), na = ".")[1] 1 231 NA 456(x <- parse_integer(c("123", "345", "abc", "123.45")))Warning: 2 parsing failures.
row col expected actual
3 -- an integer abc
4 -- no trailing characters .45[1] 123 345 NA NA
attr(,"problems")
# A tibble: 2 x 4
row col expected actual
<int> <int> <chr> <chr>
1 3 NA an integer abc
2 4 NA no trailing characters .45 To detect problems use problems().
problems(x)
# A tibble: 2 x 4
row col expected actual
<int> <int> <chr> <chr>
1 3 NA an integer abc
2 4 NA no trailing characters .45
Sometimes depending on where in the world you are you will have different conventions when it comes to numbers.
For example you may separate the integer part from the decimal part by using a . or a ,. To tell the parsing function what kind of data you’re expecting to be in a vector use locale = locale(...) in your parsing function.
parse_double("1.23")[1] 1.23parse_double("1,23", locale = locale(decimal_mark = ","))[1] 1.23parse_number("$100")[1] 100parse_number("20%")[1] 20parse_number("It cost $123.45")[1] 123.45# Used in America
parse_number("$123,456,789")[1] 123456789# Used in many parts of Europe
parse_number("123.456.789", locale = locale(grouping_mark = "."))[1] 123456789# Used in Switzerland
parse_number("123'456'789", locale = locale(grouping_mark = "'"))[1] 123456789charToRaw("Hadley")[1] 48 61 64 6c 65 79(x1 <- "El Ni\xf1o was particularly bad this year")[1] "El Niño was particularly bad this year"(x2 <- "\x82\xb1\x82\xf1\x82\xc9\x82\xbf\x82\xcd")[1] "‚±‚ñ‚É‚¿‚Í"To fix the problem you need to specify the encoding in parse_character():
parse_character(x1, locale = locale(encoding = "Latin1"))[1] "El Niño was particularly bad this year"parse_character(x2, locale = locale(encoding = "Shift-JIS"))[1] "<U+3053><U+3093><U+306B><U+3061><U+306F>"You can try the guess_encoding() to help you out:
guess_encoding(charToRaw(x1))# A tibble: 2 x 2
encoding confidence
<chr> <dbl>
1 ISO-8859-1 0.46
2 ISO-8859-9 0.23guess_encoding(charToRaw(x2))# A tibble: 1 x 2
encoding confidence
<chr> <dbl>
1 KOI8-R 0.42fruit <- c("apple", "banana")
parse_factor(c("apple", "banana", "bananana"), levels = fruit)[1] apple banana <NA>
attr(,"problems")
# A tibble: 1 x 4
row col expected actual
<int> <int> <chr> <chr>
1 3 NA value in level set bananana
Levels: apple bananaparse_datetime("2010-10-01T2010")[1] "2010-10-01 20:10:00 UTC"# If time is omitted, it will be set to midnight
parse_datetime("20101010")[1] "2010-10-10 UTC"parse_date("2010-10-01")[1] "2010-10-01"library(hms)
parse_time("01:10 am")01:10:00parse_time("20:10:01")20:10:01parse_date("01/02/15", "%m/%d/%y")[1] "2015-01-02"parse_date("01/02/15", "%d/%m/%y")[1] "2015-02-01"parse_date("01/02/15", "%y/%m/%d")[1] "2001-02-15"parse_date("1 janvier 2015", "%d %B %Y", locale = locale("fr"))[1] "2015-01-01"What are the most important arguments to locale()?
What happens if you try and set decimal_mark and grouping_mark to the same character? What happens to the default value of grouping_mark when you set decimal_mark to “,”? What happens to the default value of decimal_mark when you set the grouping_mark to “.”?
# decimal_mark` and `grouping_mark` to the same character
parse_double("123,456,78", locale = locale(decimal_mark = ",",
grouping_mark = ","))
# Error: `decimal_mark` and `grouping_mark` must be different
parse_number("123,456,78", locale = locale(decimal_mark = ",",
grouping_mark = ","))
# Error: `decimal_mark` and `grouping_mark` must be different# What happens to the default value of `grouping_mark`
# when you set `decimal_mark` to ","?
parse_number("123 456,78", locale = locale(decimal_mark = ","))[1] 123# no grouping preserved, whitespace is not considered as group
# so we get an incorrect parsing
# I would gather that since we overrode decimal_mark to be
# equal to the grouping_mark default, this removes the
# default, and hence has to be supplied for correct parsing
# if you also have a specific grouping character present
parse_number("123 456,78", locale = locale(decimal_mark = ",",
grouping_mark = " "))[1] 123456.8# both specified, number parsed correctly
parse_number("123.456,78", locale = locale(decimal_mark = ","))[1] 123456.8# even though no grouping_mark specified, parse_number
# handles the . grouping mark well
# preserve the decimals
print(parse_number("123.456,78",
locale = locale(decimal_mark = ",")),digits=10)[1] 123456.78Turns out the code sets the decimal_mark if it was not set, or vice versa. From readr code file locale.R
if (missing(grouping_mark) && !missing(decimal_mark)) {
grouping_mark <- if (decimal_mark == ".") "," else "."
} else if (missing(decimal_mark) && !missing(grouping_mark)) {
decimal_mark <- if (grouping_mark == ".") "," else "."
}# What happens to the default value of `grouping_mark`
# when you set `decimal_mark` to ","?
parse_double("123456,78", locale = locale(decimal_mark = ","))[1] 123456.8parse_double("123.456,78", locale = locale(decimal_mark = ","))[1] NA
attr(,"problems")
# A tibble: 1 x 4
row col expected actual
<int> <int> <chr> <chr>
1 1 NA no trailing characters .456,78parse_double("123.456,78", locale = locale(decimal_mark = ",",
grouping_mark = "."))[1] NA
attr(,"problems")
# A tibble: 1 x 4
row col expected actual
<int> <int> <chr> <chr>
1 1 NA no trailing characters .456,78parse_double("123 456,78", locale = locale(decimal_mark = ",",
grouping_mark = " "))[1] NA
attr(,"problems")
# A tibble: 1 x 4
row col expected actual
<int> <int> <chr> <chr>
1 1 NA no trailing characters " 456,78"Hhmm okay, so it seems like parse_double() is a bit more strict, and does not seem to like it even if we override the locale(). This Stack Overflow post confirms what we see here, so too does this post and this one. The only perplexing thing is that when I do set the grouping_mark in locale() why is this not considered? Because parse_double() also has a default locale which may be overriden by locale()? 😕
# What happens to the default value of `decimal_mark`
# when you set the `grouping_mark` to "."
parse_number("5.123.456,78", locale = locale(grouping_mark = "."))[1] 5123457# As above shows the decimal character set to , in code
parse_number("5.123.456,78", locale = locale(decimal_mark = ",",
grouping_mark = "."))[1] 5123457problems(parse_double("5.123.456,78",
locale = locale(decimal_mark = ",",
grouping_mark = ".")))# A tibble: 1 x 4
row col expected actual
<int> <int> <chr> <chr>
1 1 NA no trailing characters .123.456,78I didn’t discuss the date_format and time_format options to locale(). What do they do? Construct an example that shows when they might be useful.
If you live outside the US, create a new locale object that encapsulates the settings for the types of file you read most commonly.
What’s the difference between read_csv() and read_csv2()?
What are the most common encodings used in Europe? What are the most common encodings used in Asia? Do some googling to find out.
Generate the correct format string to parse each of the following dates and times:
d1 <- "January 1, 2010"
d2 <- "2015-Mar-07"
d3 <- "06-Jun-2017"
d4 <- c("August 19 (2015)", "July 1 (2015)")
d5 <- "12/30/14" # Dec 30, 2014
t1 <- "1705"
t2 <- "11:15:10.12 PM"
sessionInfo()R version 3.6.3 (2020-02-29)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 18363)
Matrix products: default
locale:
[1] LC_COLLATE=English_South Africa.1252 LC_CTYPE=English_South Africa.1252
[3] LC_MONETARY=English_South Africa.1252 LC_NUMERIC=C
[5] LC_TIME=English_South Africa.1252
attached base packages:
[1] stats graphics grDevices utils datasets methods base
other attached packages:
[1] hms_0.5.3 magrittr_1.5
[3] tidyquant_1.0.0 quantmod_0.4.17
[5] TTR_0.23-6 PerformanceAnalytics_2.0.4
[7] xts_0.12-0 zoo_1.8-7
[9] lubridate_1.7.8 emo_0.0.0.9000
[11] skimr_2.1.1 gt_0.2.2
[13] palmerpenguins_0.1.0 nycflights13_1.0.1
[15] flair_0.0.2 forcats_0.5.0
[17] stringr_1.4.0 dplyr_1.0.0
[19] purrr_0.3.4 readr_1.3.1
[21] tidyr_1.1.0 tibble_3.0.3
[23] ggplot2_3.3.0 tidyverse_1.3.0
[25] workflowr_1.6.2
loaded via a namespace (and not attached):
[1] httr_1.4.2 jsonlite_1.7.0 modelr_0.1.6 assertthat_0.2.1
[5] cellranger_1.1.0 yaml_2.2.1 pillar_1.4.6 backports_1.1.6
[9] lattice_0.20-38 glue_1.4.1 quadprog_1.5-8 digest_0.6.25
[13] promises_1.1.0 rvest_0.3.5 colorspace_1.4-1 htmltools_0.5.0
[17] httpuv_1.5.2 pkgconfig_2.0.3 broom_0.5.6 haven_2.2.0
[21] scales_1.1.0 whisker_0.4 later_1.0.0 git2r_0.26.1
[25] generics_0.0.2 ellipsis_0.3.1 withr_2.2.0 repr_1.1.0
[29] cli_2.0.2 crayon_1.3.4 readxl_1.3.1 evaluate_0.14
[33] fs_1.4.1 fansi_0.4.1 nlme_3.1-144 xml2_1.3.2
[37] tools_3.6.3 lifecycle_0.2.0 munsell_0.5.0 reprex_0.3.0
[41] compiler_3.6.3 rlang_0.4.7 grid_3.6.3 rstudioapi_0.11
[45] base64enc_0.1-3 rmarkdown_2.4 gtable_0.3.0 DBI_1.1.0
[49] curl_4.3 R6_2.4.1 knitr_1.28 utf8_1.1.4
[53] rprojroot_1.3-2 Quandl_2.10.0 stringi_1.4.6 Rcpp_1.0.4.6
[57] vctrs_0.3.2 dbplyr_1.4.3 tidyselect_1.1.0 xfun_0.13