New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

getSymbols("CPIAUCSL", src = "FRED") CharToDate error #141

Closed
floswald opened this Issue Feb 14, 2017 · 13 comments

Comments

Projects
None yet
3 participants
@floswald

floswald commented Feb 14, 2017

Hi,

I find this error with the most recent version of quantmod:

17.
stop("character string is not in a standard unambiguous format") 
16.
charToDate(x) 
15.
as.Date.character(as.character(x), ...) 
14.
as.Date(as.character(x), ...) 
13.
as.Date.factor(fr[, 1], origin = "1970-01-01") 
12.
as.Date(fr[, 1], origin = "1970-01-01") 
11.
inherits(x, xx) 
10.
FUN(X[[i]], ...) 
9.
lapply(X = X, FUN = FUN, ...) 
8.
sapply(c("Date", "POSIXt", "chron", "dates", "times", "timeDate", 
    "yearmon", "yearqtr", "xtime"), function(xx) inherits(x, 
    xx)) 
7.
timeBased(order.by) 
6.
xts(as.matrix(fr[, -1]), as.Date(fr[, 1], origin = "1970-01-01"), 
    src = "FRED", updated = Sys.time()) 
5.
getSymbols.FRED(Symbols = "CPIAUCSL", env = <environment>, verbose = FALSE, 
    warnings = TRUE, auto.assign = TRUE) 
4.
do.call(paste("getSymbols.", symbol.source, sep = ""), list(Symbols = current.symbols, 
    env = env, verbose = verbose, warnings = warnings, auto.assign = auto.assign, 
    ...)) 
3.
getSymbols("CPIAUCSL", src = "FRED") at makers.r#340
@joshuaulrich

This comment has been minimized.

Show comment
Hide comment
@joshuaulrich

joshuaulrich Feb 14, 2017

Owner

Please provide 1) the actual function call, 2) whatever command creates that traceback, and 3) the output from sessionInfo().

Owner

joshuaulrich commented Feb 14, 2017

Please provide 1) the actual function call, 2) whatever command creates that traceback, and 3) the output from sessionInfo().

@floswald

This comment has been minimized.

Show comment
Hide comment
@floswald

floswald Feb 14, 2017

this happens only in Rstudio:

R version 3.2.3 (2015-12-10) -- "Wooden Christmas-Tree"
Copyright (C) 2015 The R Foundation for Statistical Computing
Platform: x86_64-apple-darwin13.4.0 (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

  Natural language support but running in an English locale

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

> library(quantmod)
Loading required package: xts
Loading required package: zoo

Attaching package:zooThe following objects are masked frompackage:base:

    as.Date, as.Date.numeric

Loading required package: TTR
Version 0.4-0 included new data defaults. See ?getSymbols.
Warning messages:
1: packagequantmodwas built under R version 3.2.5 
2: packagezoowas built under R version 3.2.5 
3: packageTTRwas built under R version 3.2.4 
> getSymbols("CPIAUCSL", src = "FRED")
    As of 0.4-0, ‘getSymbolsuses env=parent.frame() and
 auto.assign=TRUE by default.

 This  behavior  will be  phased out in 0.5-0  when the call  will
 default to use auto.assign=FALSE. getOption("getSymbols.env") and 
 getOptions("getSymbols.auto.assign") are now checked for alternate defaults

 This message is shown once per session and may be disabled by setting 
 options("getSymbols.warning4.0"=FALSE). See ?getSymbols for more details.
Error in charToDate(x) : 
  character string is not in a standard unambiguous format
> traceback()
15: stop("character string is not in a standard unambiguous format")
14: charToDate(x)
13: as.Date.character(as.character(x), ...)
12: as.Date(as.character(x), ...)
11: as.Date.factor(fr[, 1], origin = "1970-01-01")
10: as.Date(fr[, 1], origin = "1970-01-01")
9: inherits(x, xx)
8: FUN(X[[i]], ...)
7: lapply(X = X, FUN = FUN, ...)
6: sapply(c("Date", "POSIXt", "chron", "dates", "times", "timeDate", 
       "yearmon", "yearqtr", "xtime"), function(xx) inherits(x, 
       xx))
5: timeBased(order.by)
4: xts(as.matrix(fr[, -1]), as.Date(fr[, 1], origin = "1970-01-01"), 
       src = "FRED", updated = Sys.time())
3: getSymbols.FRED(Symbols = "CPIAUCSL", env = <environment>, verbose = FALSE, 
       warnings = TRUE, auto.assign = TRUE)
2: do.call(paste("getSymbols.", symbol.source, sep = ""), list(Symbols = current.symbols, 
       env = env, verbose = verbose, warnings = warnings, auto.assign = auto.assign, 
       ...))
1: getSymbols("CPIAUCSL", src = "FRED")
> sessionInfo()
R version 3.2.3 (2015-12-10)
Platform: x86_64-apple-darwin13.4.0 (64-bit)
Running under: OS X 10.11.6 (El Capitan)

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] quantmod_0.4-7 TTR_0.23-1     xts_0.9-7      zoo_1.7-13    

loaded via a namespace (and not attached):
[1] tools_3.2.3     grid_3.2.3      lattice_0.20-33

R console works:

~ R

R version 3.2.3 (2015-12-10) -- "Wooden Christmas-Tree"
Copyright (C) 2015 The R Foundation for Statistical Computing
Platform: x86_64-apple-darwin13.4.0 (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

  Natural language support but running in an English locale

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

> library(quantmod)
Loading required package: xts
Loading required package: zoo

Attaching package: 'zoo'

The following objects are masked from 'package:base':

    as.Date, as.Date.numeric

Loading required package: TTR
Version 0.4-0 included new data defaults. See ?getSymbols.
Warning messages:
1: package 'quantmod' was built under R version 3.2.5 
2: package 'zoo' was built under R version 3.2.5 
3: package 'TTR' was built under R version 3.2.4 
> getSymbols("CPIAUCSL", src = "FRED")
    As of 0.4-0, 'getSymbols' uses env=parent.frame() and
 auto.assign=TRUE by default.

 This  behavior  will be  phased out in 0.5-0  when the call  will
 default to use auto.assign=FALSE. getOption("getSymbols.env") and 
 getOptions("getSymbols.auto.assign") are now checked for alternate defaults

 This message is shown once per session and may be disabled by setting 
 options("getSymbols.warning4.0"=FALSE). See ?getSymbols for more details.
[1] "CPIAUCSL"
> 
> sessionInfo()
R version 3.2.3 (2015-12-10)
Platform: x86_64-apple-darwin13.4.0 (64-bit)
Running under: OS X 10.11.6 (El Capitan)

locale:
[1] C

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] quantmod_0.4-7 TTR_0.23-1     xts_0.9-7      zoo_1.7-13    

loaded via a namespace (and not attached):
[1] grid_3.2.3      lattice_0.20-33

floswald commented Feb 14, 2017

this happens only in Rstudio:

R version 3.2.3 (2015-12-10) -- "Wooden Christmas-Tree"
Copyright (C) 2015 The R Foundation for Statistical Computing
Platform: x86_64-apple-darwin13.4.0 (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

  Natural language support but running in an English locale

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

> library(quantmod)
Loading required package: xts
Loading required package: zoo

Attaching package:zooThe following objects are masked frompackage:base:

    as.Date, as.Date.numeric

Loading required package: TTR
Version 0.4-0 included new data defaults. See ?getSymbols.
Warning messages:
1: packagequantmodwas built under R version 3.2.5 
2: packagezoowas built under R version 3.2.5 
3: packageTTRwas built under R version 3.2.4 
> getSymbols("CPIAUCSL", src = "FRED")
    As of 0.4-0, ‘getSymbolsuses env=parent.frame() and
 auto.assign=TRUE by default.

 This  behavior  will be  phased out in 0.5-0  when the call  will
 default to use auto.assign=FALSE. getOption("getSymbols.env") and 
 getOptions("getSymbols.auto.assign") are now checked for alternate defaults

 This message is shown once per session and may be disabled by setting 
 options("getSymbols.warning4.0"=FALSE). See ?getSymbols for more details.
Error in charToDate(x) : 
  character string is not in a standard unambiguous format
> traceback()
15: stop("character string is not in a standard unambiguous format")
14: charToDate(x)
13: as.Date.character(as.character(x), ...)
12: as.Date(as.character(x), ...)
11: as.Date.factor(fr[, 1], origin = "1970-01-01")
10: as.Date(fr[, 1], origin = "1970-01-01")
9: inherits(x, xx)
8: FUN(X[[i]], ...)
7: lapply(X = X, FUN = FUN, ...)
6: sapply(c("Date", "POSIXt", "chron", "dates", "times", "timeDate", 
       "yearmon", "yearqtr", "xtime"), function(xx) inherits(x, 
       xx))
5: timeBased(order.by)
4: xts(as.matrix(fr[, -1]), as.Date(fr[, 1], origin = "1970-01-01"), 
       src = "FRED", updated = Sys.time())
3: getSymbols.FRED(Symbols = "CPIAUCSL", env = <environment>, verbose = FALSE, 
       warnings = TRUE, auto.assign = TRUE)
2: do.call(paste("getSymbols.", symbol.source, sep = ""), list(Symbols = current.symbols, 
       env = env, verbose = verbose, warnings = warnings, auto.assign = auto.assign, 
       ...))
1: getSymbols("CPIAUCSL", src = "FRED")
> sessionInfo()
R version 3.2.3 (2015-12-10)
Platform: x86_64-apple-darwin13.4.0 (64-bit)
Running under: OS X 10.11.6 (El Capitan)

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] quantmod_0.4-7 TTR_0.23-1     xts_0.9-7      zoo_1.7-13    

loaded via a namespace (and not attached):
[1] tools_3.2.3     grid_3.2.3      lattice_0.20-33

R console works:

~ R

R version 3.2.3 (2015-12-10) -- "Wooden Christmas-Tree"
Copyright (C) 2015 The R Foundation for Statistical Computing
Platform: x86_64-apple-darwin13.4.0 (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

  Natural language support but running in an English locale

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

> library(quantmod)
Loading required package: xts
Loading required package: zoo

Attaching package: 'zoo'

The following objects are masked from 'package:base':

    as.Date, as.Date.numeric

Loading required package: TTR
Version 0.4-0 included new data defaults. See ?getSymbols.
Warning messages:
1: package 'quantmod' was built under R version 3.2.5 
2: package 'zoo' was built under R version 3.2.5 
3: package 'TTR' was built under R version 3.2.4 
> getSymbols("CPIAUCSL", src = "FRED")
    As of 0.4-0, 'getSymbols' uses env=parent.frame() and
 auto.assign=TRUE by default.

 This  behavior  will be  phased out in 0.5-0  when the call  will
 default to use auto.assign=FALSE. getOption("getSymbols.env") and 
 getOptions("getSymbols.auto.assign") are now checked for alternate defaults

 This message is shown once per session and may be disabled by setting 
 options("getSymbols.warning4.0"=FALSE). See ?getSymbols for more details.
[1] "CPIAUCSL"
> 
> sessionInfo()
R version 3.2.3 (2015-12-10)
Platform: x86_64-apple-darwin13.4.0 (64-bit)
Running under: OS X 10.11.6 (El Capitan)

locale:
[1] C

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] quantmod_0.4-7 TTR_0.23-1     xts_0.9-7      zoo_1.7-13    

loaded via a namespace (and not attached):
[1] grid_3.2.3      lattice_0.20-33
@joshuaulrich

This comment has been minimized.

Show comment
Hide comment
@joshuaulrich

joshuaulrich Feb 14, 2017

Owner

Thanks. Can you edit your previous comment and include the output of sessionInfo() when run at the R console?

My initial thought was that this is a locale issue, but your locale in RStudio is en_US.UTF-8, and the dates from FRED are formatted as "%Y-%m-%d", which does not have any locale-specific components.

Owner

joshuaulrich commented Feb 14, 2017

Thanks. Can you edit your previous comment and include the output of sessionInfo() when run at the R console?

My initial thought was that this is a locale issue, but your locale in RStudio is en_US.UTF-8, and the dates from FRED are formatted as "%Y-%m-%d", which does not have any locale-specific components.

@floswald

This comment has been minimized.

Show comment
Hide comment
@floswald

floswald Feb 14, 2017

hmm... i don't know why the locales are different here. I checked my ~/.zshrc, there is no locale export defined. how come those are different? anyway, the only non-locale difference between the two settings is that Rstudio attaches tools, whereas base R does not.

floswald commented Feb 14, 2017

hmm... i don't know why the locales are different here. I checked my ~/.zshrc, there is no locale export defined. how come those are different? anyway, the only non-locale difference between the two settings is that Rstudio attaches tools, whereas base R does not.

@joshuaulrich

This comment has been minimized.

Show comment
Hide comment
@joshuaulrich

joshuaulrich Feb 14, 2017

Owner

I'm not sure why the locale is different when running R at the console. Maybe a Mac/OS X thing? Regardless, FRED provides dates in ISO-8601 format, so locale shouldn't be an issue.

I'm probably not going to be much help here, since I'm not well versed in either RStudio or OS X. Calling for reinforcements: @jjallaire, @jcheng5, any ideas?

Owner

joshuaulrich commented Feb 14, 2017

I'm not sure why the locale is different when running R at the console. Maybe a Mac/OS X thing? Regardless, FRED provides dates in ISO-8601 format, so locale shouldn't be an issue.

I'm probably not going to be much help here, since I'm not well versed in either RStudio or OS X. Calling for reinforcements: @jjallaire, @jcheng5, any ideas?

@jjallaire

This comment has been minimized.

Show comment
Hide comment
@jjallaire

jjallaire Feb 14, 2017

Both R GUI on the Mac and RStudio do some locale "fixing" at startup (this is required for them to handle text correctly). Here is that code: https://github.com/rstudio/rstudio/blob/master/src/cpp/desktop-mac/Utils.mm#L25-L93

jjallaire commented Feb 14, 2017

Both R GUI on the Mac and RStudio do some locale "fixing" at startup (this is required for them to handle text correctly). Here is that code: https://github.com/rstudio/rstudio/blob/master/src/cpp/desktop-mac/Utils.mm#L25-L93

@joshuaulrich

This comment has been minimized.

Show comment
Hide comment
@joshuaulrich

joshuaulrich Feb 14, 2017

Owner

@floswald Can you please run these commands from your RStudio session where you get the error:

debugonce(getSymbols.FRED)
getSymbols("CPIAUCSL", src="FRED")

Then press [Enter] until you see:

debug: fr <- read.csv(tmp, na.string = ".")
Browse[2]> 
debug: if (verbose) cat("done.\n")
Browse[2]> 

Then provide the output of running dput(fr) at the browse prompt. You should see something like:

Browse[2]> dput(fr)
structure(list(DATE = structure(1:840, .Label = c("1947-01-01", 
"1947-02-01", "1947-03-01", "1947-04-01", "1947-05-01", "1947-06-01", 
...
...
...
    238.908, 239.362, 239.842, 239.898, 240.389, 241.006, 241.694, 
    242.199, 242.821)), .Names = c("DATE", "VALUE"), class = "data.frame",
    row.names = c(NA, -840L))
Owner

joshuaulrich commented Feb 14, 2017

@floswald Can you please run these commands from your RStudio session where you get the error:

debugonce(getSymbols.FRED)
getSymbols("CPIAUCSL", src="FRED")

Then press [Enter] until you see:

debug: fr <- read.csv(tmp, na.string = ".")
Browse[2]> 
debug: if (verbose) cat("done.\n")
Browse[2]> 

Then provide the output of running dput(fr) at the browse prompt. You should see something like:

Browse[2]> dput(fr)
structure(list(DATE = structure(1:840, .Label = c("1947-01-01", 
"1947-02-01", "1947-03-01", "1947-04-01", "1947-05-01", "1947-06-01", 
...
...
...
    238.908, 239.362, 239.842, 239.898, 240.389, 241.006, 241.694, 
    242.199, 242.821)), .Names = c("DATE", "VALUE"), class = "data.frame",
    row.names = c(NA, -840L))
@floswald

This comment has been minimized.

Show comment
Hide comment
@floswald

floswald Feb 14, 2017

d'oh. they moved the site! but how weird that in the console this still works?!

Browse[2]> dput(fr)
structure(list(X..DOCTYPE.HTML.PUBLIC....IETF..DTD.HTML.2.0..EN. = structure(c(4L, 
6L, 2L, 3L, 5L, 1L), .Label = c("</body></html>", "</head><body>", 
"<h1>Moved Permanently</h1>", "<html><head>", "<p>The document has moved <a href=https://fred.stlouisfed.org/series/CPIAUCSL/downloaddata/CPIAUCSL.csv>here</a>.</p>", 
"<title>301 Moved Permanently</title>"), class = "factor")), .Names = "X..DOCTYPE.HTML.PUBLIC....IETF..DTD.HTML.2.0..EN.", class = "data.frame", row.names = c(NA, 
-6L))

floswald commented Feb 14, 2017

d'oh. they moved the site! but how weird that in the console this still works?!

Browse[2]> dput(fr)
structure(list(X..DOCTYPE.HTML.PUBLIC....IETF..DTD.HTML.2.0..EN. = structure(c(4L, 
6L, 2L, 3L, 5L, 1L), .Label = c("</body></html>", "</head><body>", 
"<h1>Moved Permanently</h1>", "<html><head>", "<p>The document has moved <a href=https://fred.stlouisfed.org/series/CPIAUCSL/downloaddata/CPIAUCSL.csv>here</a>.</p>", 
"<title>301 Moved Permanently</title>"), class = "factor")), .Names = "X..DOCTYPE.HTML.PUBLIC....IETF..DTD.HTML.2.0..EN.", class = "data.frame", row.names = c(NA, 
-6L))
@joshuaulrich

This comment has been minimized.

Show comment
Hide comment
@joshuaulrich

joshuaulrich Feb 14, 2017

Owner

What is the URL used by try.download.file()? It should be:
"https://research.stlouisfed.org/fred2/series/CPIAUCSL/downloaddata/CPIAUCSL.csv"

Owner

joshuaulrich commented Feb 14, 2017

What is the URL used by try.download.file()? It should be:
"https://research.stlouisfed.org/fred2/series/CPIAUCSL/downloaddata/CPIAUCSL.csv"

@floswald

This comment has been minimized.

Show comment
Hide comment
@floswald

floswald Feb 14, 2017

"https://research.stlouisfed.org/fred2/series/CPIAUCSL/downloaddata/CPIAUCSL.csv"

I see a hard coded

FRED.URL <- "https://research.stlouisfed.org/fred2/series"

floswald commented Feb 14, 2017

"https://research.stlouisfed.org/fred2/series/CPIAUCSL/downloaddata/CPIAUCSL.csv"

I see a hard coded

FRED.URL <- "https://research.stlouisfed.org/fred2/series"

@joshuaulrich

This comment has been minimized.

Show comment
Hide comment
@joshuaulrich

joshuaulrich Feb 14, 2017

Owner

Well, that explains it. I'll update the URL used in getSymbols.FRED().

There must be some setting in RStudio that affects internet connections and prevents the redirect from being followed. I noticed you're using a version of R that's ~1 year old. You might try upgrading R and RStudio to see if that solves your issue. If not, you might want to let them know you're still having issues, because this could potentially affect other functions.

Owner

joshuaulrich commented Feb 14, 2017

Well, that explains it. I'll update the URL used in getSymbols.FRED().

There must be some setting in RStudio that affects internet connections and prevents the redirect from being followed. I noticed you're using a version of R that's ~1 year old. You might try upgrading R and RStudio to see if that solves your issue. If not, you might want to let them know you're still having issues, because this could potentially affect other functions.

@joshuaulrich joshuaulrich self-assigned this Feb 14, 2017

@floswald

This comment has been minimized.

Show comment
Hide comment
@floswald

floswald Feb 14, 2017

brilliant. thanks for your help!

floswald commented Feb 14, 2017

brilliant. thanks for your help!

@floswald

This comment has been minimized.

Show comment
Hide comment
@floswald

floswald Feb 14, 2017

hey @joshuaulrich, i updated R and RStudio is working again. probably a good idea to fix that link anyway though. cheers

floswald commented Feb 14, 2017

hey @joshuaulrich, i updated R and RStudio is working again. probably a good idea to fix that link anyway though. cheers

joshuaulrich added a commit that referenced this issue Feb 26, 2017

Remove double forward slash from FRED URL
Trailing slash causes the FRED server to issue a 404 response.

The trailing slash was added in:
90e2c36

See #141.

@joshuaulrich joshuaulrich added this to the Release 0.4-8 milestone Apr 10, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment