getQuote cannot open URL

[mmullinsm]

Using dby <- getQuote("AAPL",src="yahoo", auto.assign = FALSE)
gets the following response:

Error in download.file(paste("https://finance.yahoo.com/d/quotes.csv?s=", :
cannot open URL 'https://finance.yahoo.com/d/quotes.csv?s=AAPL&f=d1t1l1c1p2ohgv'
In addition: Warning message:
In download.file(paste("https://finance.yahoo.com/d/quotes.csv?s=", :
cannot open URL 'https://finance.yahoo.com/d/quotes.csv?s=AAPL&f=d1t1l1c1p2ohgv': HTTP status was '999 Request denied'

This problem started after the market close on Nov 1 but had not been an issue in prior use for several months.

Note: getSymbols, as in dbx <- getSymbols("AAPL",src="yahoo", auto.assign = FALSE), seems to work, though it is not picking up Nov 1 data.

[pjheink]

A JSON version of the call still works. You might try something like this:

`require(jsonlite)

getQuote <- function(ticks) {
qRoot <- "https://query1.finance.yahoo.com/v7/finance/quote?fields=symbol,longName,regularMarketPrice,regularMarketChange,regularMarketTime&formatted=false&symbols="
z <- fromJSON(paste(qRoot, paste(ticks, collapse=","), sep=""))
z <- z$quoteResponse$result[,c("symbol", "regularMarketTime", "regularMarketPrice", "regularMarketChange", "longName")]
row.names(z) <- z$symbol
z$symbol <- NULL
names(z) <- c("Time", "Price", "Change", "Name")
z$Time <- as.POSIXct(z$Time, origin = '1970-01-01 00:00:00')
return(z)
}
`

[mmullinsm]

That works great - thanks!

Any possibility of adding the daily open, high and low to this call?

[selgamal]

Thanks a lot, works for me too!

[pjheink]

I haven't found documentation for the API but if you just don't specify any "fields" and run this:

z <- fromJSON("https://query1.finance.yahoo.com/v7/finance/quote?formatted=false&symbols=IBM")

it look like you get just about everything and from that you can customize the URL by specifying the fields you want.

For for a nice list, just look at the names of the columns in the result. I'll transpose this below so that they're easier to read.

t(z$quoteResponse$result)

language                          "en-US"                                      
quoteType                         "EQUITY"                                     
quoteSourceName                   "Nasdaq Real Time Price"                     
currency                          "USD"                                        
priceHint                         "2"                                          
regularMarketPrice                "153.76"                                     
regularMarketTime                 "1509638054"                                 
regularMarketChange               "-0.2700043"                                 
regularMarketOpen                 "154.18"                                     
regularMarketDayHigh              "154.49"                                     
regularMarketDayLow               "152.92"                                     
regularMarketVolume               "1440836"                                    
regularMarketChangePercent        "-0.1752933"                                 
regularMarketPreviousClose        "154.03"                                     
bid                               "153.63"                                     
ask                               "153.66"                                     
bidSize                           "2"                                          
askSize                           "4"                                          
messageBoardId                    "finmb_112350"                               
fullExchangeName                  "NYSE"                                       
longName                          "International Business Machines Corporation"
financialCurrency                 "USD"                                        
averageDailyVolume3Month          "4384179"                                    
averageDailyVolume10Day           "5503657"                                    
fiftyTwoWeekLowChange             "14.62999"                                   
fiftyTwoWeekLowChangePercent      "0.1051534"                                  
fiftyTwoWeekHighChange            "-29.03"                                     
fiftyTwoWeekHighChangePercent     "-0.1588161"                                 
exchangeDataDelayedBy             "0"                                          
fiftyTwoWeekLow                   "139.13"                                     
fiftyTwoWeekHigh                  "182.79"                                     
dividendDate                      "1504915200"                                 
earningsTimestamp                 "1508270400"                                 
earningsTimestampStart            "1516222800"                                 
earningsTimestampEnd              "1516654800"                                 
trailingAnnualDividendRate        "1.4"                                        
trailingPE                        "12.82616"                                   
trailingAnnualDividendYield       "0.009089138"                                
sharesOutstanding                 "931939968"                                  
bookValue                         "21.118"                                     
fiftyDayAverage                   "149.19"                                     
fiftyDayAverageChange             "4.569992"                                   
shortName                         "International Business Machines"            
marketState                       "REGULAR"                                    
market                            "us_market"                                  
exchange                          "NYQ"                                        
fiftyDayAverageChangePercent      "0.03063203"                                 
twoHundredDayAverage              "150.346"                                    
twoHundredDayAverageChange        "3.413956"                                   
twoHundredDayAverageChangePercent "0.02270732"                                 
marketCap                         "143295086592"                               
forwardPE                         "11.03805"                                   
priceToBook                       "7.280992"                                   
sourceInterval                    "15"                                         
exchangeTimezoneName              "America/New_York"                           
exchangeTimezoneShortName         "EDT"                                        
gmtOffSetMilliseconds             "-14400000"                                  
tradeable                         "TRUE"                                       
epsTrailingTwelveMonths           "11.988"                                     
epsForward                        "13.93"                                      
symbol                            "IBM"                                        
> 

[selgamal]

@pjheink brilliant!!! thanks a lot, I was just searching for that

[jgaecke]

@pjheink This is great, will be using this as a temp solution. Thanks.

[dennisfisher]

the JSON code works great -- is it possible to obtain index quotes (e.g., Dow, S+P, NASDAQ)? I have tried various permutations of symbols for these without success.

[mmullinsm]

Have you tried ^DJI, ^GSPC and ^IXIC?

[dennisfisher]

That worked — the problem was that I needed to select different column names. Dennis Fisher MD P < (The "P Less Than" Company) Phone / Fax: 1-866-PLessThan (1-866-753-7784) www.PLessThan.com <http://www.plessthan.com/>

…

On Nov 3, 2017, at 7:17 AM, mmullinsm ***@***.***> wrote: Have you tried ^DJI, ^GSPC and ^IXIC? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#197 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AHKsUcVHWnr6l23qJkkzRP4miqcIGlaaks5syyCHgaJpZM4QPDir>.

[mmullinsm]

Here's another approach that uses the suggested URL with defined fields but without a call to jsonlite:

url <- "https://query1.finance.yahoo.com/v7/finance/quote?fields=symbol,regularMarketPrice,regularMarketChange,regularMarketChangePercent,regularMarketOpen,regularMarketDayHigh,regularMarketDayLow,regularMarketVolume,regularMarketTime&formatted=false&symbols=AAPL"

resp <- GET(url)

db <- content(resp)

[dennisfisher]

Which package is GET from? (not base R) Dennis Fisher MD P < (The "P Less Than" Company) Phone / Fax: 1-866-PLessThan (1-866-753-7784) www.PLessThan.com <http://www.plessthan.com/>

…

On Nov 3, 2017, at 7:58 AM, mmullinsm ***@***.***> wrote: Here's another approach that uses the suggested URL with defined fields but without a call to jsonlite: url <- "https://query1.finance.yahoo.com/v7/finance/quote?fields=symbol,regularMarketPrice,regularMarketChange,regularMarketChangePercent,regularMarketOpen,regularMarketDayHigh,regularMarketDayLow,regularMarketVolume,regularMarketTime&formatted=false&symbols=AAPL <https://query1.finance.yahoo.com/v7/finance/quote?fields=symbol,regularMarketPrice,regularMarketChange,regularMarketChangePercent,regularMarketOpen,regularMarketDayHigh,regularMarketDayLow,regularMarketVolume,regularMarketTime&formatted=false&symbols=AAPL>" resp <- GET(url) db <- content(resp) — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#197 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AHKsUeudJVKexo2IzZ3TgD1V6oHlmUhLks5syyn7gaJpZM4QPDir>.

[mmullinsm]

The commands are from the httr package, which allows the user to deal with URLs

[trade-notification2016]

noticed that there is no ex-div-date from the JSON API...

[trade-notification2016]

never mind, even though there is no ex-div-date from the list @pjheink provided (thanks a lot btw), it;s actually available via the API. and the true name is exDividendDate. upper D's. cheers!

[selesnow]

I have a similar error, do you can help in my issure?

getQuote(c("UAHUSD=X","RUBUSD=X","EURUSD=X"))
Error in download.file(paste("https://finance.yahoo.com/d/quotes.csv?s=", :
cannot open URL 'https://finance.yahoo.com/d/quotes.csv?s=UAHUSD=X+RUBUSD=X+EURUSD=X&f=d1t1l1c1p2ohgv'
In addition: Warning message:
In download.file(paste("https://finance.yahoo.com/d/quotes.csv?s=", :
cannot open URL 'https://finance.yahoo.com/d/quotes.csv?s=UAHUSD=X+RUBUSD=X+EURUSD=X&f=d1t1l1c1p2ohgv': HTTP status was '403 Forbidden'

sessionInfo:
R version 3.4.2 (2017-09-28)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows Server 2008 R2 x64 (build 7601) Service Pack 1

Matrix products: default

locale:
[1] LC_COLLATE=English_United States.1252 LC_CTYPE=English_United States.1252
[3] LC_MONETARY=English_United States.1252 LC_NUMERIC=C
[5] LC_TIME=English_United States.1252

attached base packages:
[1] stats graphics grDevices utils datasets methods base

other attached packages:
[1] quantmod_0.4-11 TTR_0.23-2 xts_0.10-0 zoo_1.8-0 RMySQL_0.10.13
[6] DBI_0.7

loaded via a namespace (and not attached):
[1] lattice_0.20-35 gWidgets_0.0-54 withr_2.0.0 digest_0.6.12
[5] grid_3.4.2 gWidgetstcltk_0.0-55 curl_3.0 devtools_1.13.3
[9] tools_3.4.2 yaml_2.1.14 compiler_3.4.2 tcltk_3.4.2
[13] memoise_1.1.0

[joshuaulrich]

I just pushed an update to the 197_getQuote branch. Please test and provide feedback!

There is one minor issue regarding the "Last Time" field. It's currently in your local timezone, not the exchange timezone. The fix needs to account for the possibility that users may request multiple symbols, and those symbols may be from exchanges in different timezones. You cannot mix timezones in a POSIXct vector, so we need to check whether all the timezones are the same, and convert them to a common timezone if they are not.

[pjheink]

Thanks, Josh!

I've been messing with this branch and it seems to work just fine as long as you avoid the older compact quote format codes (i.e. what='l1c1n' doesn't work now but was working last week).

While it's not really relevant anymore, I noticed that the www.gummy-stuff.org URL, which is included as a reference in the quantmod documentation is almost certainly NOT owned by Peter Ponzo anymore. The original material from that site is now stored at: http://www.financialwisdomforum.org/gummy-stuff/Yahoo-data.htm.

[parayamelo]

Thanks for the update regarding the getQuote function Josh. For me, it is working fine, but when I try to get the value of some pairs, I get the following error:

Error in [.data.frame(sq, , QF) : undefined columns selected

This happens, so far, with EURJPY, EURCAD, GBPJPY, EURAUD, AUDCAD. Do you know if there is a reason for this?

quantmod updated with branch 197_getQuote

Thanks!

[wave-electron]

I just tried to update on this branch.

devtools::install_github("joshuaulrich/quantmod", ref="197_getQuote")

and I'm getting the following download error.

Downloading GitHub repo joshuaulrich/quantmod@197_getQuote from URL https://api.github.com/repos/joshuaulrich/quantmod/zipball/197_getQuote Error in stop(github_error(request)) : 404: Not Found (404)

i also don't see the branch listed anymore?

[joshuaulrich]

@wave-electron Apologies, I deleted the branch after merging it to master, but I forgot to close this issue. The fix for this issue is currently on CRAN, so you can get the patch via install.packages().

[wave-electron]

@joshuaulrich I know this issue is now closed. But it appears that empty fields are not being handled the same way as previously by quantmod. I haven't had time to debug my own code yet... but based on the way empty fields are breaking my own code it seems something has changed there. it might also be related to @parayamelo issue above on

Error in [.data.frame(sq, , QF) : undefined columns selected

[wave-electron]

this is my code using quantmod

 what_metrics <- yahooQF(c("Name","Price/Sales", 
                        "P/E Ratio",
                        "Price/EPS Estimate Next Year",
                        "PEG Ratio",
                        "Dividend Yield", 
                        "Market Capitalization",
                        "Shares Owned","volume",
                        "EBITDA","52-week Low","52-week High",
                        "52-week Range","1 yr Target Price",
                        "Earnings/Share","Average Daily Volume",
                        "After Hours Change (Real-time)"))

 metrics <- NULL

  metrics <- getQuote(paste(tickers, sep="", collapse=";"), what=what_metrics)   

when the ticker is AAPL the returned fields all have a value so it doesn't throw the error above.

when the ticker is AMZN the Dividend Yield field is empty so it throws the error.

If I have ticker "AAPL;AMZN" the error won't throw and I get NA in the Dividend Yield. But it will if the other way around "AMZN;AAPL"

[joshuaulrich]

@wave-electron Thanks for the report! I consider the error in your comment a new issue, because this issue has been closed and is already in a CRAN release. Please open a new issue (edit: now in #208), and include the code you run and the output/error generated, as wells as your sessionInfo() output.

FWIW, I only get an error if Symbols = "AMZN", not in the other two cases.