[WIP] Yahoo Finance v7 API (since 2017-05-16) - final work (3/3)

[gadicc]

Update: All the code parts of this PR are merged, released and have been stable for years. You're welcome to check out work on the v2 candidate, in beta probably until mid-2021, at which stage we'll make some official recommendations.

---

Continuation from #37, #41. Work in Progress.

• Store cookies, retrieve crumb, cache
  • Invalidate if cookie has changed
  • Move getCrumb() usage to within download()
  • If called multiple times, queue all requests until getCrumb() completes.
• quote()
  • options sanitation
  • Support symbols option (i.e. multiple symbols in one request - this needs multiple HTTP requests in yahoo's new API).
  • tests for shortcut api
• Add note to README about first request taking longer & why.
• Do it in one request. Currently, we send two requests the first time we're run. But all the data is in the original html result too, we should consider taking it just from there. Probably not worth doing this as would require multiple parsers for each method.

[talbensimon]

Hi @gadicc / @pilwon - Really looking forward to get the new version that includes the multiple symbols in one request. This is so much needed to avoid breaking my project.

Huge thanks for the awesome work you've been doing :)

[talbensimon]

@pilwon I've just tested the recent commit made by @gadicc and both 'quote' and 'historical' are working properly with the long awaited multiple symbols in single query.

[pilwon]

@gadicc Do you think multiple symbols feature is good enough to be merged? @talbensimon seems happy with his test results.

[talbensimon]

OK, here is an updated list of issues I've discovered so far:

1. On the first run there is an error which doesn't occur afterwards.
   Unhandled rejection Error: Failed to get crumb (str.trim is not a function)
   at /app/node_modules/yahoo-finance/lib/yahooCrumb.js:72:13

2. There is still the annoying warning upon each call
   [yahoo-finance] Warning, Yahoo completely changed their API recently. quote() replaces the deprecated snapshot(), but is brand new. Please report any issues. This notice will be removed in the next release.

Other than that, it's working fine :)
@gadicc @pilwon

[gadicc]

Hi @pilwon, @talbensimon. Sorry for the late reply. Unfortunately a lot of time sensitive issues have come up for me lately and it will be another week or two until I can look at this again. From what I recall all the work I did until now was with tests, so if @talbensimon is confirming it works, we could release, and I'll address the other outstanding issues as soon as I can.

[gadicc]

@talbensimon, your first issue (about str.trim) might be from upgrading an existing install with an older version of tough-cookie. Try rm -rf node-modules and npm i (or yarn install) again and see if the error goes away.

[talbensimon]

Thanks @gadicc, it indeed solved the first issue!

[damooch]

You guys are doing awesome. I found a cool URL playing around looking at Yahoo's own code while trying to fix my own and create a new intra-day option.
If you want to take a look at it.
I think with this you guys could bring back periods and intervals.

https://query1.finance.yahoo.com/v8/finance/chart/GOOG?range=5d&includePrePost=false&interval=2m&corsDomain=finance.yahoo.com&.trsc=finance

Valid Intervals: [1m, 2m, 5m, 15m, 30m, 60m, 90m, 1h, 1d, 5d, 1wk, 1mo, 3mo]
Valid Ranges :["1d","5d","1mo","3mo","6mo","1y","2y","5y","10y","ytd","max"]
so far Ive noticed 1d acutally works in increments so you can do 1d, 2d, 9d, etc

[shortcircuit3]

Cheering everyone on in this PR!

[MichaelWhite63]

Reading through the comments it reads as if the code has been amended to handle Yahoo's changes. Is that correct? Will 'npm install --save yahoo-finance' give me the correct version?

[Luchanso]

Please, can you merge multiply symbol? Better in future create merge request for every new feature

[pilwon]

@Luchanso Please check out @gadicc's PR #47.

[gadicc]

Yeah, all, pilwon just merged PR #47 and published as v0.3.3 - that's all my work on this from earlier this year. I still wanted to leave this PR open to track the rest of the changes, which in my mind are still critical for this to be considered "finished". And as per earlier comment, I do regret how some big unexpected commitments totally halted my work on this until now. Still hoping to try to finish this off as soon as I can.

[GuilhermeZaniniMoreira]

How do I use it with ES2015?

[gadicc]

Hi @GuilhermeZaniniMoreira

If you don't pass historical() or quote() a callback as the final parameter, they'll return a promise, which can be used with ES2017 async/await (is that what you meant?).

There are some examples in the full quote docs, which includes a convenient shortcut API, e.g.

const result = await quote('TSLA');  // implies default modules below
const result = await quote('TSLA', ['summaryDetail', 'recommendationTrend']);

or as just a regular promise:

quote('TSLA').then(function(data) { console.log(data); });

[stuckinaboot]

Is this still being worked on?

[gadicc]

@stuckinaboot, @pilwon - I regret I never got to finish this exactly how I wanted, however, I may have been a bit too pedantic, as even without the extra resiliency I'd planned, all the previously submitted code has continued to work great for years.

@pilwon, I've just made some changes to the README to cleanup the notices and add a bit a note about the first request taking longer. I think let's merge and close?

[DDeme]

When it will be merged ?

[gadicc]

@DDeme do note that the only part left of this PR that hasn't been merged yet are some updates to the README, all the code is already in the released version.

[DDeme]

Great do you have any plans to migrate to typescript?

[gadicc]

Sure, why not, that's a good idea. It will be my first time using typescript 😅

[gadicc]

@DDeme, re typescript, see #75 (comment).

[kuworking]

Great that the merge seems very close, what version should we wait for?

[gadicc]

Hey @kuworking, it's only really some changes to the README we still have to do, all the code parts have been merged already and have been stable for years 😅

You're welcome to also check out the work happening at node-yahoo-finance2 although I think we'll keep that in beta for 3-6 months before making any official recommendations.

[kuworking]

Then thanks god I've asked, I could have been waiting for years 🤓

What's the difference between the two? Is it a major rewriting? And thanks a lot for this development, I'm gonna try it right now 🙌

[gadicc]

😆

Yes, exactly. The v2 candidate is a complete rewrite from scratch, whereas the current version has been stable for years. So we'll keep it in beta for some time before making any official recommendations.

[gadicc]

I made an update at the top of the this post for clarity and I'll hope we'll merge the rest soon.

[darren-outdev]

Sure, why not, that's a good idea. It will be my first time using typescript 😅

Need some assistance?

[gadicc]

Hey @darren-outdev, thanks. Did you mean assistance with TypeScript or in general?

In any case, both are very welcome. See where we're at at https://github.com/gadicc/node-yahoo-finance2.

In general: we just need help testing the library (which is in beta now).
Re typescript: I think I got up to speed quick but please let me know if you notice anything wrong or unconventional.

Thanks!

[darren-outdev]

Oh wow, you've gotten far! Well done :) Yeah I meant specifically with TS, been using it for years. I'll browse around the linked project and use the standard contributions method if I have updates

[gadicc]

Thanks and thanks, @darren-outdev! That would be awesome 😁

[pilwon]

Merged and published as v0.3.7.

[gadicc]

Thanks, @pilwon! 😀