Can not update stock prices. Get invalid cookie error. #5894

dobber1 · 2023-05-07T15:37:08Z

renato-mmex · 2023-05-07T18:11:22Z

vomikan · 2023-05-07T19:17:06Z

I'm researching this case. The link to get quotes returns an incorrect result.

https://query1.finance.yahoo.com/v7/finance/quote?symbols=IBM&fields=regularMarketPrice,currency,shortName

{"finance":{"result":null,"error":{"code":"Unauthorized","description":"Invalid Crumb"}}}

But V6 returns correct result

https://query1.finance.yahoo.com/v6/finance/quote?symbols=IBM&fields=regularMarketPrice,currency,shortName

{
    "quoteResponse": {
        "result": [{
                "language": "en-US",
                "region": "US",
                "quoteType": "EQUITY",
                "typeDisp": "Equity",
                "triggerable": true,
                "customPriceAlertConfidence": "HIGH",
                "currency": "USD",
                "marketState": "CLOSED",
                "exchangeTimezoneShortName": "EDT",
                "gmtOffSetMilliseconds": -14400000,
                "esgPopulated": false,
                "regularMarketPrice": 173.57,
                "exchange": "NMS",
                "shortName": "Apple Inc.",
                "exchangeTimezoneName": "America/New_York",
                "market": "us_market",
                "regularMarketTime": 1683316804,
                "fullExchangeName": "NasdaqGS",
                "sourceInterval": 15,
                "exchangeDataDelayedBy": 0,
                "tradeable": false,
                "cryptoTradeable": false,
                "firstTradeDateMilliseconds": 345479400000,
                "priceHint": 2,
                "symbol": "AAPL"
            }
        ],
        "error": null
    }
}

vomikan · 2023-05-07T19:21:23Z

We don't have a file format check in our code, which is received in response in the function
bool mmStocksPanel::onlineQuoteRefresh

n-stein · 2023-05-08T03:18:47Z

Swapping to v6 in mmex::weblink::YahooQuotes seems to work OK. Currency rates and stock values all update properly as far as I can see. Can we just switch over to v6?

n-stein · 2023-05-08T04:12:29Z

Alternatively we can stick with V7 but first CURL finance.yahoo.com to pull a session cookie then CURL https://query1.finance.yahoo.com/v1/test/getcrumb to get the crumb text and append it to the V7 query as seen here https://github.com/pstadler/ticker.sh/blob/acquire-yahoo-finance-session/ticker.sh#L40-L49

You can verify this works:

Go to https://finance.yahoo.com
Go to https://query1.finance.yahoo.com/v1/test/getcrumb and copy the text string
Go to https://query1.finance.yahoo.com/v7/finance/quote?symbols=AAPL&fields=regularMarketPrice,currency,shortName&crumb=<text string from step 2>

reddyshyam · 2023-05-08T11:13:57Z

Hi both (@n-stein, @vomikan), not to add pressure but v1.6.4 is long over due. 😊 Just kidding.

Hope stock update fix makes into it and sometime soon.

vomikan · 2023-05-08T15:41:27Z

In the browser, I get the value, but not in the application.

n-stein · 2023-05-08T16:45:44Z

I think the cookie from the first finance.yahoo.com GET needs to be passed to CURL with the -c option in order to pick up the crumb. The http_get_data call is not doing this.

reddyshyam · 2023-05-09T07:24:19Z

I think the cookie from the first finance.yahoo.com GET needs to be passed to CURL with the -c option in order to pick up the crumb. The http_get_data call is not doing this.

Just thinking aloud, can we not switch to Google finance api which I think is more stable and long runner?

fix(#5894): switch to yahoo v6 api

n-stein · 2023-05-25T18:56:58Z

@vomikan I think this needs to be reopened. The v6 API is no longer functioning.

n-stein · 2023-05-26T22:42:19Z

Just thinking aloud, can we not switch to Google finance api which I think is more stable and long runner?

As far as I am aware Google Finance doesn't have a public API endpoint so the only option there is web scraping. Better to get the Yahoo v7 endpoint working again.

thinkpad4 · 2023-05-26T23:51:39Z

@n-stein Is this the V7 API? https://query1.finance.yahoo.com/v7/finance/chart/RADICO.NS?&interval=1m which when I click on it seems to work

n-stein · 2023-05-27T01:13:42Z

~~Yup, seems they again removed the crumb verification check for V7. So good news is the panel button should work in 1.6.3, at least for now.~~

The chart API seems to work, but the quote API doesn't.

https://query1.finance.yahoo.com/v7/finance/quote?symbols=RADICO.NS&fields=regularMarketPrice,currency,shortName

thinkpad4 · 2023-05-28T02:56:56Z

@n-stein https://rapidapi.com/blog/best-stock-api/

Yahoo V6 and V7 are permanent shutdown it seems

n-stein · 2023-05-31T22:54:04Z

Also I noticed, history picks up current date i.e. 1st June in historical data while the trading didnot happen for the day yet! As it is 12:10am Indian Standard Time.

Which symbol? Mine don't display June 1st yet either through the History download or the quote refresh.

thinkpad4 · 2023-06-01T01:37:02Z

@n-stein the latest dev build is working for me, I can get stock prices again

reddyshyam · 2023-06-01T03:36:10Z

Also I noticed, history picks up current date i.e. 1st June in historical data while the trading didnot happen for the day yet! As it is 12:10am Indian Standard Time.

Which symbol? Mine don't display June 1st yet either through the History download or the quote refresh.

HI @n-stein, that may be because you checked the same day the trading closed. I checked next day just past midnight.

reddyshyam · 2023-06-01T03:38:56Z

Also on the stock summary, should the price date be the last day the exchange run i.e. 31st May or the current date which is 1st Jun?

It reflects the latest date in your Stock History. You have values for June 1st, hence you are seeing that date in the panel.

Trading hasn't started for the day. Like I said it is midnight on 1st June here. Also it has same value as previous trading date which is 31st May. Please see the screenshots shared in the original post where I mentioned update delays.

reddyshyam · 2023-06-01T03:44:24Z

Reference

All symbols. So generic issue I guess but if you want to check one then please use ITC.NS

reddyshyam · 2023-06-01T03:49:22Z

Ok, done. The cookie & crumb are now stored in mmexini.db3. The quote refresh tries to use these values first, and only pulls new values if it fails.

Refreshes are fast now. Give it a try.

Yes, it is. Thanks @n-stein. Back to good old days. 😎

n-stein · 2023-06-01T04:15:53Z

Also I noticed, history picks up current date i.e. 1st June in historical data while the trading didnot happen for the day yet!

The historical download isn't the one that pulls current date. Historical download only pulls close prices. The June 1st entry you see is a result of clicking the panel button to fetch the prices after midnight. That button doesn't pull close prices, it pulls the current market price.

The quote refresh button on the panel always pulls the current market price from Yahoo and saves it with the current date. It doesn't care whether the market is open or not -- it's just the price that Yahoo reports at the moment you click the button. While the market is closed the market price is obviously the same as the last close price. When the market opens you can click refresh and it will update to the new market price. This is the way it has always worked.

reddyshyam · 2023-06-01T04:51:07Z

Also I noticed, history picks up current date i.e. 1st June in historical data while the trading didnot happen for the day yet!

The historical download isn't the one that pulls current date. Historical download only pulls close prices. The June 1st entry you see is a result of clicking the panel button to fetch the prices after midnight. That button doesn't pull close prices, it pulls the current market price.

The quote refresh button on the panel always pulls the current market price from Yahoo and saves it with the current date. It doesn't care whether the market is open or not -- it's just the price that Yahoo reports at the moment you click the button. While the market is closed the market price is obviously the same as the last close price. When the market opens you can click refresh and it will update to the new market price. This is the way it has always worked.

Seems like the case/logic. Works for me, just wanted to bring to your notice in case it was not observed. Thanks for the clarification, @n-stein. 'Good to go' from myside as well. 👍👏🎉

renato-mmex · 2023-06-01T18:33:45Z

Is this Beta version also available for MAC ?

Here https://onedrive.live.com/?authkey=%21AEWtos0VyDKnlN4&id=A0FB83D788D2D744%21104142&cid=A0FB83D788D2D744
there is none.

n-stein · 2023-06-01T20:24:00Z

Not yet, @whalley will need to build it

whalley · 2023-06-01T23:05:38Z

Not yet, @whalley will need to build it

Should be there now..

fix linux build

Heny67 · 2023-06-17T07:24:22Z

Hi All!
The date is fine for me, but the prices are not correct. I looked at several stocks and the same error.

Ubuntu mate 22.04

vomikan · 2023-06-17T08:59:38Z

the prices are not correct.

If the trading session is open, the price can vary greatly. Maybe this is the reason?

n-stein · 2023-06-17T12:44:49Z

No this doesn't make sense, IUSE.L hasn't had a trading low in the 80's since June 1st.

vomikan · 2023-06-17T13:04:16Z

The value 88.34 is strangely repeated at regular intervals.

n-stein · 2023-06-17T14:33:50Z

@vomikan you made quite a few changes in f57918f when we briefly switched to the v6 API. Do we need to revert any since we moved back to v7?

vomikan · 2023-06-17T16:51:01Z

@vomikan you made quite a few changes in f57918f when we briefly switched to the v6 API. Do we need to revert any since we moved back to v7?

This changes have been made to check JSON Integrity. It shouldn't affect. But the probability of error, as practice shows, is always there.

n-stein · 2023-06-17T20:08:24Z

I think the issue could be here:

https://github.com/moneymanagerex/moneymanagerex/blob/master/src/stockdialog.cpp#L483

When the user clicks the save button whatever value is in the current price field gets applied to whatever date is in the history date text box. So if you click a line in the history panel and click save it will overwrite that line with the current price. This is why you see the same value repeated multiple times, it is overwriting on the save click. I don't think we should be doing this.

n-stein · 2023-06-17T20:17:33Z

@Heny67 If you simply download rates then view them (don't click the save or add buttons) is the price correct?

reddyshyam · 2023-06-18T05:01:18Z

I think the issue could be here:

https://github.com/moneymanagerex/moneymanagerex/blob/master/src/stockdialog.cpp#L483

When the user clicks the save button whatever value is in the current price field gets applied to whatever date is in the history date text box. So if you click a line in the history panel and click save it will overwrite that line with the current price. This is why you see the same value repeated multiple times, it is overwriting on the save click. I don't think we should be doing this.

Sorry to butt in but this is still a bug, isn't it?

vomikan · 2023-06-18T12:34:18Z

This case closed! Moving the discussion there.

#6030

n-stein · 2023-06-18T12:37:16Z

this is still a bug, isn't it?

Yes, I think that line should be using Today instead of the date from the history text control, i.e. if the user changes the current price and clicks save it updates today's price in the history data to whatever the user entered in the Current Price box, not some other historical date.

We will continue in #6030.

Heny67 · 2023-06-19T11:57:40Z

@Heny67 If you simply download rates then view them (don't click the save or add buttons) is the price correct?

Hi n-stein!

In the Edit securities investment menu, I clicked on the update rate button, the update share history window appeared, I pressed Ok and the data is not updated. The last data is from 16.06.2023. I tried with several shares.
It worked until now.

Money Manager Ex
• Verzió: 1.6.3 64 bites
• Készítési időpont: jan. 31 2023, 03:26:11
• Adatbázis verziója: 17 (aes128cbc)
MMEX a következő támogatási termékeket használja:
• wxWidgets 3.1.7 (wxGTK 3.24)
• wxSQLite3 4.9.1 (SQLite 3.40.0)
• RapidJSON 1.1.0
• Lua 5.3.6
• libcurl/7.81.0 OpenSSL/3.0.2 zlib/1.2.11 brotli/1.0.9 zstd/1.4.8 libidn2/2.3.2 libpsl/0.21.0 (+libidn2/2.3.2) libssh/0.9.6/openssl/zlib nghttp2/1.43.0 librtmp/2.3 OpenLDAP/2.5.14
• gettext 0.21
• apexcharts.js
Összeállítás a következőkkel:
• CMake 3.22.1
• GNU Make 4.3
• GCC 11.2.0
Fut a következőn:
• Ubuntu.22.04.jammy
• Ubuntu 22.04.2 LTS "jammy"
• Linux 5.19.0-45-generic x86_64
• (UTF-8)
• 0x0 0bit 96x96ppi

n-stein · 2023-06-19T17:13:01Z

Edit securities investment menu, I clicked on the update rate button, the update share history window appeared, I pressed Ok and the data is not updated. The last data is from 16.06.2023

As I mentioned up thread the history download function within the Edit Stock Investment dialog only pulls data through the prior day. You won't receive data for 6/17 until your system date is 6/18. If you want data for the current date you have to use the Tools > Download Rates menu path or press the refresh button at the bottom of the investment panel.

dobber1 closed this as completed May 7, 2023

dobber1 reopened this May 7, 2023

vomikan added the bug label May 7, 2023

vomikan added this to the v1.6.4 milestone May 7, 2023

vomikan added a commit to vomikan/moneymanagerex that referenced this issue May 8, 2023

fix(moneymanagerex#5894): switch to yahoo v6 api

f57918f

vomikan added a commit that referenced this issue May 9, 2023

Merge pull request #5898 from vomikan/master

a787d8a

fix(#5894): switch to yahoo v6 api

vomikan added the fixed label May 11, 2023

vomikan closed this as completed May 11, 2023

This was referenced May 16, 2023

Share update #5921

Closed

share update error #5890

Closed

MMEX version 1.6.3 64-bits #5910

Closed

tactilis mentioned this issue May 22, 2023

Can't Update Portfolio with the latest Price for shares in Money Manager EX #5932

Closed

vomikan reopened this May 26, 2023

vomikan added help wanted good first issue good issue for first-time contributors - please fix me ASAP and removed fixed labels May 26, 2023

n-stein mentioned this issue Jun 1, 2023

Online update in Currency Dialog doesn't work #5963

Closed

11 tasks

vomikan added a commit that referenced this issue Jun 2, 2023

Merge pull request #5965 from n-stein/fix(#5894)

230b58e

fix linux build

vomikan added fixed and removed help wanted good first issue good issue for first-time contributors - please fix me ASAP labels Jun 2, 2023

vomikan closed this as completed Jun 3, 2023

Can not update stock prices. Get invalid cookie error. #5894

Can not update stock prices. Get invalid cookie error. #5894

Comments

dobber1 commented May 7, 2023 • edited

Description of the bug

Reproduction

Additional information

renato-mmex commented May 7, 2023

vomikan commented May 7, 2023 • edited

vomikan commented May 7, 2023

n-stein commented May 8, 2023

n-stein commented May 8, 2023 • edited

reddyshyam commented May 8, 2023

vomikan commented May 8, 2023

n-stein commented May 8, 2023 • edited

reddyshyam commented May 9, 2023

n-stein commented May 25, 2023

n-stein commented May 26, 2023

thinkpad4 commented May 26, 2023

n-stein commented May 27, 2023 • edited

thinkpad4 commented May 28, 2023 • edited

n-stein commented May 31, 2023

thinkpad4 commented Jun 1, 2023

reddyshyam commented Jun 1, 2023

reddyshyam commented Jun 1, 2023

reddyshyam commented Jun 1, 2023

reddyshyam commented Jun 1, 2023

n-stein commented Jun 1, 2023

reddyshyam commented Jun 1, 2023

renato-mmex commented Jun 1, 2023

n-stein commented Jun 1, 2023

whalley commented Jun 1, 2023

Heny67 commented Jun 17, 2023 • edited

vomikan commented Jun 17, 2023

n-stein commented Jun 17, 2023

vomikan commented Jun 17, 2023

n-stein commented Jun 17, 2023

vomikan commented Jun 17, 2023

n-stein commented Jun 17, 2023

n-stein commented Jun 17, 2023

reddyshyam commented Jun 18, 2023

vomikan commented Jun 18, 2023 • edited

n-stein commented Jun 18, 2023 • edited

Heny67 commented Jun 19, 2023 • edited

n-stein commented Jun 19, 2023 • edited

dobber1 commented May 7, 2023 •

edited

vomikan commented May 7, 2023 •

edited

n-stein commented May 8, 2023 •

edited

n-stein commented May 8, 2023 •

edited

n-stein commented May 27, 2023 •

edited

thinkpad4 commented May 28, 2023 •

edited

Heny67 commented Jun 17, 2023 •

edited

vomikan commented Jun 18, 2023 •

edited

n-stein commented Jun 18, 2023 •

edited

Heny67 commented Jun 19, 2023 •

edited

n-stein commented Jun 19, 2023 •

edited