Invalid authentication credentials Google Analytics importer plugin

[niklasternstedt]

Expected Behavior

Possible to import data from Google Analytics without any errors.

Current Behavior

When importing data from Google Analytics I get this error message:
{ “error”: { “code”: 401, “message”: "Request had invalid authentication credentials. Expected OAuth 2 access token, login cookie or other valid authentication credential. See [https://developers.google.com/identity/sign-in/web/devconsole-project."](https://developers.google.com/identity/sign-in/web/devconsole-project.%22), “errors”: [ { “message”: “Invalid Credentials”, “domain”: “global”, “reason”: “authError”, “location”: “Authorization”, “locationType”: “header” } ], “status”: “UNAUTHENTICATED” } }

Your Environment

• Matomo Version: 4.6.2
• PHP Version: 7.4.3
• Browser: Brave
• Operating System: Mac iOS

[AltamashShaikh]

@niklasternstedt Have you followed all the steps mentioned here ?
Also can you try to re authorise Matomo and check ?

[JackySw]

Got the same problem. But not always. Sometimes it run fine until the expected rate-limit. But sometimes is crash with this error, same as niklasternstedt. Did a re authorize a few times, no difference. Have to start it manually every time after this. Can't figure out what the problem is since it does work half of the time.

Could it be that since Google Analytics 3 is ending soon, many people are using the export function that it is overloading?

[AltamashShaikh]

@JackySw In #337 a user noticed something that Google is sending Invalid Oauth instead of rate limit, it can be due to some change at Google's end, will check if we can find something on this more and just a Q the import if failed should start automatically after 1 hour, isn't that happening ?

[JackySw]

Thanks for looking into this. It seems that when the status is killed with this error, that it won't restart. At least, I waited over a day and no new dates were imported. When I clicked the start button again, sometimes it gives the error directly again, but many times it just resumed without a problem till it errored out again. Sometimes after hours, sometimes after 10 minutes or so. It feels a bit random, therefor hard to point into a direction for me.

[nicksherman]

For what it’s worth, I’m also getting similar errors with a more current setup:
Matomo 4.13.3
GoogleAnalyticsImporter v4.5.0 (as installed via the Plugin Marketplace)
MySQL version 8.0.28-0ubuntu0.20.04.3
PHP 8.1 FastCGI

As with others on this thread, it's happening well before 7 days from authentication. Also similarly, if I re-authenticate it seems to temporarily fix the issue but usually only for another short period of time (usually less than a day).

[MatomoForumNotifications]

This issue has been mentioned on Matomo forums. There might be relevant details there:

https://forum.matomo.org/t/analytics-import-error/41568/20

[AltamashShaikh]

@nicksherman @JackySw Can you confirm following things

1. No scopes are defined
   

2. You App is in testing mode
   

If you see no scopes and app in testing mode, can you publish the app and reauthorise via Matomo GA Import UI ? and check if this issue still persist ?

Also can you check the status value in DB
Query - SELECT * FROM matomo_option WHERE option_name='GoogleAnalyticsImporter.importStatus_{idsite}'

The import should start automatically as we have a Task which check and restarts all the failed imports

[nicksherman]

@AltamashShaikh Thanks for lending a hand! To answer your questions:

1. I can confirm there are no scopes defined.
2. The app is in Production mode, not Testing mode. I set it that way according to steps 15 & 16 from the setup guide. Should I change that? (It is an “External” app.)

I have 3 separate sites set up with GA importing in my Matomo installation, but the importStatus value for all of them matches the following (just with different dates, site IDs, and GA property specifics – the latter of which I've redacted here for privacy):

{"status":"rate_limited","idSite":2,"ga":{"import_type":"Universal Analytics","property":"[redacted]","account":"[redacted]","view":"[redacted]"},"last_date_imported":"2021-10-13","main_import_progress":"2021-10-13","import_start_time":1677109702,"import_end_time":null,"last_job_start_time":1677623299,"last_day_archived":"2021-10-13","import_range_start":null,"import_range_end":null,"extra_custom_dimensions":[],"days_finished_since_rate_limit":0,"reimport_ranges":[],"isGA4":false,"error":"Error on day 2021-10-12, {\n  \"error\": {\n    \"code\": 401,\n    \"message\": \"Request had invalid authentication credentials. Expected OAuth 2 access token, login cookie or other valid authentication credential. See https:\/\/developers.google.com\/identity\/sign-in\/web\/devconsole-project.\",\n    \"errors\": [\n      {\n        \"message\": \"Invalid Credentials\",\n        \"domain\": \"global\",\n        \"reason\": \"authError\",\n        \"location\": \"Authorization\",\n        \"locationType\": \"header\"\n      }\n    ],\n    \"status\": \"UNAUTHENTICATED\"\n  }\n}\n"}

[AltamashShaikh]

@nicksherman No need to change anything

[AltamashShaikh]

@nicksherman I see its rate_limited, do you see any value when you run this query ?
SELECT * FROM matomo_option WHERE option_name='GoogleAnalyticsImporter_nextAvailableAt'
and what does the UI show for above import?

[nicksherman]

@AltamashShaikh I'm guessing the rate_limited is because I managed to get the import(s) running again by re-authenticating, so they presumably hit their daily limit(?). If the previous few days have been any indication, the authentication will fail again the next time the import tries to restart.

The value for nextAvailableAt is 1677628800 (i.e. 2023-02-28 at 7 PM).

This is what the Matomo UI looks like (with site names and GA specifics redacted):

Is it expected that there is only one nextAvailableAt field in the database, even though I have 3 separate imports running?

[AltamashShaikh]

@nicksherman The UI is displaying correctly you have hit a Daily rate limit and the import will start after nextAvailableAt i.e 2023-02-28 at 7 PM for you and yes there should be only 1 value for nextAvailableAt since all the 3 imports uses the same Oauth and rate_limiting on 1 will apply to others also.

Can you check after 2023-02-28 at 7 PM whether the import runs automatically or not for you ?

[nicksherman]

@AltamashShaikh Since I'm in the Eastern US time zone, that nextAvailableAt time has already passed (right now it's 2023-02-28 at 11:32 PM here).

Should I force the imports to resume from the Actions buttons in the UI? In the past I've had mixed luck with that. Usually it runs for a bit before hitting another error.

[AltamashShaikh]

@nicksherman you have auto archiving cron setup right?

Link to FAQ

[nicksherman]

@AltamashShaikh I haven't set up an archiving cron via a server script (that wasn't mentioned in the GA Importer setup guide). Following the recommendations in General Settings > Archiving Settings, I do have Matomo set to "Archive reports when viewed from the browser", at most every 900 seconds (the default settings).

Is this something I should change? My server does support cron jobs.

[AltamashShaikh]

@nicksherman This is the reason, why the import doesn't resume after an error and needs a manual resume.
I need to check if tasks are executed when browser archiving is enabled, I will get back to you on this and accordingly will update the setup guide.

If you have no problem in setting a cron, I would recommend you to do that else you can manually restart the import for now

[nicksherman]

@AltamashShaikh Ok, I had some trouble getting the proper cron script working but I've at least managed to set up a web cron for now.

For what it's worth, and at the risk of going too far off the topic of this thread, the instructions on the auto-archiving instructions page are a bit misleading. When instructing how to find your token_auth code, they say:

Replace the XYZ by the super user 32 characters token_auth. To find the token_auth, log in as a super user in Matomo, click on Administration link in the top menu, then click the API link in the left and the token_auth is displayed on the page.

But there are a couple issues with that …

1. The “API” link isn't visible immediately after clicking the Administration link in the top menu. As a sub-section of the Platform section, you have to click "Platform" first.
2. Once you do get to the API page, the token_auth isn't actually displayed on the page. Instead there is a link to the Security page where you can manage tokens.
3. However, even when you get the the Security page, the token codes are still not shown. There is a listing of the tokens that have been made and when, with options to delete them, but the actual 32-character codes aren't shown.

As far as I can tell, there is no way to retrieve an existing 32-character token_auth code. I was able to get around this by creating a new token – the related code is shown in the process, but after that it is buried like the other existing tokens.

[nicksherman]

@AltamashShaikh In an interesting turn of events, I am still hitting a rate_limited barrier after triggering the imports to resume manually (even though, as we discussed, the current time is after the nextAvailableAt value). So perhaps there's an issue with how the nextAvailableAt value is set?

In the mean time I'll leave my web cron in place and see if/when that has any luck.

[AltamashShaikh]

@nicksherman so what the plugin does is whenever we encounter a 429 exception it checks if the exception message is for daily/hourly and accordingly saves the values in nextAvailableAt as +1 hour or +1day and whenever the import runs it checks this option_value and decide whether to proceed or not, I think the import might have ran again and faced the rate_limit.

Can you run this command and check the output ?
./console googleanalyticsimporter:import-reports --idsite={importIdSite}

[nicksherman]

@AltamashShaikh When I run the import-reports command, the output says:
The import was rate limited and will be restarted automatically at 2023-03-02 12:00 am

Is it unusual that the restart date is more than 24 hours after the most recent round of successful imports?

[AltamashShaikh]

@nicksherman I checked the code Its in UTC timezone, is that correct as per UTC ?

[nicksherman]

@AltamashShaikh Ah, yes that makes sense. In the mean time, however, my imports are still stalled as rate_limited even when trying to manually restart the process from the Actions buttons. This is actually a step backward from the previous situation :\

[AltamashShaikh]

@nicksherman It did run for today right automatically till it got a rate_limit ?

[nicksherman]

@AltamashShaikh Ah yes, apologies, I mistakenly was looking at an old tab where the Last Day Processed dates hadn't been updated yet.

This is a bit of a tangent, but is there some way to pause one import job without cancelling it entirely? That would be handy to allow more requests to be used on higher-priority import jobs first.

[AltamashShaikh]

@nicksherman I can think only of this solution for now quickly
Add the below code after this line in plugins/GoogleAnalyticsImporter/Commands/ImportReports.php

if ($idSite != {YOUR_ID_SITE}) {
            $output->writeln('Skipping import for idSite:' . $idSite);
            return;
        }

[nicksherman]

@AltamashShaikh Ok I may give it a shot depending on how long it takes to process these imports (it's annoying that Google no longer accepts quota increase requests for Google Analytics APIs).

Thank you so much for your help with all this stuff. I would have been totally lost otherwise so I really appreciate it a lot!

[AltamashShaikh]

@niklasternstedt @JackySw Can we close this issue ?

[JackySw]

Hi, I tested it for a few days now and so far so good! Very happy with it. I saw once an error, but it did resume. Now I've seen the rate_limited status for 3 days in a row (of course only when it hits the limit, so this is good) and it keeps importing. From this side, no problems anymore. Very happy with the outcome.

[AltamashShaikh]

@JackySw good to hear this 😄
Closing this issue for now

[MatomoForumNotifications]

This issue has been mentioned on Matomo forums. There might be relevant details there:

https://forum.matomo.org/t/analytics-import-error/41568/22