Archiving sporadically process fails with curl_exec issue (blocking whole processing) - CURL HTTP/2 configurable?

[fe-hicking]

Expected Behavior

Archiving process should execute tasks and gracefully handle exceptions on single API calls

Current Behavior

Executing the archiving process currently chokes if a single API request may fail.

Possible Solution

Maybe allow a setting to indicate if an exception should stop the whole execution. Also maybe allow to configure CURL options.

Steps to Reproduce (for Bugs)

If an API HTTP call fails (we cannot reproduce why yet), an error like this is created in the archiving cronjob output:

DEBUG [2022-11-23 01:17:17] 3204  Execute HTTP API request: https://tracking.domain.de/?module=API&method=CoreAdminHome.archiveReports&idSite=1&period=year&date=2022-01-01&format=json&segment=visitConvertedGoalId%3D%3D2&trigger=archivephp&
Uncaught exception in C:\inetpub\wwwroot\plugins\Monolog\Processor\SprintfProcessor.php line 24:
Unknown format specifier "D"

This is actually caused by an issue with the SprintfProcessor.php because it passes a raw error string to vsprintf() and the URL encoded "%3D" is tried to be interpreted, but not a valid printf command.

SprintfProcess.php actually gets this as input:

$message = "Got invalid response from API request: https://tracking.domain.de/?module=API&method=CoreAdminHome.archiveReports&idSite=1&period=year&date=2022-01-01&format=json&segment=visitConvertedGoalId%3D%3D2&trigger=archivephp&. Response was 'curl_exec: HTTP/2 stream 1 was not closed cleanly before end of the underlying stream. Hostname requested was: tracking.domain.de'";
$parameters = array('exception' => 'Exception');

So actually two things:

1. The actual error is not shown in the archiving process, because the Monolog error reporting chokes on the current input string. I wasn't able to deduce where exactly this gets passed to monolog. You may probably want to str_replace the '%' character before passing URLs to Monolog?
2. Now that we have the actual error message, we were able to see that curl_exec has an issue. We are using recent PHP8.1 on Windows with an IIS. Is there anyway to further debug why the curl request may fail? Or as an alternative, is there a way to configure Matomos curl to not use HTTP/2 but HTTP/1, without patching core files? Or, without knowing the inner workings of Matomo - would it be possible to perform Archiving not with a HTTP request but from the CLI call of the console, which is how the whole archiving already should take place?

Context

(Sorry I wasn't able to split the bug and context from each other, so my response to this is contained above.

Your Environment

• Matomo Version: 4.12.3
• PHP Version: 8.1.10
• Server Operating System: Windows Server 2016 1607 (OS Build 14393.5356), IIS 10.0.14393.0
• Browser: N/A
• Operating System: N/A

(Thank you!)

[peterhashair]

@fe-hicking thanks you for reporting this, but I don't think we can easily convert from HTTP/2 to HTTP/1, I believe you can look for the done keyword during the Archiving curl, otherwise there will be an error. But you are right, we should improve the archiving endpoint maybe with an error log send to an email or webhook.

[fe-hicking]

Thanks for getting back to me!

From what I gathered, in the core/Http.php file the method sendHttpRequestBy() seems to be the only place, where curl_init is called.

Reading https://www.php.net/manual/en/function.curl-setopt.php it seems that curl_setopt supports a constant CURLOPT_HTTP_VERSION which could be conditionally set to the constant CURL_HTTP_VERSION_1_1, depending on a matomo config version?

However: This is only a part of my actual problem. I figure that the HTTP/2 error actually stems from possibly IIS server issues.

My reasoning is, that Matomo should in case of a curl error:

• First: Ensure no PHP fatal error is thrown due to current Monolog vsprintf() issue (IMO this should be considered a bug)
• Second: Ideally, if one Exception occurs of one archiving API call, the whole archiving process should not fail completely, but instead continue and just properly log the one occurence as en error? (this is more a feature request, one could say)

Since I have only basic knowledge of Matomos inner workings I'm not able to provide a PR on my own which would cover edge cases that core developers may be aware of.

Best regards,
Garvin