Please set the user agent when doing external HTTP requests

[olasd]

Description

The plugin to retrieve license information detected by Software Heritage uses the default PHP GuzzleHttp client user agent, which makes it harder than it should be to identify it.

I don't know if there's other uses of HTTP clients in Fossology, but it would be nice if Fossology identified itself as such when crawling external resources. An ideal user agent string would:

• clearly identify itself as fossology
• provide version information on the fossology instance
• and, if possible, identify an admin contact for the fossology instance.

Thanks for considering,
Nicolas Dandrimont - Staff Engineer @ Software Heritage

How to reproduce

Unknown; This has been noticed on the receiving end of Fossology's crawling!

Versions

Unknown, because of the issue being reported.

Logs

Any logs (if any) generated in

Remote web server logs

::1 - - [23/Nov/2020:11:49:49 +0000] "GET /api/1/content/sha256:616BD8A01B0066AA1F8883B2AB2ADC1DD930CD4403E8E6AEC748016BA2B9E441/license/ HTTP/1.1" 302 - "-" "GuzzleHttp/6.3.3 curl/7.52.1 PHP/7.0.33-0+deb9u10"
::1 - - [23/Nov/2020:11:49:50 +0000] "GET /api/1/content/sha256:616bd8a01b0066aa1f8883b2ab2adc1dd930cd4403e8e6aec748016ba2b9e441/license/ HTTP/1.1" 429 95 "-" "GuzzleHttp/6.3.3 curl/7.52.1 PHP/7.0.33-0+deb9u10"
::1 - - [23/Nov/2020:11:50:25 +0000] "GET /api/1/content/sha256:616BD8A01B0066AA1F8883B2AB2ADC1DD930CD4403E8E6AEC748016BA2B9E441/license/ HTTP/1.1" 302 - "-" "GuzzleHttp/6.3.3 curl/7.52.1 PHP/7.0.33-0+deb9u10"
::1 - - [23/Nov/2020:11:50:27 +0000] "GET /api/1/content/sha256:616bd8a01b0066aa1f8883b2ab2adc1dd930cd4403e8e6aec748016ba2b9e441/license/ HTTP/1.1" 429 95 "-" "GuzzleHttp/6.3.3 curl/7.52.1 PHP/7.0.33-0+deb9u10"
::1 - - [23/Nov/2020:11:51:01 +0000] "GET /api/1/content/sha256:616BD8A01B0066AA1F8883B2AB2ADC1DD930CD4403E8E6AEC748016BA2B9E441/license/ HTTP/1.1" 302 - "-" "GuzzleHttp/6.3.3 curl/7.52.1 PHP/7.0.33-0+deb9u10"
::1 - - [23/Nov/2020:11:51:01 +0000] "GET /api/1/content/sha256:616bd8a01b0066aa1f8883b2ab2adc1dd930cd4403e8e6aec748016ba2b9e441/license/ HTTP/1.1" 429 95 "-" "GuzzleHttp/6.3.3 curl/7.52.1 PHP/7.0.33-0+deb9u10"

FOSSology logs

Not applicable

Apache logs

Not applicable

Job logs

Not applicable

[GMishx]

Hello @olasd , thanks for reporting the issue.

I have created PR #1837 which adds the User-Agent to the HTTP requests. Currently I have set it as "fossology/", example "fossology/3.9.0-rc2". Does this sounds helpful?

And for the server contact information, under Admin >> Customize, we have a field called "Support Email Address" which can be used here. But I am not sure if everyone will be okay to send this information. Therefore, I am keeping this open here for discussion.