Could not fetch newsfeed - Error: Forbidden

[Selyjohns]

Hello,

Platform: RPi 3 on Raspbian

Node Version: 16

MagicMirror Version: 2.17.1

Description: I’ve an error while i want to fetch https://actu.fr/toulouse/rss.xml on newsfeed.

Steps to Reproduce: Just add this url on newsfeed module

Actual Results:

0|mm  | Newsfeed Error. Could not fetch newsfeed:  https://actu.fr/toulouse/rss.xml Error: Forbidden
0|mm  |     at NodeHelper.checkFetchStatus (/home/pi/MagicMirror/js/node_helper.js:121:9)
0|mm  |     at processTicksAndRejections (internal/process/task_queues.js:93:5)

It's ok with some others XML :)
Thank you for your help.

[khassel]

This seems to be a problem of the provided User-Agent header.

Someone coded this line with a funny header containing node version, mm version and more.

With this header the fetch is blocked by actu.fr (and maybe by other newsfeed providers).

If I change this line to "User-Agent": "Mozilla/5.0", the above feed works.

So question at other contributors: Do we need all the stuff in current "User-Agent" or can we just provide "Mozilla/5.0"?

[rejas]

Seems like this PR #559 made the change, maybe @roramirez can weigh in on it.

I'd guess that if the simple Mozilla header works, we should use it.

[khassel]

Did some more tests:

works: "User-Agent": "Mozilla/5.0 (Node.js " + nodeVersion + ") MagicMirror/" + global.version + " (https://google.com)",
works: "User-Agent": "Mozilla/5.0 (Node.js " + nodeVersion + ") MagicMirror/" + global.version + " (https://gitlab.com)",
fails: "User-Agent": "Mozilla/5.0 (Node.js " + nodeVersion + ") MagicMirror/" + global.version + " (https://github.com)",

So github.com is blocked by the provider. So may we should omit the url part.

[rejas]

Sounds reasonable! +1 from me!

[Selyjohns]

Amazing reactivity, thank you guys 👏🏻

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.