You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Node v18.12.1, rss-parser@3.12.0. I am scraping a number of RSS feeds using rss-parser. A Nasdaq feed is invariably giving a timeout after 10 seconds, which is the configured time:
Could not read https://www.nasdaq.com/feed/rssoutbound?category=Commodities: Request timed out after 10000ms
The feed can be read without issues on a browser so I guess it is limited by user-agent on the server side. Anyway, my program does not terminate properly, and keeps the process open after everything else has been closed. Running it with wtfnode yields the following information:
As a workaround fetch the RSS's xml yourself with node's built-in fetch (since v18), axios, etc.
Example:
constaxios=require('axios');constRSSParser=require('rss-parser');constparser=newRSSParser();asyncfunctionaxiosExample(url){try{constresponse=awaitaxios.get(url);constfeed=awaitparser.parseString(response.data);}catch(error){console.error("Error fetching the RSS feed:",error);}}asyncfunctionfetchExample(url){){try{// you can also use 'node-fetch' const fetch = require('node-fetch')constresponse=awaitfetch(url);if(!response.ok){thrownewError(`Unexpected response ${response.statusText}`);}constxml=awaitresponse.text();constfeed=awaitparser.parseString(xml);}catch(error){console.error("Error fetching the RSS feed:",error);}}
Node v18.12.1, rss-parser@3.12.0. I am scraping a number of RSS feeds using
rss-parser
. A Nasdaq feed is invariably giving a timeout after 10 seconds, which is the configured time:The feed can be read without issues on a browser so I guess it is limited by user-agent on the server side. Anyway, my program does not terminate properly, and keeps the process open after everything else has been closed. Running it with wtfnode yields the following information:
As it happens, 23.214.214.41 is (not coincidentally) the IP address of the Nasdaq Akamai endpoint:
It appears that
rss-parser
is not closing the socket properly after a timeout. Any ideas?The text was updated successfully, but these errors were encountered: