You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
v1.0.1: Network binding, no telemetry, no crawler disguise
APP_BIND and APP_PORT in .env choose where the admin app is
published (default 127.0.0.1:5000, unchanged). Set APP_BIND=0.0.0.0 to
reach it from other machines on your network, the same way MCP_BIND
works for the MCP server; no override file needed. Suggested on Reddit by
u/geo38.
Changed
Meilisearch telemetry is off (MEILI_NO_ANALYTICS). It sent anonymous
usage data to Meilisearch by default. Spotted by u/geo38. Takes effect when
the container is recreated (./install.sh, or docker compose up -d meilisearch).
CNN Underscored pages are refused at ingestion (/cnn-underscored/):
they are shopping reviews with affiliate links, not news. They also answer
403, and one of those was enough to put all of cnn.com on a six-hour
scrape hold.
Removed
The scraper no longer poses as Googlebot or opens paywalled sites in a
headless browser. Seven sites were fetched with Google's crawler user
agent, and seven paywalled ones (NYT, WSJ, Washington Post, FT, Bloomberg,
The Athletic, Wired) got a headless Chromium attempt. MuckScraper now reads
only what an ordinary browser can, and a paywalled article stays
headline-only. In practice little is lost: over 30 days the crawler user
agent fetched 81 articles and the headless browser none.