Skip to content

v0.5.0

Pre-release
Pre-release

Choose a tag to compare

View all tags
@grregis grregis released this 03 Jun 02:11
v0.5.0-beta
f9b88eb

MuckScraper v0.5.0 Beta

Release Date: June 3, 2026

MuckScraper v0.5.0 moves the project into beta. The core fetch, grouping, admin, and edition workflows are now usable end-to-end on a fresh install. The remaining rough edges are mostly around source balance, scrape quality, headline tuning, and runtime optimization rather than basic project viability.

What's New

Fresh Install Bootstrap

Fresh installs are now much easier. bootstrap_admin.py initializes the database, stamps Alembic current for a fresh schema, and creates or updates the admin user from .env.

New .env values:

ADMIN_USERNAME=admin
ADMIN_EMAIL=admin@example.com
ADMIN_PASSWORD=replace_with_a_real_admin_password

Recommended first-run flow:

docker compose up -d --build postgres meilisearch app
docker compose exec app python bootstrap_admin.py
docker compose up -d scheduler

Scrape Telemetry and Quality Checks

Scrape telemetry is now more complete across ingestion, retries, and admin tooling. MuckScraper stores scrape status, method, failure reason, HTTP status, and scrape outcome history over time.

Low-value article detection now catches more weak inputs, including roundups, live updates, video pages, and repetitive bad scrape output.

Admin and Operations Improvements

This release adds or improves:

  • centralized Admin Tools page
  • Meilisearch-backed admin search support
  • rebuild/status endpoints for search
  • expanded troubleshooting documentation
  • clearer run metrics for fetch and scrape checks

Documentation Cleanup

The README and project docs were cleaned up for public use. Personal integrations like Langfuse, n8n, and Matrix notifications are no longer part of the default Docker Compose setup.

They can still be added with custom environment variables, compose overrides, or notification code.

Fixes

  • fixed fresh installs failing when create_admin.py ran before the users table existed
  • fixed Compose warnings for removed personal Langfuse/n8n variables
  • fixed editions sometimes exceeding the 20-story target
  • fixed unchanged previous-edition stories repeating too aggressively
  • fixed title-only stories producing hallucinated summaries/deep reports
  • reduced duplicate same-event clusters in published editions
  • reduced excessive retry churn on chronically blocked scrape domains
  • cleaned up misleading public docs and deployment-specific references

Upgrade Notes

For an existing install:

git pull
docker compose up -d --build
docker compose exec app flask db upgrade

For a fresh install:

git clone https://github.com/grregis/MuckScraper.git
cd MuckScraper
cp .env.sample .env
# Edit .env
docker compose up -d --build postgres meilisearch app
docker compose exec app python bootstrap_admin.py
docker compose up -d scheduler

create_admin.py still exists, but bootstrap_admin.py is now the recommended first-run setup path.

Assets 2
Loading