Skip to content

VoucherBot v1.0.0

Choose a tag to compare

View all tags
@Devathmaj Devathmaj released this 26 Jul 11:11
· 62 commits to main since this release
v1.0.0
97ee6e2

v1.0.0 — Initial Release

This is the first stable release of VoucherBot — an intelligent certification voucher aggregator that continuously monitors community and official sources for certification discounts, free exam opportunities, beta exams, and promotional campaigns, then delivers them straight to your inbox.

🌟 Highlights

  • AI-powered voucher detection — dual-provider pipeline: Groq (primary) + Google Gemini (fallback)
  • 70+ monitored sources across RSS feeds, blogs, forums, events, Pearson VUE pages, and training providers
  • Automated email notifications via Resend when a voucher is found
  • Seamless cloud deployment — Docker image published to GHCR, ready for Render
  • CI/CD pipeline — lint, type-check, test, build, and release on tag push
  • End-to-end test mode — spin up a local test server to verify the full pipeline

🚀 Features

Core Pipeline

  • Scheduler-driven ingestion loop — sequentially processes all due sources, sleeps until next due time (capped at 6 h)
  • Five-stage pipeline: collect → keyword filter → dedup/upsert → AI extraction → event matching → email notification
  • PostgreSQL-based lease system prevents duplicate processing across app instances
  • Exponential-backoff retry with advisory locking during bootstrap

Data Sources (70+)

Category Count Examples
Vendor blogs 15 AWS Training, Microsoft Learn, Google Cloud, Cisco, Red Hat, Linux Foundation, Oracle, HashiCorp, Docker, Elastic, SUSE, Canonical, Ubuntu, SAS, Neo4j
RSS feeds 16 The Register, TechTarget, Petri IT, InfoQ, Confluent, Tutorials Dojo, Packet Pilot, Microsoft Blog, Cloud Academy (disabled), Certiport, Databricks, CNCF, LF Events, etc.
Pearson VUE pages 10 AWS, Microsoft, Cisco, CompTIA, VMware/Broadcom, Fortinet, Palo Alto Networks, Salesforce, ServiceNow, Splunk
Community forums 2 Microsoft Learn Q&A, Google Cloud Training Group
Events 5 Microsoft Cloud Skills Challenge, AWS events (disabled), AWS re:Invent (disabled), Google Cloud events, Google Cloud Next, Cisco Live (disabled)
Promo pages 3 CompTIA, ISC2 (disabled), Red Hat Training (disabled), MSFT Hub, VladTalksTech
Training providers 2+ Global Knowledge, Linux Foundation Training & Promotions

AI Analysis

  • Primary provider: Groq (llama-3.1-8b-instant)
  • Fallback provider: Google Gemini
  • Structured JSON output — fields: is_voucher, confidence, discount_amount, expiry_date, exam_code, promo_code, provider, summary
  • Token usage estimation and flexible hardening

Email Notifications

  • Resend integration with HTML templates
  • Sends alerts for new vouchers and possible matches
  • Configurable EMAIL_TO recipient

Database

  • PostgreSQL with async SQLAlchemy + asyncpg
  • Full schema: sources, posts, keywords, vendor mappings, pipeline lock, events
  • VoucherPost view for public preview page
  • Deduplication via identity hash (SHA-256 of URL) + content hash

🐛 Bug Fixes

  • PgBouncer connection drops — reduced pool_recycle from 240s → 60s to stay ahead of Supabase pooler server_lifetime kills; all sessions now re-apply statement_timeout = 120s after every commit (PgBouncer resets session settings on DISCARD ALL)
  • Dead sources removed — various non-functional feeds pruned
  • RSS feed URL normalisation — Microsoft Tech Community and Google Cloud Blog redirects handled
  • Pipeline edge cases — stuck post recovery, empty feed handling, keyword filter scoring fixes
  • Scheduler resource constraints — reduced CPU/memory footprint for low-spec systems
  • Advisory lock cleanup — stale leases are reset on every startup
  • Connection refused gracefully handled — WebsiteCollector logs a warning instead of an error when local test server is not running
  • Transient error detection improved — _is_transient() now walks the full exception cause chain to catch wrapped asyncpg errors

📚 Documentation

  • Full deployment guide for Render
  • Supabase, Groq, Gemini, Resend, and Reddit API setup guides
  • Architecture overview and data flow diagrams
  • Configuration reference for all env vars
  • Schema documentation
  • Testing guide with local test server walkthrough
  • Source list and descriptions
  • Contributing guidelines and PR templates
  • Shutdown safety guidance

🛠️ Operational

Deployment

  • Dockerfile for containerised deployment
  • Render Blueprint (render.yaml) for one-click deploy
  • UptimeRobot setup guide to prevent Render free-tier spin-down

CI/CD (.github/workflows/release.yml)

Triggered on v* tag push:

  1. Verify — runs against a real PostgreSQL service container:
    • ruff format --check .
    • ruff check .
    • mypy voucherbot tests
    • pytest
  2. Docker — builds and pushes image to ghcr.io
  3. Release — creates a GitHub Release with auto-generated notes

Configuration

All features are toggled via environment variables:

Variable Default Purpose
IS_PROD false DDL-only vs full bootstrap
IS_TEST false Seeds localhost test source
REDDIT_INGESTION_ENABLED — Toggles Reddit collection
SCRAPER_RESPECT_ROBOTS true robots.txt compliance
SCRAPER_MIN_DELAY_SECONDS 2.0 Minimum per-host crawl delay

🔜 Known Limitations

  • The Register RSS feed — returns a Proof-of-Work challenge page; marked as unsupported
  • Pearson VUE scraping — HTML structure can change without notice; pages are fetched and parsed, not API-driven
  • Reddit collection — requires manual API credential setup (disabled by default)
  • Local test server — single-threaded http.server; can become stuck if a request is interrupted

🧪 Testing

# Unit / integration tests
pytest

# Lint & formatting
ruff check . && ruff format --check .

# Type checks
mypy voucherbot tests

# End-to-end with local test server
# Terminal 1: python D:\components\server.py
# Terminal 2: uvicorn voucherbot.main:app --port 9000
# Set IS_TEST=true, IS_PROD=false in .env

Full commit log: https://github.com/Devathmaj/Certification/commits/v1.0.0

New Contributors

Full Changelog: https://github.com/Devathmaj/Voucher-Tracker/commits/v1.0.0

Contributors

Devathmaj
Assets 2
Loading
0 Join discussion