Project Argus

🔒 PRIVATE REPOSITORY - Full Source Code

OSINT pipeline with advanced TikTok scraping and captcha solving capabilities.

⚠️ Important: Open Core Strategy

This repository contains the complete premium version with:

• Advanced humanized mouse movement (Bézier curves, overshoot, tremor)
• TLS fingerprinting and anti-detection
• Gaussian delay distributions
• 80-85% captcha success rate

For open source version: See OPEN_CORE_STRATEGY.md for how to extract the basic version.

---

Services

• Discovery (Go): TikTok scraper with advanced captcha solving
• Vision (Python): OpenCV-based captcha detection (can be open sourced)
• Parser (Go): Discord invite extraction and metadata persistence
• Infrastructure: NATS, Redis, PostgreSQL, Meilisearch

---

Quick Start

1. Configure

cp config/config.example.yaml config/config.yaml
# Edit config.yaml with your settings

Premium Features are controlled via config.yaml:

• Set captcha.humanized_movement.enabled: true for advanced movement
• Set captcha.delays.type: "gaussian" for anti-detection timing

2. Setup & Start Infrastructure

make setup  # Install all dependencies
make up     # Start Docker services (NATS, Redis, PostgreSQL, Meilisearch)

3. Run Services

# Terminal 1: Captcha Solver (OpenCV)
make run-captcha-solver

# Terminal 2: Discovery (TikTok Scraper)
make run-discovery

# Terminal 3: Parser (optional - for Discord extraction)
make run-parser

4. Test Captcha System

make test-captcha  # Automated test with Vision + Discovery

---

make test-scraper-browser # E2E: Tests Scraper browser automation (Playwright) make test-vision-job # Unit/Integration: Tests Vision job processing mock make send-payload # Manual: Sends a fake payload from Vision -> Parser

### 5. Verification

Check if data was inserted into Postgres:

```bash
docker exec -i banco-argus-dev psql -U argus-user -d argus-post-db -c "SELECT source_url, discord_invite_code, LEFT(raw_ocr_text, 50) as preview FROM artifacts ORDER BY processed_at DESC LIMIT 5;"

Available Make Commands

• make up: Start infrastructure
• make down: Stop infrastructure
• make logs: Detailed logs of infra
• make setup: Install all dependencies
• make run-parser: Run Parser service
• make run-scraper: Run Scraper service
• make run-vision: Run Vision service
• make test-full: Run integration full flow test
• make test-scraper-browser: Run scraper browser test
• make test-vision-job: Run vision job test
• make send-payload: Send manual test payload
• make help: List all commands

Troubleshooting

• If Parser fails to connect to Postgres with a Unix socket error, ensure the DB URL uses 127.0.0.1 and not localhost.
• Ensure containers are running: docker compose ps
• If running in open-core mode, set credentials via env vars (e.g. DATABASE_URL) instead of committing them in config.yaml.