Agentic Twitter Thread Creator ✅

An agentic, phase-based system for generating and refining social media threads (designed for Twitter). It uses a 5-step pipeline (Research → Architect → Hooks → Draft → Polisher) powered by dspy and agno, and exposes a FastAPI server for simple programmatic use.

---

🔧 Key Features

• Phase-based workflow for high-quality thread generation: Research, Architect, HookSmith, Drafter, Polisher. ✨
• FastAPI HTTP endpoints for: generating threads, editing thread drafts, and full post refinements. ⚡
• Robust LLM configuration with multi-key rotation (see dspy_module.py::MultiKeyGemini). 🔄
• Optional: compile training data and build a compiled model using compile_model.py (Supabase-backed). 📦
• Dockerfile for containerized deployment. 🐳

---

📁 Project Structure (high-level)

• main.py — FastAPI app and endpoints (/, /generate-thread, /edit-thread, /refine-full).
• run_workflow.py — The agentic Workflow orchestration (5-step pipeline). 🔁
• dspy_module.py — DSPy module & signatures for Architect, Hook, Draft, Polisher; LLM configuration and MultiKeyGemini.
• compiled_thread_creator.json — (Optional) compiled model weights used by ModelLoader.
• compile_model.py — Script to fetch training data (Supabase) and compile the dspy model.
• agents/ — Agent implementations (ResearchAgent, ArchitectAgent, HookSmithAgent, DrafterAgent, PolisherAgent, and helpers).
• utils/ — Utilities: model_loader.py, parsing.py (robust list parsing & LLM repair). 🔧
• Frontend/ — Minimal static UI (index.html) and prepare_frontend.ps1 for packaging.
• requirements.txt — Python dependencies.
• Tests & sanity checks: verify_*.py, test_google_search.py, test.py.

---

⚙️ Requirements & Environment

• Python 3.11 (recommended)
• Dependencies: install via:

python -m venv .venv
.\.venv\Scripts\activate  # Windows
pip install -r requirements.txt

Important environment variables

• gemini_api_key (required) – primary API key for the LLM (Gemini). You can add gemini_api_key_2, gemini_api_key_3, etc., for rotation.
• SERPAPI_API_KEY (recommended) – used by ResearchAgent for SerpApi (AI mode).
• SUPABASE_KEY (optional) – compile_model.py uses Supabase to fetch threads for compilation.
• PORT (optional) – port for server (defaults to 8080).

Note: Use a .env file or set environment variables in your deployment; never commit API keys to source control. 🔒

---

🚀 Running the API Server

Start the FastAPI server (development):

python main.py
# or
uvicorn main:app --reload --port 8080

Docker

Build and run with Docker:

docker build -t agentic-thread .
docker run -e gemini_api_key=YOUR_KEY -p 8080:8080 agentic-thread

---

🧪 Endpoints & Examples

1) Generate thread

POST /generate-thread

Request JSON:

{ "topic": "How to get better at debugging", "template": "Listicle", "platform": "Twitter" }

Response JSON (example):

{
  "template": "Listicle",
  "refined_topic": "How to debug faster as a developer",
  "hook": "Stop wasting time on console.log: try this instead",
  "tldr": "A fast debugging recipe",
  "cta": "Follow for more tips",
  "tweets": ["...", "..."]
}

cURL example:

curl -X POST http://localhost:8080/generate-thread \
  -H "Content-Type: application/json" \
  -d '{"topic":"AI agents in software engineering"}'

2) Edit thread (single-step edits)

POST /edit-thread

Request:

{ "tweets": ["tweet1", "tweet2"], "prompt": "Make these more concise and punchy" }

Response:

{ "edited_tweets": ["...", "..."] }

cURL example:

curl -X POST http://localhost:8080/edit-thread \
  -H "Content-Type: application/json" \
  -d '{"tweets":["First tweet","Second tweet"],"prompt":"Make them concise"}'

3) Full refine (hook + body + CTA)

POST /refine-full

Request:

{ "hook":"...","tweets":["..."],"cta":"...","prompt":"Improve the entire post focusing on clarity" }

Response: refined hook, tweets, cta.

cURL example:

curl -X POST http://localhost:8080/refine-full \
  -H "Content-Type: application/json" \
  -d '{"hook":"Old hook","tweets":["t1","t2"],"cta":"Old CTA","prompt":"Make it friendlier"}'

Docker & .env

You can pass secrets via an .env file and --env-file to docker run:

# Build
docker build -t agentic-thread .

# Run with .env (recommended to store keys in .env locally during testing)
docker run --env-file .env -p 8080:8080 agentic-thread

Frontend (static)

Prepare the static frontend for Netlify / static hosting (Windows PowerShell example):

powershell -File prepare_frontend.ps1
# This copies index.html + favicon.* to ./dist for drag-and-drop deploys

---

---

🧠 Model & Compilation Notes

• dspy_module.py contains DSPy Signatures and Modules used by agents (ArchitectSignature, BatchHookSignature, DraftThread, PolisherSignature, RefineFullPost).
• ModelLoader.get_model() tries to load compiled_thread_creator.json. If missing or load fails, the system runs in zero-shot/uncompiled mode.
• compile_model.py fetches examples from a (Supabase) database and compiles a model using DSPy teleprompt (requires SUPABASE_KEY and working DB access).

---

🔬 Tests & Debugging

• verify_optimization.py — hits /generate-thread to check for expected keys and latency. Start the server first, then run:

python verify_optimization.py

• verify_fix.py — checks parsing heuristics in utils/parsing.py:

python verify_fix.py

• test_google_search.py — manual SerpApi debug script (requires a valid SERPAPI_API_KEY):

python test_google_search.py

• test.py — minimal direct DSPy run (may refer to legacy functions; review before use). Use cautiously and inspect the imports first.

Notes:

• Some tests assume a running local server at localhost:8080 (e.g., verify_optimization.py).
• Many scripts will degrade to mock outputs if the related API keys are missing — ensure env vars are set when running tests that depend on external services.

---

Module & Agent Summary (short)

• ResearchAgent — distills user topics into search queries; manages SerpApi key rotation.
• ArchitectAgent — refines the topic, chooses a template, and creates an outline.
• HookSmithAgent — generates multiple (5) hook options.
• DrafterAgent — turns outline + hooks into a threaded draft.
• PolisherAgent — edits the thread, produces TL;DR and CTA; supports refine_full_post for manual instructions.
• utils.parsing.safe_parse_list — robust list parsing with optional DSPy LLM repair.
• utils.model_loader.ModelLoader — singleton loader for the AgenticThreadCreator and compiled weights.

---

Development & Contributing

• Keep API keys out of git. Use .env and the .env.example template.
• Follow the pipeline: if you add a new DSPy Signature, update dspy_module.py, agent code, unit tests, and the workflow in run_workflow.py.
• If you update signatures, consider re-running compile_model.py with a proper dataset.

---

Known Issues / TODOs

• test.py imports generate_thread_from_topic which may be a leftover; verify or update tests.
• Some code paths fall back to mock outputs if API keys are missing (e.g., Research step). Document required env vars before production runs.

---

License

Licensed under the MIT License. See LICENSE for details.

---

Questions / Next Steps

If you want, I can:

• Add a LICENSE file (MIT/Apache) and a short CONTRIBUTING.md. 💡
• Add a quick Makefile or invoke tasks for common dev flows (run server, run tests, build Docker). 🔧
• Create minimal automated tests for the workflow steps.

---

Thanks — this README is a first pass based on the codebase. Tell me which sections you'd like expanded or whether you want extra examples, badges, or a wiki-style breakdown of each DSPy Signature.