feat(citation-tracker): LLM citation tracker baseline + weekly cron + alerts — GEO/SEO Phase 1.5 + 5.1

[matheuspoleza]

Description

Daily measurement of how often 4 major LLMs (ChatGPT-search, Claude-web, Perplexity, Gemini) cite Vertz in response to targeted developer queries. This is the ground-truth metric for Goal 1 of the entire GEO/SEO strategy. Without it, we can't tell if anything is working.

Context

Part of GEO/SEO strategy v1 (28-day scope). Combines Phase 1 Task 5 (baseline) + Phase 5 Task 1 (weekly cron + alerts) because they share the same script + state.

• Design doc: plans/geo-seo-strategy.md
• Full task specs: plans/geo-seo-strategy/phase-01-foundation-infra.md Task 5, plans/geo-seo-strategy/phase-05-measurement-iteration.md Task 1
• Budget: $20/mo (weekly cadence, 12 queries × 4 providers = 192 API calls/month at ~$0.10/call)

Acceptance Criteria

BDD scenarios

describe('Feature: Citation tracker', () => {
  describe('Given the citation tracker runs on day 1 (baseline)', () => {
    describe('When the script executes 12 queries × 4 providers', () => {
      it('then a CSV is written with 48 rows (query_id, provider, mentioned, urls_cited, response_snippet)', () => {});
      it('then expected baseline: 0 mentions across all 48 cells', () => {});
    });
  });

  describe('Given the weekly cron is scheduled', () => {
    describe('When a new run completes', () => {
      it('then the CSV is committed to a private repo or S3', () => {});
      it('then a diff vs last week is computed', () => {});
      it('then new citations trigger a Slack alert immediately', () => {});
      it('then regressions (previously mentioned, now not) trigger a Slack alert', () => {});
    });
  });

  describe('Given a week with no changes', () => {
    describe('When the cron completes', () => {
      it('then no Slack alert fires (silence = no news)', () => {});
    });
  });
});

Checklist

• 12 baseline queries defined in scripts/citation-tracker/queries.ts covering: recommendation, comparison, gotcha, tutorial intents
• Provider wrappers: ChatGPT-search, Claude web-search, Perplexity Sonar, Gemini with grounding
• Day-1 baseline run committed (expected: 0 mentions)
• GitHub Actions workflow runs weekly at 6am BRT Monday
• Diff script compares two CSVs and flags changes
• Slack webhook integration for alerts
• README in scripts/citation-tracker/ explaining how to interpret + cost model
• Monthly cost ≤$25

Dependencies

• Blocked by: none (independent, can start day 1)
• Required credentials: Anthropic API key, OpenAI API key, Perplexity API key, Gemini API key, Slack webhook URL

Progress

• 2026-04-22: Opened