v0.3.0 — Promote feeds itself, and publishes once

397 commits since v0.2.0. Headline change: Promote campaigns now feed themselves and publish at most once — a campaign can be pointed at a keyword or a feed instead of a hand-pasted list, and a publication can no longer go out twice.

Promote — campaigns that feed themselves

A campaign used to be a fixed set of URLs someone typed in. It can now subscribe to sources: a keyword becomes an RSS Amplifier topic feed (bitcoin → https://rssamplifier.com/topics/bitcoin.rss), or any RSS/Atom URL you own or follow. Several keywords become several independent sources, never one combined URL.

Feeds are fetched once across all subscribers — two hundred users tracking "bitcoin" poll RSS Amplifier once between them, with ETag/Last-Modified conditional requests and geometric backoff on failure.

Campaigns blend owned and shared content on a configured ratio. Selection is deficit-based, not weighted-random: weighted random produces streaks, and a run of other people's content is what makes an automated account read as a content farm. A 70/30 campaign lands on 70/30 with no run longer than three.

A class the campaign has no inventory and no source for is not treated as starved — it is simply not part of that campaign's mix. Without that rule, an all-owned campaign reads the default 70/30 as "30% short on shared", covers with its own links, marks them as fallbacks, and goes silent when the daily fallback cap trips.

Promote — a publication happens at most once

The drip sweep decided and published in one pass, "claiming" a campaign by pushing next_run_at forward with no predicate on the value it had just read. That is a read-then-write: two sweeps that both read the row both won it. The worker runs the sweep on a 60s interval and out-of-band whenever someone clicks "Post now", so overlapping runs are designed in.

Nothing downstream was idempotent either. A crash between the publish call and the attempt record left no trace it had happened, and the link was still least-recently-promoted on the next tick — so it went out again. The user saw a duplicate; the logs showed nothing.

promo_job makes the intended publication the unit of work, written down before anything is sent:

• Planned before publishing, keyed on sha256(campaign, link, account, destination, kind, slot) behind a unique index. The slot is the next_run_at the sweep observed as due, not the wall clock — wall clock would give each racing sweep its own key and rebuild the bug. The loser gets nothing back and publishes nothing.
• Claimed by compare-and-swap (where state = 'queued'), so read and write are one statement and two workers cannot both take a job.

At most once, deliberately. A job interrupted mid-publish is failed, never retried. No provider we publish through accepts an idempotency key, so an interrupted publish has genuinely unknown outcome — the post may be live, and re-running it is the duplicate this exists to prevent. It is closed with the outcome recorded as unknown and left in history for a human. The credit is not refunded, because refunding a post that did land is the other way to be wrong.

Ads

• A feed ad format for RSS, Atom and JSON Feed, with artwork, a brand line, and generated prose for ads that live inside content rather than beside it.
• Impression accounting corrected: prefetch no longer reads as delivery, repeat impressions are flagged, the dedupe window is 5s rather than 60s, and free-tier delivery is counted in the stats box.
• Visitor id shared with /ad.js, with salted and rotated IP hashes.

Careers

A server-rendered /careers page and an application widget shipped as an opt-in stats.js module, with spam defences, applicant emails, sitemap coverage, and 1 credit per job posting.

Routing

Every signed-in resource moved under /dashboard/**, leaving the top level for public pages. /ads is now a public sales page that redirects signed-in visitors to /dashboard/ads.

Elsewhere

Autoblog gates drafts on quality and repairs rather than discards them, with E-E-A-T fields. Outreach stops burning the ValueSERP plan on dead queries and no longer rests a campaign over a quota failure. A connected social account that has failed thousands of times in a row is no longer retried forever. The link crawl is isolated so linkinator cannot kill the audit worker.

Upgrade notes

37 migrations land in this release. They are applied one file at a time — prod migration history has diverged from supabase/migrations/, so supabase db push would try to replay files prod already has.

20260819120000_promote_jobs.sql (promo_job) is additive and must be applied before deploying this version. The Promote sweep selects from it; a sweep meeting a missing table stops every campaign posting with nothing in the logs that looks like a failure. The code logs loudly and publishes nothing rather than failing silently, but that is a guard, not a substitute for ordering.

Both 20260818190000_promote_sources.sql and 20260819120000_promote_jobs.sql were applied to production ahead of this tag.

v0.2.0 — connect your own mailbox

First tagged release of CrawlProof. Headline change: outreach on /leads can now send from your own mailbox instead of the shared platform sender, so replies land in your inbox and deliverability follows your domain.

Mailbox autodiscovery

Enter an email address and CrawlProof works out the IMAP/SMTP settings from DNS — no hunting through your mail host's docs for a hostname and a port.

Discovery walks the most authoritative source first: SRV records (RFC 6186) → Thunderbird autoconfig → Outlook autodiscover → Mozilla ISPDB → an MX-to-provider map → a convention guess that is clearly labelled as a guess.

Verified against Forward Email, Gmail, and Fastmail.

Forward Email note: hosts that have you CNAME autoconfig.<domain> at them serve a certificate covering only their own names, so fetching the vanity hostname fails TLS. CrawlProof resolves the CNAME and re-asks the provider under a name its certificate actually covers, rather than skipping verification.

Confirm before you type a password

Discovery shows what it found and where it came from, with every field editable, and only then asks for a password. The login is verified before anything is saved — nothing is stored if the check fails.

Credentials at rest

Mailbox passwords are encrypted with AES-256-GCM. The key lives in the application environment and never in the database, so a database dump yields ciphertext and no key. This protects against database compromise, not application-server compromise.

Operators: SOCIAL_VAULT_KEY now guards mailbox credentials as well as social tokens. Rotating it invalidates stored mailbox passwords and requires reconnecting.

Upgrade notes

Migration 20260727120000_outreach_mailbox_imap.sql adds columns to organization_outreach_configs. Additive only, and it must be applied before deploying this version.