fix(paywall): audience enum handling + eval suite (follow-up to #1)#2
Merged
snapsynapse merged 1 commit intomainfrom Apr 16, 2026
Merged
fix(paywall): audience enum handling + eval suite (follow-up to #1)#2snapsynapse merged 1 commit intomainfrom
snapsynapse merged 1 commit intomainfrom
Conversation
Follow-up to #1. Two bugs surfaced while writing evals for the --detect-paywall flag: 1. Founding-tier posts misclassified as free. `audience == "only_paid"` missed "founding" (paid founding-member tier), defeating the feature's stated purpose of flagging subscriber-only content. 2. Missing `audience` field in a 200 response was silently reported as "everyone"/is_paid=False, contradicting the docstring promise of "graceful fallback to null on API errors." Fix: explicit `known_paid` + `known_free` sets. Values outside both (including a future Substack tier) return is_paid=None with the raw audience string preserved, so downstream workflows can treat it as "unknown — handle with care" instead of silently publishing as free. Also ships a 30-test eval suite covering audience decoding, HTTP failure modes, request shape, frontmatter behavior, CLI wiring, and publication-slug edge cases. Run with: pip install -r requirements.txt -r tests/requirements-dev.txt pytest tests/ -v See tests/EVALS.md for the full merge-readiness report. Credit to @drewid74 for the initial implementation in #1. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
snapsynapse
added a commit
that referenced
this pull request
Apr 16, 2026
- Add `__version__ = "1.2.0"` module constant; use it in both frontmatter
code paths (live fetch and --from-md) via f-string. Removes 3 hardcoded
"v1.1.0" occurrences that would drift on the next bump.
- Drop version from the module docstring so the constant is the single
source of truth.
- Replace `dt.datetime.utcnow()` (deprecated in 3.12, scheduled for
removal) with `dt.datetime.now(dt.timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ")`
so Python 3.14 runs warning-free while preserving the existing ISO-Z
output format.
- README: fix placeholder clone URL (yourusername -> snapsynapse).
- README: expand the Paywall Detection section to list all four known
audience values (everyone/only_free/only_paid/founding) and document
the "unknown tier -> is_paid=null" contract shipped in #2.
- README: bump example source line to v1.2.0.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
snapsynapse
added a commit
that referenced
this pull request
Apr 16, 2026
- CONTRIBUTING.md documents local test setup, PR conventions (Conventional Commits), style rules, and how to run the opt-in live smoke test. - CHANGELOG.md follows Keep a Changelog format. Captures the v1.2.0 paywall feature + fix work (PR #1, PR #2) and an Unreleased section for the current improvement pass. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Follow-up to #1 (thanks @drewid74 for the initial implementation). Fixes two bugs surfaced while building an eval suite for the new
--detect-paywallflag.Bugs fixed
1. Founding-tier posts misclassified as free
audience == "only_paid"missed"founding"— Substack's paid founding-member tier. Verified empirically across 9 publications:foundingshows up in the wild (e.g. @natesnewsletter).2. Missing
audiencefield silently reported as "everyone"data.get("audience", "everyone")contradicted the docstring promise of graceful fallback to null on API errors. A 200 response without the field now returnsNone/Noneinstead of a false-negative.New contract
is_paidaudienceonly_paid/foundingTrueeveryone/only_freeFalseNoneNoneNoneThe enum is Substack-internal — authors rename display labels in the UI but the API normalizes to this fixed set. Verified across 9 publishers / 120+ posts.
Eval suite
30-test suite covering audience decoding, HTTP failure modes, request shape, frontmatter serialization, CLI wiring, and publication-slug edge cases. Opt-in live smoke test gated on
SUBSTACK2MD_LIVE=1.See tests/EVALS.md for the full merge-readiness report.
Test plan
pip install -r requirements.txt -r tests/requirements-dev.txtpytest tests/ -v→ expect 30 passed, 1 skippedSUBSTACK2MD_LIVE=1 pytest tests/test_live_smoke.py -v -spython substack2md.py URL --detect-paywalland confirmis_paid: truein the frontmatter🤖 Generated with Claude Code