feat(sources): add Semantic Scholar adapter with citation context#22
Merged
foundatron merged 1 commit intomainfrom Mar 11, 2026
Merged
feat(sources): add Semantic Scholar adapter with citation context#22foundatron merged 1 commit intomainfrom
foundatron merged 1 commit intomainfrom
Conversation
… Semantic Scholar adapter - Add SemanticScholarSourceConfig subclass with s2_api_key, min_citations, days_back fields - Add S2_API_KEY env var override in load_config - Add validation for min_citations >= 0 and days_back >= 1 - Add x-api-key header when api_key is set - Add citationCount to fields param; filter papers below min_citations (None treated as 0) - Add publicationDateOrYear param when days_back is set (YYYY-MM-DD:YYYY-MM-DD format) - Add _fetch_with_retry with 429 retry respecting Retry-After header (max 3 retries) - Non-retryable HTTP errors are logged and the query is skipped gracefully - Update cli.py to pass new params to SemanticScholarAdapter - Add 11 new tests covering all new behaviours Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #3
Changes
1.
tentacle/config.pySemanticScholarSourceConfig(SourceConfig)dataclass with:s2_api_key: str = "",min_citations: int = 0,days_back: int | None = NoneConfig.semantic_scholartype fromSourceConfigtoSemanticScholarSourceConfigload_config: readS2_API_KEYintoconfig.semantic_scholar.s2_api_keyvalidate:min_citations >= 0,days_back >= 1(if set)DEFAULT_CONFIG_TEMPLATEwith commented-outs2_api_key,min_citations,days_back2.
tentacle/sources/semantic_scholar.py__init__(self, api_key: str = "", min_citations: int = 0, days_back: int | None = None)— store on instance_search, addx-api-keyheader toRequestwhenself._api_keyis non-emptycitationCountto thefieldsquery parameterpublicationDateOrYearparameter (verify exact name against S2 docs) usingYYYY-MM-DD:YYYY-MM-DDformat whendays_backis set_fetch_with_retry(self, req: Request) -> bytesmethod: wrapsurlopen, retries on HTTP 429 respectingRetry-Afterheader (parse asint, fallback to 1s), max 3 retries with exponential backoff. Non-retryable HTTP errors (4xx/5xx) are caught, logged, and returnNone_search, call_fetch_with_retryinstead of rawurlopen; onNonereturn, skip that querycitationCount(defaultingNoneto 0) is belowmin_citations3.
tentacle/cli.py_get_sourcesto passapi_key=config.semantic_scholar.s2_api_key,min_citations=config.semantic_scholar.min_citations,days_back=config.semantic_scholar.days_backtoSemanticScholarAdapter()4.
tentacle/models.pyReview Findings
The most actionable items are: (1) catching
URLErrorin_fetch_with_retryto handle network-level failures gracefully, (2) adding a log line whenRetry-Afterheader parsing fails, and (3) adding a test for non-integerRetry-Aftervalues. The overall implementation is solid — retry logic, config validation, env var override, and test coverage are all well-structured.