Temperature-departure lit review: rag-build + 10 papers + cite map (1/3)#60
Merged
Conversation
Lit-review temperature-departure methodology + interpretation backing. Issue 1 of 3 in the climate-departure 3-split lit reviews (temperature, precip+drying, interpretation framing). Mirrors the #53/#54/v0.1.7 snowpack-lit-review pattern verbatim. Phase 0 renames existing rag scripts to noun_verb convention before new work begins. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Switch the 3 existing ragnar build/query scripts from verb_noun to noun_verb so the scripts/ directory reads cleanly for human users and matches the cd_* function naming convention (cd_cog_write, not cd_write_cog): rag_build_snow_methodology.R -> rag_snow_methodology_build.R rag_query_snow_methodology.R -> rag_snow_methodology_query.R rag_build_departure_framing.R -> rag_departure_framing_build.R Also updates internal docstring usage lines, CLAUDE.md script references, and the snow-lit-review archive README how-to-reproduce section. Phase 0 prep for #58 — establishes the naming convention before adding scripts/rag_temp_methodology_build.R + _query.R in Phase 3. NEWS.md and the archive task_plan/findings/progress files left as-is — historical records of what shipped at v0.1.7 stay named for that version. Refs #58. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Companion to 521084f (rename commit). Updates internal references that point at the old script names — lost from 521084f because the preceding heredoc-style commit message failed to parse and the `git add -A` never ran: - 3 renamed scripts: docstring header + Usage line - CLAUDE.md (lines 107-108): script-reference bullets + naming- convention note - planning/archive/2026-05-issue-53-snow-lit-review/README.md: how-to-reproduce section - planning/active/{task_plan,findings,progress}.md: PWF tracking Refs #58.
…cross-rag) Targeted web search confirmed DOIs + OA paths for 10 new papers covering: DTR / day-night asymmetry methodology (Karl 93, Easterling 97, Vose 05), Canadian / BC temperature trends (Vincent 18), elevation-dependent warming (Pepin 15, Rangwala & Miller 12), BC downscaling (Wang 12 ClimateWNA), climate-fish bridge (Mantua 10), and salmonid thermal envelope (Eaton & Scheller 96, Richter & Kolmes 05). Deep screen of the existing 19 items in NewGraphEnvironment/climate identified 7 reuse-relevant items (Mora 13, Hersbach 20, Munoz-Sabater 21, Isaak 17 NorWeST, Dierauer 20, Warkentin 22, Moore 22) — these get linked by item key without re-adding. Two cross-rag references from the snow methodology rag (Najafi 17, Yue & Wang 02) cover trend-test methodology and BC attribution without needing duplicate entries. PDF acquisition strategy: 4 papers OA-fetchable directly (UNL DigitalCommons, AMS post-embargo, NOAA-hosted PDFs, Canada.gov T&F landing), 6 paywalled and flagged for user ResearchGate download in Phase 2. 12-topic coverage matrix in findings.md maps every #58 vignette claim type to its primary + supporting citations, ready for the Phase 5 cite-this-for-that map. Refs #58.
POSTed all 10 candidate papers to NewGraphEnvironment/climate (key 8MH9LCC9) via Zotero Web API with PDFs attached via 4-step S3 upload. CrossRef-driven metadata; tags temperature-departure-methodology + cd-issue-58. 3 fresh PDF uploads, 7 deduped via md5 against existing Zotero S3 storage. PDFs sourced: 1 via curl (Wang 12 from UAlberta), 9 user-provided via ResearchGate. Karl 93 + Richter & Kolmes 05 needed OCR (image-only scans). All 10 in data/rag/temp_methodology_pdfs/ with text layers verified, gitignored. Initial run mistakenly stuffed Citation Key: <clean_key> overrides into the extra field — caught on review since NGE convention is BBT- auto-derived keys, not manual. PATCH'd all 10 items to clear the override. soul#43 filed to update /lit-search + /zotero-api skills so future runs avoid this pattern. User action needed: restart Zotero desktop so BBT generates the citation keys for the 10 new items (sync alone does not trigger key generation for Web-API-created items per CLAUDE.md). Once restarted, keys get captured into findings.md and the Phase 3 rag build script. Phase 2 deliverable: parent itemKey + attachKey table for all 10 papers in findings.md, PDFs in local cache ready for ragnar ingestion. Refs #58.
Adds scripts/rag_temp_methodology_build.R cloning the snow build script with a 10-paper pdf_specs map (label + Zotero attachKey + note). Reads PDFs out of data/rag/temp_methodology_pdfs/ — same local-cache pattern as snow, sidesteps the "download files at sync" Zotero desktop dependency. Runs in ~28 s on Ollama nomic-embed-text: Found 10 / 10 PDFs Chunks: 677 Sources: 10 Sanity-tested retrieval on a DTR query — returns Karl 1993 abstract + DTR variable construction, Rangwala & Miller 2012 alpine trends, and Vincent 2018 nighttime asymmetry. All on-topic for the temperature methodology citation backbone. The Zotero attachKeys hardcoded in pdf_specs serve downstream auditing; the BBT-auto-derived citation keys (which is what the vignette will cite via [@key] markers) get captured in findings.md once the user restarts Zotero desktop and BBT generates keys for the 10 Web-API-created items. Next: Phase 4 — write rag_temp_methodology_query.R + mine the store for methodology quotes across 8 topics. Refs #58.
Phase 4 — adds scripts/rag_temp_methodology_query.R (clone of the snow query script) running 24 queries across 8 topics and capturing top-5 chunks each to planning/active/temp_methodology_quotes.md (637 lines). Topics cover DTR asymmetry, Tmax/Tmin globe trends, Canadian/BC temperature trends, BC downscaling, elevation-dependent warming, climate-stream-temp bridge, salmonid thermal envelope, and trend methodology cross-checks. Phase 5 — synthesis in findings.md: per-topic methodology quotes selected from the rag retrieval, cross-cutting methodology section (baseline window, trend test cross-check vs Vincent 18's AC-iterative procedure, ERA5-Land 2m temperature validation gap), 4 documented deviations (UTC-day tmax/tmin per issue #37, raw MK vs AC-iterative, no direct ERA5-Land 2m T validation paper for BC, regional DTR ratio may differ from global), and an 18-row "cite this for that" menu mapping vignette claim types to citation keys. Philosophy preface added to findings.md per user feedback (2026-05-05): this is a library, not a prescription. The downstream vignette branch picks sparingly — citations only for authorities on findings actually visible in the AOI's graphs/tables, plain language, spelled-out acronyms, no over-decoration. Captured in memory as feedback_vignette_citations_sparse.md so future runs don't bloat findings.md with vignette-write expectations. Phase 5 also documented: - 7 existing items in NewGraphEnvironment/climate that are reuse-relevant (no re-add): mora_etal2013, hersbach_etal2020, munoz-sabater_etal2021, isaak_etal2017, dierauer_etal2020, warkentin_etal2022, moore_schindler2022 - 2 cross-rag references from data/rag/snow_methodology.duckdb that cover trend-test methodology and BC attribution without duplicate Zotero entries: najafi_etal2017, yue_wang2002 Next: Phase 6 — code-check, PR, archive. User action pending: Zotero desktop restart so BBT generates citation keys for the 10 new items; keys captured into findings.md before PR. Refs #58.
After Zotero restart and BBT plugin update from 8.0.25 to 9.x (Z8/9 line; Z9 had auto-disabled BBT 8.x as incompatible), all 10 new climate-collection items now have BBT-auto-derived keys following the firstauthor_etalYEAR + TitlePrefix convention. Pepin 2015 required a creators-PATCH first: CrossRef returned only the corporate author "Mountain Research Initiative EDW Working Group" with no individuals, so BBT initially fell back to a title-derived key. Added the 21 individual working-group members listed in the paper roster (N. Pepin first as corresponding author). With BBT alive again, the key regenerated to pepin_etal2015Elevationdependentwarming. Final 10 BBT keys captured in findings.md Phase 2 table: karl_etal1993NewPerspective easterling_etal1997MaximumMinimum vose_etal2005Maximumminimum vincent_etal2018ChangesCanadas pepin_etal2015Elevationdependentwarming rangwala_miller2012Climatechange wang_etal2012ClimateWNAHighb mantua_etal2010Climatechange eaton_scheller1996Effectsclimate richter_kolmes2005MaximumTemperaturea cite-this-for-that map updated to use full BBT keys (replacing the local-label placeholders) so the downstream vignette branch can copy [@key] markers verbatim. Also captures keys for the 7 existing reuse-relevant climate-collection items + 2 cross-rag references (najafi_etal2017AttributionObserved, yue_wang2002Applicabilityprewhitening) for completeness. Refs #58.
NewGraphEnvironment
added a commit
that referenced
this pull request
May 5, 2026
…/3) (#64) * Initialize PWF baseline for #63 * Phase 1: interpretation framing candidate list (4 new + 6 reuse + 4 cross-rag) * Phases 2+3+4+5: framing rag store + query + synthesis (with auto-restart) Phase 2 — POSTed 4 candidates to NewGraphEnvironment/climate (8MH9LCC9) via Web API with PDFs attached via S3. CrossRef-driven metadata; tags interpretation-framing-methodology + cd-issue-63. 2 fresh PDF uploads (Arguez & Vose, Hawkins & Sutton), 2 md5- deduped (Hansen 2012, Livezey 2007). Auto-restart fired and all 4 BBT keys captured cleanly: arguez_vose2011DefinitionStandard livezey_etal2007EstimationExtrapolation hawkins_sutton2012Timeemergence hansen_etal2012Perceptionclimate Phase 3 — adds scripts/rag_interpretation_framing_build.R cloning the precip+drying build script with a 4-paper pdf_specs map. Runs in ~10 s on Ollama nomic-embed-text: Found 4 / 4 PDFs Chunks: 291 Sources: 4 Phase 4 — adds scripts/rag_interpretation_framing_query.R running 16 queries across 6 topics (narrower than #58/#61's 24 queries since framing topic surface is smaller). Captures top-5 chunks each to planning/active/interpretation_framing_quotes.md (373 lines). Topics: baseline window methodology, normals when trends exist, time of emergence, cumulative-impact / loaded dice, shifting baseline climate, departure from recent variability. Phase 5 — synthesis in findings.md: per-topic methodology quotes selected from the rag retrieval, cross-cutting methodology section (Hansen 2012's choice of 1951-1980 base period validates cd's choice for cumulative-impact reporting — strongest direct precedent across all three lit reviews), 3 documented deviations (1951-1980 vs WMO 1961-1990 baseline, no AC correction, ToE not quantified per-AOI), and an 11-row cite-this-for-that menu mapping vignette framing claim types to BBT-auto-derived citation keys. 3-split scoreboard added to findings.md — pointer to the four findings.md files (#53/#54 snow, #58/#60 temp, #61/#62 precip+ drying, this) for the downstream vignette wire-up branch. Refs #63.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
[@key]markers.NewGraphEnvironment/climateZotero collection (key8MH9LCC9) with PDFs attached: Karl 1993 (DTR foundational), Easterling 1997 / Vose 2005 (DTR globe trends), Vincent 2018 (Canada/BC trends), Pepin 2015 / Rangwala & Miller 2012 (elevation-dependent warming), Wang 2012 (ClimateWNA BC downscaling), Mantua 2010 (PNW salmon climate-stress bridge), Eaton & Scheller 1996 / Richter & Kolmes 2005 (salmonid thermal envelope). Plus 7 reuse-relevant existing climate-collection items + 2 cross-rag references from the snow methodology rag.data/rag/temp_methodology.duckdb(677 chunks, 10 sources, ~28 s ingest via Ollama nomic-embed-text), built by the newscripts/rag_temp_methodology_build.R. 24-query mining viascripts/rag_temp_methodology_query.Rproducedplanning/active/temp_methodology_quotes.md(637 lines), synthesized intofindings.mdwith methodology-quotes-by-topic, cross-cutting methodology, deviations, and an 18-row "cite this for that" map (BBT-auto-derived keys, ready for downstream[@key]insertion).rag_build_*.R/rag_query_*.Rscripts torag_*_build.R/rag_*_query.R(noun_verbto match the cd_* function convention). 3 file renames + docstring/CLAUDE.md/archive-README updates.soul#43documents the BBT-auto-derived citation key convention so future/lit-search+/zotero-apiruns avoid manualCitation Key:overrides;cd#59is the BEC zone-shift lit-review tracker (placeholder, post-3-split).Headline finding:
cd_trend()'s raw Mann-Kendall + Theil-Sen aligns with Vincent et al. 2018 (Canadian temperature trend methodology — Sen slope + Kendall's τ), modulo their iterative lag-1 AC handling. Yue & Wang 2002 (cross-rag) supports our raw form for 76-year strong-trend series.Relates to NewGraphEnvironment/sred-2025-2026#23.
Fixes #58.
Test plan
devtools::test()clean on58-temperature-lit-review(verified: 166 PASS, 0 FAIL)lintr::lint('scripts/rag_temp_methodology_build.R')+_query.R→ 0 lints (verified)Rscript scripts/rag_temp_methodology_build.Rreproduces the store (10 sources, ~677 chunks) given the local PDF cache populated from ZoteroRscript scripts/rag_temp_methodology_query.Rregeneratesplanning/active/temp_methodology_quotes.mdfindings.mdPhase 2 table resolve to items inNewGraphEnvironment/climate(verified via local sqlite)planning/active/task_plan.mdmatch landed work🤖 Generated with Claude Code