fix: pin consumer LLM workflow installs - #2730
Merged
Merged
Conversation
stranske
added
automation
📝 WalkthroughWalkthroughStandardizes Python dependency installation to ChangesPinned LLM Install Standardization
Test Command Fallback
Estimated code review effort: 2 (Simple) | ~12 minutes Possibly related PRs
Suggested labels: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Comment |
stranske
enabled auto-merge (squash)
July 7, 2026 06:11
Contributor
Workflow source neededPR #2730 needs either a linked GitHub issue or one valid non-issue Workflow Source before PR metadata automation can manage it safely. Please do one of:
Once a valid source is present, this warning will not be reposted. |
Contributor
Automated Status SummaryHead SHA: 15af0d0
Coverage Overview
Coverage Trend
Top Coverage Hotspots (lowest coverage)
Low Coverage Files (<50.0%)
Updated automatically; will refresh on subsequent CI/Docker completions. Keepalive checklistScopeNo scope information available Tasks
Acceptance criteria
|
stranske
temporarily deployed
to
agent-high-privilege
July 7, 2026 06:11 — with 
GitHub Actions
Inactive
GitHub Actions
Inactive
2 tasks
stranske
force-pushed
the
codex/fix-sync-llm-installs
branch
from
July 7, 2026 06:13
d81134f to
07adcac
Compare
stranske
temporarily deployed
to
agent-high-privilege
July 7, 2026 06:14 — with
GitHub Actions
Inactive
GitHub Actions
Inactive
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: d81134f1f1
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| # Install langchain dependencies | ||
| pip install langchain langchain-core langchain-openai \ | ||
| langchain-anthropic langchain-community | ||
| python -m pip install -r tools/requirements-llm.txt |
There was a problem hiding this comment.
In this consumer template, the job checks out stranske/Workflows into workflows-scripts and later runs python workflows-scripts/scripts/langchain/..., but this line installs the consumer checkout's tools/requirements-llm.txt. When Workflows main updates the optimizer scripts or their pinned LLM dependencies before the consumer sync lands, the workflow executes newer workflows-scripts code with stale consumer pins and can fail at import/runtime; install workflows-scripts/tools/requirements-llm.txt here or pin the Workflows checkout to the same synced revision.
Useful? React with 👍 / 👎.
Contributor
There was a problem hiding this comment.
Pull request overview
This PR tightens the consumer-facing agent workflows’ LLM dependency installation so they consistently install the pinned tools/requirements-llm.txt set (via python -m pip) and avoid pre-installing a consumer repo’s requirements.txt ahead of LLM deps, with tests/docs updated to reflect the new contract.
Changes:
- Update
agents-auto-pilot.ymlandagents-issue-optimizer.yml(root + consumer template copies) to install LLM deps viapython -m pip install -r tools/requirements-llm.txtand remove the prior floating/consumerrequirements.txtinstalls. - Extend workflow/tests + snippet tests to cover the optimizer workflow and accept
python -m pipin install snippets. - Update workflow-update docs/snippets and local tooling output (
check_test_dependencies.sh) to match the new guidance.
Reviewed changes
Copilot reviewed 12 out of 12 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/workflows/test_workflow_llm_installs.py | Adds coverage for pinned LLM installs in agents-issue-optimizer.yml and updates the expected install command to python -m pip. |
| tests/docs/test_workflow_snippets_yaml.py | Accepts python -m pip install -r ... in snippet validation (but needs indentation cleanup for stable formatting). |
| templates/consumer-repo/.github/workflows/agents-issue-optimizer.yml | Switches optimizer to install pinned LLM deps from tools/requirements-llm.txt (removing repo requirements.txt + floating langchain installs). |
| templates/consumer-repo/.github/workflows/agents-auto-pilot.yml | Switches auto-pilot to python -m pip install -r tools/requirements-llm.txt and removes the conditional consumer requirements.txt install. |
| scripts/generate_llm_workflow_update_comment.py | Updates the needs-human guidance text to reference python -m pip install -r tools/requirements-llm.txt (but currently omits issue-optimizer despite this PR changing it). |
| scripts/check_test_dependencies.sh | Recommends python -m pytest when ./scripts/run_tests.sh is absent. |
| docs/workflow-updates/README.md | Updates verification guidance for auto-pilot’s pinned install (but currently doesn’t include issue-optimizer). |
| docs/workflow-updates/agents-auto-pilot-changes.yml | Updates the protected-workflow snippet to use python -m pip install -r tools/requirements-llm.txt. |
| docs/workflow-snippets/agents-auto-pilot-install.yml | Updates the snippet to use python -m pip install -r tools/requirements-llm.txt. |
| .github/workflows/agents-issue-optimizer.yml | Switches optimizer to install pinned LLM deps from tools/requirements-llm.txt (removing repo requirements.txt + floating langchain installs). |
| .github/workflows/agents-auto-pilot.yml | Switches auto-pilot to python -m pip install -r tools/requirements-llm.txt and removes the conditional consumer requirements.txt install. |
Comment on lines
+70
to
+74
| assert any( | ||
| isinstance(step, dict) | ||
| and isinstance(step.get("run"), str) | ||
| and any( | ||
| line.strip() == f"pip install -r {requirements_path}" | ||
| for line in step["run"].splitlines() | ||
| ) | ||
| and any( | ||
| line.strip() in { |
Comment on lines
19
to
22
| **Verification** | ||
| 1. Confirm `.github/workflows/agents-auto-pilot.yml` contains `pip install -r tools/requirements-llm.txt` and no unpinned `langchain` install commands. | ||
| 1. Confirm `.github/workflows/agents-auto-pilot.yml` contains `python -m pip install -r tools/requirements-llm.txt` and no unpinned `langchain` install commands. | ||
| 2. Confirm `.github/workflows/reusable-agents-verifier.yml` contains `pip install -r .workflows-lib/tools/requirements-llm.txt` in both evaluate and compare paths. | ||
| 3. Confirm both workflows include `actions/cache@v4` steps with keys that include `python-version` and the relevant `hashFiles(...)` call. |
stranske
temporarily deployed
to
agent-high-privilege
July 7, 2026 06:22 — with
GitHub Actions
Inactive
GitHub Actions
Inactive
stranske
temporarily deployed
to
agent-high-privilege
July 7, 2026 06:32 — with
GitHub Actions
Inactive
GitHub Actions
Inactive
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@config/template-drift-allowlist.txt`:
- Around line 93-98: The allowlist entry for pair.11 is out of date because the
template hash no longer matches the current contents of
templates/consumer-repo/.github/workflows/agents-issue-optimizer.yml. Regenerate
or update the pair.11 hashes in config/template-drift-allowlist.txt so
template_sha256 reflects the latest template while keeping main_sha256
unchanged, and preserve the existing reason text unless the divergence meaning
has changed.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: fbb49f6d-32e1-444a-ba9b-286ed149a73e
📒 Files selected for processing (14)
.github/workflows/agents-auto-pilot.yml.github/workflows/agents-issue-optimizer.ymlagents/codex-1447.mdconfig/template-drift-allowlist.txtdocs/workflow-snippets/agents-auto-pilot-install.ymldocs/workflow-updates/README.mddocs/workflow-updates/agents-auto-pilot-changes.ymlscripts/check_test_dependencies.shscripts/generate_llm_workflow_update_comment.pytemplates/consumer-repo/.github/workflows/agents-auto-pilot.ymltemplates/consumer-repo/.github/workflows/agents-issue-optimizer.ymltests/docs/test_workflow_snippets_yaml.pytests/scripts/test_generate_llm_workflow_update_comment.pytests/workflows/test_workflow_llm_installs.py
Comment on lines
93
to
+98
| [pair.11] | ||
| main = .github/workflows/agents-issue-optimizer.yml | ||
| template = templates/consumer-repo/.github/workflows/agents-issue-optimizer.yml | ||
| main_sha256 = 26806e76cd620e227c81566f4db0cb9a4eca2ddd4c3e05a941d2c4072b2fe3b5 | ||
| template_sha256 = ea5a33238a2dfc8bbc5e4661f66751dde1e9ca853b2cf77ed4eb286865fb9c93 | ||
| reason = Intentional divergence (re-baselined 2026-06-14): consumer template SHA-pins third-party actions per the fleet action-pin contract (docs/HISTORY.md, PR #1925) and sets LangSmith tracing env; root uses floating major tags with repo-internal concurrency + sparse-checkout (docs/fixes/sparse-checkout-audit-2026-02-03.csv). Fingerprints refreshed after #2391/#2394 bumps. Do not align: would strip consumer action pins. | ||
| main_sha256 = 1e2fb1b588d00f694fd9d02816944e68a4d1a8041f4f052730b1f30f3ad696db | ||
| template_sha256 = bbd9d30e392d3df884ebabd9738fcd5864d79754dcfc04068466132e4c2e769d | ||
| reason = Intentional divergence re-baselined 2026-07-07: root and consumer issue-optimizer workflows keep different auth plumbing/action pin surfaces, but both now use the pinned tools/requirements-llm.txt install path with python -m pip. Do not align wholesale because that would strip consumer action pins/token setup. |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🔴 Critical | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Recompute sha256 for the actual current workflow files to compare against the allowlist entries.
sha256sum .github/workflows/agents-issue-optimizer.yml templates/consumer-repo/.github/workflows/agents-issue-optimizer.yml
fd check_template_drift.py --exec cat {}Repository: stranske/Workflows
Length of output: 25459
🏁 Script executed:
#!/bin/bash
set -euo pipefail
python3 - <<'PY'
from pathlib import Path
import hashlib
def normalize_text(text: str) -> str:
normalized = text.replace("\r\n", "\n").replace("\r", "\n")
lines = [line.rstrip() for line in normalized.split("\n")]
while lines and lines[-1] == "":
lines.pop()
if not lines:
return ""
return "\n".join(lines) + "\n"
def normalized_sha256(path: Path) -> str:
text = path.read_text(encoding="utf-8")
return hashlib.sha256(normalize_text(text).encode("utf-8")).hexdigest()
paths = [
Path("config/template-drift-allowlist.txt"),
Path(".github/workflows/agents-issue-optimizer.yml"),
Path("templates/consumer-repo/.github/workflows/agents-issue-optimizer.yml"),
]
for path in paths:
print(f"== {path} ==")
if path.exists():
text = path.read_text(encoding="utf-8").splitlines()
for i, line in enumerate(text[92:98], start=93):
print(f"{i}: {line}")
if path.suffix == ".yml":
print(f"normalized_sha256={normalized_sha256(path)}")
print(f"line_count={len(text)}")
else:
print("MISSING")
PYRepository: stranske/Workflows
Length of output: 1616
🏁 Script executed:
#!/bin/bash
set -euo pipefail
python3 - <<'PY'
from pathlib import Path
import hashlib
import configparser
def normalize_text(text: str) -> str:
normalized = text.replace("\r\n", "\n").replace("\r", "\n")
lines = [line.rstrip() for line in normalized.split("\n")]
while lines and lines[-1] == "":
lines.pop()
if not lines:
return ""
return "\n".join(lines) + "\n"
def normalized_sha256(text: str) -> str:
return hashlib.sha256(normalize_text(text).encode("utf-8")).hexdigest()
repo = Path('.').resolve()
allowlist_path = repo / 'config/template-drift-allowlist.txt'
main_path = repo / '.github/workflows/agents-issue-optimizer.yml'
template_path = repo / 'templates/consumer-repo/.github/workflows/agents-issue-optimizer.yml'
for p in [allowlist_path, main_path, template_path]:
print(f"== {p.relative_to(repo)} ==")
print(f"exists={p.exists()}")
if p.exists():
txt = p.read_text(encoding='utf-8')
print(f"normalized_sha256={normalized_sha256(txt)}")
print(f"raw_bytes={len(txt.encode('utf-8'))}")
if p == allowlist_path:
parser = configparser.ConfigParser(interpolation=None)
parser.read(p, encoding='utf-8')
for section in parser.sections():
if section == 'pair.11':
print("pair.11 parsed values:")
for key in ['main', 'template', 'main_sha256', 'template_sha256', 'reason']:
print(f" {key}={parser.get(section, key)}")
PY
python3 - <<'PY'
from pathlib import Path
import hashlib
import configparser
import yaml
def normalize_text(text: str) -> str:
normalized = text.replace("\r\n", "\n").replace("\r", "\n")
lines = [line.rstrip() for line in normalized.split("\n")]
while lines and lines[-1] == "":
lines.pop()
if not lines:
return ""
return "\n".join(lines) + "\n"
def normalized_sha256(text: str) -> str:
return hashlib.sha256(normalize_text(text).encode("utf-8")).hexdigest()
repo = Path('.').resolve()
allowlist = configparser.ConfigParser(interpolation=None)
allowlist.read(repo / 'config/template-drift-allowlist.txt', encoding='utf-8')
entry = allowlist['pair.11']
main_path = repo / entry['main']
template_path = repo / entry['template']
main_text = main_path.read_text(encoding='utf-8')
template_text = template_path.read_text(encoding='utf-8')
print("pair.11 match status:")
print(" main_hash_matches=", entry['main_sha256'] == normalized_sha256(main_text))
print(" template_hash_matches=", entry['template_sha256'] == normalized_sha256(template_text))
print(" content_equal=", normalize_text(main_text) == normalize_text(template_text))
# Reproduce checker results for all pairs using the repo's discovery rules
manifest_path = repo / '.github/sync-manifest.yml'
manifest = yaml.safe_load(manifest_path.read_text(encoding='utf-8')) if manifest_path.exists() else {}
workflow_names = set()
for e in (manifest or {}).get('workflows', []) or []:
if isinstance(e, dict):
src = str(e.get('source', ''))
if src.startswith('.github/workflows/'):
workflow_names.add(src.removeprefix('.github/workflows/'))
alias = {"agents-63-issue-intake.yml": "agents-issue-intake.yml"}
main_dir = repo / '.github/workflows'
template_dir = repo / 'templates/consumer-repo/.github/workflows'
pairs = []
for main in sorted(main_dir.glob('agents-*.yml')):
template_name = alias.get(main.name, main.name)
template = template_dir / template_name
if (not template.exists()) and (main.name not in alias) and (template_name not in workflow_names):
continue
pairs.append((main, template))
drifts = []
for main, template in pairs:
main_text = main.read_text(encoding='utf-8')
if not template.exists():
drifts.append((main.name, template.name, 'missing-template'))
continue
template_text = template.read_text(encoding='utf-8')
if normalize_text(main_text) != normalize_text(template_text):
drifts.append((main.name, template.name, 'diff'))
print("drift_pairs=", drifts)
PYRepository: stranske/Workflows
Length of output: 2446
Refresh pair.11's template hash config/template-drift-allowlist.txt:93-98 — main_sha256 still matches, but template_sha256 is stale for templates/consumer-repo/.github/workflows/agents-issue-optimizer.yml, so this pair will keep tripping the drift check until the allowlist entry is regenerated.
🧰 Tools
🪛 LanguageTool
[uncategorized] ~94-~94: The official name of this software platform is spelled with a capital “H”.
Context: ...s digest to ebef44a. [pair.11] main = .github/workflows/agents-issue-optimizer.yml te...
(GITHUB)
[uncategorized] ~95-~95: The official name of this software platform is spelled with a capital “H”.
Context: ...yml template = templates/consumer-repo/.github/workflows/agents-issue-optimizer.yml ma...
(GITHUB)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@config/template-drift-allowlist.txt` around lines 93 - 98, The allowlist
entry for pair.11 is out of date because the template hash no longer matches the
current contents of
templates/consumer-repo/.github/workflows/agents-issue-optimizer.yml. Regenerate
or update the pair.11 hashes in config/template-drift-allowlist.txt so
template_sha256 reflects the latest template while keeping main_sha256
unchanged, and preserve the existing reason text unless the divergence meaning
has changed.
Source: Pipeline failures
This was referenced Jul 24, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
requirements.txtbefore LLM dependenciestools/requirements-llm.txtset withpython -m pipin root and consumer-template workflow copiescheck_test_dependencies.shrecommendpython -m pytestwhen a repo does not provide./scripts/run_tests.shValidation
PYTHONPYCACHEPREFIX=/tmp/workflows-syncfix-pycache python3 -m pytest tests/workflows/test_workflow_llm_installs.py tests/scripts/test_validate_dependency_test_setup.py tests/scripts/test_generate_llm_workflow_update_comment.py tests/docs/test_workflow_snippets_yaml.py -qpython3 scripts/validate_template_sync.pypython3 scripts/validate_template_completeness.pygit diff --checkRelated sync blocker: stranske/Manager-Database#1388.
Summary by CodeRabbit
Bug Fixes
Documentation
Tests