Skip to content

Fix: explain the nltk 3.10.1 CWD import guard instead of leaking it - #151

Open
AmaadMartin wants to merge 2 commits into
mainfrom
fix/nltk-cwd-guard-import-diagnostic
Open

Fix: explain the nltk 3.10.1 CWD import guard instead of leaking it#151
AmaadMartin wants to merge 2 commits into
mainfrom
fix/nltk-cwd-guard-import-diagnostic

Conversation

@AmaadMartin

@AmaadMartin AmaadMartin commented Aug 6, 2026

Copy link
Copy Markdown
Owner

Please ensure you have read the contribution guide before creating a pull request.

Link to Issue or Description of Change

  1. Link to an existing issue (if applicable):
    No existing issue. I searched google/adk-python for nltk and for Blocked import and found none.

  2. Or, if no issue exists, describe the change:

Problem: nltk 3.10.1 installs a meta-path finder that blocks any import whose file resolves under the current working directory while nltk is on the call stack. site-packages resolves under the current working directory when the virtual environment lives inside the project, so nltk blocks its own dependency and rouge_score fails to import. The user sees Blocked import of regex from current working directory for security reasons, which names neither nltk nor ADK. The -P and PYTHONSAFEPATH remedy that message suggests does not work, because the finder tests where the file lives, not what is on sys.path.

Solution: google.adk.dependencies.rouge_scorer is the only place ADK imports rouge_score, so I translate that one failure there. The new ImportError names nltk 3.10.1 and the fix pip install --upgrade "nltk!=3.10.1", and it chains the original nltk error as __cause__. Every other import failure re-raises unchanged, so an absent rouge_score still raises ModuleNotFoundError and the existing except ModuleNotFoundError handlers in cli_eval.py, dev_server.py and agent_evaluator.py keep working. pyproject.toml already excludes 3.10.1, but that does not help an environment that already installed 3.10.1 or pinned it in a lockfile.

I chose the diagnostic over setting NLTK_DISABLE_IMPORT_SECURITY=1. A library must not switch off a dependency's security control on the user's behalf, and that variable is read only at nltk's first import, so the assignment is a silent no-op if anything imported nltk earlier.

Grounding: I inspected both wheels. nltk/inisec.py exists in the 3.10.1 wheel and nltk/__init__.py:22 imports it. Both are absent in 3.10.2. The marker string this change matches is the verbatim text inisec.NLTKSafeImportFinder.find_spec raises.

Collision check: I listed all 150 open pull requests on the fork and inspected their file lists. None touches src/google/adk/dependencies/, tests/unittests/test_optional_dependencies.py, or final_response_match_v1.py. PR #114 shares the nltk root cause but changes only tests/unittests/conftest.py, which this pull request does not touch. This branch is one commit ahead of main and zero behind.

Testing Plan

Please describe the tests that you ran to verify your changes. This is required for all PRs that are not small documentation or typo fixes.

Unit Tests:
[x] I have added or updated unit tests for my change.
[x] All unit tests pass locally.

Four tests in tests/unittests/test_optional_dependencies.py cover the guard-matched branch, the cause chain, the re-raise branch, and the success path.

pytest tests/unittests/test_optional_dependencies.py -q
  9 passed, 6 skipped

pytest tests/unittests/evaluation/test_final_response_match_v1.py \
       tests/unittests/evaluation/test_response_evaluator.py \
       tests/unittests/evaluation/test_metric_evaluator_registry.py -q
  95 passed

pytest ... -k rouge_scorer_shim --cov=src/google/adk/dependencies --cov-branch
  src/google/adk/dependencies/rouge_scorer.py  8 stmts, 0 miss, 2 branch, 0 partial, 100%

mypy src/google/adk/dependencies/rouge_scorer.py    Success: no issues found
ruff check src/google/adk/dependencies/rouge_scorer.py    All checks passed
pyink --check / isort --check-only    both clean on the two changed files
scripts/compliance_checks.py, scripts/check_new_py_files.sh    both exit 0

Mutation results. I ran each new test against broken code and confirmed it fails.

Mutation Test that failed Failure message
Restore the two plain from rouge_score import ... lines ..._explains_nltk_cwd_guard assert 'nltk' in "Blocked import of regex from current working directory ..."
Restore the two plain from rouge_score import ... lines ..._chains_original_nltk_error assert None is ImportError("Blocked import of regex ...")
Delete the marker if ... not in str(e): raise guard ..._reraises_unrelated_import_error assert 'nltk' not in 'Failed to import rouge_score because nltk 3.10.1 ...'
Delete the tokenizers re-export ..._reexports_when_import_succeeds AttributeError: module '_test_adk_rouge_scorer_shim' has no attribute 'tokenizers'

Manual End-to-End (E2E) Tests:
Please provide instructions on how to manually test your changes, including any necessary setup or configuration.

I ran this against real nltk 3.10.1, with the packages installed under the working directory.

mkdir -p /tmp/adkproj/.venvlibs && cd /tmp/adkproj
uv pip install --python <repo>/.venv/bin/python --target ./.venvlibs \
  "nltk==3.10.1" "rouge-score==0.1.2"

# Baseline, the raw dependency import:
PYTHONPATH=./.venvlibs <repo>/.venv/bin/python -c "from rouge_score import rouge_scorer"
# ImportError: Blocked import of regex from current working directory for
# security reasons. Use '-P' or set PYTHONSAFEPATH ...

# The ADK eval import chain, with this change:
PYTHONPATH=./.venvlibs <repo>/.venv/bin/python -c \
  "import google.adk.evaluation.final_response_match_v1"
# ImportError: Failed to import rouge_score because nltk 3.10.1 blocked one of
# its imports. ... pip install --upgrade "nltk!=3.10.1". ...
# The traceback keeps the nltk error: "The above exception was the direct cause
# of the following exception".

# The remedy the message names really recovers:
uv pip install --python <repo>/.venv/bin/python --target ./.venvlibs \
  --upgrade "nltk!=3.10.1"
PYTHONPATH=./.venvlibs <repo>/.venv/bin/python -c \
  "import nltk, google.adk.evaluation.final_response_match_v1 as m; \
   print(nltk.__version__, m.RougeEvaluator.__name__)"
# 3.10.2 RougeEvaluator

I also checked the no-false-positive path: with rouge_score mapped to None in sys.modules, the import still raises ModuleNotFoundError: import of rouge_score halted; None in sys.modules.

CI: every test job passes. Unit Tests on Python 3.10, 3.11, 3.12, 3.13 and 3.14, Mypy Check on 3.10 to 3.13, and A2A v0.3 Tests on 3.10 to 3.14 are all green.

The Pre-commit Linter job fails, and it does not fail because of this change. The update-constraints hook fails the same way on main at commit a95b008f, the parent of this branch. Both logs report hook id: update-constraints and Resolution failed even without constraints. I left it alone because it is unrelated breakage.

Checklist

[x] I have read the CONTRIBUTING.md document.
[x] I have performed a self-review of my own code.
[x] I have commented my code, particularly in hard-to-understand areas.
[x] I have added tests that prove my fix is effective or that my feature works.
[x] New and existing unit tests pass locally with my changes.

GWeale added 2 commits August 5, 2026 20:32
…g it

nltk 3.10.1 installs a meta-path finder that refuses any import whose file
resolves under the current working directory while nltk is on the stack. With
the virtualenv inside the project - the common '.venv/' layout - site-packages
is under the CWD, so nltk blocks its own dependency and rouge_score, which ADK
imports through google.adk.dependencies.rouge_scorer for the ROUGE eval metric,
fails to import.

The error the user sees names 'regex', never nltk and never ADK, and the '-P' /
PYTHONSAFEPATH remedy it suggests does not work: the finder tests where the
file lives, not what is on sys.path.

pyproject.toml already excludes 3.10.1, and nltk reverted the hook in 3.10.2
(nltk/nltk#3732), but neither helps an environment that already has 3.10.1
installed or pinned in a lockfile. Translate that one failure at the dependency
shim into a message naming the cause and the fix, chaining the original error;
every other import failure propagates unchanged.
…ests

Both module constants had exactly one reader each, and the module comment
repeated what the user-facing message already says. Move the comment into the
except block, keep only the facts the message does not carry, and inline both
strings at their single use site.

The tests set attributes on a fake module through setattr(). mypy excludes
tests/ (pyproject.toml), so there was no checker to route around. Use plain
attribute assignment.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants