depverify

Checks whether the packages an LLM told you to pip install actually exist.

Advisory only. Zero LLM calls. Never installs, deletes, halts, or quarantines anything.

---

The problem, in 30 seconds

An LLM suggests this fix for you:

To build a quick CLI progress tracker with retry logic, install these:

pip install requests
pip install retry-backoff-pro
pip install tqdm

Then in your script:

import requests
import tqdm
from retry_backoff_pro import RetryPolicy

policy = RetryPolicy(max_attempts=3)

requests and tqdm are real. retry-backoff-pro reads like a real, small retry-helper library. It isn't -- it does not exist on PyPI. If an attacker registers that exact name with malware before you run the install (a real technique known as slopsquatting), pip install will happily fetch it and your script will happily import it.

$ depverify check llm_answer.txt
✅ requests  [pip_install]  EXISTS
✅ tqdm  [pip_install]  EXISTS
➖ retry_backoff_pro  [import]  LOCAL_OR_UNKNOWN
❌ retry-backoff-pro  [pip_install]  NOT_FOUND

checked=4  not_found=1  suspicious=0  cant_verify=0

That's the whole tool. Deterministic report, one command.

Why

LLM coding models hallucinate package names. USENIX Security 2025 (Spracklen et al.) measured ~19.7% of recommended packages as hallucinated across 16 models (~5% commercial, ~21% open-source); a May 2026 replication found frontier models compressed to ~4.6-6.1%, but quantized local models remain far worse. Attackers register hallucinated names on PyPI with malware ("slopsquatting").

Every verdict is deterministic. Extraction is regex/AST, existence is a PyPI lookup, reputation is arithmetic over metadata. No LLM judging calls anywhere in this codebase -- the tool that checks for hallucinations doesn't itself hallucinate.

	depverify	Content-scanning tools (Socket, Snyk, etc.)
Checks name exists	✅	usually assumed, not the focus
Checks package content/behavior	❌ (by design -- see What this does NOT do)	✅
Blocks installs / fails CI	❌ (report only)	often yes
Model calls involved	0	varies by product
npm / yarn	❌ (Python/PyPI only)	often yes

Install

pip install depverify

Dev: pip install -e ".[dev]" (pytest, pandas).

Usage

depverify check path/to/answer.txt
cat answer.txt | depverify check -
depverify check answer.txt --json
depverify check answer.txt --symbols
depverify check answer.txt --symbols --symbols-isolated
depverify check answer.txt --ignore my-internal-lib --ignore 'acme-*'

A successful check's exit code is always 0, regardless of what it finds (NOT_FOUND, suspicious, etc.) -- this is an advisory tool, not a linter that fails your build over the contents of a report. That guarantee covers verdicts, not invocation: a CLI usage error (bad arguments, a missing/unreadable input file, non-UTF-8 input) is reported as a clean one-line message on stderr and exits non-zero, the same way depverify check with a missing required argument already does -- it is not itself a verdict, so it is not covered by the "always 0" guarantee.

If depverify finds nothing at all, it says so and explains why -- check only reads literal pip install <name> lines and import x / from x import y statements from raw text. Point it at the actual text an LLM gave you (chat output, a code block, a shell command list). If you want to check a real project's dependency declarations instead -- a requirements.txt or a PEP 621 pyproject.toml already committed to a project -- use depverify manifest (below), not check.

--symbols additionally checks statically-resolvable attribute chains (e.g. pandas.DataFrame.flatten) against packages already installed in the local environment. Security note: this is the one depverify feature that imports locally installed code, which can execute that code's import-time side effects -- see Security note: --symbols and local code execution before enabling it. Off by default; normal depverify check never imports or executes anything locally installed.

--symbols-isolated (only meaningful together with --symbols) runs each attribute-chain check in a throwaway child process instead of depverify's own process, so a package's import-time side effects land in a process that exits immediately after reporting one result rather than persisting in depverify's own process for the rest of the run. This reduces blast radius; it is not a sandbox -- the child still runs as the same OS user with the same filesystem/network access as depverify itself. See the security note below for the full picture.

Manifest mode: checking a real project's dependencies

depverify check reads LLM-answer text. depverify manifest reads an actual project dependency manifest already sitting on disk -- requirements.txt or a PEP 621 pyproject.toml:

depverify manifest requirements.txt
depverify manifest pyproject.toml
depverify manifest .                    # directory: see discovery rule below
depverify manifest . --json
depverify manifest requirements.txt --ignore my-internal-lib --ignore 'acme-*'

Pointed at a directory, depverify prefers pyproject.toml if present, otherwise falls back to requirements.txt, and exits with a clear error if neither exists in that directory. It never merges or reads both -- whichever one is chosen is printed to stderr (depverify: using /path/to/pyproject.toml) so it's never ambiguous which file's dependencies you're looking at.

Same core promise as check, no exceptions: manifest mode is pure text/TOML parsing. It never runs pip, never installs anything, never executes any code from the project being checked (not even to import a package for inspection -- that's what --symbols does elsewhere, and manifest mode doesn't touch it), never deletes files, and never resolves the transitive dependency tree. Only the names a manifest declares directly are checked -- if requests pulls in urllib3 and idna under the hood, those never appear in a manifest-mode report unless they're also declared directly. Getting the true resolved dependency graph would require actually running a resolver (uv, pip, Poetry, etc.) against real package metadata, which is exactly the kind of environment- mutating operation this tool refuses to do.

requirements.txt support

Parses plain PEP 508 requirement lines: bare names, extras, exact pins, and version ranges/markers --

requests==2.32.0
flask>=3.0
rich[markdown]>=13
tomli>=2.0; python_version < "3.11"

Blank lines and #-prefixed (or inline #) comments are ignored. A recognized set of ordinary pip options (--index-url, --extra-index-url, --find-links, --hash, --no-index, --no-binary, --pre, etc.) are silently skipped -- they configure how pip would install, which is irrelevant since depverify never invokes pip, but they're not package declarations either, so they must not be misparsed as one.

Unsupported declarations are detected and reported, never silently dropped and never a crash:

Kind	Example	Reported as
Editable install	-e ./local-pkg	editable_install
Local path	./vendor/mypkg, C:\vendor\pkg	local_path
Direct URL	https://example.com/foo.whl, foo @ https://...	direct_url
VCS dependency	git+https://github.com/foo/bar.git	vcs_dependency
Nested include	-r other-requirements.txt, -c constraints.txt	nested_include

Each shows up as a structured warning in the report (see JSON example below) -- the rest of a valid file is still fully checked. Nested -r/-c includes are flagged, not followed: depverify does not recursively read and parse a referenced file in this pass (see Limitations).

pyproject.toml support (PEP 621)

Parses the standard [project].dependencies array and [project.optional-dependencies] table:

[project]
dependencies = ["requests==2.32.0", "rich[markdown]>=13"]

[project.optional-dependencies]
dev = ["pytest>=7.0", "black"]

Each optional-dependency group name (dev, test, etc.) is recorded on the corresponding result ("group": "dev") and on its own source (pyproject_optional_dependency vs. plain pyproject_dependency for the un-grouped list). Extras and the full PEP 508 requirement string are preserved the same way as requirements.txt. Uses tomllib (standard library) on Python 3.11+ and the tomli backport on 3.10, same as the rest of depverify's TOML handling (.depverify.toml).

Only PEP 621 is parsed. Poetry ([tool.poetry.dependencies], [tool.poetry.group.*]), Hatch ([tool.hatch.envs.*]), PDM ([tool.pdm.dev-dependencies]), and setuptools' dynamic-fields table are each a materially different schema depverify does not attempt to understand -- when recognized, they produce an explicit unsupported_tool_table warning naming the tool, rather than silently producing an empty or wrong result. A project using [project.dependencies] normally is handled fully regardless of what unrelated [tool.*] tables (ruff, mypy, pytest config, etc.) also happen to be present. Malformed TOML raises a clear error before any parsing is attempted; a single malformed entry inside an otherwise-valid dependencies array degrades to a warning instead of aborting the whole file.

Realistic JSON example

{
  "packages": [
    {
      "name_raw": "requests", "resolved": "requests",
      "source": "requirements", "verdict": "EXISTS",
      "risk": {"level": "ok", "reasons": []},
      "version_requested": "2.32.0", "specifier": "==2.32.0",
      "source_file": "requirements.txt"
    },
    {
      "name_raw": "rich", "resolved": "rich",
      "source": "requirements", "verdict": "EXISTS",
      "risk": {"level": "ok", "reasons": []},
      "extras": ["markdown"], "specifier": ">=13",
      "source_file": "requirements.txt"
    },
    {
      "name_raw": "retry-backoff-pro", "resolved": "retry-backoff-pro",
      "source": "requirements", "verdict": "NOT_FOUND", "risk": null,
      "specifier": "", "source_file": "requirements.txt"
    }
  ],
  "summary": {"checked": 3, "not_found": 1, "suspicious": 0, "cant_verify": 0, "warnings": 1},
  "warnings": [
    {"kind": "editable_install", "detail": "-e ./local-pkg", "source_file": "requirements.txt"}
  ]
}

extras, specifier, source_file, and group are only ever present for manifest-sourced results (requirements / pyproject_dependency / pyproject_optional_dependency) -- omitted entirely, not even as null, for check-sourced (pip_install/import) results, so existing check JSON consumers see byte-identical output. warnings (both the top-level list and the summary.warnings count) is likewise only present when at least one unsupported declaration was actually encountered.

Internal / private packages

If you run a private package index (Artifactory, Nexus, an internal PyPI mirror), your own package names will otherwise come back NOT_FOUND or LOCAL_OR_UNKNOWN -- depverify only ever checks public PyPI, and has no way to know a name is legitimate on your infrastructure. Recognize those names so they're reported as INTERNAL instead, never queried against PyPI at all:

depverify check answer.txt --ignore my-internal-lib
depverify check answer.txt --ignore 'foo,bar,acme-*'   # repeatable and/or comma-separated
depverify check answer.txt --ignore-file .depverify-ignore
depverify manifest requirements.txt --ignore my-internal-lib   # works identically for manifest mode

Or via a .depverify.toml in your project root (auto-discovered; no flag needed), or an explicit path with --config:

[internal]
names = ["my-internal-lib", "acme-*"]

All three sources are mergeable, and all three work identically for both depverify check and depverify manifest. Matching is name-only (never a network call), case-insensitive, PEP-503-normalized (so My_Internal.Lib, my-internal-lib, and MY-INTERNAL-LIB are all the same pattern), and supports shell-style glob wildcards via fnmatch (acme-*, not regex). INTERNAL applies to a matched name from a pip install line, a bare import, or any manifest source (requirements / pyproject_dependency / pyproject_optional_dependency) -- unlike LOCAL_OR_UNKNOWN, it's a confirmed verdict, not a "might be yours" guess. This feature is entirely opt-in: with no --ignore/--ignore-file/--config/.depverify.toml in play, behavior, verdicts, and Report.summary()'s shape are unchanged from before this feature existed (summary()'s internal key is present only when at least one INTERNAL verdict actually occurs).

Policy / CI mode (opt-in)

By default depverify never fails your build -- check and manifest always exit 0 regardless of what's found. If you want CI to deliberately fail when specific risks show up, opt in with --fail-on:

depverify manifest requirements.txt --fail-on not-found
depverify manifest pyproject.toml --fail-on not-found,suspicious
depverify check llm-answer.txt --fail-on not-found,cant-verify

Rules: not-found, suspicious, cant-verify, version-missing, warnings. Configurable via .depverify.toml's [policy] table too (merges with --fail-on, doesn't replace it), with a JSON report field and a ready-to-copy GitHub Actions example -- see docs/ci.md for the full reference. With no --fail-on and no [policy] config, behavior is unchanged from before this feature existed.

Library

from depverify import verify_text

report = verify_text(llm_answer_text)
print(report.to_json())

Verdict model

Per detected dependency:

Verdict	Meaning
EXISTS	Confirmed on PyPI. Carries a risk object (see below).
NOT_FOUND	Confirmed not on PyPI. Assigned to pip install lines and every manifest source (requirements, pyproject_dependency, pyproject_optional_dependency) -- all explicit declarations, same asymmetry reasoning as pip install (see below).
STDLIB_SKIPPED	Part of the Python standard library -- not a PyPI lookup at all.
LOCAL_OR_UNKNOWN	Unresolved import. May be a local module, a relative import, or an unconfigured private package (see INTERNAL below to stop guessing and configure it). Never assigned to a manifest source -- a manifest declaration is always explicit, never "might be a local module."
INTERNAL	Matched a configured internal/private-package name or pattern (--ignore/--ignore-file/.depverify.toml, see Internal / private packages) -- not a PyPI lookup at all, applies to pip install, import, and every manifest source.
CANT_VERIFY	Lookup itself failed (network, rate limit, etc.) -- not a verdict on the package.

source also identifies how a name was found: import / pip_install for depverify check; requirements / pyproject_dependency / pyproject_optional_dependency for depverify manifest (see Manifest mode above).

EXISTS packages additionally carry a risk object: level ∈ ok | suspicious, with reasons ⊆ {young_package, low_downloads, near_name:<popular-package>, version_missing}.

Existence and reputation are orthogonal. A package that EXISTS can still be suspicious. depverify never folds "sketchy" into "missing" -- those are different questions with different implications.

The core asymmetry

A name that 404s on PyPI is interpreted differently depending on how it was detected:

Detected via	Verdict	Why
pip install <name>	NOT_FOUND	The LLM told you to install this; it doesn't exist; that's the signal slopsquatting exploits.
requirements.txt / pyproject.toml entry	NOT_FOUND	Same reasoning as pip install: a manifest entry is an explicit declaration, not a guess.
import <name>	LOCAL_OR_UNKNOWN	Might be your own module, a relative import target, or a name that isn't on PyPI for a legitimate reason. Flagging every unresolvable import as "not found" would bury real warnings under false positives from ordinary project code (import myapp.models, etc.).

Report shape (click to expand)

{
  "packages": [
    {"name_raw": "cv2", "resolved": "opencv-python", "source": "import",
     "verdict": "EXISTS", "risk": {"level": "ok", "reasons": []}},
    {"name_raw": "pdfreader-pro", "resolved": "pdfreader-pro", "source": "pip_install",
     "verdict": "NOT_FOUND", "risk": null}
  ],
  "summary": {"checked": 2, "not_found": 1, "suspicious": 0, "cant_verify": 0}
}

summary.cant_verify counts only package-level CANT_VERIFY verdicts. It does not include symbol-level CANT_VERIFY results (e.g. a --symbols chain that couldn't be checked because the package isn't installed locally) -- those live under each package's own symbols list (see --symbols below) and are a separate count from the top-level summary. This is deliberate: summary() predates the symbol-checking feature and was kept unchanged so existing consumers of Report.summary()/to_dict() see byte-identical output when check_symbols is left at its default.

Each package entry also carries "resolution_uncertain": true when (and only when) resolved came from locally-installed package metadata rather than the static mapping table -- see Limitations's mapping-table entry for what that means and why it's surfaced. Omitted entirely (not even as false) otherwise, same additive-only discipline as symbols above, so existing consumers of to_dict()/to_json() see byte-identical output for any result that never hits that fallback path.

Reputation signals (EXISTS packages only)

Signal	How it's computed
Age	Days since the earliest PyPI release.
Downloads	Last-30-day count from pypistats.org. If that API is unreachable or rate-limited, depverify silently omits download-based reasons -- a missing download count is never treated as CANT_VERIFY.
Near-name	difflib.SequenceMatcher.ratio() >= 0.88 (tunable, see reputation.py) against a vendored list of popular package names (depverify/top_packages.json), flagged as near_name:<popular-package> when the candidate itself isn't already a popular package.

Default suspicion rule (a tunable default, not ground truth): suspicious if (age < 60 days AND downloads < 1000/month) OR any near_name hit. Tune the constants in depverify/reputation.py for your risk tolerance.

What this does NOT do

• No package-content scanning. depverify checks whether a name exists and looks reputable by metadata -- it does not download, sandbox, or static-analyze package code. For that, see dedicated tools like Socket or Snyk.
• No blocking by default. Nothing here gates a pip install, a CI job, or an LLM response unless you explicitly opt in with --fail-on/ [policy] (see docs/ci.md) -- and even then, all it does is set an exit code based on counts of existing verdict/warning categories. It's still a report, not a scanner or installer.
• No npm/yarn. Python/PyPI only, by design.
• No LLM-based judging. Every verdict is deterministic: extraction is regex/AST, existence is a PyPI lookup, reputation is arithmetic over metadata. No model calls anywhere in this codebase.
• No pip invocation, ever -- including in manifest mode. depverify manifest parses requirements.txt/pyproject.toml as text/TOML. It never shells out to pip, uv, poetry, or any other installer, and never installs, upgrades, or removes anything, even to resolve a version range or discover what a dependency actually pulls in.
• No code execution, in either mode. Neither check nor manifest ever executes any code from the input being checked -- not the LLM answer's code snippets, and not the checked project's own source files. (The one narrow exception, --symbols, imports already-installed packages to introspect their attributes -- see its own security note below. It never executes anything from the file/project being checked itself, and manifest mode doesn't use it at all.)
• No transitive dependency resolution. depverify manifest only checks the names a manifest declares directly. It does not compute what those packages would themselves depend on -- that requires an actual resolver run against real package metadata (what pip, uv, or poetry lock do), which this tool deliberately does not perform.

Security note: --symbols and local code execution (click to expand)

Security note: --symbols and local code execution

Normal depverify check (no --symbols) never imports or executes any locally installed package code. Existence checks are PyPI metadata lookups over HTTP; reputation checks are pypistats.org lookups and arithmetic. Nothing in that path touches your local Python environment's installed packages, and nothing ever executes the LLM-generated code being scanned.

--symbols is different, and deliberately scoped narrowly because of it:

• To check whether an attribute chain like pandas.DataFrame.flatten really exists, --symbols uses importlib.import_module() to import the already-installed package and getattr() to walk the chain. Importing a Python module runs that module's top-level code -- this is ordinary Python behavior, not something depverify adds, but it means --symbols is the one code path in this tool that executes local code as a side effect of scanning.
• This is not the same as executing the LLM's code. --symbols never runs the snippet being checked; it only imports packages by name using importlib, and only ever calls getattr/hasattr on the resulting module or class objects -- never instantiates a class, never calls a function or method. (Confirmed in tests/test_symbols.py: walking a chain that reaches a property or method never triggers the property getter, the method body, or the class's __init__.)
• --symbols never installs anything. If a package exists on PyPI but is not already installed locally, the result is CANT_VERIFY, not an install-then-import. depverify's "never installs, deletes, halts, or quarantines anything" guarantee holds for --symbols too.
• The residual risk: if a malicious or already-compromised package happens to be installed in the same environment running depverify, --symbols will trigger that package's import-time code the same way a plain import thatpackage anywhere else in that environment would. depverify does not sandbox, isolate, or vet locally installed packages before importing them for a symbol check.
• Consequently: only enable --symbols in an environment where the installed package set is already trusted -- the same trust you'd already extend to running python -c "import <installed package>" in that environment. Do not run --symbols as a way to safely inspect an environment whose installed packages you don't already trust, and do not present or rely on --symbols as a sandbox or security boundary around untrusted local installs -- it is not one.
• --symbols-isolated narrows, but does not remove, this risk. Each attribute-chain check runs in its own child process (python -m depverify.symbols <dotted_path>) instead of depverify's own process. That process exits immediately after reporting one result, so a hang, crash, or lingering global state from a package's import-time code doesn't outlive the single check or take depverify's own process down with it (bounded by a 10-second default timeout). This is not OS-level sandboxing. The child process runs as the same user, with the same filesystem and network access, as depverify itself -- a malicious package's import-time code can still do anything that OS user can do. Depverify does not implement containers, seccomp, restricted users, or network namespaces around --symbols; that is a materially larger engineering and security-review effort than a single-process-vs-child- process distinction, and was deliberately left out of this pass rather than half-implemented. Treat --symbols-isolated as "reduces accidental blast radius," never as "safe to run against untrusted installs."

Limitations (click to expand)

• Dynamic imports are not detected. importlib.import_module("name") string literals are invisible to the AST/regex extraction in extract.py. Only literal import x / from x import y statements and pip install lines are found.
• depverify check (text mode) only reads pip install lines and import statements. It does not parse requirements.txt, pyproject.toml, setup.py, or Conda environment files -- use depverify manifest for those (see Manifest mode).
• depverify manifest (project mode) only parses requirements.txt and PEP 621 pyproject.toml. setup.py, Conda environment files, Pipfile/Pipfile.lock, and every build-tool-specific dependency table (Poetry, Hatch, PDM, setuptools dynamic fields) are explicitly out of scope -- pyproject.toml tables belonging to those tools are detected and reported via an unsupported_tool_table warning rather than parsed (each has a materially different schema this tool does not attempt to understand), and a setup.py/Conda/Pipfile file passed directly is rejected with a clear "unrecognized manifest filename" error rather than silently producing an empty or wrong result.
• depverify manifest never resolves the transitive dependency graph. Only the names a manifest declares directly are checked. A dependency's own dependencies (what would appear in a lockfile after actually running a resolver) are invisible to this tool by design -- computing them would require running pip/uv/poetry lock or equivalent against real package metadata, which depverify does not do.
• depverify manifest does not follow nested -r/-c includes. A -r other-requirements.txt or -c constraints.txt line is reported as a nested_include warning, not automatically read and merged in. Point depverify at each file directly if you need all of them checked.
• The import-name -> distribution-name mapping table is incomplete by nature. depverify/mapping_table.py is hand-curated (71 entries as of this writing) and will always miss some real-world aliases. depverify resolves an import-sourced name in this order: the mapping table first (deterministic, environment-independent); then, only for names the table doesn't cover, whatever importlib.metadata.packages_distributions() reports for packages installed alongside depverify itself; then, if neither has an answer, the import name unchanged. Only that middle step -- the locally-installed-metadata fallback -- is environment- dependent (it can differ machine to machine depending on what's pip-installed there), and results resolved through it carry "resolution_uncertain": true in the JSON report (and an inline note in the table output) so that dependency is visible rather than silently presented with the same confidence as the deterministic paths. The mapping table is always checked first and always wins if it has an entry, specifically to keep that dependency as rare as possible.
• Verification is point-in-time. A name that 404s right now can be registered on PyPI minutes later -- including by an attacker watching for exactly this kind of hallucinated name (slopsquatting). Nothing here caches a "safe" verdict indefinitely; the cache TTLs (24h for EXISTS, 6h for NOT_FOUND) reflect that names that don't exist yet are the more time-sensitive case.
• EXISTS ≠ safe: compromised legitimate packages are invisible to this tool. A package that exists, is old, and has millions of downloads can still ship malware in a compromised release. depverify's existence and reputation checks say nothing about supply-chain compromise of an otherwise-legitimate package.
• depverify/top_packages.json was hand-vendored, not fetched live, because the environment this project was built in could not reach raw.githubusercontent.com, api.github.com, or any CDN mirror (only pypi.org / files.pythonhosted.org / bare github.com were reachable). It's a ~450-name curated list of well-known packages written from training knowledge, not the real top-5000 hugovk/top-pypi-packages dataset -- depverify does not claim otherwise anywhere in its own output. reputation.py's near-name check only ever reads this file from disk, so this does not affect depverify's offline usability: no network call is made to build or use this list at runtime, regardless of whether it's ever refreshed. scripts/fetch_top_packages.py documents exactly what the real source is, how to refresh the file (python scripts/fetch_top_packages.py, from a network that can reach raw.githubusercontent.com), and how to validate the file already on disk without any network access (python scripts/fetch_top_packages.py --validate-only); the same shape checks that validator runs also live as unit tests in tests/test_top_packages_dataset.py. Run the refresh before relying on near-name detection for anything beyond obvious, well-known package names.
• Attribute-chain verification (--symbols) only covers simple, statically-resolvable chains rooted directly in an imported name (e.g. pandas.DataFrame.flatten). It does not do instance attribute inference, signature/arity checking, type inference, .pyi stub parsing, or follow chains past a function call (requests.get(url).json is not checked past requests.get) or past a reassigned import. See depverify/symbols.py's module docstring for the full scope, and Security note: --symbols and local code execution before enabling it -- unlike the rest of depverify, it imports locally installed code.

Project layout

depverify/
├── depverify/                 # the library + CLI -- this is what `pip install depverify` ships
├── docs/                      # ci.md: policy/CI-mode reference and GitHub Actions example
├── tests/                     # unit tests (mocked, zero network) + eval/ (live network)
└── scripts/                   # fetch_top_packages.py: regenerate/validate the vendored top-package list
                                # check_release_tag.py: used by the publish workflow's pre-publish gate

Development

pip install -e ".[dev]"
pytest tests/ --ignore=tests/eval     # unit tests, zero network
python tests/eval/run_eval.py         # eval, live network against PyPI/pypistats

---

No blocking. No LLM calls. Just an honest answer to "does this package exist?"