Skip to content

Rust panic (index out of bounds) scanning a from x import y of a missing module under a dotted root package #308

Description

@nathan-gilbert

Summary

grimp.build_graph() panics (pyo3_runtime.PanicException) instead of raising a Python
exception (or handling the import gracefully) when all of the following hold:

  1. the root package name is dotted (a child of a namespace package, e.g. ns.inner),
  2. include_external_packages=True,
  3. a scanned file contains a from x import y statement where x is a missing module
    inside the root package (so both x and its parent fail to resolve as internal modules).
thread '<unnamed>' (73) panicked at src/import_scanning.rs:330:74:
index out of bounds: the len is 0 but the index is 0
...
pyo3_runtime.PanicException: index out of bounds: the len is 0 but the index is 0

Reproduction (grimp 3.15)

mkdir -p ns/inner
echo 'from ns.inner.missing_module import something' > ns/inner/__init__.py
python -c "import grimp; grimp.build_graph('ns.inner', include_external_packages=True)"

The import statement may equivalently be relative (from .missing_module import something) —
it panics after resolution to the same absolute name.

Boundary conditions (all verified on 3.15)

Variation Result
from ns.inner.missing_module import something panic
from .missing_module import something panic
import ns.inner.missing_module ok
from ns.inner import missing_module ok
same import, include_external_packages=False ok
same import, single-component root (mypackage) ok

The import / single-level from forms survive because _get_internal_module resolves them
via the one-level parent trim; the deeper from form fails both the exact and parent lookup,
falls through to the external branch, and reaches _distill_external_module with a module
name that shares its full prefix with the root package.

Analysis

In rust/src/import_scanning.rs, _distill_external_module walks the shared namespace
prefix:

while external_path_components[0] == internal_path_components[0] {
    external_namespace_components.push(external_path_components.remove(0));
    internal_path_components.remove(0);
}
external_namespace_components.push(external_path_components[0]);

For an "external" module that is actually inside the root package (here
ns.inner.missing_module.something vs. root ns.inner), the loop consumes every component of
one of the vectors and the next [0] access panics. The earlier
"parent of an internal package" guard only catches proper ancestors of the root, not
descendants-that-failed-internal-resolution, and _get_internal_module only trims one
trailing component before giving up.

Real-world impact

We hit this in CI in two independent ways, both reducing to "a .py module referenced by a
from … import … doesn't exist on disk":

  • A setuptools-scm-generated version file (from ._version import __version__ in the package
    root) that doesn't exist yet when lint-imports runs — whether the file exists depends on
    whether the build backend ran, so the job fails only on package-cache hits, which made this
    look like nondeterministic CI flakiness for months.
  • Running import-linter against a package installed from a binary-compiled wheel (Nuitka),
    where most modules exist only as .so: every from <pkg>.<module> import x in the
    remaining .py files references a module grimp cannot see.

Because the panic is a pyo3_runtime.PanicException rather than a normal exception, it also
aborts the rayon worker threads mid-scan and can't be caught meaningfully downstream
(import-linter shows the raw traceback).

Expected behavior

Either of these would be fine:

  • Treat an unresolvable module that is a descendant of a found package as internal (matching
    the behavior of the plain-import form, which squashes to the nearest existing ancestor); or
  • At minimum, bounds-check the loop and raise a proper Python exception naming the offending
    module and file.

Environment

  • grimp: 3.15 (also reproduced via import-linter 2.13 lint-imports)
  • Python: 3.13.9 (CPython)
  • OS: reproduced on both macOS 26 (arm64) and Linux x86_64 (Debian-based CI image)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions