Skip to content

fix: importlens optimization bugs#1641

Merged
KRRT7 merged 11 commits intomainfrom
fix-importlens-bugs
Feb 23, 2026
Merged

fix: importlens optimization bugs#1641
KRRT7 merged 11 commits intomainfrom
fix-importlens-bugs

Conversation

@KRRT7
Copy link
Collaborator

@KRRT7 KRRT7 commented Feb 23, 2026

Summary

  • Fix comparator failing on __slots__-only objects (e.g. textual.cache.LRUCache) with "Unknown comparator input type" warnings, causing spurious test mismatches
  • Skip pytest-internal JUnit XML entries and optimize pytest plugin hot loop
  • Preserve comment position by passing CST module directly to import adder
  • Cache module scan in _clear_lru_caches
  • Skip identical and duplicate candidates early before processing

Test plan

  • Existing comparator tests pass (145 passed)
  • New TestSlotsObjects tests cover equality, inequality, inheritance, and nesting
  • Existing context extractor and pytest plugin tests pass

KRRT7 and others added 10 commits February 22, 2026 01:03
- Skip test entries with classname "pytest" or "_pytest" in JUnit XML
  parsing to avoid spurious warnings from pytest internal error reports
- Use find_spec instead of eager numpy import in pytest plugin
- Move protected_modules set to module-level frozenset to avoid
  recreating it on every _clear_lru_caches call
Cache inspect.getmembers() results per module so repeated loop
iterations skip the expensive rescan. Add tests for get_runtime_from_stdout,
should_stop, _set_nodeid, _get_total_time, _timed_out, logreport, and
setup/teardown hooks.
…rt adder

parse_code_and_prune_cst now returns cst.Module instead of str.
add_needed_imports_from_module accepts cst.Module | str, skipping re-parse
when a Module is passed. This eliminates the string round-trip that caused
comments to migrate from statement leading_lines to Module.header,
resulting in comments appearing above imports instead of at their
original position.
Move duplicate detection and add original-code detection before logging
and code printing, avoiding unnecessary work for candidates that would
be discarded anyway.
Objects with __slots__ but no __dict__ (e.g. textual.cache.LRUCache)
fell through all comparator branches, logging "Unknown comparator input
type" and returning False — causing spurious test mismatches.
@claude
Copy link
Contributor

claude bot commented Feb 23, 2026

PR Review Summary

Prek Checks

✅ All checks passed — ruff check and ruff format clean.

Mypy

⚠️ 247 mypy errors across the changed source files — all pre-existing (these files have extensive pre-existing type issues). No new type errors introduced by this PR.

Code Review

No critical issues found. Changes reviewed:

  1. parse_code_and_prune_cst returns cst.Module instead of str — Preserves the CST tree so add_needed_imports_from_module can skip re-parsing, fixing comment position drift. HASHING normalization correctly moved to the caller. The return ""raise ValueError change is safely caught by the existing except Exception in process_file_context.

  2. add_needed_imports_from_module accepts str | cst.Module — Clean handling with dst_code_fallback for error paths. When a cst.Module is passed, the re-parse is skipped correctly.

  3. Comparator __slots__ support — Correctly collects slots across the full MRO and compares attribute values. This only fires for pure-slots objects (those without __dict__), since hasattr(orig, "__dict__") is checked earlier in the function.

  4. Comparator hot-path optimizations — Module-level imports (guarded by HAS_* flags), module-level _EQUALITY_TYPES tuple, isinstance for dict view types, and isinstance(orig.__eq__, types.MethodType) replacing string comparison.

  5. _clear_lru_caches caching — Module scan results are cached in _module_clearables dict. The _scan_module_clearables method cleanly separates scanning from clearing. Protected modules moved to module-level frozenset.

  6. Skip identical/duplicate candidates earlynormalized_original computed once and compared before logging/compilation. Good optimization.

  7. Skip pytest-internal JUnit XML entries — Simple guard for pytest/_pytest class paths.

  8. Regex pre-compilation_NODEID_LOOP_PATTERN compiled once; uses subn to avoid double regex call.

Test Coverage

File main PR Δ
code_context_extractor.py 72% 72%
code_extractor.py 69% 69%
function_optimizer.py 18% 18%
comparator.py 57% 59% +2%
parse_test_output.py 58% 58%
pytest_plugin.py 33% 56% +23%
Total (changed files) 50% 52% +2%

Coverage improved across all changed files:

  • comparator.py: +2% from new TestSlotsObjects tests
  • pytest_plugin.py: +23% from comprehensive new tests covering get_runtime_from_stdout, should_stop, _set_nodeid, _get_total_time, _timed_out, _get_delay_time, pytest_runtest_logreport, pytest_runtest_setup/teardown, and _clear_lru_caches with module scanning
  • No coverage regressions

Note: 8 pre-existing test failures in test_tracer.py (same on both main and PR branch).


Last updated: 2026-02-23

@codeflash-ai
Copy link
Contributor

codeflash-ai bot commented Feb 23, 2026

⚡️ Codeflash found optimizations for this PR

📄 440% (4.40x) speedup for _apply_deterministic_patches in codeflash/verification/pytest_plugin.py

⏱️ Runtime : 2.32 milliseconds 430 microseconds (best of 250 runs)

A dependent PR with the suggested changes has been created. Please review:

If you approve, it will be merged into this PR (branch fix-importlens-bugs).

Static Badge

perf: optimize comparator with module-level constants and identity check
- Hoist _equality_types tuple and conditional library imports to module level
- Use isinstance for dict view type checks instead of string comparison
- Use isinstance for user-defined __eq__ detection instead of str(type(...))
- Add identity short-circuit (orig is new) at the top of comparator()

style: suppress false-positive PD011 on tf.SparseTensor.values
@KRRT7 KRRT7 force-pushed the fix-importlens-bugs branch from 15ed9a9 to c32abd1 Compare February 23, 2026 11:39
@KRRT7 KRRT7 merged commit f91278c into main Feb 23, 2026
23 of 28 checks passed
@KRRT7 KRRT7 deleted the fix-importlens-bugs branch February 23, 2026 11:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant