Skip to content

Conversation

@Peter554
Copy link
Collaborator

@Peter554 Peter554 commented Jun 10, 2025

A sketch of bulk import scanning via rust.

Ideas to make this nicer:

  • This change makes the existing ImportScanner python interface obsolete. We should likely instead introduce a BulkImportScanner interface, which processes all modules at once.
  • Many tests fail, mainly because rust does not respect the mock/stub filesystem. I'm unsure what to do about this / how much I care about this (the added snapshot test gives me a lot of confidence in the change, so maybe those older tests can just be deleted).

This seems to make import scanning a little bit faster on my machine, but the change doesn't seem to be really big.

@Peter554 Peter554 force-pushed the bulk-rust-import-scanning branch from c7bee51 to f0d4871 Compare June 10, 2025 19:57
@codspeed-hq
Copy link

codspeed-hq bot commented Jun 10, 2025

CodSpeed Instrumentation Performance Report

Merging #222 will degrade performances by 77.57%

Comparing Peter554:bulk-rust-import-scanning (f0d4871) with master (ce0d32f)

Summary

⚡ 3 improvements
❌ 2 regressions
✅ 17 untouched benchmarks

⚠️ Please fix the performance issues or acknowledge them on CodSpeed.

Benchmarks breakdown

Benchmark BASE HEAD Change
test_deep_layers_large_graph_kept 19.8 ms 17.2 ms +15.19%
test_no_chain 1.2 ms 1.1 ms +12.45%
test_no_chains 1.2 ms 1.1 ms +12.43%
test_build_django_from_cache_a_few_misses[350] 298.4 ms 726.3 ms -58.92%
test_build_django_uncached 145.2 ms 647.5 ms -77.57%

@Peter554
Copy link
Collaborator Author

Peter554 commented Aug 6, 2025

Closing this one since we're working on it in other PRs

@Peter554 Peter554 closed this Aug 6, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant