Skip to content

chore: move LMDB imports into methods for performance#945

Draft
njzjz wants to merge 3 commits intodeepmodeling:masterfrom
njzjz:move-lmdb-imports
Draft

chore: move LMDB imports into methods for performance#945
njzjz wants to merge 3 commits intodeepmodeling:masterfrom
njzjz:move-lmdb-imports

Conversation

@njzjz
Copy link
Member

@njzjz njzjz commented Feb 27, 2026

Summary by CodeRabbit

  • Improvements
    • Enhanced LMDB data format initialization to ensure proper library dependency handling during object construction.

njzjz and others added 3 commits February 27, 2026 09:02
@codspeed-hq
Copy link

codspeed-hq bot commented Feb 27, 2026

Merging this PR will not alter performance

⚠️ Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

✅ 2 untouched benchmarks
⏩ 2 skipped benchmarks1


Comparing njzjz:move-lmdb-imports (dc22e44) with devel (ae134fe)

Open in CodSpeed

Footnotes

  1. 2 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

@coderabbitai
Copy link

coderabbitai bot commented Feb 27, 2026

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between ae134fe and dc22e44.

📒 Files selected for processing (1)
  • dpdata/lmdb/format.py

📝 Walkthrough

Walkthrough

The LMDBFormat class now includes an __init__ constructor that applies msgpack_numpy patching during instantiation. Module-level imports of lmdb and msgpack are deferred to individual methods, ensuring patching occurs at the appropriate initialization time.

Changes

Cohort / File(s) Summary
LMDBFormat Initialization & Import Deferral
dpdata/lmdb/format.py
Added __init__ constructor to trigger msgpack_numpy patching during object construction. Moved lmdb and msgpack imports from module-level to method-level in to_multi_systems, _dump_to_txn, and from_multi_systems to control patch timing.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~12 minutes

🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 50.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (2 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately describes the main change: moving LMDB imports (lmdb, msgpack) from module-level to inside methods for performance optimization.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@codecov
Copy link

codecov bot commented Feb 27, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 86.37%. Comparing base (ae134fe) to head (dc22e44).

Additional details and impacted files
@@           Coverage Diff           @@
##           master     #945   +/-   ##
=======================================
  Coverage   86.37%   86.37%           
=======================================
  Files          86       86           
  Lines        8086     8090    +4     
=======================================
+ Hits         6984     6988    +4     
  Misses       1102     1102           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant