Performance fix: lazier download of repodata#12
Conversation
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #12 +/- ##
=======================================
Coverage 99.29% 99.30%
=======================================
Files 15 15
Lines 853 859 +6
=======================================
+ Hits 847 853 +6
Misses 6 6 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
There was a problem hiding this comment.
Pull Request Overview
This PR refactors CondaReleaseSource to lazily fetch and cache repository data, improving performance by avoiding redundant downloads.
- Added a
_repodata_cacheattribute and replaced direct_repodataassignment with a lazy-loaded@property. - Updated merge loops to populate the cache instead of modifying the attribute directly.
- Maintained existing merge logic while deferring network and file operations until first access.
Comments suppressed due to low confidence (1)
src/spec0/releasesource.py:269
- Add unit tests for the
_repodataproperty to verify that repodata is fetched only once and that subsequent accesses return the cached result.
self._repodata_cache = None
| self._channel_platforms = channel_platforms | ||
| self._repodata_cache = None | ||
|
|
||
| @property |
There was a problem hiding this comment.
[nitpick] Consider using functools.cached_property instead of a manual cache to simplify the implementation and reduce boilerplate.
There was a problem hiding this comment.
hmm... I hadn't been aware of that. Nice trick to know, but I don't think I'll use it here. We might invalidate this cache at some point, and that's considerably uglier and less obvious with a functools.cached_property (you have to delete the entry from the instance's __dict__, as opposed to setting self._cache = None)
This pull request refactors the
CondaReleaseSourceclass insrc/spec0/releasesource.pyto introduce a caching mechanism for repository data, enhancing performance by delaying the download until needed. The key changes include replacing the_repodataattribute with a cached property and updating related logic to utilize this cache.Refactoring and Performance Improvements:
_repodata_cacheattribute and a_repodataproperty to manage repository data caching. This ensures that repository data is only fetched and processed once, improving efficiency._repodata_cacheinstead of directly modifying the_repodataattribute. This change aligns with the new caching mechanism.