New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix: filtering missing keys accounts for values #1694
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
When this passes, please pay attention to next jina hub
update since ZarrIndexer
may be sensitive to this change
@@ -162,17 +162,31 @@ def flush(self): | |||
except: | |||
pass | |||
|
|||
def _filter_nonexistent_keys(self, keys: Iterator, existent_keys: Iterator, check_path: str): | |||
indices_to_drop = [] | |||
def _filter_nonexistent_keys_values(self, keys: Iterator, values: Iterator, existent_keys: Iterator, check_path: str) -> Tuple[List, List]: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
can the typehints be more specific as per which kind of iterators it gets?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not really. Each of the indexers have their own types. I had to add IgnoreWarnings comment on some
Codecov Report
@@ Coverage Diff @@
## master #1694 +/- ##
==========================================
+ Coverage 84.82% 85.07% +0.24%
==========================================
Files 133 133
Lines 6833 6833
==========================================
+ Hits 5796 5813 +17
+ Misses 1037 1020 -17
Continue to review full report at Codecov.
|
Latency summaryCurrent PR yields:
Breakdown
Backed by latency-tracking. Further commits will update this comment. |
5c8a5e3
to
76f9ef9
Compare
76f9ef9
to
1d196d1
Compare
No description provided.