Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: doc content hash #1616

Merged
merged 2 commits into from
Jan 7, 2021
Merged

refactor: doc content hash #1616

merged 2 commits into from
Jan 7, 2021

Conversation

hanxiao
Copy link
Member

@hanxiao hanxiao commented Jan 7, 2021

refactor on #1611

@github-actions
Copy link

github-actions bot commented Jan 7, 2021

This PR closes: #1615

@jina-bot jina-bot added the size/S label Jan 7, 2021
@jina-bot jina-bot added area/core This issue/PR affects the core codebase component/type labels Jan 7, 2021
@github-actions
Copy link

github-actions bot commented Jan 7, 2021

Latency summary

Current PR yields:

  • 😶 index QPS at 1680, delta to last 3 avg.: -1%
  • 😶 query QPS at 31, delta to last 3 avg.: -4%

Breakdown

Version Index QPS Query QPS
current 1680 31
0.9.2 1718 32
0.9.1 1712 32
0.8.22 1706 32

Backed by latency-tracking. Further commits will update this comment.

@jina-bot jina-bot added the area/testing This issue/PR affects testing label Jan 7, 2021
@codecov
Copy link

codecov bot commented Jan 7, 2021

Codecov Report

Merging #1616 (d808f8c) into master (ccbd74b) will increase coverage by 0.91%.
The diff coverage is 100.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #1616      +/-   ##
==========================================
+ Coverage   84.11%   85.02%   +0.91%     
==========================================
  Files         127      127              
  Lines        6654     6699      +45     
==========================================
+ Hits         5597     5696      +99     
+ Misses       1057     1003      -54     
Impacted Files Coverage Δ
jina/types/document/multimodal.py 98.00% <100.00%> (+0.08%) ⬆️
jina/types/document/uid.py 84.61% <100.00%> (+2.79%) ⬆️
jina/logging/sse.py 92.75% <0.00%> (-2.56%) ⬇️
jina/logging/profile.py 69.84% <0.00%> (-0.56%) ⬇️
jina/flow/base.py 86.58% <0.00%> (-0.41%) ⬇️
jina/drivers/craft.py 100.00% <0.00%> (ø)
jina/types/ndarray/generic.py 100.00% <0.00%> (ø)
jina/drivers/encode.py 94.91% <0.00%> (+0.08%) ⬆️
jina/enums.py 96.59% <0.00%> (+0.09%) ⬆️
jina/jaml/__init__.py 95.93% <0.00%> (+0.09%) ⬆️
... and 16 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e763f17...d808f8c. Read the comment docs.

@hanxiao hanxiao merged commit 031b74e into master Jan 7, 2021
@hanxiao hanxiao deleted the fix-1615 branch January 7, 2021 04:49
def update_content_hash(self):
"""Update the document hash according to its content."""
self._document.content_hash = get_content_hash(self._document)
def update_content_hash(self, mask: Tuple[str] = ('id', 'chunks', 'matches', 'content_hash')) -> None:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I opened the issue to have exactly the opposite logic, I want the mask to be the fields that we want to include and not the ones we want to exclude.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I opened the issue to have exactly the opposite logic, I want the mask to be the fields that we want to include and not the ones we want to exclude.

I also think this is more resilient to any future adding of new fields

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/core This issue/PR affects the core codebase area/testing This issue/PR affects testing component/type size/S
Projects
None yet
4 participants