Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: detach BaseExecutable from Recursivity #1939

Merged
merged 4 commits into from
Feb 16, 2021
Merged

Conversation

JoanFM
Copy link
Member

@JoanFM JoanFM commented Feb 13, 2021

Changes introduced
Try to clarify the inheritance structure of FastRecursiveMixin and BaseRecursiveDriver by turniong the latter in another mixin.

Also try to change some Drivers into the new FastRecursiveMixin since seems to be an easy way to go.

Benefits

  • Current classes using FastRecursiveMixin also inherit from BaseRecursiveDriver, which makes their inheritance structure very ambiguous
  • Prepares the work for easy transition when possible from BaseRecursiveDriver to FastRecursiveMixin by putting recursivity traversal of the documents as a mixin deattaching from the core of an ExecutableDriver

@JoanFM JoanFM requested a review from a team as a code owner February 13, 2021 19:57
@jina-bot jina-bot added size/S area/core This issue/PR affects the core codebase component/driver labels Feb 13, 2021
@JoanFM JoanFM marked this pull request as draft February 13, 2021 20:02
@JoanFM JoanFM force-pushed the refactor-craftdriver branch 2 times, most recently from eade6a3 to e2159e3 Compare February 13, 2021 20:05
@github-actions
Copy link

github-actions bot commented Feb 13, 2021

Latency summary

Current PR yields:

  • 😶 index QPS at 1115, delta to last 3 avg.: +1%
  • 🐢🐢 query QPS at 18, delta to last 3 avg.: -8%

Breakdown

Version Index QPS Query QPS
current 1115 18
1.0.1 1097 18
1.0.0 1104 20

Backed by latency-tracking. Further commits will update this comment.

@codecov
Copy link

codecov bot commented Feb 13, 2021

Codecov Report

Merging #1939 (dd430ad) into master (e2971d2) will increase coverage by 2.72%.
The diff coverage is 100.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #1939      +/-   ##
==========================================
+ Coverage   83.95%   86.67%   +2.72%     
==========================================
  Files         148      148              
  Lines        7092     7093       +1     
==========================================
+ Hits         5954     6148     +194     
+ Misses       1138      945     -193     
Impacted Files Coverage Δ
jina/drivers/convertdriver.py 93.93% <100.00%> (ø)
jina/drivers/craft.py 100.00% <100.00%> (ø)
jina/drivers/encode.py 94.20% <100.00%> (ø)
jina/drivers/evaluate.py 98.27% <100.00%> (ø)
jina/drivers/multimodal.py 91.89% <100.00%> (+0.22%) ⬆️
jina/drivers/querylang/filter.py 100.00% <100.00%> (ø)
jina/drivers/querylang/reverse.py 100.00% <100.00%> (ø)
jina/drivers/querylang/select.py 89.65% <100.00%> (ø)
jina/drivers/querylang/slice.py 93.75% <100.00%> (ø)
jina/drivers/querylang/sort.py 100.00% <100.00%> (ø)
... and 10 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e2971d2...dd430ad. Read the comment docs.

@jina-bot jina-bot added the area/testing This issue/PR affects testing label Feb 13, 2021
@JoanFM JoanFM marked this pull request as ready for review February 13, 2021 20:14
@jina-bot jina-bot added size/M and removed size/S labels Feb 13, 2021
@JoanFM JoanFM closed this Feb 13, 2021
@JoanFM JoanFM reopened this Feb 13, 2021
Copy link
Member

@nan-wang nan-wang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

jina/drivers/__init__.py Show resolved Hide resolved
nan-wang
nan-wang previously approved these changes Feb 15, 2021
@maximilianwerk
Copy link
Member

I would not add the FastRecursiveMixin that often, but rather stay with the old one. Reason: When we change the traversal in the DocumentSet as a follow up, we don't need to fix that many drivers immediately. But I am also fine with merging away and using the SegmentDriver as an example how to do the following refactoring.

@JoanFM
Copy link
Member Author

JoanFM commented Feb 15, 2021

I would not add the FastRecursiveMixin that often, but rather stay with the old one. Reason: When we change the traversal in the DocumentSet as a follow up, we don't need to fix that many drivers immediately. But I am also fine with merging away and using the SegmentDriver as an example how to do the following refactoring.

Let's keep it as it is now, there are 2 or 3 now, and the change should be inmediate

Copy link
Member

@nan-wang nan-wang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM👍

@nan-wang nan-wang merged commit d153381 into master Feb 16, 2021
@nan-wang nan-wang deleted the refactor-craftdriver branch February 16, 2021 06:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants