Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: optional disable uses_b/a if parallel is 1 #1711

Merged
merged 1 commit into from
Jan 18, 2021

Conversation

JoanFM
Copy link
Member

@JoanFM JoanFM commented Jan 16, 2021

Changes introduced
Right now if we use uses_before or uses_after it is added even when parallel is 1. Although it can be desired to have a uses_after or uses_before it is more common to only want it when there are multiple Peas inside a Pod.

Add an option to disable them if parallel is 1. Like this one can keep the same Flow structure for sharding and no sharding and just parametrize the number of parallels or other options while not wasting CPU and network hops by these extra Peas

@JoanFM JoanFM requested a review from a team as a code owner January 16, 2021 11:44
@jina-bot jina-bot added size/S area/core This issue/PR affects the core codebase area/network This issue/PR affects network functionality component/peapod labels Jan 16, 2021
@github-actions
Copy link

github-actions bot commented Jan 16, 2021

Latency summary

Current PR yields:

  • 😶 index QPS at 1624, delta to last 3 avg.: -1%
  • 🐢🐢 query QPS at 31, delta to last 3 avg.: -6%

Breakdown

Version Index QPS Query QPS
current 1624 31
0.9.15 1648 33

Backed by latency-tracking. Further commits will update this comment.

@JoanFM JoanFM closed this Jan 16, 2021
@JoanFM JoanFM reopened this Jan 16, 2021
@codecov
Copy link

codecov bot commented Jan 16, 2021

Codecov Report

Merging #1711 (f53633e) into master (2fb2a5e) will decrease coverage by 0.05%.
The diff coverage is 100.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #1711      +/-   ##
==========================================
- Coverage   84.78%   84.73%   -0.06%     
==========================================
  Files         133      133              
  Lines        6856     6857       +1     
==========================================
- Hits         5813     5810       -3     
- Misses       1043     1047       +4     
Impacted Files Coverage Δ
jina/parsers/peapods/pods/base.py 100.00% <100.00%> (ø)
jina/drivers/querylang/select.py 79.31% <0.00%> (-6.90%) ⬇️
jina/flow/base.py 86.57% <0.00%> (-0.55%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update fe93542...f53633e. Read the comment docs.

@JoanFM JoanFM force-pushed the refactor-do-not-force-uses-before-after branch 2 times, most recently from 3fac3af to a6e79fa Compare January 17, 2021 18:48
Copy link
Member

@florian-hoenicke florian-hoenicke left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we add a small test here?

@JoanFM
Copy link
Member Author

JoanFM commented Jan 17, 2021

Can we add a small test here?

sure, will

@JoanFM JoanFM force-pushed the refactor-do-not-force-uses-before-after branch from a6e79fa to f53633e Compare January 18, 2021 08:38
@JoanFM
Copy link
Member Author

JoanFM commented Jan 18, 2021

Can we add a small test here?

Done!

@jina-bot jina-bot added the area/testing This issue/PR affects testing label Jan 18, 2021
Copy link
Member

@nan-wang nan-wang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM👍

@JoanFM JoanFM merged commit 1d5df92 into master Jan 18, 2021
@JoanFM JoanFM deleted the refactor-do-not-force-uses-before-after branch January 18, 2021 19:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/core This issue/PR affects the core codebase area/network This issue/PR affects network functionality area/testing This issue/PR affects testing component/peapod size/S
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants