feat: remove head for non sharded deployments #4517

jacobowitz · 2022-03-18T12:48:55Z

Remove heads for non sharded Deployments

Description

This PR removes the head for non sharded deployments. This means if shards are set to 1 for a Deployment, there will be no head created. In these cases the Gateway communicates directly with the workers.
Also reducing requests in case of multiple needs is moved to the Gateway now as we cant rely on the fact to due it at the head anymore.

This is a breaking change 💥 It breaks uses_before/uses_after for non sharded Deploments. It also changes the way we interact with external Executors. Additionally rolling_update and scale are removed from the Flow.

Before we do a release we should do the follow up issues around (external) Deployments as first class citizens.

Remove RollingUpdate/Scale feature from Flow/Deployment/Pod/JinaD
Set all worker Pods in the Gateway (Outside K8s)
Make the gateway send to worker Pods
Dont create heads for Deployments with no shards
Implement reducing after multiple needs in the Gateway
Adapt tests
Change docs

Closes #4503 (issue)

codecov · 2022-03-18T12:54:06Z

Codecov Report

Merging #4517 (23aec28) into master (70c931b) will increase coverage by 1.52%.
The diff coverage is 91.95%.

@@            Coverage Diff             @@
##           master    #4517      +/-   ##
==========================================
+ Coverage   85.95%   87.47%   +1.52%     
==========================================
  Files         115      115              
  Lines        8353     8298      -55     
==========================================
+ Hits         7180     7259      +79     
+ Misses       1173     1039     -134

Flag	Coverage Δ
jina	`87.47% <91.95%> (+7.28%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
jina/excepts.py	`100.00% <ø> (ø)`
jina/parsers/orchestrate/deployment.py	`100.00% <ø> (ø)`
jina/serve/runtimes/head/__init__.py	`92.42% <ø> (-0.12%)`	⬇️
jina/orchestrate/deployments/__init__.py	`87.46% <80.95%> (+4.90%)`	⬆️
jina/serve/networking.py	`84.31% <83.33%> (-2.81%)`	⬇️
jina/orchestrate/flow/base.py	`89.95% <95.23%> (+7.35%)`	⬆️
...a/orchestrate/deployments/config/docker_compose.py	`100.00% <100.00%> (+0.57%)`	⬆️
jina/orchestrate/deployments/config/k8s.py	`100.00% <100.00%> (+0.64%)`	⬆️
jina/parsers/orchestrate/runtimes/remote.py	`100.00% <100.00%> (ø)`
jina/serve/runtimes/gateway/__init__.py	`100.00% <100.00%> (ø)`
... and 22 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0a75d9c...23aec28. Read the comment docs.

github-actions · 2022-03-18T12:54:52Z

Latency summary

Current PR yields:

🐢🐢 index QPS at 992, delta to last 2 avg.: -18%
🐢🐢 query QPS at 48, delta to last 2 avg.: -24%
🐎🐎🐎🐎 avg flow time within 1.509 seconds, delta to last 2 avg.: +31%
🐎🐎🐎🐎 import jina within 0.5272 seconds, delta to last 2 avg.: +18%

Breakdown

Version	Index QPS	Query QPS	Avg Flow Time (s)	Import Time (s)
current	992	48	1.509	0.5272
`3.2.10`	1068	56	1.2347	0.4858
`3.2.9`	1371	71	1.0578	0.4008

Backed by latency-tracking. Further commits will update this comment.

daemon/stores/deployments.py

github-actions · 2022-03-21T11:08:29Z