Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Expose Executors as first class citizens #4597

Merged
merged 5 commits into from
Apr 12, 2022

Conversation

jacobowitz
Copy link
Contributor

@jacobowitz jacobowitz commented Apr 11, 2022

Expose Executors as first class citizens

Description

This adds to_k8s_yaml() and to_docker_compose_yaml() for the Executors. It works similar to serve() in the sense that it creates a Flow behind the scenes and uses its existing methods to provide a similar user experience.
These features are also added to the docs and some basic tests have been added.

  • Implement to_k8s_yaml() and to_docker_compose_yaml() as a static convenience wrapper for Executors
  • Update the docs: Executors can now be deployed via the k8s/docker-compose mechanism. Explain the difference between external/shared Executors.

Closes # (4502)

@github-actions github-actions bot added size/M area/core This issue/PR affects the core codebase area/testing This issue/PR affects testing labels Apr 11, 2022
@github-actions
Copy link

github-actions bot commented Apr 11, 2022

Latency summary

Current PR yields:

  • 🐎🐎🐎🐎 index QPS at 1292, delta to last 2 avg.: +17%
  • 🐎🐎🐎🐎 query QPS at 76, delta to last 2 avg.: +29%
  • 🐢🐢 avg flow time within 1.1495 seconds, delta to last 2 avg.: -20%
  • 🐢🐢 import jina within 0.4077 seconds, delta to last 2 avg.: -12%

Breakdown

Version Index QPS Query QPS Avg Flow Time (s) Import Time (s)
current 1292 76 1.1495 0.4077
3.2.10 1264 72 1.6016 0.3838
3.2.9 933 44 1.2827 0.5481

Backed by latency-tracking. Further commits will update this comment.

@github-actions github-actions bot added the area/docs This issue/PR affects the docs label Apr 12, 2022
@codecov
Copy link

codecov bot commented Apr 12, 2022

Codecov Report

Merging #4597 (25d12e2) into master (5e0ee17) will decrease coverage by 1.87%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master    #4597      +/-   ##
==========================================
- Coverage   87.70%   85.83%   -1.88%     
==========================================
  Files         117      117              
  Lines        8411     8479      +68     
==========================================
- Hits         7377     7278      -99     
- Misses       1034     1201     +167     
Flag Coverage Δ
jina 85.83% <100.00%> (-1.85%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
jina/orchestrate/flow/base.py 89.78% <100.00%> (-0.10%) ⬇️
jina/serve/executors/__init__.py 85.44% <100.00%> (+0.72%) ⬆️
jina/helloworld/multimodal/app.py 0.00% <0.00%> (-88.14%) ⬇️
jina/helloworld/multimodal/my_executors.py 69.79% <0.00%> (-27.52%) ⬇️
jina/serve/networking.py 76.53% <0.00%> (-9.23%) ⬇️
jina/serve/runtimes/head/__init__.py 83.70% <0.00%> (-8.73%) ⬇️
jina/types/mixin.py 62.50% <0.00%> (-6.25%) ⬇️
jina/types/request/data.py 92.92% <0.00%> (-5.06%) ⬇️
jina/clients/request/__init__.py 69.56% <0.00%> (-4.35%) ⬇️
.../runtimes/request_handlers/data_request_handler.py 89.47% <0.00%> (-3.51%) ⬇️
... and 11 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b05212d...25d12e2. Read the comment docs.

@jacobowitz jacobowitz marked this pull request as ready for review April 12, 2022 09:19
@github-actions
Copy link

📝 Docs are deployed on https://feat-executor-first-class--jina-docs.netlify.app 🎉

@JoanFM JoanFM merged commit 8a8c293 into master Apr 12, 2022
@JoanFM JoanFM deleted the feat-executor-first-class branch April 12, 2022 09:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/core This issue/PR affects the core codebase area/docs This issue/PR affects the docs area/testing This issue/PR affects testing size/M
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Expose Executor as a Service as First class citizen in Documentation.
2 participants