Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: export Flow into set of k8s yamls #4089

Merged
merged 218 commits into from
Jan 10, 2022
Merged

feat: export Flow into set of k8s yamls #4089

merged 218 commits into from
Jan 10, 2022

Conversation

JoanFM
Copy link
Member

@JoanFM JoanFM commented Dec 22, 2021

No description provided.

jacobowitz and others added 30 commits December 13, 2021 09:42
* feat(proto): adjust proto for star routing

* feat(proto): generate proto files
* feat(k8s): add more labels to k8s deployments

* feat(network): refactor connection pool

* feat(network): refactor k8s pool
* feat(runtimes): refactor grpc dataruntime

* fix(tests): adapt worker runtime tests

* fix(import): fix import
* feat(head): new head runtime

* feat(head): new head runtime

* style: fix overload and cli autocomplete

* feat(network): improve proto comments

Co-authored-by: Jina Dev Bot <dev-bot@jina.ai>
* feat(worker): merge docs in worker runtime

* feat(tests): assert after clean up
* feat(zmq): purge zmq, zed, routing_table

* style: fix overload and cli autocomplete

* feat(zmq): adapt comment in dependency list

* style: fix overload and cli autocomplete

* fix(tests): fix type tests

Co-authored-by: Jina Dev Bot <dev-bot@jina.ai>
* feat(pea): adapt peas for star routing

* style: fix overload and cli autocomplete

* feat(pea): add tests

* feat(tests): add failing head pea test

Co-authored-by: Jina Dev Bot <dev-bot@jina.ai>
* feat(tests): integration tests for peas

* feat(pea): remove _inner_pea function
* feat(pod): adapt pods for star routing

* feat(pods): adapt basepod to star routing

* feat(pod): merge pod and compound pod

* feat(tests): fix tests

* style: fix overload and cli autocomplete

* feat(test): add container pea int test

* feat(ci): remove more unnecessary tests

* fix(tests): remove jinad runtime

* feat(ci): remove latency tracking

* fix(ci): fix ci def

* fix(runtime): enable runtime to be exited

* fix(tests): wrap runtime test in process

* fix(runtimes): remove unused runtimes

* feat(runtimes): improve cancel wait

* fix(ci): build test pip again in ci

* fix(tests): fix a test

* fix(test): run async in its own process

* feat(pod): include shard in activate msg

* fix(pea): dont join

* feat(pod): more debug out

* feat(grpc): manage channels properly

* feat(pods): remove exitfifo

* feat(network): add simple send retry mechanism

* fix(network): await pool close

* fix(test): always close grpc server in worker

* fix(tests): remove container pea from tests

* fix(tests): reorder tests

* fix(ci): split tests

* fix(ci): allow alias setting

* fix(test): skip a test

* feat(pods): address comments

Co-authored-by: Jina Dev Bot <dev-bot@jina.ai>
* feat: jinad pea

* feat: jinad pea

* test: remote peas

* test: toplogy tests with jinad

* ci: parallel jobs
* feat(tests): add pod integration tests

* fix(tests): make tests less flaky

* fix(test): fix test
* test(pea): remote pea simple topology

* test: remote pea topologies
* test: optimize k8s test

* test: increase timeout and use different namespace

* test: optimize k8s test

* test: build and load image when needed

* test: refactor k8s test

* test: fix image name error

* test: fix k8s image load

* test: fix typoe port expose

* test: update tests in connection pool and handling

* test: remove unused fixture

* test: parameterize docker images

* test: parameterize docker images

* test: parameterize docker images

* feat(k8s): adapt k8s pod for star routing

* fix(k8s): dont overwrite add/remove function in pool

* fix(k8s): some fixes

* fix(k8s): some more fixes

* fix(k8s): linting

* fix(tests): fix tests

* fix(tests): fix k8s unit tests

* feat(k8s): complete k8s integration test

* feat(k8s): finish k8s tests

* feat(k8s): fix test

* fix(tests): fix test with no name

* feat(k8s): unify create/replace interface

* feat(k8s): extract k8s port constants

* fix(tests): fix tests

* fix(tests): wait for runtime being ready in tests

* feat(k8s): address comments

Co-authored-by: bwanglzu <bo.wang@jina.ai>
* feat(flow): add routes

* feat(flow): adapt flow to star routing

* style: fix overload and cli autocomplete

* feat(flow): handle empty topologies

* feat(k8s): allow k8s pool disabling

* style: fix overload and cli autocomplete

* fix(test): fix test with mock

* fix(tests): fix more tests

* feat(flow): clean up tests

* style: fix overload and cli autocomplete

* fix(tests): fix more tests

* feat: add plot function (#3994)

* fix(tests): avoid hanging tests

* feat(flow): add type hinting

* fix(test): fix duplicate exec name in test

* fix(tests): fix more tests

* fix(tests): enable jinad test again

* fix(tests): random port fixture

* fix(style): replace quotes

Co-authored-by: Jina Dev Bot <dev-bot@jina.ai>
Co-authored-by: Joan Fontanals <joan.martinez@jina.ai>
jina/clients/mixin.py Show resolved Hide resolved
jina/flow/base.py Show resolved Hide resolved
tests/k8s/test_custom_resource_dir.py Show resolved Hide resolved
jina/flow/base.py Show resolved Hide resolved
jina/parsers/peapods/base.py Show resolved Hide resolved
jina/parsers/peapods/pea.py Outdated Show resolved Hide resolved
jina/peapods/pods/config/k8s.py Show resolved Hide resolved
scripts/get-all-test-paths.sh Outdated Show resolved Hide resolved
@github-actions github-actions bot added the area/setup This issue/PR affects setting up Jina label Jan 10, 2022
@JoanFM JoanFM closed this Jan 10, 2022
@JoanFM JoanFM reopened this Jan 10, 2022
@github-actions github-actions bot removed the area/setup This issue/PR affects setting up Jina label Jan 10, 2022
@JoanFM JoanFM closed this Jan 10, 2022
@JoanFM JoanFM reopened this Jan 10, 2022
jacobowitz
jacobowitz previously approved these changes Jan 10, 2022
Copy link
Contributor

@jacobowitz jacobowitz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good to me. I assume there is nothing we can do here to clean up the commit history? I find it always scary to let git figure out these things

@JoanFM
Copy link
Member Author

JoanFM commented Jan 10, 2022

looks good to me. I assume there is nothing we can do here to clean up the commit history? I find it always scary to let git figure out these things

I just smashed when merging

@github-actions
Copy link

📝 Docs are deployed on https://to_k8s_yaml--jina-docs.netlify.app 🎉

@JoanFM JoanFM merged commit 6e9e7ef into master Jan 10, 2022
@JoanFM JoanFM deleted the to_k8s_yaml branch January 10, 2022 17:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/cli This issue/PR affects the command line interface area/core This issue/PR affects the core codebase area/daemon area/docs This issue/PR affects the docs area/helper This issue/PR affects the helper functionality area/network This issue/PR affects network functionality area/testing This issue/PR affects testing component/client component/flow component/peapod component/resource size/XL
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

8 participants