control-service: Add python_version to Control Service API #1806
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ivakoleva
approved these changes
Mar 31, 2023
There was a problem hiding this comment.
LGTM;
May you link to PR description for convenience, the VEP-1739 described API and use cases?
For example, having a pinned python version corresponds to an exact python version requirement. Is there a use case of applying a lower bound only, so you still get your data job automatically upgraded to latest?
projects/control-service/projects/model/apidefs/datajob-api/api.yaml
Outdated
Show resolved
Hide resolved
projects/control-service/projects/model/apidefs/datajob-api/api.yaml
Outdated
Show resolved
Hide resolved
...lines_control_service/src/main/java/com/vmware/taurus/datajobs/DeploymentModelConverter.java
Show resolved
Hide resolved
...cts/pipelines_control_service/src/main/java/com/vmware/taurus/service/KubernetesService.java
Show resolved
Hide resolved
.../pipelines_control_service/src/test/java/com/vmware/taurus/service/JobImageDeployerTest.java
Show resolved
Hide resolved
|
python_version validation will be added in a separate PR, right? |
|
Shouldn't we return a default python version in case of empty python-version annotation? |
doks5
force-pushed
the
person/andonova/api-job-deployment
branch
from
9f5b111
to
45c1d4f
Compare
doks5
force-pushed
the
person/andonova/api-job-deployment
branch
from
45c1d4f
to
f2573cb
Compare
Added in this PR. |
In case of empty python-version, we set the defaultPythonVersion. |
doks5
force-pushed
the
person/andonova/api-job-deployment
branch
from
85f8c32
to
3470770
Compare
antoniivanov
reviewed
Apr 4, 2023
...lines_control_service/src/main/java/com/vmware/taurus/datajobs/DeploymentModelConverter.java
Outdated
Show resolved
Hide resolved
doks5
force-pushed
the
person/andonova/api-job-deployment
branch
from
3470770
to
37dc043
Compare
mivanov1988
reviewed
Apr 4, 2023
...elines_control_service/src/main/java/com/vmware/taurus/service/deploy/DeploymentService.java
Outdated
Show resolved
Hide resolved
mivanov1988
reviewed
Apr 4, 2023
...elines_control_service/src/main/java/com/vmware/taurus/service/deploy/DeploymentService.java
Outdated
Show resolved
Hide resolved
|
Have you seen the Codacy issues? |
antoniivanov
reviewed
Apr 4, 2023
...elines_control_service/src/main/java/com/vmware/taurus/service/deploy/DeploymentService.java
Outdated
Show resolved
Hide resolved
antoniivanov
reviewed
Apr 4, 2023
...elines_control_service/src/main/java/com/vmware/taurus/service/deploy/DeploymentService.java
Outdated
Show resolved
Hide resolved
As part of the effort to provide users with the ability to specify the python release version, which they want their data jobs to be deployed with, we need to expose a property in the Control Service's API and ensure that the configuration is stored in the database and the kubernetes objects. This change exposes a `python_version` property in the api schemas of the DataJobDeployment and DataJobDeploymentStatus components, and propagates the property to the database model and kubernetes resources. All of this is done as one change, to avoid unexpected behaviour or regressions. Testing Done: New and modified existing tests. Signed-off-by: Andon Andonov <andonova@vmware.com>
Signed-off-by: Andon Andonov <andonova@vmware.com>
Signed-off-by: Andon Andonov <andonova@vmware.com>
doks5
force-pushed
the
person/andonova/api-job-deployment
branch
from
37dc043
to
68e53cd
Compare
As discussed offline, these are fixed now. Thanks |
mivanov1988
reviewed
Apr 4, 2023
...elines_control_service/src/main/java/com/vmware/taurus/service/deploy/DeploymentService.java
Outdated
Show resolved
Hide resolved
mivanov1988
approved these changes
Apr 4, 2023
Signed-off-by: Andon Andonov <andonova@vmware.com>
antoniivanov
approved these changes
Apr 4, 2023
There was a problem hiding this comment.
The change looks good to me.
But I'd like to discuss following questions (at some point before the feature is considered implemented)
:
- What happens when administrators change the supported versions?
What is the right behaviour ?
- the same goes for the default version. What happens when administrators change teh default version ?
I don't think we've thought about and discussed it.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
As part of the effort to provide users with the ability to specify the python release version, which they want their data jobs to be deployed with, we need to expose a property in the Control Service's API and ensure that the configuration is stored in the database and the kubernetes objects.
This change exposes a
python_versionproperty in the api schemas of the DataJobDeployment and DataJobDeploymentStatus components, and propagates the property to the database model and kubernetes resources. All of this is done as one change, to avoid unexpected behaviour or regressions.VEP for Reference: https://github.com/vmware/versatile-data-kit/tree/main/specs/vep-1739-multiple-python-versions
Testing Done: New and modified existing tests.