Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add management of Helm applications #354

Open
wants to merge 34 commits into
base: main
Choose a base branch
from

Conversation

konstan
Copy link
Contributor

@konstan konstan commented Apr 26, 2024

@konstan konstan marked this pull request as draft April 26, 2024 16:14
@konstan konstan marked this pull request as ready for review May 1, 2024 13:32
@konstan konstan marked this pull request as draft May 1, 2024 13:32
@konstan konstan requested review from schaubl and 0xbase12 May 27, 2024 08:07
@konstan
Copy link
Contributor Author

konstan commented May 28, 2024

@giovannibianco because we don't use Roles and ClusterRoles I removed them from the code. To have them still available I created this gist https://gist.github.com/konstan/ebcc78ece247a6febc3d1da8176b04c5

Copy link
Contributor

github-actions bot commented May 28, 2024

Test Results

34 tests  ±0   34 ✅ ±0   1s ⏱️ ±0s
 1 suites ±0    0 💤 ±0 
 1 files   ±0    0 ❌ ±0 

Results for commit 25c6e89. ± Comparison against base commit 484a926.

♻️ This comment has been updated with latest results.

@konstan konstan marked this pull request as ready for review May 29, 2024 17:34
Copy link
Contributor

@0xbase12 0xbase12 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved, just a comment about a fixme note

nuvla/job_engine/connector/kubernetes.py Show resolved Hide resolved
nuvla/job_engine/job/actions/utils/deployment_utils.py Outdated Show resolved Hide resolved
nuvla/job_engine/connector/utils.py Outdated Show resolved Hide resolved
nuvla/job_engine/connector/utils.py Show resolved Hide resolved
nuvla/job_engine/connector/utils.py Outdated Show resolved Hide resolved
nuvla/job_engine/connector/utils.py Outdated Show resolved Hide resolved
nuvla/job_engine/connector/kubernetes.py Outdated Show resolved Hide resolved
Comment on lines -79 to +95
_, services = connector.update(**kwargs)
if connector_name == HELM_CONNECTOR_KIND:
result, services, release = connector.update(deployment, **kwargs)
self.app_helm_release_params_update(release)
else:
result, services = connector.update(**kwargs)
if result:
self.job.set_status_message(result)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Having the same method returning different number of values is disturbing me.

The idea of having this connector interface is that (as much as possible) we don't have to write code specific to a connector in deployment_*.py files.
But here we have almost everywhere "if HELM_CONNECTOR".

Is there a way to improve that ?

For example we could update the interface of update() and start() to always return something like (result, services, extra: dict).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed. But to be precise, there are only two places where the Helm app deployment is treated special way

Screenshot 2024-06-13 at 10 23 00

One is where you are pointing and another one is in getting application state.

I thought about changing the return value signature of the interface, but in the end resorted to the if as it was least intrusive.
If you are as well suggesting the three-tuple as return value for start() and update(), I will do that.
I'll see what can be done with get_application_state().

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TODO: refactor. Look for TODOs in the code.

Comment on lines 46 to 51
def store_yaml_file(data: dict, file_name, directory_path='.',
allow_unicode=True) -> str:
file_path = os.path.join(directory_path, file_name)
with open(file_path, 'w') as fd:
yaml.safe_dump(data, fd, allow_unicode=allow_unicode)
return file_path
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please see my other comments about writing to files that are not deleted or that are not ensured to be deleted.

In this case this method seems to be used only from one place (line 193) which give a temporary directory as directory_path but I think we need to ensure here that the directory is a temporary one.
(for example by creating the temporary directory here and returning its path instead of getting it as a function argument)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is handled by contract by the caller. Docu will be provided.

"""
def __init__(self, **kwargs):
super().__init__(**kwargs)
self.helm = Helm(_NUVLAEDGE_SHARED_PATH, **kwargs)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of _NUVLAEDGE_SHARED_PATH, job.nuvlaedge_shared_path should be used.

⚠️ Using this path will not work for helm deployment on a cloud k8s cluster (not a NE).
This class is used for both deployment on NE and on Cloud.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will be done.

@konstan konstan changed the title add management of Helm applications feat: add management of Helm applications Jun 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants