Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix rendering of flyte-core and flyteagent charts #5048

Merged
merged 2 commits into from
Mar 26, 2024

Conversation

pbrogan12
Copy link
Contributor

While testing out v1.11.0 ran into a couple of issues when enabling flyteagent from the flyte-core chart.

We define the templates flyteagent.* in both flyte-core and flyteagent charts.
Since template names are global across top level charts/subcharts, I ran into an issue dereferencing podValues.
In this case it was using the flyte-core definition of the template instead of flyteagent's.

Error: template: flyte-core/charts/flyteagent/templates/agent/deployment.yaml:17:17: executing "flyte-core/charts/flyteagent/templates/agent/deployment.yaml" at <include "flyteagent.podLabels" .>: error calling include: template: flyte-core/templates/_helpers.tpl:122:16: executing "flyteagent.podLabels" at <.Values.flyteagent.podLabels>: nil pointer evaluating interface {}.podLabels

There is also a small typo in the flyteagent template flyteagent.podLabels, which results in the flyteagent chart not being able to render if podValues is set.

Tracking issue

https://github.com/flyteorg/flyte/issues/

Why are the changes needed?

What changes were proposed in this pull request?

How was this patch tested?

Setup process

Screenshots

Check all the applicable boxes

  • I updated the documentation accordingly.
  • All new and existing tests passed.
  • All commits are signed-off.

Related PRs

Docs link

Copy link

welcome bot commented Mar 13, 2024

Thank you for opening this pull request! 🙌

These tips will help get your PR across the finish line:

  • Most of the repos have a PR template; if not, fill it out to the best of your knowledge.
  • Sign off your commits (Reference: DCO Guide).

@dosubot dosubot bot added size:S This PR changes 10-29 lines, ignoring generated files. bug Something isn't working labels Mar 13, 2024
Signed-off-by: Patrick Brogan <pbrogan12@gmail.com>
Copy link
Member

@pingsutw pingsutw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for fixing it. cc @ddl-ebrown did you run into this issue?

Copy link

codecov bot commented Mar 13, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 58.99%. Comparing base (e07084c) to head (edc72ed).
Report is 15 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #5048      +/-   ##
==========================================
- Coverage   59.00%   58.99%   -0.02%     
==========================================
  Files         645      645              
  Lines       55578    55578              
==========================================
- Hits        32792    32786       -6     
- Misses      20194    20200       +6     
  Partials     2592     2592              
Flag Coverage Δ
unittests 58.99% <ø> (-0.02%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@yini7777
Copy link
Contributor

@pingsutw I encountered this issue when trying to upgrade to v1.11.0 too.

@pingsutw
Copy link
Member

@yini7777 did you enable the flyte agent as well?

@yini7777
Copy link
Contributor

@pingsutw Yes, I did enable the flyteagent.

@pbrogan12
Copy link
Contributor Author

generate_helm workflow failing due to the spark-operator helm repo not being not available.
kubeflow/spark-operator#1926 looks like it is becoming a part of the kubeflow org.
@pingsutw ok if update to the new location? should I make a separate PR?

@pingsutw
Copy link
Member

@pingsutw ok if update to the new location?

yes, please do it, thanks!

@dosubot dosubot bot added size:M This PR changes 30-99 lines, ignoring generated files. and removed size:S This PR changes 10-29 lines, ignoring generated files. labels Mar 14, 2024
Signed-off-by: Patrick Brogan <pbrogan12@gmail.com>
@ddl-ebrown
Copy link
Contributor

Thanks for fixing it. cc @ddl-ebrown did you run into this issue?

No we haven't run across this issue that I'm aware of - we're using the charts as-is without any issues right now.

We are running forks of the charts internally and:

  • we're a little ahead in some cases b/c of PRs I've put up that haven't landed in this repo yet
  • there are a few extra bits we need (like network policies)

I'll run through an upgrade to 1.11 shortly and report back

@ddl-ebrown
Copy link
Contributor

ddl-ebrown commented Mar 14, 2024

I should also note that the last published flyte/flyteagent chart is v0.1.10. Is that chart just going to be a part of flyte-core going forward or should it be shipped separately? The reason we probably didn't run into this problem is because we're deploying from flyte/flyteagent and NOT flyte/flyte-core

From a consumer standpoint that needs to install multiple agents, being able to use a single parameterized agent chart that ships standalone is easier IMHO.

I'll also note that we use Helmfile -- so by having a standalone Flyte agent chart that's configurable, it's very easy to install multiple releases of the chart that each correspond to a different agent.

UPDATE: I see that the chart with v0.1.10 is getting pushed over with different contents. This is probably not a good idea @davidmirror-ops

@@ -31,7 +31,7 @@ app.kubernetes.io/managed-by: {{ .Release.Service }}
{{- define "flyteagent.podLabels" -}}
{{ include "flyteagent.labels" . }}
{{- with .Values.podLabels }}
{{- toYaml . }}
{{ toYaml . }}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 I can confirm that this change is correct

This is what I have in our local copy of the chart, so it would appear that I accidentally did not carry it over into my PR at #4756 (fb9ffd5#diff-40d80930ba32a1baa6eaca2a0694d9b0d21b8d72fc71fd71405b2973fde1a5aaR34)

Apologies!

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will a new version of the chart be published before the 1.12 release w/ these fixes?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we can certainly have this out as a patch release before 1.12.

@dosubot dosubot bot added the lgtm This PR has been approved by a maintainer label Mar 22, 2024
@davidmirror-ops davidmirror-ops merged commit cf7c638 into flyteorg:master Mar 26, 2024
48 of 49 checks passed
Copy link

welcome bot commented Mar 26, 2024

Congrats on merging your first pull request! 🎉

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working lgtm This PR has been approved by a maintainer size:M This PR changes 30-99 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants