tpl function not performant

[himmakam]

I have a scenario of having one umbrella chart with many sub-charts as dependencies. When I have the templates folder in umbrella chart, helm lint takes more time like almost 30 to 40 minutes. When this folder is not there, helm lint returns very fast. Can I know the reason.running in debug mode not giving any logs. It is helm v3.

[hickeyma]

@himmakam Thanks for raising the issue.

How many sub-charts in the umbrella chart?

I have tested with 30 sub-charts and having a templates folder in the umbrella chart. helm lint is running and returning as expected. Would you be able to test with different number of sub-charts to see when the delay starts occurring? Have you nested sub-charts within sub-charts?

[technosophos]

Providing a reproducible case would be great. I am currently working this sprint on helm lint issues, so I can take a look if someone can point me to a reproducible case.

[technosophos]

I did recently uncover an issue where if you use tpl in your templates (a notoriously poorly performing function that should be avoided whenever possible), you can drive lint times quite high. Could that be the problem here?

[himmakam]

@himmakam Thanks for raising the issue.

How many sub-charts in the umbrella chart?

I have tested with 30 sub-charts and having a templates folder in the umbrella chart. helm lint is running and returning as expected. Would you be able to test with different number of sub-charts to see when the delay starts occurring? Have you nested sub-charts within sub-charts?

@hickeyma I have a total of 120 sub-charts and do not have any nested sub-charts.

[himmakam]

I did recently uncover an issue where if you use tpl in your templates (a notoriously poorly performing function that should be avoided whenever possible), you can drive lint times quite high. Could that be the problem here?

Might be because I use tpl many times in these charts. Also helm install also takes almost 30 to 45 minutes to parse all the 120 sub-charts I have.

[technosophos]

I did a test with a massive chart with hundreds of subcharts. I changed the tpl function to return an empty string. It cut that chart's helm lint down from 2.5 hours to 2.08 seconds. If you want to test reproducing this, all you have to do is comment out this: https://github.com/helm/helm/blob/master/pkg/engine/engine.go#L127-L152 and then recompile

[himmakam]

So are we saying helm lint has issue with umbrella chart and having massive sub-charts whereby it is taking 2.5 hrs? Can someone fix this?

[hickeyma]

@himmakam I think @technosophos is saying that the tpl function is not very performant. This becomes an issue when using the function in an umbrella chart containing a massive number of sub-charts, and then you run helm lint on it.

helm lint does not have an issue with an umbrella chart having massive number of sub-charts.

[himmakam]

ok. so can I know the alternative where helm lint works fine with umbrella and massive sub-charts having this tpl function. I see that upgrade/install/template does not take this much time. Only lint with tpl is the issue. I would like to know if there is any other alternative.

[himmakam]

Is there a plan to fix the performance issue related to tpl in future helm releases? Right now it is becoming very difficult to use this as it takes lot of time. Is there any other alternative for tpl?

[technosophos]

There is no plan to change the way tpl works.

[himmakam]

Is there any alternative for this, please

[himmakam]

@himmakam I think @technosophos is saying that the tpl function is not very performant. This becomes an issue when using the function in an umbrella chart containing a massive number of sub-charts, and then you run helm lint on it.

helm lint does not have an issue with an umbrella chart having massive number of sub-charts.

[himmakam]

We tried replacing tpl function with some other logic, but we are not successful. Basically would like to know what it takes to fix the tpl function. Also would like to know if there are any other alternatives for this so that we can proceed. Please do let me know if you need my sample code to look at it.

[hickeyma]

@himmakam Would you be able to show a snippet of how you are using tpl function?

[himmakam]

Sure. Lets assume that I have this below line in my properties file which needs to be mounted to the pod as part of the configmap -
memcache.server.url=${memcache.server.url}
Basically the above is a place-holder and dynamically I would like to pass the values for the above such that the final solution in pod could be as below -

memcache.server.url="https://10.10.11.11/remote-control
So in my tpl file, I have defined a property as below where 10.10.11.11 comes from parent values.yaml.

{{- define "memcacheserverurl" -}} {{- printf "https://%s/remote-control" .Values.global.service_url -}} {{- end -}}

And I have an another function, where am parsing all the property files and search for $, replacing with actual value using below logic.

{{- $updatedline := regexReplaceAll "\$\{(.*?)\}" $line "{{ include "${1}" $}}" }}
{{ tpl $updatedline $}}

Please do let me know if you need more information.

[himmakam]

Basically helm upgrade --install command takes 40 minutes to 1 hour for any update. When I comment the tpl, it just runs in 2 or 3 minutes. I would like to know if it is issue with using tpl function?

[technosophos]

I have already explained that it is DEFINITELY an issue involving tpl. To function in "proper isolation", tpl has to re-render the entire template tree on each invocation, creating a sandbox for each call. That is expensive. The bigger the chart, the more expensive the re-render is. And the more calls you make to tpl, the more frequently this re-render will be performed. tpl is simply incapable of scaling and should be avoided. It's basically the eval of Helm templates.

If someone wants to take another crack at implementing tpl while still maintaining the security model, I'm fine with that. But in lieu of that, the answer is simple: Don't use tpl. Use Helmfile or some other external tool for pregenerating, or ask in the Slack channel how other people did similar things.

[himmakam]

ok. thanks a lot

[hickeyma]

Thanks for the feedback @technosophos. I have edited the issue to describe the problem better.

It is open if someone in the community would like to try and improve the function.

[diafour]

Hello! Here is "another crack at implementing tpl" : #8371

I think it is compliant with "still maintaining the security model" as it uses Template.Clone to run tpl in isolation.

[github-actions]

This issue has been marked as stale because it has been open for 90 days with no activity. This thread will be automatically closed in 30 days if no further activity occurs.