Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: include func support #1187

Merged
merged 30 commits into from
Dec 4, 2023
Merged

feat: include func support #1187

merged 30 commits into from
Dec 4, 2023

Conversation

yxxhero
Copy link
Member

@yxxhero yxxhero commented Dec 2, 2023

@yxxhero yxxhero linked an issue Dec 2, 2023 that may be closed by this pull request
pkg/tmpl/context.go Outdated Show resolved Hide resolved
pkg/app/app.go Outdated Show resolved Hide resolved
docs/templating_funcs.md Outdated Show resolved Hide resolved
@yxxhero
Copy link
Member Author

yxxhero commented Dec 3, 2023

@mumoshu As a first-time implementation, do you think this is acceptable?

@yxxhero
Copy link
Member Author

yxxhero commented Dec 3, 2023

@mumoshu I will try to implement the nested mode.

@mumoshu
Copy link
Contributor

mumoshu commented Dec 3, 2023

@yxxhero See #1187 (comment). I think you need less code to achieve more!

@yxxhero
Copy link
Member Author

yxxhero commented Dec 3, 2023

@yxxhero See #1187 (comment). I think you need less code to achieve more!

@mumoshu ok. got it.

@yxxhero
Copy link
Member Author

yxxhero commented Dec 3, 2023

@mumoshu Another issue is that when loading the values of the environment or the values of the release, the basePath will change to the directory where the values file is located, which is also a problem.

@mumoshu
Copy link
Contributor

mumoshu commented Dec 3, 2023

the basePath will change to the directory where the values file is located

Yes! That's the expected behavior.

, which is also a problem.

Why is that?
Each template file, either it is a helmfile.yaml template or values.yaml template, is rendered in the context where the current directory / basePath.

That's also how each file locates colocated files with readFile.
You never want a values.yaml.gotmpl to behave differently depending on when you read the values.yaml.gotmpl from.

@yxxhero
Copy link
Member Author

yxxhero commented Dec 3, 2023

, which is also a problem.

because it lost the helmfile state location information. @mumoshu

@mumoshu
Copy link
Contributor

mumoshu commented Dec 3, 2023

@yxxhero Please see my updated reply! It is never lost. It must not depend on the "irrelevant" state. Each helmfile.yaml and values.yaml template need to be independently runnable and processable.

@mumoshu
Copy link
Contributor

mumoshu commented Dec 3, 2023

Otherwise, you break helmfile's premise of "each helmfile.yaml is independently runnable and reusable".

@yxxhero
Copy link
Member Author

yxxhero commented Dec 3, 2023

Otherwise, you break helmfile's premise of "each helmfile.yaml is independently runnable and reusable".

got it.

Signed-off-by: yxxhero <aiopsclub@163.com>
Signed-off-by: yxxhero <aiopsclub@163.com>
Signed-off-by: yxxhero <aiopsclub@163.com>
Signed-off-by: yxxhero <aiopsclub@163.com>
Signed-off-by: yxxhero <aiopsclub@163.com>
Signed-off-by: yxxhero <aiopsclub@163.com>
Signed-off-by: yxxhero <aiopsclub@163.com>
Signed-off-by: yxxhero <aiopsclub@163.com>
Signed-off-by: yxxhero <aiopsclub@163.com>
Signed-off-by: yxxhero <aiopsclub@163.com>
Signed-off-by: yxxhero <aiopsclub@163.com>
Signed-off-by: yxxhero <aiopsclub@163.com>
yxxhero and others added 2 commits December 3, 2023 14:01
Co-authored-by: Yusuke Kuoka <ykuoka@gmail.com>
Signed-off-by: yxxhero <aiopsclub@163.com>
Signed-off-by: yxxhero <aiopsclub@163.com>
Signed-off-by: yxxhero <aiopsclub@163.com>
Signed-off-by: yxxhero <aiopsclub@163.com>
Signed-off-by: yxxhero <aiopsclub@163.com>
Signed-off-by: yxxhero <aiopsclub@163.com>
Signed-off-by: yxxhero <aiopsclub@163.com>
Signed-off-by: yxxhero <aiopsclub@163.com>
Signed-off-by: yxxhero <aiopsclub@163.com>
Signed-off-by: yxxhero <aiopsclub@163.com>
Signed-off-by: yxxhero <aiopsclub@163.com>
Signed-off-by: yxxhero <aiopsclub@163.com>
Signed-off-by: yxxhero <aiopsclub@163.com>
@yxxhero
Copy link
Member Author

yxxhero commented Dec 3, 2023

@mumoshu all is ok.

if err != nil {
return nil, fmt.Errorf("failed to get working directory: %v", err)
}
files, err := c.fs.Glob(filepath.Join(wd, "_*.tpl"))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit but this could be simpified:

Suggested change
files, err := c.fs.Glob(filepath.Join(wd, "_*.tpl"))
files, err := c.fs.Glob(filepath.Join(c.basePath, "_*.tpl"))

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wd == c.basePath? I am not sure when to render the value file of release and the value file of env. @mumoshu

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

see: pkg/state/envvals_loader.go 62

r := tmpl.NewFileRenderer(ld.fs, filepath.Dir(f), tmplData)

// f is the path of the env value file. not helm statefile file path.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so I think wd will be better. @mumoshu

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah! Sorry for the confusion. The rule is that you shouldn't use wd at all.

Our readFile, for example, reads the file relative to the basePath, not wd, you know:

path = filepath.Join(c.basePath, filename)

The NewFileRenderer you pointed out takes basePath as the second arg, instead of wd, which aligns with the rule.

func NewFileRenderer(fs *filesystem.FileSystem, basePath string, data any) *FileRenderer {

If you used wd here, it ends up with that tpl files are loaded from the directory where helmfile.yaml is located, whereas readFile reads files from the directory where the (env) values.yaml template file is located, which is inconsistent.

var buf strings.Builder
if v, ok := includedNames[name]; ok {
if v > recursionMaxNums {
return "", errors.Wrapf(fmt.Errorf("unable to execute template"), "rendering template has a nested reference name: %s", name)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good job!
A test case for this error would make it perfect!

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mumoshu that logic comes from helm. I just copied and changed a bit.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It doesn't justify not having a test! We might change or refactor it in the future and test prevents breakage.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mumoshu make sense.

Signed-off-by: yxxhero <aiopsclub@163.com>
@yxxhero
Copy link
Member Author

yxxhero commented Dec 4, 2023

@mumoshu I added a test case for wd

see: test/integration/test-cases/include-template-func/input/helmfile.yaml.gotmpl

values/configmap.gotmpl can't be rendered. because helmfile can find the _helpers.tpl when use c.basePath.

Signed-off-by: yxxhero <aiopsclub@163.com>
Signed-off-by: yxxhero <aiopsclub@163.com>
@yxxhero
Copy link
Member Author

yxxhero commented Dec 4, 2023

@mumoshu @xiaomudk all is ok. please review and merge.

Copy link
Contributor

@mumoshu mumoshu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome work! Thank you very much for your contribution ☺️

@yxxhero yxxhero merged commit 7d6ed97 into main Dec 4, 2023
14 checks passed
@yxxhero yxxhero deleted the feat--include-func-support branch December 4, 2023 13:51
JadeHayes pushed a commit to Secretions/helmfile that referenced this pull request Dec 7, 2023
* feat: include func support

Signed-off-by: yxxhero <aiopsclub@163.com>
Signed-off-by: Jade Hayes <jade.hayes@dominodatalab.com>
@plevart
Copy link

plevart commented Dec 12, 2023

Eagerly awaiting this feature to be released.

@yxxhero
Copy link
Member Author

yxxhero commented Dec 25, 2023

@plevart released. please enjoy it. thanks so much.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add 'include' function identical to helm's
4 participants