Proposal: selector for applications to be installed

[uesyn]

I use aqua to bootstrap my deveopment environment which be cleaned up periodically.
I don't want take a time to bootstrap, so I install applications in advance which I want to use them immediately, but I install the others lazily.

To achieve this, I have two aqua.yaml files, one is to be used to install in advance, another is to be used to install lazily.

$ aqua --config=/path/to/aqua_in_advance.yaml install
$ aqua --config=/path/to/aqua_lazy.yaml install --only-links

Then, I use AQUA_GLOBAL_CONFIG to merge them.
If aqua would be able to select applications to be installed, I can manage applications in one configuration file, and I don't have to use AQUA_GLOBAL_CONFIG to merge it.

However, I also understand that this is not a common situation.

So, If there are a lot of usecases to select applications to install like I do, I would like you to consider it.
In this Issue, I would like to collect many such use cases.

[suzuki-shunsuke]

Thank you for your proposal.
ref. #437 (comment)

[sheldonhull]

I ran into this with trying to use with github actions. I want to pass it aqua.ci.yaml and no matter what I did I found it tried to install every package, including the aqua.yaml + the ci.yml version. This is pretty important, and the lazy install didn't seem to work smoothly with calling certain cli tasks. I'll try again, but worth mentioning!

[suzuki-shunsuke]

I ran into this with trying to use with github actions. I want to pass it aqua.ci.yaml and no matter what I did I found it tried to install every package, including the aqua.yaml + the ci.yml version.

Even if -c option is set aqua reads aqua.yaml so this is an expected behavior.
When -c option is set maybe aqua should not read aqua.yaml, but I don't want to change the behavior to keep the compatibility.
One idea is to add a new option instead of -c and if the option is set aqua reads only the specified file. In that case,
I'm not sure whether global configuration should be read or not.
This idea should be handled in the other issue.

Or as a workaround, you can rename aqua.ci.yaml to aqua.yaml in CI before running aqua i.

$ mv aqua.ci.yaml aqua.yaml

the lazy install didn't seem to work smoothly with calling certain cli tasks

If Lazy Install has an issue, please create a new issue.

[suzuki-shunsuke]

I've designed the specification based on your proposal.

• Provide a tag attribute for a package and allow this to filter the packages installed in CI or other environments #1330
• feat: add lazy install option to package config #437 (comment)

And I've created the pull request. #1336

Note

aqua should be simple, so we should be careful to add features, especially adding command line options and fields in aqua.yaml.
If you feel this specification is too complicated, please let us know.

Specification

Add the optional field tags to aqua.yaml's package.
This field is a string list of tags.

e.g.

---
registries:
- type: standard
  ref: v3.79.0 # renovate: depName=aquaproj/aqua-registry
packages:
- name: suzuki-shunsuke/tfcmt@v3.2.0
  tags:
    - test
    - foo
- name: suzuki-shunsuke/github-comment@v4.0.0
- name: cli/cli@v2.0.0
  tags:
    - bar
    - foo

And add the following command line options to the commands aqua install and aqua cp.

• --tags (-t) (string): When this option is set, only packages that have specifed tags are installed. You can specify multiple tags joining with , (e.g. -t ci,test)
• --exclude-tags (string): When this option is set, packages that have specifed tags aren't installed. You can specify multiple tags joining with , (e.g. -exclude-tags ci,test)
• --ignore-tags (boolean): When this option is set, package tags are ignored and all packages are installed or copied

$ aqua i # Install suzuki-shunsuke/github-comment@v4.0.0
$ aqua i -t test # Install suzuki-shunsuke/tfcmt@v3.2.0
$ aqua i --ignore-tags # Install suzuki-shunsuke/github-comment@v4.0.0 and suzuki-shunsuke/tfcmt@v3.2.0 and cli/cli@v2.0.0
$ aqua i -t foo,bar # Install suzuki-shunsuke/tfcmt@v3.2.0 and cli/cli@v2.0.0
$ aqua i --exclude-tags test # Install suzuki-shunsuke/github-comment@v4.0.0 and cli/cli@v2.0.0
$ aqua i --exclude-tags test -t foo # Install cli/cli@v2.0.0

Note that symbolic links of all packages are created regardless tags, so that you can execute all tools by Lazy Install and assure that tools are managed by aqua.

[suzuki-shunsuke]

@sheldonhull @uesyn Sorry to bother you, but do you have any opinion to the proposal? #441 (comment)
Does this proposal solve your problem?
I've published the prerelease version https://github.com/aquaproj/aqua/releases/tag/v1.22.0-0 , so you can try this feature.

Thanks

[uesyn]

@suzuki-shunsuke

Your proporsal is exactly what I have wanted!
In my usecase, --tags and --exclude-tags options is needed, and I tried to those flags. It worked perfectly:+1:
Thank you!

I think --ignore-tags option is complicated. In my intuition, --ignore-tags=true is default behavior, and I think the flag is unnecessary, because we can express the condition with --exclude-tags flag instead of --ignore-tags. (However, if there are few tags.)

[sheldonhull]

@sheldonhull @uesyn Sorry to bother you, but do you have any opinion to the proposal? #441 (comment)

Does this proposal solve your problem?

I've published the prerelease version https://github.com/aquaproj/aqua/releases/tag/v1.22.0-0 , so you can try this feature.

Thanks

I'm excited to try this. It will be at least 1 week minimum before I can try the prerelease. I am thankful for your effort on this and will definitely get back to you regardless.

I'm leveraging aqua as the setup for my go tooling in github actions so this will cut down the wasted installs and simplify things greatly.

I need to write up a blog post on hours I'm using with devcontainers and chezmoi. I can setup my full dev environment from scratch now in a min or so. so much better and easier to contribute to than asdf.

Even if the yaml gets big I can see this scaling. Only a few minor things remaining and I'd say it's a near flawless experience 🥂

[suzuki-shunsuke]

Sorry for late reply.

I think --ignore-tags option is complicated. In my intuition, --ignore-tags=true is default behavior, and I think the flag is unnecessary, because we can express the condition with --exclude-tags flag instead of --ignore-tags. (However, if there are few tags.)

Thank you for your feedback.
I agree.
Updated.

• Remove --ignore-tags
• When -t and --exclude-tags aren't set, all packages are installed

Specification

Add the optional field tags to aqua.yaml's package.
This field is a string list of tags.

e.g.

---
registries:
- type: standard
  ref: v3.79.0 # renovate: depName=aquaproj/aqua-registry
packages:
- name: suzuki-shunsuke/tfcmt@v3.2.0
  tags:
    - test
    - foo
- name: suzuki-shunsuke/github-comment@v4.0.0
- name: cli/cli@v2.0.0
  tags:
    - bar
    - foo

And add the following command line options to the commands aqua install and aqua cp.

• --tags (-t) (string): When this option is set, only packages that have specifed tags are installed. You can specify multiple tags joining with , (e.g. -t ci,test)
• --exclude-tags (string): When this option is set, packages that have specifed tags aren't installed. You can specify multiple tags joining with , (e.g. -exclude-tags ci,test)

$ aqua i # Install suzuki-shunsuke/tfcmt@v3.2.0 and suzuki-shunsuke/github-comment@v4.0.0 and cli/cli@v2.0.0
$ aqua i -t test # Install suzuki-shunsuke/tfcmt@v3.2.0
$ aqua i -t foo,bar # Install suzuki-shunsuke/tfcmt@v3.2.0 and cli/cli@v2.0.0
$ aqua i --exclude-tags test # Install suzuki-shunsuke/github-comment@v4.0.0 and cli/cli@v2.0.0
$ aqua i --exclude-tags test -t foo # Install cli/cli@v2.0.0

Note that symbolic links of all packages are created regardless tags, so that you can execute all tools by Lazy Install and assure that tools are managed by aqua.

[suzuki-shunsuke]

I'm excited to try this. It will be at least 1 week minimum before I can try the prerelease. I am thankful for your effort on this and will definitely get back to you regardless.

I'm leveraging aqua as the setup for my go tooling in github actions so this will cut down the wasted installs and simplify things greatly.

I need to write up a blog post on hours I'm using with devcontainers and chezmoi. I can setup my full dev environment from scratch now in a min or so. so much better and easier to contribute to than asdf.

Even if the yaml gets big I can see this scaling. Only a few minor things remaining and I'd say it's a near flawless experience 🥂

Thank you. I'm looking forward to your feedback and blog post!

[suzuki-shunsuke]

Published the prerelease version v1.22.0-1.
#441 (comment)

[uesyn]

@suzuki-shunsuke

Thank you!
As far as I confirmed v1.22.0-1, it works as expected.
This is a great feature! LGTM!

[suzuki-shunsuke]

Document: aquaproj/aquaproj.github.io#203

TODO: Add the description.

• Motivation, Background
• Usecase (Example)

[suzuki-shunsuke]

I'll release this feature this week (Tuesday or Wednesday).

[suzuki-shunsuke]

Released. https://github.com/aquaproj/aqua/releases/tag/v1.23.0
Thank you for your contribution!

[suzuki-shunsuke]

Published the document. https://aquaproj.github.io/docs/tutorial-extras/package-tag

I'll really appreciate if you share your usecase and how you use this feature for other users.
Thanks.