Notify user of a new recommended CLI version

[marckhouzam]

What this PR does / why we need it

This PR may seem very big but notice it has 1750 lines of new unit/e2e tests.

This PR teaches the CLI to detect a new release of the CLI itself and notify the user they should upgrade.
The notification is printed at most once every 24 hours by default.

The PR also implements a yaml data-store that the CLI can use to store different information that does not belong in the configuration file. This new file is located in $HOME/.config/.data-store.yaml. For this feature, the CLI stores the timestamp of the last time the user was notified to upgrade.

The PR also implements an API to read the Central Configuration obtained from the Central Repository of plugins, as introduced in #700. For this feature, the Central Configuration is read to detect any new version of the CLI having been released. The PR also updates the test central repo to improve the format of the central config.

Things to know:

1. The version notification includes the recommended version for Major, Minor and Patch (see the testing section below for an example)
   • In practice, there should never be a line for Major since we expect all users to be using some v1 version. If a user is still using the v0.90 version, then the line for the MAJOR version will be shown.
   • In practice, we mostly release minor versions of the CLI (the only patch up to date has been v0.90.1) so in most cases there will be a single line showing the new minor
2. Pre-releases are only recommended if the current CLI version is also a pre-release
3. The notification can be triggered by any CLI command, including plugin commands, except for tanzu __complete, tanzu completion and tanzu version
4. The notification is printed after the execution of the current command.
5. The version notification output is sent to stderr so that it does not affect potential parsing of the real command output
6. The variable TANZU_CLI_RECOMMEND_VERSION_DELAY_DAYS can be used to change the period of the notification (default 1 day) or to disable the feature by setting it to 0. If this variable is set to a negative number then the delay will be in seconds instead of days; this allows for testing.
7. If any of the recommended versions (Minor or Patch) is lower than the current version, it is ignored.
8. The format of the value for cli.core.cli_recommended_versions in the central config is changed from a comma-separated list to a yaml array of structs. With the latest central config API this is easier to parse, it is also easier for a human to read, and it even allows to add comments. The use of structs makes it future-proof as new fields can be added in the future without breaking existing CLIs.

When to update the central config

Whenever there is a new release of the CLI, be it major, minor, patch or even pre-release, we should add the new version to the production Central Config in the existing array of key cli.core.cli_recommended_versions and publish a new image of the production central repo. There is no rush to do it but no message about a new version will be printed until it is published. We probably only want to publish once the version is GA (except for pre-releases).

Real scenarios

When v1.3.0 is released with this feature

No notification will be printed since existing CLIs (pre v1.3.0) don't have this feature.
A notification of a new recommended version will only happen once we release a new minor or patch after the v1.3.0 version.

The central config should still be updated with the list of recommended version:

cli.core.cli_recommended_versions:
- version: v1.3.0
- version: v1.2.0
- version: v1.1.0
- version: v1.0.0
- version: v0.90.1

A new minor release of the CLI is published

As soon as the release is available, users can install it. However, no new message will be shown until we update the Central Configuration with this new version.

If a user installs the new version, no message will be printed either although the current central config recommends an older version, it is for this use case that we don't print a message to downgrade to an older version, as it would not be valid in this case.

When GA happens we then add the new minor version to the production Central Config in the existing array of key cli.core.cli_recommended_versions and publish a new image of the production central repo. At this time a running older CLI will notice the new version BUT only when it refreshes its central repo data, which will happen at most within 24 hours of usage (thanks to the essential plugins threshold of #637); once the CLI refreshes its central repo it will print the notification of the new version, at most once every 24 hours.

A new patch release of the CLI is published

Same as for a minor except that in the central config, the existing patch version of the same minor should be replaced. For example if the central config recommends v1.3.0 and we release v1.3.1, then the entry v1.3.0 should be replaced by v1.3.1.

A new major release of the CLI is published

Same as for a minor.
Note that although the CLI won't recommend a new major, we do want to include the new major in the central config so that people that start using that major can eventually get notifications of a new minor/patch for that major.

A new pre-release of the CLI is published

Depending on if this pre-release is for a new minor or not, the pre-release should be added (new minor) or updated (existing minor) into the central config.

If the pre-release is for a new patch version, then the recommended version of the previous patch should NOT be removed, and the pre-release should be added as a new recommended entry in the central config. For example, if v1.3.4 is a recommended version and a v1.3.5-alpha.1 is released, then the v1.3.4 entry should not be removed until the final v1.3.5 is available; the v1.3.5-alpha.1 version should be added as a new entry in the central config.

No need to wait for GA for a pre-release.

Which issue(s) this PR fixes

Fixes #706

Describe testing done for PR

Added unit tests. and E2E tests.

Manual testing:

Setup. Notice the recommended versions that are part of the test central repo (at the end of the setup):

$ make start-test-central-repo
[...]
$ tz config set env.TANZU_CLI_PLUGIN_DISCOVERY_IMAGE_SIGNATURE_VERIFICATION_SKIP_LIST localhost:9876/tanzu-cli/plugins/central:small
$ tz plugin source update default -u localhost:9876/tanzu-cli/plugins/central:small
[i] Refreshing plugin inventory cache for "localhost:9876/tanzu-cli/plugins/central:small", this will take a few seconds.
[i] Reading plugin inventory for "localhost:9876/tanzu-cli/plugins/central:small", this will take a few seconds.
[!] Skipping the plugins discovery image signature verification for "localhost:9876/tanzu-cli/plugins/central:small"

[ok] updated discovery source default
$ grep recommended_versions ~/.cache/tanzu/plugin_inventory/default/central_config.yaml -A10
cli.core.cli_recommended_versions:
- version: v2.1.0-alpha.2
- version: v2.0.2
- version: v1.5.0-beta.0
- version: v1.4.4
- version: v1.3.3
- version: v1.2.2
- version: v1.1.1
- version: v0.90.0
cli.core.some-string: "the meaning of life, the universe, and everything"
cli.core.some-int: 42

Run a CLI with a version that is not recommended:

$ make build BUILD_VERSION=v1.4.3

# Any command triggers the check for recommended versions.
# This is the very first check, so it will happen and then the 24 hours delay will start for the next check
$ tz plugin list
[i] The tanzu cli essential plugins are outdated and are being updated now. The update may take a few seconds.
[i] Installing plugins from plugin group 'vmware-tanzucli/essentials:v9.9.9'
[i] Installed plugin 'telemetry:v9.9.9' with target 'global' (from cache)

Standalone Plugins
  NAME         DESCRIPTION                                             TARGET      VERSION        STATUS
  appsv2       Applications on Kubernetes for TAP (SaaS distribution)  kubernetes  v0.1.1-beta.2  installed
  supplychain  supplychain management                                  kubernetes  v0.1.0-beta.2  installed
  telemetry    telemetry functionality                                 global      v9.9.9         installed
  workload     create, update, view and list Tanzu Workloads.          kubernetes  v0.1.0-beta.2  installed

==
Note: A new version of the Tanzu CLI is available. You are at version: v1.4.3.
To benefit from the latest security and features, please update to a recommended version:
  - v1.4.4

This message will print at most once per 24 hours until you update the CLI.
Set TANZU_CLI_RECOMMEND_VERSION_DELAY_DAYS to adjust this period (0 to disable).

# Run the command again (or any other) and notice there is no notification since the
# last notification was less than 24 hours ago
$ tz plugin list
Standalone Plugins
  NAME         DESCRIPTION                                             TARGET      VERSION        STATUS
  appsv2       Applications on Kubernetes for TAP (SaaS distribution)  kubernetes  v0.1.1-beta.2  installed
  supplychain  supplychain management                                  kubernetes  v0.1.0-beta.2  installed
  telemetry    telemetry functionality                                 global      v9.9.9         installed
  workload     create, update, view and list Tanzu Workloads.          kubernetes  v0.1.0-beta.2  installed

Run the CLI with a version that is the latest minor but not the latest patch, and notice only the newer patch is recommended:

$ make build BUILD_VERSION=v1.4.2

$ TANZU_CLI_RECOMMEND_VERSION_DELAY_DAYS=-5 tanzu plugin source list
  NAME     IMAGE
  default  localhost:9876/tanzu-cli/plugins/central:small

==
Note: A new version of the Tanzu CLI is available. You are at version: v1.4.2.
To benefit from the latest security and features, please update to a recommended version:
  - v1.4.4

This message will print at most once per 5 seconds until you update the CLI.
Set TANZU_CLI_RECOMMEND_VERSION_DELAY_DAYS to adjust this period (0 to disable).

Test that nothing is recommended if the recommended version is older

$ make build BUILD_VERSION=v1.4.5

$ TANZU_CLI_RECOMMEND_VERSION_DELAY_DAYS=-5 tanzu plugin source list
  NAME     IMAGE
  default  localhost:9876/tanzu-cli/plugins/central:small

Test the configuration of the notification delay

$ make build BUILD_VERSION=v1.3.0

# Remove the data-store.yaml file to pretend this is the first version check
$ rm ~/.config/tanzu/.data-store.yaml; \
tz plugin group search; \
sleep 7; \
TANZU_CLI_RECOMMEND_VERSION_DELAY_DAYS=-8 tz plugin search --name cluster -t k8s; \
sleep 1; \
TANZU_CLI_RECOMMEND_VERSION_DELAY_DAYS=-8 tz plugin search --name cluster -t k8s

# The first prints the notification
  GROUP                       DESCRIPTION                                 LATEST
  vmware-tanzucli/essentials  Desc for vmware-tanzucli/essentials:v9.9.9  v9.9.9
  vmware-tkg/default          Desc for vmware-tkg/default:v9.9.9          v9.9.9
  vmware-tkg/shortversion     Desc for vmware-tkg/shortversion:v9.9.9     v9.9.9
  vmware-tmc/tmc-user         Desc for vmware-tmc/tmc-user:v9.9.9         v9.9.9

Note: To view all plugin group versions available, use 'tanzu plugin group search --show-details'.

==
Note: A new version of the Tanzu CLI is available. You are at version: v1.3.0.
To benefit from the latest security and features, please update to a recommended version:
  - v1.4.4
  - v1.3.3

This message will print at most once per 24 hours until you update the CLI.
Set TANZU_CLI_RECOMMEND_VERSION_DELAY_DAYS to adjust this period (0 to disable).

# The next command is run 7 seconds later but we set TANZU_CLI_RECOMMEND_VERSION_DELAY_DAYS
# to 8 seconds, so there is no notification
  NAME     DESCRIPTION            TARGET      LATEST
  cluster  cluster functionality  kubernetes  v9.9.9

# One second later, when we again set TANZU_CLI_RECOMMEND_VERSION_DELAY_DAYS
# to 8 seconds, we get the notification as expected
  NAME     DESCRIPTION            TARGET      LATEST
  cluster  cluster functionality  kubernetes  v9.9.9

Note: A new version of the Tanzu CLI is available. You are at version: v1.3.0.
To benefit from the latest security and features, please update to a recommended version:
  - v1.4.4
  - v1.3.3

This message will print at most once per 8 seconds until you update the CLI.
Set TANZU_CLI_RECOMMEND_VERSION_DELAY_DAYS to adjust this period (0 to disable).

Run a CLI with a pre-release version and notice that pre-release recommendations are now shown:

$ make build BUILD_VERSION=v1.3.0-alpha.1

$ TANZU_CLI_RECOMMEND_VERSION_DELAY_DAYS=-5 tanzu context list
  NAME  ISACTIVE  TYPE

[i] Use '--wide' flag to view additional columns.

==
Note: A new version of the Tanzu CLI is available. You are at version: v1.3.0-alpha.1.
To benefit from the latest security and features, please update to a recommended version:
  - v1.5.0-beta.0
  - v1.3.3

This message will print at most once per 5 seconds until you update the CLI.
Set TANZU_CLI_RECOMMEND_VERSION_DELAY_DAYS to adjust this period (0 to disable).

Check the 3 commands that don't trigger the check and notice there is no notification:

$ TANZU_CLI_RECOMMEND_VERSION_DELAY_DAYS=-1 tanzu version
version: v1.3.0-alpha.1
buildDate: 2024-03-18
sha: cd9962ab5
arch: arm64

$ TANZU_CLI_RECOMMEND_VERSION_DELAY_DAYS=-1 tanzu completion bash
# bash completion V2 for tanzu                                -*- shell-script -*-
[...]
if [[ $(type -t compopt) = "builtin" ]]; then
    complete -o default -F __start_tanzu tanzu
else
    complete -o default -o nospace -F __start_tanzu tanzu
fi

# ex: ts=4 sw=4 et filetype=sh

$ TANZU_CLI_RECOMMEND_VERSION_DELAY_DAYS=-1 tanzu __complete c
completion	Output shell completion code
config	Configuration for the CLI
context	Configure and manage contexts for the Tanzu CLI
:4
Completion ended with directive: ShellCompDirectiveNoFileComp

Check that the notification is on stderr:

$ TANZU_CLI_RECOMMEND_VERSION_DELAY_DAYS=-1 tanzu plugin list > /dev/null

==
Note: A new version of the Tanzu CLI is available. You are at version: v1.3.0-alpha.1.
To benefit from the latest security and features, please update to a recommended version:
  - v1.5.0-beta.0
  - v1.3.3

This message will print at most once per 1 seconds until you update the CLI.
Set TANZU_CLI_RECOMMEND_VERSION_DELAY_DAYS to adjust this period (0 to disable).

Release note

On execution, the CLI will notify the user of an available new version of the CLI itself, at most once every 24 hours.

Additional information

Special notes for your reviewer

If you want to test this, you will need to use the latest test central repo.
If you haven't yet, don't forget to cleanup your test central repo cache to get the latest one:

$ cd tanzu-cli
$ \rm -rf hack/central-repo/registry-content
$ make start-test-central-repo

[anujc25]

I am still reviewing the PR but adding some early comments.

[vuil]

Based on a few comments, it might be good to understand procedurally how time-sensitive it is with regards to when to publish a CLI version and when to publish the recommendation data in the Central Configuration. Maybe update the PR description with this info?

[marckhouzam]

I was thinking about TANZU_CLI_RECOMMEND_VERSION_DELAY_SECONDS yesterday and I don't like that it is in seconds. This variable, unlike our many others, is directly meant for the users to set and for the feature of version detection, they will never set things to seconds. I even think that hours is too granular. I was thinking of changing this to TANZU_CLI_RECOMMEND_VERSION_DELAY_DAYS.

For testing, where we need a much smaller delay, instead of adding another variable, I was thinking of using negative values to indicate seconds, so TANZU_CLI_RECOMMEND_VERSION_DELAY_DAYS=7 would mean 7 days but TANZU_CLI_RECOMMEND_VERSION_DELAY_DAYS=-7 would mean 7 seconds.

Let me know if you disagree.

[marckhouzam]

Based on a few comments, it might be good to understand procedurally how time-sensitive it is with regards to when to publish a CLI version and when to publish the recommendation data in the Central Configuration. Maybe update the PR description with this info?

I have added a Real scenarios and a When to update the central config section to the PR description.

One scenario that may not be obvious is for pre-releases:

If the pre-release is for a new patch version, then the recommended version of the previous patch should NOT be removed, and the pre-release should be added as a new recommended entry in the central config. For example, if v1.3.4 is a recommended version and a v1.3.5-alpha.1 is released, then the v1.3.4 entry should not be removed until the final v1.3.5 is available; the v1.3.5-alpha.1 version should instead be added as a new entry in the central config.

[marckhouzam]

I've updated the API for the new data store to use the same approach than the Central Configuration API (marshall/unmarshall the value stored) as it is a nicer approach.

I have also added documentation about:

• the central configuration
• the CLI data store
• the detection of new CLI releases

[vuil]

Looks good to me. This is a really nice set of changes, @marckhouzam !

A few things:
I assume you plan to do some rebase/squashing. If you do, please retain a handful of logical commits so it will be easier for future reference.

Also, something to consider as a followup:
Over time, it will become more and more important that a CLI has up-to-date information about the central configuration, else might start to exhibit different/possibly unwanted behavior.
The situation that is really important to get right (though it is likely not the only one) is when the user picks up a Central-Config aware CLI for the first time.
I suggest that any caching behavior we have today with regards to the fetching and updating of the data from the centrl OCI image be examined.
The conservative thing to do would be to invalidate the cache on any new CLI version.
Few options if we are to do this:
track in the data store file any CLI version(s) that the CLI host has seen, and perform any cleanup/invalidation (and update of the data store file with the new version, of course) if we encounter a new CLI version.
Alternatively, store in the cache the version of the CLI responsible for populating it and treat any new cli version as an indication that the cache should be invalidate.

There could be more uncommon situations where the cache data needs to be invalidated as well. One could be when a discovery source is updated. It is less likely to happen in production, but might be worth verifying as well.

[marckhouzam]

Looks good to me. This is a really nice set of changes, @marckhouzam !

Thank you @vuil and @anujc25 for working with me to reach this final iteration!

A few things: I assume you plan to do some rebase/squashing. If you do, please retain a handful of logical commits so it will be easier for future reference.

Good point I have squashed on my machine and kept the following three commits (most recent at the top of the list):

• Have the CLI notify the user of a new version
• Provide an API to read the Central Configuration
• Implement a yaml data store

Also, something to consider as a followup: Over time, it will become more and more important that a CLI has up-to-date information about the central configuration, else might start to exhibit different/possibly unwanted behavior. The situation that is really important to get right (though it is likely not the only one) is when the user picks up a Central-Config aware CLI for the first time. I suggest that any caching behavior we have today with regards to the fetching and updating of the data from the centrl OCI image be examined.

Excellent point!
I will track this as a follow-up. Thanks!