Update test central repo with some test Central Config #700

marckhouzam · 2024-03-13T16:17:22Z

What this PR does / why we need it

This PR starts things off for the implementation of the Central Configuration feature of #699.

The first commit of the PR updates the test central repository images to contain a new central_config.yaml file.
This will make it simpler to work in the implementation of the Central Configuration feature.

The second commit updates the CLI to include the new central_config.yaml when setting up the plugin source cache.

With this PR, whenever the CLI uses any of the test central repo images a new file central_config.yaml will be part of the $HOME/.cache/tanzu/plugin_inventory/default directory.

Which issue(s) this PR fixes

Part of #699

Describe testing done for PR

Make sure things still work with the real central repo.

$ tz plugin source init
[i] Refreshing plugin inventory cache for "projects.registry.vmware.com/tanzu_cli/plugins/plugin-inventory:latest", this will take a few seconds.
[i] Reading plugin inventory for "projects.registry.vmware.com/tanzu_cli/plugins/plugin-inventory:latest", this will take a few seconds.
[ok] successfully initialized discovery source
$ ls -l ~/.cache/tanzu/plugin_inventory/default
total 1440
-rw-r--r--  1 kmarc  staff      70 13 Mar 11:03 digest.b6c4a21c4da1764327406642b1bf2591638ae05a020305bd1395de2f323a9ab9
-rw-r--r--  1 kmarc  staff       0 13 Mar 11:03 metadata.digest.none
-rw-------  1 kmarc  staff  733184 13 Mar 11:03 plugin_inventory.db
$ tz plugin search --name builder
[i] The tanzu cli essential plugins have not been installed and are being installed now. The install may take a few seconds.
[i] Installing plugins from plugin group 'vmware-tanzucli/essentials:v1.0.0'
[i] Installed plugin 'telemetry:v1.1.0' with target 'global' (from cache)

  NAME     DESCRIPTION             TARGET  LATEST
  builder  Build Tanzu components  global  v1.2.0

Now try with the new test central:small image

$ tz config set env.TANZU_CLI_PLUGIN_DISCOVERY_IMAGE_SIGNATURE_VERIFICATION_SKIP_LIST localhost:9876/tanzu-cli/plugins/central:small

$ tz plugin source update default -u localhost:9876/tanzu-cli/plugins/central:small
[i] Refreshing plugin inventory cache for "localhost:9876/tanzu-cli/plugins/central:small", this will take a few seconds.
[i] Reading plugin inventory for "localhost:9876/tanzu-cli/plugins/central:small", this will take a few seconds.
[!] Skipping the plugins discovery image signature verification for "localhost:9876/tanzu-cli/plugins/central:small"

[ok] updated discovery source default

$ ll /Users/kmarc/.cache/tanzu/plugin_inventory/default
total 776
-rw-------  1 kmarc  staff   678B 13 Mar 12:10 central_config.yaml
-rw-r--r--  1 kmarc  staff    46B 13 Mar 12:10 digest.ef1e7cef1bd799ddfc1befe0d6aa68226af56578446f57a6aaf4624fca919731
-rw-r--r--  1 kmarc  staff     0B 13 Mar 12:03 metadata.digest.none
-rw-------  1 kmarc  staff   380K 13 Mar 12:10 plugin_inventory.db

$ tz plugin search
  NAME                DESCRIPTION                       TARGET           LATEST
  account             account functionality             mission-control  v9.9.9
  apply               apply functionality               mission-control  v9.9.9
  audit               audit functionality               mission-control  v9.9.9
  cluster             cluster functionality             kubernetes       v9.9.9
  [...]
  telemetry           telemetry functionality           global           v9.9.9
  telemetry           telemetry functionality           kubernetes       v9.9.9
  workspace           workspace functionality           mission-control  v9.9.9

Below we makes sure that when the CLI points to each image in the test repo, we now get a central_config.yaml file.

# Start the new test central repo
$ make stop-test-central-repo
Stopped docker test central repo

# Remove the previous cached content since the repo has changed
$ \rm -rf hack/central-repo/registry-content

$ make start-test-central-repo
echo "Adding docker test central repo cert to the config file"
Adding docker test central repo cert to the config file
TANZU_CLI_CEIP_OPT_IN_PROMPT_ANSWER="No" TANZU_CLI_EULA_PROMPT_ANSWER="Yes" /Users/kmarc/git/tanzu-cli/bin/tanzu config cert delete localhost:9876 || true
[ok] deleted certificate data for host localhost:9876
/Users/kmarc/git/tanzu-cli/bin/tanzu config cert add --host localhost:9876 --ca-cert /Users/kmarc/git/tanzu-cli/hack/central-repo/certs/localhost.crt
[ok] successfully added certificate data for host localhost:9876
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
Started docker test central repo with images:
=> localhost:9876/tanzu-cli/plugins/central:small
    - a small amount of plugins matching product plugin names
    - only versions v0.0.1 and v9.9.9 can be installed
=> localhost:9876/tanzu-cli/plugins/central:large with 100 plugins
    - the same content as the small image with extra plugins for a total of 100
    - none of the 'stubXY' plugins can be installed
    - only versions v0.0.1 and v9.9.9 can be installed
=> localhost:9876/tanzu-cli/plugins/sandbox1:small
    - an extra v11.11.11 version of the plugins of the small image
=> localhost:9876/tanzu-cli/plugins/sandbox2:small
    - an extra v22.22.22 version of the plugins of the small image
=> localhost:9876/tanzu-cli/plugins/airgapped:small
    - a small amount of plugins matching product plugin names
    - contains only versions v0.0.1 and v9.9.9 of plugins and all of them can be installed
=> localhost:9876/tanzu-cli/plugins/airgapped:large
    - all plugins matching product plugin names
    - contains only versions v0.0.1 and v9.9.9 of plugins and all of them can be installed
=> localhost:9876/tanzu-cli/plugins/shas:small
    - a small amount of plugins matching product plugin names
    - contains only versions v0.0.1 and v9.9.9 of plugins and all of them can be installed
    - SHAs are used to reference the v9.9.9 plugin binaries and tags for v0.0.1
=> localhost:9876/tanzu-cli/plugins/extra:small
    - same DB as central:small but with an extra column both for plugins and groups

# Build the CLI from this PR
$ make build
build darwin-arm64 CLI with version: v1.3.0-dev
mkdir -p bin
cp /Users/kmarc/git/tanzu-cli/artifacts/darwin/arm64/cli/core/v1.3.0-dev/tanzu-cli-darwin_arm64 ./bin/tanzu

# For every image in the test repo, empty the cache, set the plugin source to the image
# and check that a central_config.yaml file appears
$ for image in localhost:9876/tanzu-cli/plugins/central:empty localhost:9876/tanzu-cli/plugins/central:emptytable localhost:9876/tanzu-cli/plugins/central:small localhost:9876/tanzu-cli/plugins/central:large localhost:9876/tanzu-cli/plugins/sandbox1:small localhost:9876/tanzu-cli/plugins/sandbox2:small localhost:9876/tanzu-cli/plugins/airgapped:small localhost:9876/tanzu-cli/plugins/airgapped:large localhost:9876/tanzu-cli/plugins/shas:small localhost:9876/tanzu-cli/plugins/extra:small;do
\rm -rf ~/.cache/tanzu/plugin_inventory
tz config set env.TANZU_CLI_PLUGIN_DISCOVERY_IMAGE_SIGNATURE_VERIFICATION_SKIP_LIST $image
tz plugin source update default -u $image
ls -l ~/.cache/tanzu/plugin_inventory/default
done

# No central config file for the empty image

[i] Refreshing plugin inventory cache for "localhost:9876/tanzu-cli/plugins/central:empty", this will take a few seconds.
[i] Reading plugin inventory for "localhost:9876/tanzu-cli/plugins/central:empty", this will take a few seconds.
[!] Skipping the plugins discovery image signature verification for "localhost:9876/tanzu-cli/plugins/central:empty"

[x] : unable to setup DB query for DB at '/Users/kmarc/.cache/tanzu/plugin_inventory/default/plugin_inventory.db': SQL logic error: no such table: PluginBinaries (1)
total 8
-rw-r--r--  1 kmarc  staff  46 13 Mar 11:12 digest.bfad969329ced12ac1ddbefdeef3dc377254b9a67ecb2a64cb5fd2f702bb6655
-rw-r--r--  1 kmarc  staff   0 13 Mar 11:12 metadata.digest.none
-rw-------  1 kmarc  staff   0 13 Mar 11:12 plugin_inventory.db


# An empty central config file for the emptytable image

[i] Refreshing plugin inventory cache for "localhost:9876/tanzu-cli/plugins/central:emptytable", this will take a few seconds.
[i] Reading plugin inventory for "localhost:9876/tanzu-cli/plugins/central:emptytable", this will take a few seconds.
[!] Skipping the plugins discovery image signature verification for "localhost:9876/tanzu-cli/plugins/central:emptytable"

[ok] updated discovery source default
total 48
-rw-------  1 kmarc  staff      0 13 Mar 11:12 central_config.yaml
-rw-r--r--  1 kmarc  staff     51 13 Mar 11:12 digest.66e8cc46a754d991c4e984ffa28ec0302aada047a1900c6eb5375313244f8a20
-rw-r--r--  1 kmarc  staff      0 13 Mar 11:12 metadata.digest.none
-rw-------  1 kmarc  staff  20480 13 Mar 11:12 plugin_inventory.db

# The same central config file for the other images

[i] Refreshing plugin inventory cache for "localhost:9876/tanzu-cli/plugins/central:small", this will take a few seconds.
[i] Reading plugin inventory for "localhost:9876/tanzu-cli/plugins/central:small", this will take a few seconds.
[!] Skipping the plugins discovery image signature verification for "localhost:9876/tanzu-cli/plugins/central:small"

[ok] updated discovery source default
total 776
-rw-------  1 kmarc  staff     678 13 Mar 11:12 central_config.yaml
-rw-r--r--  1 kmarc  staff      46 13 Mar 11:12 digest.ef1e7cef1bd799ddfc1befe0d6aa68226af56578446f57a6aaf4624fca919731
-rw-r--r--  1 kmarc  staff       0 13 Mar 11:12 metadata.digest.none
-rw-------  1 kmarc  staff  389120 13 Mar 11:12 plugin_inventory.db


[i] Refreshing plugin inventory cache for "localhost:9876/tanzu-cli/plugins/central:large", this will take a few seconds.
[i] Reading plugin inventory for "localhost:9876/tanzu-cli/plugins/central:large", this will take a few seconds.
[!] Skipping the plugins discovery image signature verification for "localhost:9876/tanzu-cli/plugins/central:large"

[ok] updated discovery source default
total 2200
-rw-------  1 kmarc  staff      678 13 Mar 11:12 central_config.yaml
-rw-r--r--  1 kmarc  staff       46 13 Mar 11:12 digest.32ee213fc1a12f62ee7d557f75de3715e65bee564722cb7ac128538c82f79a91
-rw-r--r--  1 kmarc  staff        0 13 Mar 11:12 metadata.digest.none
-rw-------  1 kmarc  staff  1118208 13 Mar 11:12 plugin_inventory.db


[i] Refreshing plugin inventory cache for "localhost:9876/tanzu-cli/plugins/sandbox1:small", this will take a few seconds.
[i] Reading plugin inventory for "localhost:9876/tanzu-cli/plugins/sandbox1:small", this will take a few seconds.
[!] Skipping the plugins discovery image signature verification for "localhost:9876/tanzu-cli/plugins/sandbox1:small"

[ok] updated discovery source default
total 120
-rw-------  1 kmarc  staff    678 13 Mar 11:12 central_config.yaml
-rw-r--r--  1 kmarc  staff     47 13 Mar 11:12 digest.92e9800fac818c56470b3fe46248c98fff569caa22ddbc252d4fae5b2bc2e332
-rw-r--r--  1 kmarc  staff      0 13 Mar 11:12 metadata.digest.none
-rw-------  1 kmarc  staff  53248 13 Mar 11:12 plugin_inventory.db


[i] Refreshing plugin inventory cache for "localhost:9876/tanzu-cli/plugins/sandbox2:small", this will take a few seconds.
[i] Reading plugin inventory for "localhost:9876/tanzu-cli/plugins/sandbox2:small", this will take a few seconds.
[!] Skipping the plugins discovery image signature verification for "localhost:9876/tanzu-cli/plugins/sandbox2:small"

[ok] updated discovery source default
total 120
-rw-------  1 kmarc  staff    678 13 Mar 11:12 central_config.yaml
-rw-r--r--  1 kmarc  staff     47 13 Mar 11:12 digest.1d6d440361eab1b5b48e975aae5af5b9f187804c59b03f9382fe903ed2fc64aa
-rw-r--r--  1 kmarc  staff      0 13 Mar 11:12 metadata.digest.none
-rw-------  1 kmarc  staff  53248 13 Mar 11:12 plugin_inventory.db


[i] Refreshing plugin inventory cache for "localhost:9876/tanzu-cli/plugins/airgapped:small", this will take a few seconds.
[i] Reading plugin inventory for "localhost:9876/tanzu-cli/plugins/airgapped:small", this will take a few seconds.
[!] Skipping the plugins discovery image signature verification for "localhost:9876/tanzu-cli/plugins/airgapped:small"

[ok] updated discovery source default
total 80
-rw-------  1 kmarc  staff    678 13 Mar 11:12 central_config.yaml
-rw-r--r--  1 kmarc  staff     48 13 Mar 11:12 digest.dcf741ce4aae0ae9e50a8d17f70babce679111f6a5f46be6cc421ae5b258aac9
-rw-r--r--  1 kmarc  staff      0 13 Mar 11:12 metadata.digest.none
-rw-------  1 kmarc  staff  32768 13 Mar 11:12 plugin_inventory.db


[i] Refreshing plugin inventory cache for "localhost:9876/tanzu-cli/plugins/airgapped:large", this will take a few seconds.
[i] Reading plugin inventory for "localhost:9876/tanzu-cli/plugins/airgapped:large", this will take a few seconds.
[!] Skipping the plugins discovery image signature verification for "localhost:9876/tanzu-cli/plugins/airgapped:large"

[ok] updated discovery source default
total 208
-rw-------  1 kmarc  staff    678 13 Mar 11:12 central_config.yaml
-rw-r--r--  1 kmarc  staff     48 13 Mar 11:12 digest.931d51aa63770bf6bc6ac378e3193c560be4e176700d1822eed28711d083d64a
-rw-r--r--  1 kmarc  staff      0 13 Mar 11:12 metadata.digest.none
-rw-------  1 kmarc  staff  98304 13 Mar 11:12 plugin_inventory.db


[i] Refreshing plugin inventory cache for "localhost:9876/tanzu-cli/plugins/shas:small", this will take a few seconds.
[i] Reading plugin inventory for "localhost:9876/tanzu-cli/plugins/shas:small", this will take a few seconds.
[!] Skipping the plugins discovery image signature verification for "localhost:9876/tanzu-cli/plugins/shas:small"

[ok] updated discovery source default
total 72
-rw-------  1 kmarc  staff    678 13 Mar 11:12 central_config.yaml
-rw-r--r--  1 kmarc  staff     43 13 Mar 11:12 digest.685e88632a76efd3911b2265dc5e6bd8c6a293845b131e10a3d31f5ad12b3e91
-rw-r--r--  1 kmarc  staff      0 13 Mar 11:12 metadata.digest.none
-rw-------  1 kmarc  staff  28672 13 Mar 11:12 plugin_inventory.db

[i] Refreshing plugin inventory cache for "localhost:9876/tanzu-cli/plugins/extra:small", this will take a few seconds.
[i] Reading plugin inventory for "localhost:9876/tanzu-cli/plugins/extra:small", this will take a few seconds.
[!] Skipping the plugins discovery image signature verification for "localhost:9876/tanzu-cli/plugins/extra:small"

[ok] updated discovery source default
total 864
-rw-------  1 kmarc  staff     678 13 Mar 12:03 central_config.yaml
-rw-r--r--  1 kmarc  staff      44 13 Mar 12:03 digest.96fe6766f4d6c8d04bfbfdb0b99e67703e1f44167ce217a39b10d2d6de869b19
-rw-r--r--  1 kmarc  staff       0 13 Mar 12:03 metadata.digest.none
-rw-------  1 kmarc  staff  434176 13 Mar 12:03 plugin_inventory.db

Release note

Update test central repo with some test Central Configuration

Additional information

Special notes for your reviewer

There are no unit tests for downloading the OCI image from the Central Repo.
But we will have E2E tests by the end of the feature, which will cover this.

With this commit, whenever the CLI uses any of the test central repo images a new file "central_config.yaml" will be part of the "$HOME/.cache/tanzu/plugin_inventory/default" directory. This file can be used to test the Central Configuration feature. Signed-off-by: Marc Khouzam <marc.khouzam@broadcom.com>

Signed-off-by: Marc Khouzam <marc.khouzam@broadcom.com>

anujc25

LGTM. Very nice piece of addition. 👍

) With this commit, whenever the CLI uses any of the test central repo images a new file "central_config.yaml" will be part of the "$HOME/.cache/tanzu/plugin_inventory/default" directory. This file can be used to test the Central Configuration feature. Signed-off-by: Marc Khouzam <marc.khouzam@broadcom.com>

With this commit, whenever the CLI uses any of the test central repo images a new file "central_config.yaml" will be part of the "$HOME/.cache/tanzu/plugin_inventory/default" directory. This file can be used to test the Central Configuration feature. Signed-off-by: Marc Khouzam <marc.khouzam@broadcom.com>

marckhouzam added 2 commits March 13, 2024 12:03

Include the central_config.yaml file in the cache

a4bae16

Signed-off-by: Marc Khouzam <marc.khouzam@broadcom.com>

vmwclabot added the cla-not-required label Mar 13, 2024

marckhouzam marked this pull request as ready for review March 13, 2024 16:17

marckhouzam requested a review from a team as a code owner March 13, 2024 16:17

anujc25 approved these changes Mar 15, 2024

View reviewed changes

marckhouzam added this to the v1.3.0 milestone Mar 15, 2024

marckhouzam merged commit 73fa46a into vmware-tanzu:main Mar 15, 2024
8 checks passed

marckhouzam deleted the feat/centralCfgTestRepo branch March 15, 2024 10:04

marckhouzam mentioned this pull request Mar 18, 2024

Notify user of a new recommended CLI version #707

Merged

marckhouzam mentioned this pull request Apr 30, 2024

Asynchronously updatable, centrally hosted CLI configuration #699

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update test central repo with some test Central Config #700

Update test central repo with some test Central Config #700

marckhouzam commented Mar 13, 2024

anujc25 left a comment

Update test central repo with some test Central Config #700

Update test central repo with some test Central Config #700

Conversation

marckhouzam commented Mar 13, 2024

What this PR does / why we need it

Which issue(s) this PR fixes

Describe testing done for PR

Release note

Additional information

Special notes for your reviewer

anujc25 left a comment

Choose a reason for hiding this comment