feat: support extra catalog index images for Extensions UI [RHIDP-12933]

[rm3l]

Description

This PR introduces a new EXTRA_CATALOG_INDEX_IMAGES env var in the install-dynamic-plugins.py script, so that additional catalog index images can be listed alongside the primary CATALOG_INDEX_IMAGE.
This is a comma-separated list of images.

Each extra image's catalog entities are extracted to an isolated sub-directory in the /extensions extraction dir, making those plugins automatically discoverable in the Extensions UI.

Notes:

• Only the existing CATALOG_INDEX_IMAGE is considered primary source for extracting and handling the dynamic-plugins.default.yaml (DPDY). Extra catalog images contribute catalog entities only, since there is no use case yet for supporting multiple DPDY files
• Extra images are extracted under <extraction_dir>/extra/, which should prevent entries from accidentally overwriting the primary index's catalog entities
• Supports both name=<image_ref> and plain image_ref formats: explicit names produce cleaner directory names (e.g., /extra/community/), while plain refs auto-derive names
• Duplicate sub-directory names warn but don't error out; the later entry overwrites the earlier one, with a warning printed

Assisted-by: Claude

Which issue(s) does this PR fix

• Fixes https://redhat.atlassian.net/browse/RHIDP-12933

PR acceptance criteria

Please make sure that the following steps are complete:

• GitHub Actions are completed and successful
• Unit Tests are updated and passing
• E2E Tests are updated and passing
• Documentation is updated if necessary (requirement for new features)
• Add a screenshot if the change is UX/UI related

How to test changes / Special notes to the reviewer

Try running the install-dynamic-plugins script locally after setting the EXTRA_CATALOG_INDEX_IMAGES env var.
You can also deploy the image from this PR with the EXTRA_CATALOG_INDEX_IMAGES env var in the install-dynamic-plugins container:

❯ EXTRA_CATALOG_INDEX_IMAGES=quay.io/rhdh/plugin-catalog-index:1.10,community=quay.io/rhdh/plugin-catalog-index:1.9.1,partner=quay.io/rhdh/plugin-catalog-index:1.9.3,community=quay.io/rhdh/plugin-catalog-index:1.9.4,quay.io/rhdh/plugin-catalog-index:1.10

Logs

======= Created lock file: /dynamic-plugins-root/install-dynamic-plugins.lock

======= Extracting catalog index from quay.io/rhdh/plugin-catalog-index:1.9
        ==> Copying catalog index image to local filesystem
        ==> Extracting catalog index layers
        ==> Extracting layer a9837d06aa6340c55fe90c8ba4f4310ec114d362a3b93da55a7ff75ac0d2d1b1
        ==> Successfully extracted dynamic-plugins.default.yaml from catalog index image
        ==> Extracting extensions catalog entities to /extensions
        ==> Successfully extracted extensions catalog entities from index image

======= Extracting extra catalog index 'quay.io_rhdh_plugin-catalog-index_1.10' from quay.io/rhdh/plugin-catalog-index:1.10
        ==> Copying extra catalog index image to local filesystem
        ==> Extracting extra catalog index layers
        ==> Extracting layer 641a777bf41068b766fd520fd20c4d17eb202389dccd00eb5a35b85bea0d78df
        ==> Extracting extensions catalog entities to /extensions/extra/quay.io_rhdh_plugin-catalog-index_1.10
        ==> Successfully extracted extensions catalog entities from extra index image to /extensions/extra/quay.io_rhdh_plugin-catalog-index_1.10

======= Extracting extra catalog index 'community' from quay.io/rhdh/plugin-catalog-index:1.9.1
        ==> Copying extra catalog index image to local filesystem
        ==> Extracting extra catalog index layers
        ==> Extracting layer 79f2b46a48b5b2dffec5466be8e06638708e473484fbde053482a591a71ffab0
        ==> Extracting extensions catalog entities to /extensions/extra/community
        ==> Successfully extracted extensions catalog entities from extra index image to /extensions/extra/community

======= Extracting extra catalog index 'partner' from quay.io/rhdh/plugin-catalog-index:1.9.3
        ==> Copying extra catalog index image to local filesystem
        ==> Extracting extra catalog index layers
        ==> Extracting layer b47a90f180ba8d747fbcdce9af2b34e16101ee47a8ca895d8184057ebfc5cde1
        ==> Extracting extensions catalog entities to /extensions/extra/partner
        ==> Successfully extracted extensions catalog entities from extra index image to /extensions/extra/partner

======= Extracting extra catalog index 'community' from quay.io/rhdh/plugin-catalog-index:1.9.4
        ==> WARNING: Subdirectory 'community' was already used by 'quay.io/rhdh/plugin-catalog-index:1.9.1'. The previous extraction will be overwritten.
        ==> Copying extra catalog index image to local filesystem
        ==> Extracting extra catalog index layers
        ==> Extracting layer a9837d06aa6340c55fe90c8ba4f4310ec114d362a3b93da55a7ff75ac0d2d1b1
        ==> Extracting extensions catalog entities to /extensions/extra/community
        ==> Successfully extracted extensions catalog entities from extra index image to /extensions/extra/community

======= Extracting extra catalog index 'quay.io_rhdh_plugin-catalog-index_1.10' from quay.io/rhdh/plugin-catalog-index:1.10
        ==> WARNING: Subdirectory 'quay.io_rhdh_plugin-catalog-index_1.10' was already used by 'quay.io/rhdh/plugin-catalog-index:1.10'. The previous extraction will be overwritten.
        ==> Copying extra catalog index image to local filesystem
        ==> Extracting extra catalog index layers
        ==> Extracting layer 641a777bf41068b766fd520fd20c4d17eb202389dccd00eb5a35b85bea0d78df
        ==> Extracting extensions catalog entities to /extensions/extra/quay.io_rhdh_plugin-catalog-index_1.10
        ==> Successfully extracted extensions catalog entities from extra index image to /extensions/extra/quay.io_rhdh_plugin-catalog-index_1.10

======= Replacing dynamic-plugins.default.yaml with catalog index: /dynamic-plugins-root/.catalog-index-temp/dynamic-plugins.default.yaml

======= Including dynamic plugins from /dynamic-plugins-root/.catalog-index-temp/dynamic-plugins.default.yaml

[...]

And in the /extensions dir:

❯ tree -L 2 /tmp/extensions
/extensions
├── catalog-entities
│   ├── collections
│   ├── packages
│   └── plugins
└── extra
    ├── community
    ├── partner
    └── quay.io_rhdh_plugin-catalog-index_1.10

❯ tree -L 2 /extensions/extra
/extensions/extra
├── community
│   └── catalog-entities
│       ├── collections
│       ├── packages
│       └── plugins
├── partner
│   └── catalog-entities
│       ├── collections
│       ├── packages
│       └── plugins
└── quay.io_rhdh_plugin-catalog-index_1.10
    └── catalog-entities
        ├── collections
        ├── packages
        └── plugins

[openshift-ci]

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

[github-actions]

Image was built and published successfully. It is available at:

• quay.io/rhdh-community/rhdh:pr-4655
• quay.io/rhdh-community/rhdh:pr-4655-eb312c2a

[github-actions]

Image was built and published successfully. It is available at:

• quay.io/rhdh-community/rhdh:pr-4655
• quay.io/rhdh-community/rhdh:pr-4655-76e51264

[github-actions]

Image was built and published successfully. It is available at:

• quay.io/rhdh-community/rhdh:pr-4655
• quay.io/rhdh-community/rhdh:pr-4655-06598797

[rm3l]

/test

[rm3l]

/agentic_review

[rhdh-qodo-merge]

Code Review by Qodo

🐞 Bugs (1) 📘 Rule violations (0) 📎 Requirement gaps (1)

1. Unsafe extra dir name 🐞 Bug ⛨ Security

Description

EXTRA_CATALOG_INDEX_IMAGES explicit names are used as raw path segments, allowing values like '..'
or '/tmp' to escape the intended <CATALOG_ENTITIES_EXTRACT_DIR>/extra root and then be recursively
deleted/overwritten. This can cause unintended data loss on the container filesystem (via
shutil.rmtree) and write catalog entities outside the Extensions extraction directory.

Code

scripts/install-dynamic-plugins/install-dynamic-plugins.py[R1195-1207]

+        subdirectory_parent = os.path.join(catalog_entities_parent_dir, subdirectory)
+        print(f"\t==> Extracting extensions catalog entities to {subdirectory_parent}", flush=True)
+
+        extensions_dir_from_catalog_index = os.path.join(catalog_index_temp_dir, 'catalog-entities', 'extensions')
+        if not os.path.isdir(extensions_dir_from_catalog_index):
+            extensions_dir_from_catalog_index = os.path.join(catalog_index_temp_dir, 'catalog-entities', 'marketplace')
+
+        if os.path.isdir(extensions_dir_from_catalog_index):
+            os.makedirs(subdirectory_parent, exist_ok=True)
+            catalog_entities_dest = os.path.join(subdirectory_parent, 'catalog-entities')
+            if os.path.exists(catalog_entities_dest):
+                shutil.rmtree(catalog_entities_dest, ignore_errors=True, onerror=None)
+            shutil.copytree(extensions_dir_from_catalog_index, catalog_entities_dest, dirs_exist_ok=True)

Relevance

⭐⭐⭐ High

Unsanitized subdirectory can escape base and trigger rmtree/copytree outside intended dir; team
accepts hardening fixes.

PR-#3737
PR-#3970

ⓘ Recommendations generated based on similar findings in past PRs

Evidence

parse_extra_catalog_index_images() returns the user-provided explicit name without any
validation/sanitization, and extract_extra_catalog_index() uses that subdirectory directly in
os.path.join() to form the destination path and then performs a recursive delete+copy. Because
os.path.join(base, subdirectory) will honor absolute paths (e.g., subdirectory='/tmp') and parent
directory segments (e.g., subdirectory='..'), this can escape the intended extraction directory and
apply rmtree()/copytree() outside of it.

scripts/install-dynamic-plugins/install-dynamic-plugins.py[1217-1247]
scripts/install-dynamic-plugins/install-dynamic-plugins.py[1148-1208]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
`EXTRA_CATALOG_INDEX_IMAGES` supports `name=image_ref`, but the `name` is currently used as a raw filesystem path segment. This allows absolute paths (e.g. `/tmp`) or traversal (e.g. `..`) to escape `<CATALOG_ENTITIES_EXTRACT_DIR>/extra`, after which the code will recursively delete (`shutil.rmtree`) and overwrite (`shutil.copytree`) paths outside the intended extraction directory.

### Issue Context
- `parse_extra_catalog_index_images()` accepts explicit names and returns them unchanged.
- `extract_extra_catalog_index()` does `os.path.join(catalog_entities_parent_dir, subdirectory)` and then deletes/copies into `<that>/catalog-entities`.

### Fix Focus Areas
- scripts/install-dynamic-plugins/install-dynamic-plugins.py[1148-1208]
- scripts/install-dynamic-plugins/install-dynamic-plugins.py[1217-1247]

### Suggested fix
1. Reject or sanitize explicit `name` values:
  - Disallow empty names.
  - Disallow absolute paths.
  - Disallow any path separators (`/` and `\\`).
  - Disallow `.` and `..` segments.
  - Optionally restrict to a safe regex like `^[A-Za-z0-9][A-Za-z0-9._-]{0,63}$`.
2. Add a defensive check in `extract_extra_catalog_index()`:
  - Resolve the computed destination path and ensure it is within `extra_parent_dir` before calling `rmtree()`/`copytree()`.
3. Add unit tests covering bad names such as `/tmp`, `..`, `a/../b`, and empty name (e.g. `=quay.io/img:1.0`) to ensure they error/skip safely.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

2. Primary catalog not subdirectory 📎 Requirement gap ≡ Correctness

Description

The primary CATALOG_INDEX_IMAGE catalog entities are extracted directly into
CATALOG_ENTITIES_EXTRACT_DIR/catalog-entities rather than a dedicated per-catalog subdirectory
under /extensions. This does not meet the requirement that the primary and each extra catalog
index extract into separate /extensions/<catalog>/... directories for isolation.

Code

scripts/install-dynamic-plugins/install-dynamic-plugins.py[R1392-1394]

+    catalog_entities_parent_dir = os.environ.get("CATALOG_ENTITIES_EXTRACT_DIR", os.path.join(tempfile.gettempdir(), "extensions"))
    if catalog_index_image:
-        # default to a temporary directory if the env var is not set
-        catalog_entities_parent_dir = os.environ.get("CATALOG_ENTITIES_EXTRACT_DIR", os.path.join(tempfile.gettempdir(), "extensions"))
        catalog_index_default_file = extract_catalog_index(catalog_index_image, dynamic_plugins_root, catalog_entities_parent_dir)

Relevance

⭐ Low

Repo previously merged primary extraction directly into <extract_dir>/catalog-entities; per-catalog
subdir not required historically.

PR-#3970
PR-#3984

ⓘ Recommendations generated based on similar findings in past PRs

Evidence

PR Compliance ID 3 requires the primary and each extra index image to extract into separate
subdirectories under /extensions. The PR continues to call extract_catalog_index(...) with
catalog_entities_parent_dir and that function writes to `os.path.join(catalog_entities_parent_dir,
'catalog-entities'), meaning the primary index is not placed into its own /extensions/<name>/...`
subdirectory while extras are.

Extract each catalog index image into a separate subdirectory under /extensions
scripts/install-dynamic-plugins/install-dynamic-plugins.py[1392-1394]
scripts/install-dynamic-plugins/install-dynamic-plugins.py[1126-1136]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
Primary catalog entities are extracted into the root `CATALOG_ENTITIES_EXTRACT_DIR/catalog-entities` instead of a dedicated per-catalog subdirectory under `/extensions`, so the primary index is not isolated the same way as extra catalogs.

## Issue Context
Compliance requires that the primary and each extra catalog index image extract into separate subdirectories under `/extensions` (e.g., `/extensions/<catalog>/...`) to ensure isolation between catalogs.

## Fix Focus Areas
- scripts/install-dynamic-plugins/install-dynamic-plugins.py[1392-1405]
- scripts/install-dynamic-plugins/install-dynamic-plugins.py[1126-1139]
- docs/dynamic-plugins/installing-plugins.md[88-113]
- scripts/install-dynamic-plugins/test_install-dynamic-plugins.py[3432-3885]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

[rm3l]

/test e2e-ocp-helm

[openshift-ci]

New changes are detected. LGTM label has been removed.

[sonarqubecloud]

Quality Gate passed

Issues
3 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[rm3l]

Re-applying lgtm label as it was previously approved. I've only merged main to retrigger the E2E tests.

[github-actions]

Image was built and published successfully. It is available at:

• quay.io/rhdh-community/rhdh:pr-4655
• quay.io/rhdh-community/rhdh:pr-4655-16e77117