Add workflow step catalog — community-installable step types#2394
Add workflow step catalog — community-installable step types#2394
Conversation
…nd tests Agent-Logs-Url: https://github.com/github/spec-kit/sessions/2885e646-477d-4df8-b9a3-06d8cb29e748 Co-authored-by: mnriem <15701806+mnriem@users.noreply.github.com>
…e-effect' Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Adds a catalog/registry system for community-installable workflow step types, plus CLI commands to discover, install, and manage them alongside built-in steps.
Changes:
- Introduces
StepRegistryandStepCatalog(multi-source resolution + SHA256 cache) for step type distribution/management. - Adds dynamic filesystem-based loading of installed custom step packages into
STEP_REGISTRY. - Expands CLI with
specify workflow step …and adds tests and initial (empty) catalog JSON files.
Show a summary per file
| File | Description |
|---|---|
| workflows/step-catalog.json | Adds the built-in “official” step catalog scaffold (currently empty). |
| workflows/step-catalog.community.json | Adds the built-in “community” step catalog scaffold (currently empty). |
| src/specify_cli/workflows/catalog.py | Implements StepRegistry + StepCatalog with config resolution and caching. |
| src/specify_cli/workflows/init.py | Adds load_custom_steps(project_root) dynamic import/registration for installed step packages. |
| src/specify_cli/init.py | Adds Typer subcommands for listing/searching/installing/removing steps and managing step catalogs. |
| tests/test_workflows.py | Adds unit tests covering registry CRUD, catalog resolution/validation, search/info, and dynamic loading behavior. |
Copilot's findings
Tip
Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comments suppressed due to low confidence (1)
src/specify_cli/init.py:5555
workflow_step_remove()buildsstep_dirfrom unvalidatedstep_idand thenshutil.rmtree(step_dir). A malicious value like../...could delete arbitrary directories outside.specify/workflows/steps. Add the same resolved-pathrelative_to()guard used byworkflow_remove/workflow_addbefore performing deletions.
step_dir = project_root / ".specify" / "workflows" / "steps" / step_id
if step_dir.exists():
import shutil
shutil.rmtree(step_dir)
- Files reviewed: 6/6 changed files
- Comments generated: 4
| if not info: | ||
| console.print(f"[red]Error:[/red] Step type '{step_id}' not found in catalog") | ||
| raise typer.Exit(1) | ||
|
|
||
| if not info.get("_install_allowed", True): | ||
| console.print( | ||
| f"[yellow]Warning:[/yellow] Step type '{step_id}' is from a discovery-only catalog" | ||
| ) | ||
| console.print("Direct installation is not enabled for this catalog source.") | ||
| raise typer.Exit(1) |
There was a problem hiding this comment.
This command allows installing a step whose step_id collides with an already-registered built-in step type. In that case the install will succeed, but load_custom_steps() will later skip loading it because the key is already in STEP_REGISTRY, leaving the user with an unusable “installed” step. Add an explicit check that step_id is not already present in STEP_REGISTRY (and likely not already installed in StepRegistry) and fail fast with a clear error.
| for key in sorted(STEP_REGISTRY.keys()): | ||
| if key in custom_keys: | ||
| custom.append(key) | ||
| else: | ||
| built_in.append(key) | ||
|
|
There was a problem hiding this comment.
workflow_step_list() only iterates over STEP_REGISTRY.keys(), so custom steps that are recorded as installed in StepRegistry but fail to import/validate (and therefore aren’t registered) won’t be shown at all. Since the help text says it lists installed custom types, consider also listing registry entries that are not currently registered (e.g., under a “Custom (installed, failed to load)” section) so users can discover and remove/repair them.
| fetched_at = meta.get("fetched_at", 0) | ||
| return (time.time() - fetched_at) < self.CACHE_DURATION | ||
| except (json.JSONDecodeError, OSError): |
There was a problem hiding this comment.
_is_url_cache_valid() assumes meta['fetched_at'] is numeric; if the meta file is corrupted and fetched_at is a string/null, (time.time() - fetched_at) will raise TypeError and break catalog operations. Coerce fetched_at to a float (or return False when it’s not an int/float) and include TypeError in the handled exceptions. (The same pattern exists in WorkflowCatalog._is_url_cache_valid, so consider updating both for consistency.)
| fetched_at = meta.get("fetched_at", 0) | |
| return (time.time() - fetched_at) < self.CACHE_DURATION | |
| except (json.JSONDecodeError, OSError): | |
| fetched_at = float(meta.get("fetched_at", 0)) | |
| return (time.time() - fetched_at) < self.CACHE_DURATION | |
| except (json.JSONDecodeError, OSError, TypeError, ValueError): |
| raise ValueError(f"Redirect to non-HTTPS URL: {final_url}") | ||
| return resp.read() | ||
|
|
||
| step_dir = project_root / ".specify" / "workflows" / "steps" / step_id |
There was a problem hiding this comment.
step_id is used directly to build step_dir under .specify/workflows/steps/. Without a safety check, values like ../x or absolute paths can escape the intended directory and lead to arbitrary file writes/deletes (especially since failure cleanup rmtree() follows). Validate that the resolved step_dir stays within the intended base directory (similar to the workflow_add path-traversal guard) before creating the directory.
This issue also appears on line 5551 of the same file.
| step_dir = project_root / ".specify" / "workflows" / "steps" / step_id | |
| steps_base_dir = (project_root / ".specify" / "workflows" / "steps").resolve() | |
| step_dir = (steps_base_dir / step_id).resolve() | |
| try: | |
| step_dir.relative_to(steps_base_dir) | |
| except ValueError: | |
| console.print(f"[red]Error:[/red] Invalid step id '{step_id}'") | |
| raise typer.Exit(1) |
The workflow engine shipped with a dynamic
STEP_REGISTRYbut no distribution mechanism for community-authored step types. This adds a full catalog system for discovering, installing, and managing custom step types, following the same patterns as the workflow/extension catalogs.New classes (
workflows/catalog.py)StepRegistry— persists installed custom steps in.specify/workflows/steps/step-registry.jsonStepCatalog— multi-source catalog stack with SHA256-based caching; resolvesSPECKIT_STEP_CATALOG_URLenv var →.specify/step-catalogs.yml→~/.specify/step-catalogs.yml→ built-in defaults (step-catalog.json+step-catalog.community.json)StepCatalogError/StepValidationError/StepCatalogEntrysupporting typesDynamic step loading (
workflows/__init__.py)load_custom_steps(project_root)scans.specify/workflows/steps/, dynamically imports each package's__init__.py, finds theStepBasesubclass matching the declaredtype_key, and registers it intoSTEP_REGISTRY. Broken packages are silently skipped.CLI surface (
specify workflow step …)addvalidates that the downloadedstep.yml'stype_keymatches the catalog ID and that all fetches use HTTPS before writing anything to disk.Catalog files
workflows/step-catalog.json— official catalog (empty, ready for entries)workflows/step-catalog.community.json— community catalog (empty, ready for entries)Tests
27 new tests across
TestStepRegistryCustom,TestStepCatalog, andTestLoadCustomStepscovering CRUD, catalog resolution (env var / project / user / default), URL validation, search, and dynamic loading edge cases (missing files, broken imports, already-registered keys).