Add skip plugins functionality, and list-plugins subcommand#23
Merged
sseago merged 1 commit intomigtools:mainfrom Aug 11, 2021
Merged
Add skip plugins functionality, and list-plugins subcommand#23sseago merged 1 commit intomigtools:mainfrom
sseago merged 1 commit intomigtools:mainfrom
Conversation
shawn-hurley
requested changes
Aug 10, 2021
Contributor
shawn-hurley
left a comment
shawn-hurley
left a comment
There was a problem hiding this comment.
I think that we have a small merge conflict (sorry merged that one before this one)
|
|
||
| func GetFilteredPlugins(pluginDir, skipPlugins string) ([]transform.Plugin, error) { | ||
| var filteredPlugins []transform.Plugin | ||
| plugins, err := GetBinaryPlugins(pluginDir) |
Contributor
There was a problem hiding this comment.
I think this is going to require an update. I think that we should consider GetFilteredPlugins moving to GetPlugins.
Also, can skipplugins be a vardic? and the caller responsible for the split ?
Contributor
Author
There was a problem hiding this comment.
@shawn-hurley I expect that GetFilteredPlugins will call GetPlugins now instead of GetBinaryPlugins, as we need to be able to skip internal plugins as well.
Contributor
Author
There was a problem hiding this comment.
As for vardic, the skipPlugins string comes from the user. it's a comma-separated list, and this method splits it out. It's a CLI option, so it's a string.
Contributor
Author
There was a problem hiding this comment.
I think in an earlier iteration, the caller was responsible for the split, but the result was that I had 3 identical split calls to pass into this func, since all 3 callers are different transform subcommands, so it was less code to just split it in here.
Contributor
There was a problem hiding this comment.
Fair enough, I realized after the fact that the API was internal so makes sense.
The only other time is if that was to ever be used by something else maybe we want to do this in like validate flags or something?
Contributor
Author
There was a problem hiding this comment.
Makes sense. If a second use for this comes up, it should be pulled out of here into a util method of some sort.
Contributor
Author
|
@shawn-hurley no worries on the merge conflict. I was expecting it -- and yes, we need to make sure that KubernetesPlugin can be skipped too. I'll work out the update. |
shawn-hurley
approved these changes
Aug 10, 2021
aufi
added a commit
to aufi/crane
that referenced
this pull request
Mar 26, 2026
Implements tasks migtools#21 and migtools#23: Transform command (cmd/transform/transform.go): - Detects if new Kustomize workflow should be used based on flags - runKustomizeWorkflow(): Uses Orchestrator for single/multi-stage execution - runLegacyWorkflow(): Preserves existing JSONPatch workflow - Supports --stage, --from-stage, --to-stage, --stages flags - Validates mutually exclusive flag usage Apply command (cmd/apply/apply.go): - Detects if new Kustomize workflow should be used - runKustomizeWorkflow(): Uses KustomizeApplier with kubectl kustomize - runLegacyWorkflow(): Preserves existing apply workflow - Supports --stage, --from-stage, --to-stage, --stages, --final-only flags - Includes preflight validation with --validate flag - Validates mutually exclusive flag usage Both commands maintain backward compatibility by defaulting to legacy workflow when multi-stage flags are not used. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
aufi
added a commit
that referenced
this pull request
Apr 10, 2026
* Add stage discovery mechanism for multi-stage pipeline
Implements Phase 6-F1:
- Stage discovery scans transform directory for subdirectories matching
pattern <number>_<pluginName> (e.g., 10_kubernetes, 20_openshift)
- Stages sorted by priority (numeric prefix) for deterministic ordering
- FilterStages supports multiple selectors: specific stage, from-stage,
to-stage, or list of stages
- Helper functions for stage navigation: GetFirstStage, GetLastStage,
GetPreviousStage, GetNextStage
- ValidateStageName and GenerateStageName utilities
Comprehensive test coverage for all discovery and filtering scenarios.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* Add KustomizeWriter and report generation for transform output
Implements Phase 2-B1 and B2:
- KustomizeWriter orchestrates writing all transform artifacts to
Kustomize layout (resources/, patches/, kustomization.yaml)
- Groups resources by type and writes multi-doc YAML files
- Generates patches with target selectors
- Creates whiteout and ignored-patches reports in YAML format
- Implements dirty check integration to prevent overwriting user edits
- Writes metadata with SHA256 content hashes for change tracking
- Helper functions to parse ResourceGroup TypeKey and extract
JSONPatch operation details
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* Add transform orchestrator for multi-stage execution
Implements Phase 6-F2:
- Orchestrator coordinates single-stage and multi-stage transform execution
- RunSingleStage: Default mode creating one transform stage
- RunMultiStage: Executes pipeline with stage chaining
- Stage chaining: output of stage N becomes input for stage N+1
- Integrates with KustomizeWriter to write Kustomize layout
- Parses JSONPatch from existing Runner.Run() response
- loadStageOutput reads resources from completed stage
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* Add KustomizeApplier for kubectl kustomize integration
Implements Phase 3-C1 and C2:
- KustomizeApplier orchestrates applying transformations via kubectl kustomize
- ApplySingleStage: Applies a specific stage to produce output YAML
- ApplyMultiStage: Applies multiple stages with selector support
- ApplyFinalStage: Applies the last stage in pipeline (typical use case)
- Integrates with stage discovery to find and process stages
- Validates kubectl availability before execution
- Writes combined output YAML for each stage
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* Add preflight validation for apply operations
Implements Phase 3-C3:
- ValidateStage: Checks stage directory structure, kustomization.yaml,
resources, patches, and metadata
- ValidateAllStages: Validates all discovered stages
- ValidatePipeline: End-to-end pipeline validation with error reporting
- ValidateOutputDirectory: Ensures output directory is suitable
- ValidateStageChaining: Verifies stages can be chained correctly
- Integrates dirty check to warn about user modifications
- Validates kustomization.yaml by running kubectl kustomize build
- Comprehensive error and warning reporting
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* Add stage-aware CLI flags to transform and apply commands
Implements Phase 6-F3:
- Transform command flags:
--stage: Run specific stage only
--from-stage: Run from stage onwards
--to-stage: Run up to and including stage
--stages: Run specific stages (comma-separated)
--stage-name: Name for output stage (default: '10_transform')
--plugin-name: Plugin name for metadata (default: 'transform')
--force: Force overwrite of existing stages with user modifications
- Apply command flags:
--stage, --from-stage, --to-stage, --stages: Stage selectors
--final-only: Apply only final stage (default: true)
--validate: Run preflight validation (default: true)
--force: Force overwrite of output directory
Flags support full multi-stage pipeline control and integrate with
stage discovery and filtering mechanisms.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* Add plugin priority auto-assignment algorithm
Implements Phase 6-F4:
- AutoAssignPriorities: Generates priority map from discovered stages
- AutoAssignPrioritiesFromDir: Discovers stages and assigns priorities
- MergePriorities: Merges user-specified and auto-assigned priorities
- SuggestStageName: Suggests next available stage name for a plugin
- ParsePluginPrioritiesFromStrings: Parses priority strings ("plugin:10")
- GetPriorityAssignments: Returns all plugin priority assignments
- ValidatePriorityAssignments: Checks for priority conflicts
- RecommendPriorityOrder: Heuristic-based priority recommendations
Features:
- Fills significant gaps (> 5) in priority sequence
- Appends new stages with increment of 10
- Intelligent plugin categorization (kubernetes:10, openshift:20, etc.)
- Comprehensive test coverage for all functions
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* Add integration tests for transform workflow
Implements task #11:
- TestKustomizeWriterIntegration: End-to-end test of KustomizeWriter
creating stage directory structure, resources, kustomization.yaml,
and metadata
- TestStageDiscoveryIntegration: Tests stage discovery and filtering
with multiple stages
- TestDirtyCheckIntegration: Tests dirty check workflow including
metadata generation, modification detection, and EnsureCleanDirectory
Test fixtures:
- deployment.yaml: Sample Kubernetes Deployment
- service.yaml: Sample Kubernetes Service
All integration tests pass with -tags=integration flag.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* Add comprehensive documentation for Kustomize multi-stage pipeline
Implements task #12:
docs/kustomize-multistage.md:
- Complete feature overview and key concepts
- Stage directory structure and resource grouping
- CLI usage for transform and apply commands
- Priority assignment (auto, manual, recommended)
- Stage chaining workflow
- Migration guide from JSONPatch workflow
- Advanced features and API reference
- Troubleshooting guide and best practices
docs/TRANSFORM_README.md:
- Quick start guide for transform directory
- Explanation of directory contents
- Common workflows (review, customize, re-run, chain)
- Git best practices
- Troubleshooting common issues
- Advanced Kustomize customization examples
Documentation includes:
- Complete CLI flag reference
- Code examples in Go
- Command-line examples
- Directory structure diagrams
- Migration paths
- Best practices
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* Integrate Kustomize workflow into transform and apply commands
Implements tasks #21 and #23:
Transform command (cmd/transform/transform.go):
- Detects if new Kustomize workflow should be used based on flags
- runKustomizeWorkflow(): Uses Orchestrator for single/multi-stage execution
- runLegacyWorkflow(): Preserves existing JSONPatch workflow
- Supports --stage, --from-stage, --to-stage, --stages flags
- Validates mutually exclusive flag usage
Apply command (cmd/apply/apply.go):
- Detects if new Kustomize workflow should be used
- runKustomizeWorkflow(): Uses KustomizeApplier with kubectl kustomize
- runLegacyWorkflow(): Preserves existing apply workflow
- Supports --stage, --from-stage, --to-stage, --stages, --final-only flags
- Includes preflight validation with --validate flag
- Validates mutually exclusive flag usage
Both commands maintain backward compatibility by defaulting to legacy
workflow when multi-stage flags are not used.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* Add crane transform validate subcommand
Implements task #24:
New subcommand: crane transform validate
- Validates all stages in transform directory
- Checks stage directory structure
- Validates kustomization.yaml syntax
- Verifies resource and patch file references
- Validates stage chaining correctness
- Checks dirty status of stages
- Provides clear success/warning/error output with emoji indicators
Features:
- --transform-dir flag to specify transform directory
- --verbose flag to show all warnings
- Validates each stage individually
- Validates stage chaining across pipeline
- User-friendly output with ✓, ⚠️ , ✗ indicators
- Returns non-zero exit code on validation failure
Usage:
crane transform validate
crane transform validate --transform-dir transform
crane transform validate --verbose
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* Tranform service methods like dirtycheck
Signed-off-by: Marek Aufart <maufart@redhat.com>
* Simplify implementation - remove non-essential features
Remove helper features while keeping core Kustomize and multi-stage functionality:
Removed files:
- priority.go, priority_test.go - auto-assignment not needed for MVP
- dirtycheck.go, dirtycheck_test.go, metadata.go - can add later
- validation.go, validate/ subcommand - can add later
- reports.go - whiteout/ignored patches reports not critical
- ordering.go, ordering_test.go - deterministic ordering not critical
- integration_test.go - depends on removed features
Simplified:
- writer.go: Removed metadata, dirty check, reports generation
Now just writes resources, patches, and kustomization.yaml
- orchestrator.go: Updated to match simplified writer signature
- apply.go: Removed validation flags and checks
- transform.go: Removed validate subcommand reference
Core functionality preserved:
✓ Kustomize layout generation (resources/, patches/, kustomization.yaml)
✓ Multi-stage pipeline support
✓ Stage discovery and filtering
✓ Resource grouping by type
✓ Integration with crane transform and apply commands
All tests passing. Implementation is now simpler and focused on core features.
* Upgrade crane-lib dependency with kustomize
Signed-off-by: Marek Aufart <maufart@redhat.com>
* Address feedback, fix problems
Signed-off-by: Marek Aufart <maufart@redhat.com>
* Address feedback primary options
Signed-off-by: Marek Aufart <maufart@redhat.com>
* Update feedback, stages fix
Signed-off-by: Marek Aufart <maufart@redhat.com>
* Remove jsonpatch alternative and final-stage only
Signed-off-by: Marek Aufart <maufart@redhat.com>
* Minor cleanup and stage unittests
Signed-off-by: Marek Aufart <maufart@redhat.com>
* Use kustomize only in transform and apply
Signed-off-by: Marek Aufart <maufart@redhat.com>
* Remove unused temp test fixtures
Signed-off-by: Marek Aufart <maufart@redhat.com>
* Unify jsonpatch via kustomize, docs update
Signed-off-by: Marek Aufart <maufart@redhat.com>
* Filter non-matching patch ops
Signed-off-by: Marek Aufart <maufart@redhat.com>
* Keep original output file structure even w/kustomize
Signed-off-by: Marek Aufart <maufart@redhat.com>
* Update transform flags parsing, harden jsonpatch usage check
Signed-off-by: Marek Aufart <maufart@redhat.com>
* Remove export-dir from apply, fix kubectl validation
orchestration unit test
Signed-off-by: Marek Aufart <maufart@redhat.com>
* Fix output YAML indentation 2 spaces
Signed-off-by: Marek Aufart <maufart@redhat.com>
* Use full plugin name in transform directory names
Signed-off-by: Marek Aufart <maufart@redhat.com>
* Re-add export-dir to apply until decision on flow
Signed-off-by: Marek Aufart <maufart@redhat.com>
---------
Signed-off-by: Marek Aufart <maufart@redhat.com>
Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #21