Skip to content

Feat: GitOps YAML Migration Tool#32237

Merged
Illbjorn merged 55 commits intofleetdm:mainfrom
Illbjorn:feat/gitops-yaml-migration-tooling
Sep 8, 2025
Merged

Feat: GitOps YAML Migration Tool#32237
Illbjorn merged 55 commits intofleetdm:mainfrom
Illbjorn:feat/gitops-yaml-migration-tooling

Conversation

@Illbjorn
Copy link
Copy Markdown
Contributor

@Illbjorn Illbjorn commented Aug 22, 2025

Overview

This pull request resolves #31165, implementing command-line tooling to migrate GitOps YAML files following the changes introduced in the upcoming 4.74 release.

Aligning with the recommended steps in the README; this is an example of the first step (gitops-migrate format) and this is an example of the second step (gitops-migrate migrate).

Comment thread cmd/fleetctl/gitops-migrate/backup.go Fixed
@codecov
Copy link
Copy Markdown

codecov Bot commented Aug 22, 2025

Codecov Report

❌ Patch coverage is 45.14523% with 661 lines in your changes missing coverage. Please review.
✅ Project coverage is 63.84%. Comparing base (35df59b) to head (0dfa86b).
⚠️ Report is 133 commits behind head on main.

Files with missing lines Patch % Lines
cmd/gitops-migrate/cmd_migrate.go 57.24% 151 Missing and 23 partials ⚠️
cmd/gitops-migrate/cmd_format.go 0.00% 109 Missing ⚠️
cmd/gitops-migrate/backup.go 51.63% 73 Missing and 16 partials ⚠️
cmd/gitops-migrate/restore.go 42.96% 61 Missing and 12 partials ⚠️
cmd/gitops-migrate/cmd_usage.go 0.00% 35 Missing ⚠️
cmd/gitops-migrate/log/log.go 77.16% 29 Missing ⚠️
cmd/gitops-migrate/cmd.go 0.00% 25 Missing ⚠️
cmd/gitops-migrate/fs.go 40.47% 21 Missing and 4 partials ⚠️
cmd/gitops-migrate/main.go 0.00% 23 Missing ⚠️
cmd/gitops-migrate/args.go 0.00% 22 Missing ⚠️
... and 6 more
Additional details and impacted files
@@            Coverage Diff             @@
##             main   #32237      +/-   ##
==========================================
- Coverage   64.02%   63.84%   -0.19%     
==========================================
  Files        1987     2032      +45     
  Lines      194728   200488    +5760     
  Branches     6484     6484              
==========================================
+ Hits       124683   127999    +3316     
- Misses      60314    62353    +2039     
- Partials     9731    10136     +405     
Flag Coverage Δ
backend 65.03% <45.14%> (-0.25%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Comment thread cmd/fleetctl/gitops-migrate/restore.go Fixed
@Illbjorn
Copy link
Copy Markdown
Contributor Author

Illbjorn commented Aug 23, 2025

Sample command usage text (gitops-migrate usage):

Code_pZuToqWGe4

@mostlikelee
Copy link
Copy Markdown
Contributor

I think there's enough "meat" here to justify bringing this to backend sync to get more opinions on this architecture

Signed-off-by: Illbjorn <am@hades.so>
Signed-off-by: Illbjorn <am@hades.so>
Signed-off-by: Illbjorn <am@hades.so>
Signed-off-by: Illbjorn <am@hades.so>
Signed-off-by: Illbjorn <am@hades.so>
…), cont'd. tidying

Signed-off-by: Illbjorn <am@hades.so>
Signed-off-by: Illbjorn <am@hades.so>
Signed-off-by: Illbjorn <am@hades.so>
Signed-off-by: Illbjorn <am@hades.so>
Signed-off-by: Illbjorn <am@hades.so>
Signed-off-by: Illbjorn <am@hades.so>
Signed-off-by: Illbjorn <am@hades.so>
Signed-off-by: Illbjorn <am@hades.so>
Signed-off-by: Illbjorn <am@hades.so>
Signed-off-by: Illbjorn <am@hades.so>
Signed-off-by: Illbjorn <am@hades.so>
Signed-off-by: Illbjorn <am@hades.so>
… make this concurrent

Signed-off-by: Illbjorn <am@hades.so>
Signed-off-by: Illbjorn <am@hades.so>
Signed-off-by: Illbjorn <am@hades.so>
Signed-off-by: Illbjorn <am@hades.so>
Signed-off-by: Illbjorn <am@hades.so>
Signed-off-by: Illbjorn <am@hades.so>
@Illbjorn
Copy link
Copy Markdown
Contributor Author

Illbjorn commented Sep 4, 2025

After engaging our team members in infrastructure, concerns were expressed regarding building these binaries outside of GHA. To address these concerns a Workflow has been added to:

  • Build gitops-migrate for the appropriate OS/architecture permutations.
  • SHA-256 hash the built binary (sha256sum).
  • Upload the binary + hash file to an R2 bucket where they will be accessible via https://custom-tools.fleetdm.com (ex: https://custom-tools.fleetdm.com/gitops-migrate-windows-amd64.exe).

The Workflow was successfully tested end-to-end via Act (including the R2 upload):
Code_bIErMrykZA

Following the Workflow run I was able to confirm the artifacts existed in S3:
Code_ojuDynhyEe

And lastly, I confirmed that after pulling down the binary and SHA-256 hash file the hashes verified:
Code_BCI9QaD6SL

@Illbjorn Illbjorn requested a review from iansltx September 4, 2025 13:02
Signed-off-by: Illbjorn <am@hades.so>
…s in GHA)

Signed-off-by: Illbjorn <am@hades.so>
Signed-off-by: Illbjorn <am@hades.so>
Signed-off-by: Illbjorn <am@hades.so>
iansltx
iansltx previously approved these changes Sep 4, 2025
Signed-off-by: Illbjorn <am@hades.so>
iansltx
iansltx previously approved these changes Sep 5, 2025
…o we remove the `len` check and nest the `[]any` assertion to ensure the map key delete still happens

Signed-off-by: Illbjorn <am@hades.so>
iansltx
iansltx previously approved these changes Sep 5, 2025
Comment thread cmd/fleetctl/gitops-migrate/README.md
@Illbjorn Illbjorn requested a review from iansltx September 8, 2025 16:20
Comment thread cmd/gitops-migrate/restore.go Dismissed
@Illbjorn Illbjorn merged commit 288ea58 into fleetdm:main Sep 8, 2025
59 of 61 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

GitOps software/setup experience restructure: YAML migration tool

5 participants