Skip to content

fleetctl gitops#16535

Merged
getvictor merged 25 commits intomainfrom
victor/13643-fleetctl-gitops
Feb 9, 2024
Merged

fleetctl gitops#16535
getvictor merged 25 commits intomainfrom
victor/13643-fleetctl-gitops

Conversation

@getvictor
Copy link
Copy Markdown
Member

@getvictor getvictor commented Feb 1, 2024

Add fleetctl gitops command for #13643

Code review video: https://www.loom.com/share/7941c51c709b44ccafd618dd05837d99?sid=27b923d7-1393-4396-bac7-30616b2d6de9

fleet-gitops PR that also needs review: fleetdm/fleet-gitops#26

Working global/team gitops configs that can be used for testing: https://github.com/fleetdm/fleet-gitops/tree/victor/fixing-configs

Checklist for submitter

If some of the following don't apply, delete the relevant line.

  • Changes file added for user-visible changes in changes/ or orbit/changes/.
  • Added/updated tests
  • Manual QA for all new/changed functionality

@codecov
Copy link
Copy Markdown

codecov Bot commented Feb 1, 2024

Codecov Report

Attention: 85 lines in your changes are missing coverage. Please review.

Comparison is base (3d92503) 65.60% compared to head (c0b16db) 65.79%.
Report is 90 commits behind head on main.

Files Patch % Lines
server/service/client.go 67.89% 45 Missing and 16 partials ⚠️
pkg/spec/gitops.go 96.17% 9 Missing and 5 partials ⚠️
cmd/fleetctl/gitops.go 94.73% 2 Missing and 1 partial ⚠️
ee/server/service/teams.go 70.00% 2 Missing and 1 partial ⚠️
server/service/client_policies.go 89.65% 2 Missing and 1 partial ⚠️
server/service/scripts.go 50.00% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main   #16535      +/-   ##
==========================================
+ Coverage   65.60%   65.79%   +0.18%     
==========================================
  Files        1132     1138       +6     
  Lines       99069   100094    +1025     
  Branches     2448     2448              
==========================================
+ Hits        64996    65852     +856     
- Misses      29198    29330     +132     
- Partials     4875     4912      +37     
Flag Coverage Δ
backend 66.86% <87.53%> (+0.18%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@getvictor getvictor marked this pull request as ready for review February 6, 2024 23:14
@getvictor getvictor requested a review from a team as a code owner February 6, 2024 23:14
Copy link
Copy Markdown
Member

@lucasmrod lucasmrod left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall looks great! Left some questions.

I've reviewed all non-test code. Am approving and will review tests tomorrow.

Comment thread cmd/fleetctl/gitops.go Outdated
Comment thread cmd/fleetctl/gitops.go
Comment on lines +59 to +68
err = fleetClient.DoGitOps(c.Context, config, baseDir, logf, flDryRun)
if err != nil {
return err
}
if flDryRun {
_, _ = fmt.Fprintf(c.App.Writer, "[!] gitops dry run succeeded\n")
} else {
_, _ = fmt.Fprintf(c.App.Writer, "[!] gitops succeeded\n")
}
return nil
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The issue says:

fleetctl gitops does a dry run first by default. If there's an error, show a helpful error message and don't make any changes to Fleet.

Shouldn't the non-dry run do a dry-run first?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That comment refers to the GitHub action flow, which is the intended use case for fleetctl gitops. I kept the dry-run behavior consistent with how fleetctl apply works. FYI: @noahtalerman

Comment thread pkg/spec/gitops.go Outdated
Comment thread pkg/spec/gitops.go
Comment thread pkg/spec/gitops.go
Comment thread server/service/client.go Outdated
group.AppConfig = config.OrgSettings
group.EnrollSecret = &fleet.EnrollSecretSpec{Secrets: config.OrgSettings["secrets"].([]*fleet.EnrollSecret)}
group.AppConfig.(map[string]interface{})["agent_options"] = config.AgentOptions
delete(config.OrgSettings, "secrets")
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Curious why delete?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Secrets are applied separately with Client.ApplyGroup, and it will trigger a validation fail if I don't delete them here.

Comment thread server/service/client_policies.go
Comment thread server/service/client_policies.go
Comment thread server/service/mdm.go
return ctxerr.Wrap(ctx, err, "getting app config")
}
if assumeEnabled {
appCfg.MDM.WindowsEnabledAndConfigured = true
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What fails if you don't do this?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Dry run fails when I try to enable and configure MDM at the same time.

Comment thread pkg/spec/test_data/agent-options.yml
@lucasmrod
Copy link
Copy Markdown
Member

In the user story it says:

All keys are required. If a top-level key is empty, then remove all settings for that key.

What does it mean? (by remove all settings for that key?)

@lucasmrod
Copy link
Copy Markdown
Member

lucasmrod commented Feb 9, 2024

On the demo I see that in some occasions you need to run fleetctl gitops twice (to create the teams first IIRC)

Should this behavior be documented?

Copy link
Copy Markdown
Member

@lucasmrod lucasmrod left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, double checking something first:
Are you running fleetctl gitops as a user with GitOps role?

Or are we ok with running gitops with admin/maintainer roles? IIRC we use the gitops role for dogfood.

@getvictor
Copy link
Copy Markdown
Member Author

In the user story it says:

All keys are required. If a top-level key is empty, then remove all settings for that key.

What does it mean? (by remove all settings for that key?)

If user supplies empty policies, then we remove all policies.
If user supplies empty org settings, then we set all settings to defaults.

@getvictor
Copy link
Copy Markdown
Member Author

On the demo I see that in some occasions you need to run fleetctl gitops twice (to create the teams first IIRC)

Should this behavior be documented?

That shouldn't be the case. I think I run dry-run first, and then regular run. There is one MDM bug that caused me to run it twice: #16636

@getvictor
Copy link
Copy Markdown
Member Author

Ah, double checking something first: Are you running fleetctl gitops as a user with GitOps role?

Or are we ok with running gitops with admin/maintainer roles? IIRC we use the gitops role for dogfood.

Yes, gitops role should run fleetctl gitops
I updated the role to allow read of queries/policies, and write/read of scripts. Tested locally.

@getvictor
Copy link
Copy Markdown
Member Author

I will have a separate PR for gitops role: #16710
It requires a bunch of test updates.

@getvictor getvictor merged commit e4d5e27 into main Feb 9, 2024
@getvictor getvictor deleted the victor/13643-fleetctl-gitops branch February 9, 2024 19:34
georgekarrv pushed a commit that referenced this pull request Feb 13, 2024
Add `fleetctl gitops` command for #13643 

Code review video:
https://www.loom.com/share/7941c51c709b44ccafd618dd05837d99?sid=27b923d7-1393-4396-bac7-30616b2d6de9

fleet-gitops PR that also needs review:
fleetdm/fleet-gitops#26

Working global/team gitops configs that can be used for testing:
https://github.com/fleetdm/fleet-gitops/tree/victor/fixing-configs

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants