Manage labels via YAML (GitOps)

[noahtalerman]

Goal

User story
As a Fleet user,
I want to manage labels in Fleet using Fleet's YAML files
so that I can manage custom targets for my software, configuration profiles, and queries using best practice GitOps.

Key result

Deliver prioritized customer requests

Original requests

• Ability to manage labels via best practice GitOps #17944

Context

• Product designer: @rachaelshaw

Changes

Product

• UI changes: Figma here.
• CLI (fleetctl) usage changes: No changes.
• YAML changes: PR here.
• REST API changes: PR here.
• Fleet's agent (fleetd) changes: No changes.
• GitOps mode changes: No changes. (GitOps mode does NOT disable labels, because it is an optional key in default.yml)
• Activity changes: No changes.
• Permissions changes: PR here.
• Changes to paid features or tiers: Fleet Premium and Fleet Free
• Transparency changes: No changes.
• First draft of test plan added
• Other reference documentation changes: No chanages.
• Once shipped, requester has been notified
  • @noahtalerman: Leaving this story open until we resolve this bug: GitOps: can't add labels and use labels in the same commit #28154
  • More context here: Ability to manage labels via best practice GitOps #17944 (comment)
• Once shipped, dogfooding issue has been filed

Engineering

• Feature guide changes: Need to find a relevant guide and add usage instructions. (Could be part of a help menu)
• Database schema migrations: No need
• Load testing: No need

ℹ️  Please read this issue carefully and understand it. Pay special attention to UI wireframes, especially "dev notes".

Technical Design Plan #

• In the GitOpsFromFile implementation in pkg/spec/gitops.go:

  • Parse label: key in YAML file (if it exists), read in any files referenced by path, validate syntax and add labels to gitops config var.

• In the gitopsCommand implementation in cmd/fleetctl/gitops.go (where gitops YAML is parsed):

  • Get the current set of labels persisted in the db, for use in determining whether to complain about in-use labels being removed, or about inexistent labels being referenced by other sections
  • Check whether the labels: key exists at the top level of the global YML file. If not, populate config.Labels with the currently-persisted labels as if they'd been specified in the YAML. This is for backwards-compatibility so that customers don't blow away all their labels the next time they run gitops because they haven't populated labels: yet.
  • Examine the areas of the YAML where labels may be used (software installers, profiles and queries), and gather info about their usage in a map of label name -> uses.
  • Compare the usage map to the set of labels in config.Labels. If any used labels would not exist after the gitops run, bail out and output warning like Label XXX would be not exist after this run, but is referenced by the following entities: software installer with url "http://foo.bar", query "Get outdated software", etc.

• In the DoGitOps implementation in server/service/client.go (where the parsed spec is prepped for sending to the API, and deletions are processed)

  • Compare current labels names to proposed label names and do any label deletion (if in dry-run mode, just output a message about what would be deleted

• In the ApplyGroup implementation in server/service/client.go

  • No changes needed -- we already support applying Labels via the API, so this should already work.

QA

Risk assessment

• Requires load testing: No
• Risk level: Medium
• Risk description: Could potentially affect other labels. Need a thorough QA

Manual testing steps

• QA engineer signed-off on the test-plan below. ( @xpkoala )

Test plan

• Defining labels inline: Add a label inline default.yml. Run GitOps.
  • Label should be added
• Defining labels in a file: Create a YAML file with one label in it and reference it in default.yml. Run GitOps.
  • Label should be added
• Multiple labels in one YAML file: Add another label to the referenced labels.yml file. Then reference both labels when applying custom target to software.
  • Both labels should exist
  • Software should target both of these labels
• Deleting labels: in default.yml: include the labels key but do not define any labels.
  • All labels should be deleted.
• Errors:
  • Add a label to default.yml and reference it in configuration profile targets. Run GitOps. Remove the label and run GitOps again.
    • Should get an error removing the label.
  • Add a label to default.yml and reference it in software installation targets. Run GitOps. Remove the label and run GitOps again.
    • Should get an error removing the label.
  • Add a label to default.yml and reference it in query targets. Run GitOps. Remove the label and run GitOps again.
    • Should get an error removing the label.
• labels is an optional key: remove the labels key from default.yml. Now, create some labels in the UI. After applying GitOps changes, labels created via the UI should still exist.
• Team admin permissions: As a team admin, create a label in the UI.
  • Filter hosts by the label. Should see edit/delete button.
  • Edit label. Changes should be saved.
  • Delete label. Should be able to delete.
  • Filter hosts by a label authored by another user. Should not see edit/delete button.
    • Should also not be able to navigate directly to the edit page
    • Should not be able to edit via the API
    • Should not be able to delete via the API
• Team maintainer permissions: As a team maintainer, create a label in the UI.
  • Filter hosts by the label. Should see edit/delete button.
  • Edit label. Changes should be saved.
  • Delete label. Should be able to delete.
  • Filter hosts by a label authored by another user. Should not see edit/delete button.
    • Should also not be able to navigate directly to the edit page
    • Should not be able to edit via the API
    • Should not be able to delete via the API
• API changes: hit the add, update, get, and list labels API endpoints and verify that author_id is included.

Testing notes

Confirmation

1. Engineer (@____): Added comment to user story confirming successful completion of QA.
2. QA (@____): Added comment to user story confirming successful completion of QA.

[mna]

@noahtalerman note for when that story will be worked on, the #22813 story will have been completed and will prevent deletion of labels referenced by software installers. It will be important for the gitops approach to ensure a similar validation/error message when deleting labels. (for whoever will work on this - this is enforced at the database level via a FK in software_installer_labels)

[noahtalerman]

when that story will be worked on, the #22813 story will have been completed and will prevent deletion of labels referenced by software installers. It will be important for the gitops approach to ensure a similar validation/error message when deleting labels. (for whoever will work on this - this is enforced at the database level via a FK in software_installer_labels)

FYI @rachaelshaw