Configuration profiles: Custom targets with "Include any" and "Exclude any"

[noahtalerman]

Goal

User story
As an IT admin that's targeting specific hosts by adding a label (include) to a configuration profile,
I want to exclude some hosts within that label
so that I can have more control over which hosts get that profile without having to write a long label query.

Changes

Product

• UI changes:
  • Figma
  • Error messages should only reject attempts to combine labels_include_all and labels_include_any:
    • any error message for configuration profiles that shows for "only one" label (For each profile, only one of "labels_exclude_any", "labels_include_all", "labels_include_any" or "labels" can be included.) update to: Couldn't edit configuration profiles. For each profile, only one of "labels_include_all" or "labels_include_any" can be included.
  • Error when you attempt to remove a label already tied to configuration profile:
    • Couldn't delete. A configuration profile targets this label. Please delete the profile and try again.

• CLI (fleetctl) usage changes: fleetctl apply and fleetctl generate-gitops will need to handle new functionality of combined labels.
• YAML changes: [YAML] Update configuration for macOS profiles example #44327
• REST API changes: [API] Update label inclusion/exclusion rules in API docs #44328
• Fleet's agent (fleetd) changes: No changes
• GitOps mode UI changes: No changes Currently, "Add profile" is disabled in GitOps mode.
• GitOps generation changes: No changes Currently, existing profile generation code exists independently for each label.
• Activity changes: No changes
• Permissions changes: No changes
• Changes to paid features or tiers: No changes
• My device and fleetdm.com/better changes: No changes
• Other reference documentation changes: No changes
• First draft of test plan added
• Once shipped, requester has been notified
• Once shipped, dogfooding issue has been filed

Engineering

• Test plan is finalized
• Contributor API changes: No changes
• Feature guide changes: [Guide] add combined include/exclude targeting for configuration profiles #44795
• Database schema migrations: See subtasks
• Load testing: See subtasks We will want to load test profile reconciliation compared to v4.86
• Load testing/osquery-perf improvements: No changes

ℹ️  Please read this issue carefully and understand it. Pay special attention to UI wireframes, especially "dev notes".

Test plan

Make sure to go through the list and consider all events that might be related to this story, so we catch edge cases earlier.

Note that in all test cases below "all 6 cases" means:

1. No labels
2. Include-any
3. Include-all
4. Exclude-any
5. Include-any + exclude-any
6. Include-all + exclude-any

"All 4 profile types" means:

1. Windows profile
2. Apple profile
3. Apple declaration
4. Android profile

"All 3 label types" means:

1. Manual
2. Dynamic(e.g. query)
3. IDP

Finally: When testing any exclude-any rules with dynamic labels, note that the rule can't apply until the host has run queries and its labels have been recalculated AFTER the creation of the exclued any label, meaning if for instance:

1. Host does a refetch, calculating labels
2. User creates a label, e.g. "Macs on Tahoe" with query "mac running os >= 26"
3. User creates a profile referencing "Macs on tahoe" as an exclude label
4. Profile gets applied to a mac running macOS 15

A profile with that label on it shouldn't get installed on macs running a build lower than tahoe until after the next refetch. So if 1 happened, then 2-3 happened one minute later, 4 shouldn't happen until a bit less than 1 hour later

Core flow

Include any + exclude any via API

• Upload all 4 profile types with a combination of include_any and exclude_any and verify profile is created successfully and both fields returned in response. Verify include-any list labels can't be included on the exclude list

Include all + exclude any via API

• Upload all 4 profile types with a combination of include_all and exclude_any and verify profile is created successfully and both fields returned in response. Verify labels are shown properly. Verify include-all list labels can't be included on the exclude list

Profiles batch endpoint:

• Upload all 4 profile types with all 6 cases for Windows, Apple and Android including declarations(doesn't have to be same request but make sure endpoint supports all 24 cases in the matrix properly)
• Upload all 4 profile types mixing include_any and include_all, verify proper errors are reported
• Upload all 4 profile types mixing include_any, include_all, exclude_any and verify proper errors are reported

Profile targeting UI

Run all tests for all 4 profile types

• Save only include_all and verify profile is created successfully and displayed properly.
• Save only include_any and verify profile is created successfully and displayed properly.
• Save only exclude_any and verify profile is created successfully and displayed properly.
• Save include all + exclude any and verify profile is created successfully and displayed properly
• Save include any + exclude any and verify profile is created successfully and displayed properly
• Make sure the UI doesn't allow you to specify the same labels for include any + exclude any
• Make sure the UI doesn't allow you to specify the same labels for include all + exclude any
• Make sure the UI doesn't allow you to specify include all + include any at the same time

Profile targeting backend

• Confirm profile delivery, or non-delivery with all 4 profile types and all 6 cases, so at least 24 cases total
• Confirm profile removal if exclusion label applied to host after a profile was originally delivered in exclude-any, exclude-any+include-any, exclude-any+include-all, with all 4 profile types
• Make sure each of the 3 label types(manual, dynamic, host vitals/IDP) can be used in all cases 1 and 2 and can be mixed

Confirm that any time exclude-any rules are used, the existing exclusion logic is used such that the host must check in and have its labels calculated AFTER(see note above)

No labels UI

• Confirm when no labels, we see empty state with button to "Add label" that takes you to create a new label.
  • After new label is created, user taken to labels page.

Delete label

• Confirm that attempting to delete a label tied to any of the 4 types of configuration profile as an include-any, include-all or exclude label you get appropriate error:
  • UI
  • API
  • GitOps

Fleet Premium & Fleet Free

• Free: verify no label/target options when adding a profile in UI
• verify you can't target via API or GitOps

Platform testing(Covered above, listed here as a checklist)

• macOS
  • Declaration
  • Profiles
• Windows (Profiles)
• Android

Label types(Covered above, listed here as a checklist)

• Manual labels
• Dynamic labels
• Host vitals based labels (IdP)

UI

• Verify that all UI changes specified in the Figma wireframes are correctly implemented
  • Verify the small screen experience is as expected on Controls > OS settings > Configuration profiles
• Verify that many labels for include and/or exclude scrolls on overflow-x (similar to MDM command details modal).
• Verify expected UI states (loading, empty, error states as applicable)
• Verify Error messages should only reject attempts to combine labels_include_all and labels_include_any:
  • Couldn't edit configuration profiles. For each profile, only one of "labels_include_all" or "labels_include_any" can be included.
• Verify error when you attempt to remove a label already tied to configuration profile:
  • Couldn't delete. A configuration profile targets this label. Please delete the profile and try again.

GitOps (generate + run)

• Confirm that profiles of all 4 types can be uploaded for all 6 cases via gitops. Verify label settings in the UI
• Confirm the profile upload modal behaves as expected with Gitops enabled
• Confirm that generate-gitops properly exprots all 4 types with all 6 cases
• Confirm proper error is returned when mixing labels_include_any and labels_include_all in gitops
• (Also covered above) Confirm an error happens in gitops if a label referenced by a profile is deleted
• fleetctl apply with labels_include_any + labels_exclude_any on a profile. Verify success.
• fleetctl generate-gitops for a fleet with a combined-label profile. Verify output includes both keys.

Supplemental testing

Testing notes

Confirmation

1. Engineer: Added comment to user story confirming successful completion of test plan (include any special setup, test data, or configuration used during development/testing if applicable).
2. QA: Added comment to user story confirming successful completion of test plan.