Skip to content

Apple ACME profile validation + accept CERTIFICATE_RENEWAL_ID variable (PR 6/9)#45043

Merged
mostlikelee merged 15 commits into
40639-cert-renewfrom
pr-2.3-apple-validation-rename
May 12, 2026
Merged

Apple ACME profile validation + accept CERTIFICATE_RENEWAL_ID variable (PR 6/9)#45043
mostlikelee merged 15 commits into
40639-cert-renewfrom
pr-2.3-apple-validation-rename

Conversation

@mostlikelee
Copy link
Copy Markdown
Contributor

@mostlikelee mostlikelee commented May 8, 2026

Related issue: Resolves #44346

What this PR does

Adds a top-level Apple ACME profile validator: any profile containing a com.apple.security.acme payload must carry the renewal-ID marker variable in the cert Subject's CN or OU, or the upload is rejected.

Adds \$FLEET_VAR_CERTIFICATE_RENEWAL_ID as the preferred name for the renewal-ID marker, alongside the legacy \$FLEET_VAR_SCEP_RENEWAL_ID. Both are accepted by validation and both substitute to fleet-<profile_uuid>. Validation error messages reference only the new name.

Why this is needed

Phase 2 (#40639) auto-renews certs from non-proxied CAs. Renewal works by matching ingested certs against fleet-<profile_uuid> markers in their Subject — so the marker variable has to actually be in the profile. The ACME validator forces customers to include it before the profile can be uploaded.

The variable rename matches the customer-facing docs (PR #44069); back-compat with the legacy name avoids breaking already-deployed profiles.

Checklist for submitter

  • Input data is properly validated, SELECT * is avoided, SQL injection is prevented (using placeholders for values in statements)

Testing

  • Added/updated automated tests

Summary by CodeRabbit

  • New Features

    • Added ACME certificate renewal ID validation support for configuration profiles
    • Introduced CERTIFICATE_RENEWAL_ID as the preferred Fleet variable for certificate renewal configuration
  • Chores

    • Updated error messages and validation logic to reference the new CERTIFICATE_RENEWAL_ID variable name; legacy SCEP_RENEWAL_ID remains supported for backward compatibility

Review Change Stack

Two related changes that prepare Apple profile uploads for the non-proxied
renewal flow:

1. Add `$FLEET_VAR_CERTIFICATE_RENEWAL_ID` as the preferred name for the
   renewal-ID marker variable (per docs PR #44069). The legacy
   `$FLEET_VAR_SCEP_RENEWAL_ID` is still accepted by validation and
   substitution for back-compat with existing profiles. Both names
   substitute to the same value: "fleet-" + profile_uuid.

2. Add additionalACMEValidation: any profile containing a
   com.apple.security.acme payload must include either renewal-ID
   variable in the cert Subject's CN or OU, otherwise the upload is
   rejected. Without the marker the resulting cert can't be linked back
   to its profile and would never auto-renew.

The shared SCEP-renewal-ID-without-URL/Challenge guard in
mdm_profiles.go skips its check when an ACME payload is present —
ACME profiles use the marker variable alone (no SCEP URL/Challenge
companions).

Validation error messages reference only the new
CERTIFICATE_RENEWAL_ID name to steer new authoring toward it.
@codecov
Copy link
Copy Markdown

codecov Bot commented May 8, 2026

Codecov Report

❌ Patch coverage is 86.95652% with 6 lines in your changes missing coverage. Please review.
⚠️ Please upload report for BASE (40639-cert-renew@b1d165b). Learn more about missing BASE report.

Files with missing lines Patch % Lines
server/service/apple_mdm.go 80.64% 3 Missing and 3 partials ⚠️
Additional details and impacted files
@@                 Coverage Diff                 @@
##             40639-cert-renew   #45043   +/-   ##
===================================================
  Coverage                    ?   66.83%           
===================================================
  Files                       ?     2726           
  Lines                       ?   219217           
  Branches                    ?    10717           
===================================================
  Hits                        ?   146514           
  Misses                      ?    59526           
  Partials                    ?    13177           
Flag Coverage Δ
backend 68.70% <86.95%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@sharon-fdm
Copy link
Copy Markdown
Collaborator

@mostlikelee , removing the PR from the board (The ticket is there)

The validation error messages in mdm_profiles.go switched to reference
the preferred CERTIFICATE_RENEWAL_ID name. The unit and integration
test assertions still required the legacy SCEP_RENEWAL_ID text.
Updating them to match.
Decision 2.7 requires profile validation to keep accepting the
pre-rename variable name. The existing failure-case tests don't
guard that contract because they all exercise error paths. Add a
happy-path case asserting that a Custom SCEP profile authored with
$FLEET_VAR_SCEP_RENEWAL_ID validates successfully when all other
required variables are present.
@mostlikelee mostlikelee changed the title Apple ACME profile validation + accept CERTIFICATE_RENEWAL_ID variable (PR 6/8) Apple ACME profile validation + accept CERTIFICATE_RENEWAL_ID variable (PR 6/9) May 12, 2026
@mostlikelee
Copy link
Copy Markdown
Contributor Author

@coderabbitai full review

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented May 12, 2026

✅ Actions performed

Full review triggered.

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented May 12, 2026

Walkthrough

This PR introduces FleetVarCertificateRenewalID as the preferred Fleet variable for SCEP/NDES/Smallstep renewal identifiers, replacing the legacy FleetVarSCEPRenewalID while maintaining back-compat. New regex patterns match both variable names equivalently. Apple profile validation now enforces renewal-ID markers in ACME payload certificate subjects before variable scanning. SCEP/NDES/Smallstep validation error messages are updated to reference the new variable name. Profile processors across Apple and Windows platforms treat both renewal ID variables as interchangeable during variable substitution. Shared MDM validation logic normalizes error messages to always surface the preferred name and suppresses renewal-ID-only validation for ACME profiles.

Possibly related PRs

  • fleetdm/fleet#44372: Both PRs introduce and modify ACME payload detection and validation logic in Apple MDM handling, touching the same feature area for certificate renewal and ACME profile support.
🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 42.11% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly and concisely describes the main changes: adding Apple ACME profile validation and accepting the new CERTIFICATE_RENEWAL_ID variable.
Description check ✅ Passed The PR description covers the main objectives (ACME validation, new variable name) and testing, but the provided checklist is incomplete with only 2 items checked, missing sections like database migrations and changes files.
Linked Issues check ✅ Passed The PR successfully addresses issue #44346 by adding ACME profile validation that rejects profiles lacking the renewal-ID marker in the certificate Subject's CN or OU, and introduces CERTIFICATE_RENEWAL_ID as the preferred variable name while maintaining backward compatibility.
Out of Scope Changes check ✅ Passed All changes are directly aligned with the objective of adding ACME validation and renaming the renewal-ID variable. No out-of-scope modifications detected beyond the stated scope.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch pr-2.3-apple-validation-rename

Warning

Review ran into problems

🔥 Problems

Git: Failed to clone repository. Please run the @coderabbitai full review command to re-trigger a full review. If the issue persists, set path_filters to include or exclude specific files.

Tip

💬 Introducing Slack Agent: The best way for teams to turn conversations into code.

Slack Agent is built on CodeRabbit's deep understanding of your code, so your team can collaborate across the entire SDLC without losing context.

  • Generate code and open pull requests
  • Plan features and break down work
  • Investigate incidents and troubleshoot customer tickets together
  • Automate recurring tasks and respond to alerts with triggers
  • Summarize progress and report instantly

Built for teams:

  • Shared memory across your entire org—no repeating context
  • Per-thread sandboxes to safely plan and execute work
  • Governance built-in—scoped access, auditability, and budget controls

One agent for your entire SDLC. Right inside Slack.

👉 Get started


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@server/service/apple_mdm.go`:
- Around line 663-664: The validation errors incorrectly reference only the OU
while the code accepts the renewal marker in either the certificate Subject CN
or OU; update the error messages near the checks that use
fleet.FleetVarRenewalIDRegexp against scepPayloadContent.CommonName and
scepPayloadContent.OrganizationalUnit so they mention both CN and OU (e.g.,
"must be in the SCEP certificate's subject CN or organizational unit (OU)").
Make this change for the occurrences around the existing check using
fleet.FleetVarRenewalIDRegexp and returning &fleet.BadRequestError{Message: ...}
(the instances at the current locations and the other similar checks referenced
in the file).
- Around line 782-783: The ACME validation currently only checks
fleet.FleetVarCertificateRenewalIDRegexp against commonName and orgUnit,
rejecting the older $FLEET_VAR_SCEP_RENEWAL_ID; replace that single-regex check
with the unified renewal-ID regex used elsewhere (the combined FleetVar
renewal-ID regex) so both $FLEET_VAR_CERTIFICATE_RENEWAL_ID and
$FLEET_VAR_SCEP_RENEWAL_ID are accepted when validating commonName.String() and
orgUnit.String().

In `@server/service/mdm_profiles.go`:
- Around line 511-517: The check uses a brittle substring search
(strings.Contains(profileContents, mobileconfig.ACMEPayloadType)) to skip
renewal-only validation; replace this with a structured ACME payload detection
by parsing the mobileconfig/profile payloads and verifying any payload has Type
== mobileconfig.ACMEPayloadType (i.e., implement or call a helper like
ProfileHasPayloadType(profileContents, mobileconfig.ACMEPayloadType) that
unmarshals the plist/JSON profile and inspects payload dictionaries). Update the
conditional to use that helper (referencing profileContents and
mobileconfig.ACMEPayloadType) so the renewal-ID-only bypass only triggers when
an actual ACME payload is present.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: ec5a257b-f08c-4376-a2e3-b9379e1586df

📥 Commits

Reviewing files that changed from the base of the PR and between 1c0edda and b67f0e2.

📒 Files selected for processing (10)
  • server/fleet/errors.go
  • server/fleet/mdm.go
  • server/fleet/mdm_test.go
  • server/mdm/apple/profile_processor.go
  • server/mdm/microsoft/profile_variables.go
  • server/service/apple_mdm.go
  • server/service/apple_mdm_test.go
  • server/service/integration_mdm_profiles_test.go
  • server/service/mdm_profiles.go
  • server/service/mdm_profiles_test.go

Comment thread server/service/apple_mdm.go
Comment thread server/service/apple_mdm.go Outdated
Comment thread server/service/mdm_profiles.go Outdated
The renewal-ID-only bypass in validateProfileCertificateAuthorityVariables
checked the profile body with strings.Contains, which would false-positive
on "com.apple.security.acme" appearing in a description, identifier, or
custom string field. Switch to the existing Mobileconfig.HasPayloadType
helper for a structured plist parse. Parse failure (e.g., Windows-path
content that isn't a mobileconfig) is treated as no-ACME, so the SCEP-only
error fires normally on that path.
@mostlikelee mostlikelee marked this pull request as ready for review May 12, 2026 15:00
@mostlikelee mostlikelee requested a review from a team as a code owner May 12, 2026 15:00
Copy link
Copy Markdown

@claude claude Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This repository is configured for manual code reviews. Comment @claude review to trigger a review and subscribe this PR to future pushes, or @claude review once for a one-time review.

Tip: disable this comment in your organization's Code Review settings.

Copy link
Copy Markdown
Contributor

@sgress454 sgress454 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense, one nit question.

Comment thread server/fleet/mdm.go Outdated
@@ -118,9 +119,17 @@ var (
FleetVarNDESSCEPProxyURLRegexp = regexp.MustCompile(fmt.Sprintf(`(\$FLEET_VAR_%s)|(\${FLEET_VAR_%[1]s})`, FleetVarNDESSCEPProxyURL))
FleetVarHostEndUserIDPFullnameRegexp = regexp.MustCompile(fmt.Sprintf(`(\$FLEET_VAR_%s)|(\${FLEET_VAR_%[1]s})`, FleetVarHostEndUserIDPFullname))
FleetVarSCEPRenewalIDRegexp = regexp.MustCompile(fmt.Sprintf(`(\$FLEET_VAR_%s)|(\${FLEET_VAR_%[1]s})`, FleetVarSCEPRenewalID))
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we still need this or is it completely superseded by FleetVarRenewalIDRegexp?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good catch, it removed

Net-new validation surface, so no back-compat for CN placement (per
Decision 2.7). Error message already said "must be in the OU"; this
aligns behavior with the message.

Also drops dead FleetVarSCEPRenewalIDRegexp — superseded by
FleetVarRenewalIDRegexp (unified, for back-compat sites) and
FleetVarCertificateRenewalIDRegexp (preferred-only, for net-new
validators). Zero callers.
@mostlikelee
Copy link
Copy Markdown
Contributor Author

@sgress454 note I made one more change after your review. The guides all point customer to do OU=renewalID, so I assume all the existing CN support is for some undocumented backwards compatibility. Since Acme and Non-proxied SCEP renewals are both net new functionality, I removed support for CN=renewalID on those.

@qodo-free-for-open-source-projects
Copy link
Copy Markdown

CI Feedback 🧐

A test triggered by this PR failed. Here is an AI-generated analysis of the failure:

Action: test-go (mysql, mysql:8.0.44) / test

Failed stage: Run Go Tests [❌]

Failed test name: TestIncrementalMigrationStep/progress_output

Failure summary:

The action failed because Go tests in ./server/datastore/mysql/... failed (job exited from make
.run-go-tests with non-zero status).
- Failing test: server/datastore/mysql/migrations/tables
TestIncrementalMigrationStep/progress_output.
- Failure details: assertion in
server/datastore/mysql/migrations/tables/migration_helpers_test.go:180 expected 6 progress output
entries but only got 5:
"[ 30% complete 50% complete 70% complete 90% complete
100% complete]" should have 6 item(s), but has 5.

Relevant error logs:
1:  ##[group]Runner Image Provisioner
2:  Hosted Compute Agent
...

917:  �[36;1mattempt=1�[0m
918:  �[36;1m�[0m
919:  �[36;1mwhile [ $attempt -le $max_attempts ]; do�[0m
920:  �[36;1m  echo "Attempt $attempt of $max_attempts"�[0m
921:  �[36;1m�[0m
922:  �[36;1m  # Try to connect to MySQL�[0m
923:  �[36;1m  if wait_for_mysql "mysql_test"; then�[0m
924:  �[36;1m    # If MySQL is ready, try to connect to MySQL replica�[0m
925:  �[36;1m    if wait_for_mysql "mysql_replica_test"; then�[0m
926:  �[36;1m      # Both are ready, we're done�[0m
927:  �[36;1m      echo "All MySQL connections successful"�[0m
928:  �[36;1m      exit 0�[0m
929:  �[36;1m    fi�[0m
930:  �[36;1m  fi�[0m
931:  �[36;1m�[0m
932:  �[36;1m  # If we get here, at least one connection failed�[0m
933:  �[36;1m  echo "Failed to connect to MySQL on attempt $attempt"�[0m
934:  �[36;1m�[0m
935:  �[36;1m  if [ $attempt -lt $max_attempts ]; then�[0m
936:  �[36;1m    echo "Restarting containers and trying again..."�[0m
937:  �[36;1m    restart_containers�[0m
938:  �[36;1m  else�[0m
939:  �[36;1m    echo "Maximum attempts reached. Failing the job."�[0m
940:  �[36;1m    exit 1�[0m
...

1040:  NEED_DOCKER: 1
1041:  ARTIFACT_PREFIX: mysql-mysql8.0.44
1042:  GOTOOLCHAIN: local
1043:  FLEET_PREVIEW_TAG: main
1044:  ##[endgroup]
1045:  make .run-go-tests PKG_TO_TEST="./server/datastore/mysql/..."
1046:  make[1]: Entering directory '/home/runner/work/fleet/fleet'
1047:  Running Go tests with gotestsum:
1048:  gotestsum --format=testdox --jsonfile=/tmp/test-output.json -- -tags full,fts5,netgo -run=  -v -race=false -timeout=20m  -parallel 8 -coverprofile=coverage.txt -covermode=atomic -coverpkg=github.com/fleetdm/fleet/v4/... ././server/datastore/mysql/... 
1049:  go: downloading github.com/DATA-DOG/go-sqlmock v1.5.0
1050:  go: downloading pgregory.net/rapid v1.2.0
1051:  github.com/fleetdm/fleet/v4/server/datastore/mysql/migrations/data:
1052:  github.com/fleetdm/fleet/v4/server/datastore/mysql/rdsauth:
1053:  github.com/fleetdm/fleet/v4/server/datastore/mysql/migrations/tables:
1054:  �[32m✓�[0m Basic migration step (0.00s)
1055:  �[32m✓�[0m Basic migration step error (0.00s)
1056:  �[32m✓�[0m Basic migration step success (0.00s)
1057:  �[32m✓�[0m Collation (10.46s)
1058:  �[31m✖�[0m Incremental migration step (0.08s)
1059:  �[32m✓�[0m Incremental migration step count error is returned (0.00s)
1060:  �[32m✓�[0m Incremental migration step executor error is returned (0.00s)
1061:  �[32m✓�[0m Incremental migration step increment updates progress (0.03s)
...

1352:  �[32m✓�[0m Up 20260401153503 many assignments (35.76s)
1353:  �[32m✓�[0m Up 20260401153503 no assignment (30.44s)
1354:  �[32m✓�[0m Up 20260401153503 some assignments (31.99s)
1355:  �[32m✓�[0m Up 20260409153715 (31.40s)
1356:  �[32m✓�[0m Up 20260409153717 (32.58s)
1357:  �[32m✓�[0m Up 20260423161823 (25.52s)
1358:  �[32m✓�[0m Up 20260427134220 fresh install (21.19s)
1359:  �[32m✓�[0m Up 20260427134220 upgraded install (14.15s)
1360:  �[32m✓�[0m Up 20260428125634 (12.74s)
1361:  �[32m✓�[0m Up 20260429180725 (13.56s)
1362:  �[32m✓�[0m Up 20260505115111 (15.60s)
1363:  �[32m✓�[0m Up 20260506132626 (12.32s)
1364:  �[32m✓�[0m Up 20260507160833 (9.25s)
1365:  �[32m✓�[0m With steps (0.00s)
1366:  �[32m✓�[0m With steps empty steps succeeds (0.00s)
1367:  �[32m✓�[0m With steps error stops execution (0.00s)
1368:  �[32m✓�[0m With steps integration with basic migration step (0.00s)
...

1406:  �[32m✓�[0m Activity list host upcoming activities 1: fleet. list options{ page: 0x 1, per page: 0x 4, order key:"", order direction: 0, match query:"", after:"", include metadata:false, test secondary order key:"", test secondary order direction: 0} (0.01s)
1407:  �[32m✓�[0m Activity list host upcoming activities 1: fleet. list options{ page: 0x 2, per page: 0x 2, order key:"", order direction: 0, match query:"", after:"", include metadata:false, test secondary order key:"", test secondary order direction: 0} (0.01s)
1408:  �[32m✓�[0m Activity list host upcoming activities 1: fleet. list options{ page: 0x 2, per page: 0x 4, order key:"", order direction: 0, match query:"", after:"", include metadata:false, test secondary order key:"", test secondary order direction: 0} (0.01s)
1409:  �[32m✓�[0m Activity list host upcoming activities 1: fleet. list options{ page: 0x 3, per page: 0x 2, order key:"", order direction: 0, match query:"", after:"", include metadata:false, test secondary order key:"", test secondary order direction: 0} (0.02s)
1410:  �[32m✓�[0m Activity list host upcoming activities 1: fleet. list options{ page: 0x 3, per page: 0x 4, order key:"", order direction: 0, match query:"", after:"", include metadata:false, test secondary order key:"", test secondary order direction: 0} (0.01s)
1411:  �[32m✓�[0m Activity list host upcoming activities 1: fleet. list options{ page: 0x 4, per page: 0x 2, order key:"", order direction: 0, match query:"", after:"", include metadata:false, test secondary order key:"", test secondary order direction: 0} (0.01s)
1412:  �[32m✓�[0m Activity list host upcoming activities 2: fleet. list options{ page: 0x 0, per page: 0x 5, order key:"", order direction: 0, match query:"", after:"", include metadata:false, test secondary order key:"", test secondary order direction: 0} (0.01s)
1413:  �[32m✓�[0m Activity list host upcoming activities 3: fleet. list options{ page: 0x 0, per page: 0x 0, order key:"", order direction: 0, match query:"", after:"", include metadata:false, test secondary order key:"", test secondary order direction: 0} (0.01s)
1414:  �[32m✓�[0m Activity list host upcoming activities 4: fleet. list options{ page: 0x 0, per page: 0x 0, order key:"", order direction: 0, match query:"", after:"", include metadata:false, test secondary order key:"", test secondary order direction: 0} (0.01s)
1415:  �[32m✓�[0m Activity list host upcoming activities rejects unknown order key (0.00s)
1416:  �[32m✓�[0m Activity set result after cancel upcoming activity (5.57s)
1417:  �[32m✓�[0m Activity unblock hosts upcoming activity queue (5.60s)
1418:  �[32m✓�[0m Activity username change (4.89s)
1419:  �[32m✓�[0m Aggregated stats (5.67s)
1420:  �[32m✓�[0m Aggregated stats queries (0.88s)
1421:  �[32m✓�[0m Already exists error (0.00s)
1422:  �[32m✓�[0m Already exists error is error (0.00s)
1423:  �[32m✓�[0m Already exists error with team ID (0.00s)
1424:  �[32m✓�[0m Already exists error with team name (0.00s)
1425:  �[32m✓�[0m Android (114.92s)
1426:  �[32m✓�[0m Android add delete android app with configuration (1.89s)
1427:  �[32m✓�[0m Android android BYOD detection (3.15s)
1428:  �[32m✓�[0m Android android BYOD detection company enrollment (0.02s)
1429:  �[32m✓�[0m Android android BYOD detection personal enrollment with UUID (0.03s)
1430:  �[32m✓�[0m Android android BYOD detection update existing host enrollment status (0.04s)
1431:  �[32m✓�[0m Android android MDM stats (4.42s)
1432:  �[32m✓�[0m Android android app configuration cascade delete team (1.65s)
1433:  �[32m✓�[0m Android android app configuration global vs team (1.56s)
1434:  �[32m✓�[0m Android android host storage data (4.30s)
1435:  �[32m✓�[0m Android batch set MDM android profiles associations (3.11s)
1436:  �[32m✓�[0m Android bulk delete MDM android host profiles (3.22s)
1437:  �[32m✓�[0m Android bulk delete MDM android host profiles deletes pending or failed remove profiles with policy version lower than or equal to passed (0.02s)
1438:  �[32m✓�[0m Android bulk delete MDM android host profiles does not delete install operation types (0.01s)
...

1473:  �[32m✓�[0m Android has android app configuration changed (1.58s)
1474:  �[32m✓�[0m Android has android app configuration changed boolean instead of object (0.00s)
1475:  �[32m✓�[0m Android has android app configuration changed empty compared to non-existing (0.00s)
1476:  �[32m✓�[0m Android has android app configuration changed empty managed configuration (0.00s)
1477:  �[32m✓�[0m Android has android app configuration changed empty new config (0.00s)
1478:  �[32m✓�[0m Android has android app configuration changed empty object (0.00s)
1479:  �[32m✓�[0m Android has android app configuration changed expanded different config (0.02s)
1480:  �[32m✓�[0m Android has android app configuration changed same config (0.00s)
1481:  �[32m✓�[0m Android has android app configuration changed slightly different config (0.00s)
1482:  �[32m✓�[0m Android has android app configuration changed some config compared to non-existing (0.00s)
1483:  �[32m✓�[0m Android has android app configuration changed very different config (0.00s)
1484:  �[32m✓�[0m Android insert and get android app configuration (2.73s)
1485:  �[32m✓�[0m Android list MDM android profiles to send (3.87s)
1486:  �[32m✓�[0m Android list MDM android profiles to send with exclude any (3.96s)
1487:  �[32m✓�[0m Android list host MDM android profiles pending install with version (2.57s)
1488:  �[32m✓�[0m Android list host MDM android profiles pending install with version does list pending install profiles and failed install profiles with can reverify (0.01s)
1489:  �[32m✓�[0m Android list host MDM android profiles pending install with version does list pending install profiles with version less than or equal to applied policy version (0.01s)
...

1548:  �[32m✓�[0m Campaigns cleanup completed campaign targets (4.03s)
1549:  �[33m∅�[0m Campaigns cleanup completed campaign targets large batch (3.58s)
1550:  �[32m✓�[0m Campaigns cleanup distributed query (3.84s)
1551:  �[32m✓�[0m Campaigns completed campaigns (4.49s)
1552:  �[32m✓�[0m Campaigns distributed query (3.93s)
1553:  �[32m✓�[0m Campaigns save distributed query (3.99s)
1554:  �[32m✓�[0m Carves (24.07s)
1555:  �[32m✓�[0m Carves blocks (3.89s)
1556:  �[32m✓�[0m Carves cleanup (4.16s)
1557:  �[32m✓�[0m Carves list (4.07s)
1558:  �[32m✓�[0m Carves list allowed order block count (0.00s)
1559:  �[32m✓�[0m Carves list allowed order block size (0.00s)
1560:  �[32m✓�[0m Carves list allowed order carve id (0.01s)
1561:  �[32m✓�[0m Carves list allowed order carve size (0.01s)
1562:  �[32m✓�[0m Carves list allowed order created at (0.00s)
1563:  �[32m✓�[0m Carves list allowed order error (0.00s)
1564:  �[32m✓�[0m Carves list allowed order expired (0.00s)
...

1566:  �[32m✓�[0m Carves list allowed order id (0.00s)
1567:  �[32m✓�[0m Carves list allowed order max block (0.00s)
1568:  �[32m✓�[0m Carves list allowed order name (0.01s)
1569:  �[32m✓�[0m Carves list allowed order request id (0.00s)
1570:  �[32m✓�[0m Carves list allowed order session id (0.00s)
1571:  �[32m✓�[0m Carves list rejects unknown key (0.00s)
1572:  �[32m✓�[0m Carves metadata (3.89s)
1573:  �[32m✓�[0m Carves update (3.89s)
1574:  �[32m✓�[0m Certificate authority (28.10s)
1575:  �[32m✓�[0m Certificate authority create certificate authority (4.23s)
1576:  �[32m✓�[0m Certificate authority delete (4.11s)
1577:  �[32m✓�[0m Certificate authority get all certificate authorities (3.89s)
1578:  �[32m✓�[0m Certificate authority get certificate authority by ID (4.11s)
1579:  �[32m✓�[0m Certificate authority list certificate authorities (3.82s)
1580:  �[32m✓�[0m Certificate authority update certificate authority by ID (3.78s)
1581:  �[32m✓�[0m Certificate authority update certificate authority by ID fails if certificate authority is not found (0.00s)
1582:  �[32m✓�[0m Certificate authority update certificate authority by ID successfully updates custom SCEP proxy CA (0.01s)
1583:  �[32m✓�[0m Certificate authority update certificate authority by ID successfully updates custom est proxy CA (0.01s)
1584:  �[32m✓�[0m Certificate authority update certificate authority by ID successfully updates digicert CA (0.01s)
1585:  �[32m✓�[0m Certificate authority update certificate authority by ID successfully updates hydrant CA (0.01s)
1586:  �[32m✓�[0m Certificate authority update certificate authority by ID successfully updates ndes scep proxy CA (0.01s)
1587:  �[32m✓�[0m Certificate authority update certificate authority by ID successfully updates smallstep SCEP proxy CA (0.01s)
1588:  �[32m✓�[0m Certificates (85.98s)
1589:  �[32m✓�[0m Certificates batch delete certificate templates (5.89s)
1590:  �[32m✓�[0m Certificates batch delete certificate templates delete existing certificates (3.35s)
1591:  �[32m✓�[0m Certificates batch delete certificate templates empty slice (2.55s)
1592:  �[32m✓�[0m Certificates batch upsert certificates (8.30s)
1593:  �[32m✓�[0m Certificates batch upsert certificates create certificates (2.82s)
1594:  �[32m✓�[0m Certificates batch upsert certificates empty slice (2.73s)
1595:  �[32m✓�[0m Certificates batch upsert certificates upsert existing certificates (2.75s)
1596:  �[32m✓�[0m Certificates create certificate template (7.43s)
1597:  �[32m✓�[0m Certificates create certificate template certificate template exists, fails to create (3.66s)
1598:  �[32m✓�[0m Certificates create certificate template create certificate template (3.76s)
1599:  �[32m✓�[0m Certificates delete certificate template (6.10s)
1600:  �[32m✓�[0m Certificates delete certificate template delete existing certificate template (3.34s)
1601:  �[32m✓�[0m Certificates delete certificate template delete non-existing certificate template (2.75s)
1602:  �[32m✓�[0m Certificates get certificate template by id (16.28s)
1603:  �[32m✓�[0m Certificates get certificate template by id get existing certificate template (4.18s)
1604:  �[32m✓�[0m Certificates get certificate template by id no existing certificate template (3.83s)
1605:  �[32m✓�[0m Certificates get certificate template by id template with pending host certificate template (3.81s)
1606:  �[32m✓�[0m Certificates get certificate template by id template with verified host certificate template (4.47s)
1607:  �[32m✓�[0m Certificates get certificate template for host (3.07s)
1608:  �[32m✓�[0m Certificates get certificate template for host returns certificate template for host with host certificate template record (0.00s)
1609:  �[32m✓�[0m Certificates get certificate template for host returns certificate template for host without host certificate template record (0.00s)
1610:  �[32m✓�[0m Certificates get certificate template for host returns error for non-existent certificate template (0.00s)
1611:  �[32m✓�[0m Certificates get certificate template for host returns error for non-existent host (0.00s)
1612:  �[32m✓�[0m Certificates get certificate template for host returns error when certificate template doesn't belong to host's team (0.00s)
1613:  �[32m✓�[0m Certificates get certificate templates by ids and team (9.64s)
...

1618:  �[32m✓�[0m Certificates get certificate templates by team ID get existing certificate templates for team (3.58s)
1619:  �[32m✓�[0m Certificates get certificate templates by team ID no existing certificate templates for team (3.85s)
1620:  �[32m✓�[0m Certificates get certificate templates by team ID pagination works (4.77s)
1621:  �[32m✓�[0m Certificates get host certificate template record (3.32s)
1622:  �[32m✓�[0m Certificates get host certificate template record returns not found for non-existent host (0.00s)
1623:  �[32m✓�[0m Certificates get host certificate template record returns not found for non-existent template (0.00s)
1624:  �[32m✓�[0m Certificates get host certificate template record returns record even after parent certificate template is deleted (0.02s)
1625:  �[32m✓�[0m Certificates get host certificate template record returns record when it exists (0.00s)
1626:  �[32m✓�[0m Certificates get host certificate templates (3.20s)
1627:  �[32m✓�[0m Certificates get host certificate templates host UUID is not provided (0.00s)
1628:  �[32m✓�[0m Certificates get host certificate templates no certificate templates found (0.00s)
1629:  �[32m✓�[0m Certificates get host certificate templates returns the certificates available for the host (0.00s)
1630:  �[32m✓�[0m Certificates resend host certificate template (3.12s)
1631:  �[32m✓�[0m Certificates resend host certificate template clears validity fields and deletes challenge (0.04s)
1632:  �[32m✓�[0m Certificates resend host certificate template from delivered (0.01s)
1633:  �[32m✓�[0m Certificates resend host certificate template from failed (0.02s)
1634:  �[32m✓�[0m Certificates resend host certificate template from verified (0.01s)
1635:  �[32m✓�[0m Certificates resend host certificate template returns error for non-existent host (0.00s)
1636:  �[32m✓�[0m Certificates resend host certificate template returns error for non-existent template (0.00s)
1637:  �[32m✓�[0m Cleanup cron stats (4.47s)
...

1649:  �[32m✓�[0m Compare versions equal (0.00s)
1650:  �[32m✓�[0m Compare versions equal-out-of-order (0.00s)
1651:  �[32m✓�[0m Compare versions known-unknown (0.00s)
1652:  �[32m✓�[0m Compare versions missing (0.00s)
1653:  �[32m✓�[0m Compare versions missing-and-unknown (0.00s)
1654:  �[32m✓�[0m Compare versions unknown (0.00s)
1655:  �[32m✓�[0m Compare versions unknowns (0.00s)
1656:  �[32m✓�[0m Conditional access (11.42s)
1657:  �[32m✓�[0m Conditional access SCEP (19.69s)
1658:  �[32m✓�[0m Conditional access SCEP certificate lifecycle (4.02s)
1659:  �[32m✓�[0m Conditional access SCEP expired certs not returned (3.94s)
1660:  �[32m✓�[0m Conditional access SCEP get cert by serial and created at (3.90s)
1661:  �[32m✓�[0m Conditional access SCEP revoked certs not returned (3.80s)
1662:  �[32m✓�[0m Conditional access bypass (39.76s)
1663:  �[32m✓�[0m Conditional access bypass conditional access bypass allowed with CA enabled non critical policy (4.35s)
1664:  �[32m✓�[0m Conditional access bypass conditional access bypass allowed with non CA failing critical policy (4.11s)
1665:  �[32m✓�[0m Conditional access bypass conditional access bypass deleted with host (4.17s)
...

1683:  �[32m✓�[0m Disk encryption (12.88s)
1684:  �[32m✓�[0m Disk encryption test cleanup disk encryption keys on team change (4.65s)
1685:  �[32m✓�[0m Disk encryption test delete LUKS data (4.03s)
1686:  �[32m✓�[0m Email changes (8.38s)
1687:  �[32m✓�[0m Email changes confirm (4.11s)
1688:  �[32m✓�[0m Extract windows build version (0.00s)
1689:  �[32m✓�[0m Extract windows build version empty string (0.00s)
1690:  �[32m✓�[0m Extract windows build version leading whitespace (0.00s)
1691:  �[32m✓�[0m Extract windows build version no spaces (0.00s)
1692:  �[32m✓�[0m Extract windows build version single word (0.00s)
1693:  �[32m✓�[0m Extract windows build version trailing whitespace (0.00s)
1694:  �[32m✓�[0m Extract windows build version windows 10 pro (0.00s)
1695:  �[32m✓�[0m Extract windows build version windows server 2025 datacenter (0.00s)
1696:  �[32m✓�[0m Get MDM apple OS updates settings by host serial (4.31s)
1697:  �[32m✓�[0m Get context try stmt (0.00s)
1698:  �[32m✓�[0m Get context try stmt get with other error (0.00s)
1699:  �[32m✓�[0m Get context try stmt get with unknown statement error (0.00s)
1700:  �[32m✓�[0m Get host operating system (4.07s)
1701:  �[32m✓�[0m Get latest cron stats (4.12s)
1702:  �[32m✓�[0m Health check detects read only (0.00s)
1703:  �[32m✓�[0m Host DEPAs signments (5.12s)
1704:  �[32m✓�[0m Host DEPAs signments DEP enrollment (0.25s)
1705:  �[32m✓�[0m Host DEPAs signments DEP enrollment with migration (0.18s)
1706:  �[32m✓�[0m Host DEPAs signments manual enrollment (0.07s)
1707:  �[32m✓�[0m Host certificate templates (206.42s)
1708:  �[32m✓�[0m Host certificate templates bulk insert and delete host certificate templates (20.82s)
1709:  �[32m✓�[0m Host certificate templates bulk insert and delete host certificate templates bulk inserts and deletes specific records (4.61s)
1710:  �[32m✓�[0m Host certificate templates bulk insert and delete host certificate templates deletes multiple records at once (4.29s)
1711:  �[32m✓�[0m Host certificate templates bulk insert and delete host certificate templates no error when deleting non-existent records (4.42s)
1712:  �[32m✓�[0m Host certificate templates bulk insert and delete host certificate templates no error with empty list (3.88s)
1713:  �[32m✓�[0m Host certificate templates certificate template full state machine (4.07s)
1714:  �[32m✓�[0m Host certificate templates certificate template reinstalled after transfer back to original team (3.27s)
1715:  �[32m✓�[0m Host certificate templates create pending certificate templates for existing hosts (15.71s)
1716:  �[32m✓�[0m Host certificate templates create pending certificate templates for existing hosts creates pending records for all enrolled android hosts in team (4.00s)
1717:  �[32m✓�[0m Host certificate templates create pending certificate templates for existing hosts does not create records for non-android hosts (3.72s)
1718:  �[32m✓�[0m Host certificate templates create pending certificate templates for existing hosts does not create records for unenrolled hosts (4.10s)
1719:  �[32m✓�[0m Host certificate templates create pending certificate templates for new host (16.14s)
1720:  �[32m✓�[0m Host certificate templates create pending certificate templates for new host creates pending records for newly enrolled host (4.03s)
1721:  �[32m✓�[0m Host certificate templates create pending certificate templates for new host no-op when team has no certificate templates (4.02s)
1722:  �[32m✓�[0m Host certificate templates create pending certificate templates for new host resets verified certs to pending on re-enrollment (4.04s)
1723:  �[32m✓�[0m Host certificate templates delete host certificate template (11.95s)
1724:  �[32m✓�[0m Host certificate templates delete host certificate template no error when deleting non-existent record (4.28s)
1725:  �[32m✓�[0m Host certificate templates delete host certificate template only deletes specified record (4.17s)
1726:  �[32m✓�[0m Host certificate templates get and transition certificate templates to delivering includes removal (3.34s)
1727:  �[32m✓�[0m Host certificate templates get android certificate templates for renewal (2.78s)
1728:  �[32m✓�[0m Host certificate templates get certificate template for host no team (17.20s)
1729:  �[32m✓�[0m Host certificate templates get certificate template for host no team returns certificate template for host with team (4.41s)
1730:  �[32m✓�[0m Host certificate templates get certificate template for host no team returns certificate template for no team host (4.40s)
1731:  �[32m✓�[0m Host certificate templates get certificate template for host no team returns not found for non-existent host (4.35s)
1732:  �[32m✓�[0m Host certificate templates get or create fleet challenge for certificate template (2.59s)
1733:  �[32m✓�[0m Host certificate templates get or create fleet challenge for certificate template creates challenge on first call and returns same on subsequent calls (0.03s)
1734:  �[32m✓�[0m Host certificate templates get or create fleet challenge for certificate template returns error for non-delivered status (0.01s)
1735:  �[32m✓�[0m Host certificate templates get or create fleet challenge for certificate template returns error for non-existent template (0.00s)
1736:  �[32m✓�[0m Host certificate templates list android host UUIDs with deliverable certificate templates (26.21s)
...

1744:  �[32m✓�[0m Host certificate templates list android host UUIDs with pending certificate templates includes removal (2.97s)
1745:  �[32m✓�[0m Host certificate templates list android host UUIDs with pending certificate templates respects pagination (4.98s)
1746:  �[32m✓�[0m Host certificate templates list android host UUIDs with pending certificate templates returns hosts with pending install certificates (4.32s)
1747:  �[32m✓�[0m Host certificate templates list certificate templates for hosts (15.67s)
1748:  �[32m✓�[0m Host certificate templates list certificate templates for hosts host with existing host certificate templates (3.60s)
1749:  �[32m✓�[0m Host certificate templates list certificate templates for hosts host with no existing host certificate templates (4.40s)
1750:  �[32m✓�[0m Host certificate templates list certificate templates for hosts includes removal after team transfer (3.12s)
1751:  �[32m✓�[0m Host certificate templates list certificate templates for hosts no team host returns no team certificate templates (3.78s)
1752:  �[32m✓�[0m Host certificate templates retry host certificate template (2.54s)
1753:  �[32m✓�[0m Host certificate templates revert stale certificate templates (14.74s)
1754:  �[32m✓�[0m Host certificate templates revert stale certificate templates does not revert non-delivering statuses (4.62s)
1755:  �[32m✓�[0m Host certificate templates revert stale certificate templates returns zero when no stale templates (3.34s)
1756:  �[32m✓�[0m Host certificate templates revert stale certificate templates reverts stale delivering templates (4.09s)
1757:  �[32m✓�[0m Host certificate templates set android certificate templates for renewal (2.69s)
1758:  �[32m✓�[0m Host certificate templates set host certificate templates to pending remove (11.07s)
1759:  �[32m✓�[0m Host certificate templates set host certificate templates to pending remove deletes pending and failed rows and updates others to pending remove (2.85s)
1760:  �[32m✓�[0m Host certificate templates set host certificate templates to pending remove for host (3.42s)
1761:  �[32m✓�[0m Host certificate templates set host certificate templates to pending remove for host deletes pending and failed installs, updates other installs, leaves removes unchanged (0.07s)
1762:  �[32m✓�[0m Host certificate templates set host certificate templates to pending remove handles no matching rows gracefully (2.90s)
1763:  �[32m✓�[0m Host certificate templates set host certificate templates to pending remove only affects rows for the specified template (2.98s)
1764:  �[32m✓�[0m Host certificate templates upsert host certificate template status (4.02s)
1765:  �[32m✓�[0m Host certificate templates upsert host certificate template status creates new record with install operation type (0.02s)
1766:  �[32m✓�[0m Host certificate templates upsert host certificate template status creates new record with remove operation type (0.01s)
1767:  �[32m✓�[0m Host certificate templates upsert host certificate template status invalid status (0.00s)
1768:  �[32m✓�[0m Host certificate templates upsert host certificate template status update operation type to remove (0.00s)
1769:  �[32m✓�[0m Host certificate templates upsert host certificate template status valid update with details (0.01s)
1770:  �[32m✓�[0m Host certificate templates upsert host certificate template status valid update with install operation type (0.01s)
1771:  �[32m✓�[0m Host certificates (46.34s)
1772:  �[32m✓�[0m Host certificates count matches main query (4.29s)
1773:  �[32m✓�[0m Host certificates create certificates with long country code (4.37s)
1774:  �[32m✓�[0m Host certificates insert host mdm managed certificates from non-proxied ingestion (4.90s)
1775:  �[32m✓�[0m Host certificates matcher recovers stuck hmmc rows (6.08s)
1776:  �[32m✓�[0m Host certificates matcher recovers stuck hmmc rows digi cert not clobbered (0.19s)
1777:  �[32m✓�[0m Host certificates matcher recovers stuck hmmc rows failed profile skipped (0.20s)
1778:  �[32m✓�[0m Host certificates matcher recovers stuck hmmc rows grace boundary (0.27s)
...

1842:  �[32m✓�[0m Hosts enroll orbit scenario c ubuntu (0.02s)
1843:  �[32m✓�[0m Hosts enroll orbit scenario c windows (0.03s)
1844:  �[32m✓�[0m Hosts enroll orbit scenario d darwin (3.25s)
1845:  �[32m✓�[0m Hosts enroll orbit scenario d ubuntu (3.17s)
1846:  �[32m✓�[0m Hosts enroll orbit scenario d windows (3.13s)
1847:  �[32m✓�[0m Hosts enroll orbit scenario e from darwin to darwin (3.24s)
1848:  �[32m✓�[0m Hosts enroll orbit scenario e from darwin to ubuntu (3.22s)
1849:  �[32m✓�[0m Hosts enroll orbit scenario e from darwin to windows (3.26s)
1850:  �[32m✓�[0m Hosts enroll orbit scenario e from ubuntu to darwin (3.20s)
1851:  �[32m✓�[0m Hosts enroll orbit scenario e from ubuntu to ubuntu (3.25s)
1852:  �[32m✓�[0m Hosts enroll orbit scenario e from ubuntu to windows (3.26s)
1853:  �[32m✓�[0m Hosts enroll orbit scenario e from windows to darwin (3.19s)
1854:  �[32m✓�[0m Hosts enroll orbit scenario e from windows to ubuntu (3.12s)
1855:  �[32m✓�[0m Hosts enroll orbit scenario e from windows to windows (3.15s)
1856:  �[32m✓�[0m Hosts enroll updates missing info (0.88s)
1857:  �[32m✓�[0m Hosts failing policies count (1.36s)
1858:  �[32m✓�[0m Hosts failing policies count no policies (0.02s)
1859:  �[32m✓�[0m Hosts failing policies count with policies and memberships (0.26s)
1860:  �[32m✓�[0m Hosts generate status statistics (3.85s)
1861:  �[32m✓�[0m Hosts generate status statistics ABM pending exclusion (4.11s)
1862:  �[32m✓�[0m Hosts generate status statistics DEP errors (3.92s)
1863:  �[32m✓�[0m Hosts get device auth token (1.18s)
...

1879:  �[32m✓�[0m Hosts host IDs by OS version (1.25s)
1880:  �[32m✓�[0m Hosts host IDs by OS version filtering by os version (0.03s)
1881:  �[32m✓�[0m Hosts host IDs by OS version no match (0.00s)
1882:  �[32m✓�[0m Hosts host IDs by OSID (4.69s)
1883:  �[32m✓�[0m Hosts host IDs by OSID no OS (0.00s)
1884:  �[32m✓�[0m Hosts host IDs by OSID returns empty if no more pages (3.37s)
1885:  �[32m✓�[0m Hosts host IDs by OSID returns matching entries (0.03s)
1886:  �[32m✓�[0m Hosts host MDM and munki (1.28s)
1887:  �[32m✓�[0m Hosts host device mapping (1.21s)
1888:  �[32m✓�[0m Hosts host health (1.00s)
1889:  �[32m✓�[0m Hosts host list options android OS settings (4.70s)
1890:  �[32m✓�[0m Hosts host list options android certificate templates OS settings (4.66s)
1891:  �[32m✓�[0m Hosts host list options android certificate templates OS settings combined profile and cert template (0.08s)
1892:  �[32m✓�[0m Hosts host list options android certificate templates OS settings delivered (0.06s)
1893:  �[32m✓�[0m Hosts host list options android certificate templates OS settings delivering (0.11s)
1894:  �[32m✓�[0m Hosts host list options android certificate templates OS settings failed (0.07s)
1895:  �[32m✓�[0m Hosts host list options android certificate templates OS settings pending (0.09s)
...

1908:  �[32m✓�[0m Hosts hostnames by identifiers no match (0.00s)
1909:  �[32m✓�[0m Hosts hostnames by identifiers single match (0.00s)
1910:  �[32m✓�[0m Hosts hostnames by identifiers two matches (0.00s)
1911:  �[32m✓�[0m Hosts hosts add to team cleans up team query results (0.97s)
1912:  �[32m✓�[0m Hosts hosts all pack stats (1.67s)
1913:  �[32m✓�[0m Hosts hosts enroll orbit with platform like (0.85s)
1914:  �[32m✓�[0m Hosts hosts expiration (2.74s)
1915:  �[32m✓�[0m Hosts hosts includes scheduled queries in pack stats (1.59s)
1916:  �[32m✓�[0m Hosts hosts list batch script execution (3.23s)
1917:  �[32m✓�[0m Hosts hosts list by OS name and version (3.03s)
1918:  �[32m✓�[0m Hosts hosts list by disk encryption status (4.14s)
1919:  �[32m✓�[0m Hosts hosts list by operating system ID (3.01s)
1920:  �[32m✓�[0m Hosts hosts list by software (3.03s)
1921:  �[32m✓�[0m Hosts hosts list by software changed at (5.66s)
1922:  �[32m✓�[0m Hosts hosts list by vulnerability (3.05s)
1923:  �[32m✓�[0m Hosts hosts list failing policies (3.07s)
1924:  �[32m✓�[0m Hosts hosts no seen time (2.20s)
1925:  �[32m✓�[0m Hosts hosts pack stats for platform (1.34s)
1926:  �[32m✓�[0m Hosts hosts pack stats multiple hosts (1.25s)
1927:  �[32m✓�[0m Hosts hosts pack stats no duplication (1.21s)
1928:  �[33m∅�[0m Hosts hosts reads less rows (1.16s)
1929:  �[32m✓�[0m Hosts list MDM (4.26s)
1930:  �[32m✓�[0m Hosts list MDM android (4.61s)
1931:  �[32m✓�[0m Hosts list by policy (3.56s)
1932:  �[32m✓�[0m Hosts list filter additional (3.97s)
1933:  �[32m✓�[0m Hosts list hosts DEP filters (0.71s)
1934:  �[32m✓�[0m Hosts list hosts DEP filters deleted DEP assignment is excluded from dep assign profile response filter (0.01s)
1935:  �[32m✓�[0m Hosts list hosts DEP filters deleted DEP assignment is excluded from dep profile error=true (0.03s)
1936:  �[32m✓�[0m Hosts list hosts DEP filters dep assign profile response with no matching hosts returns empty (0.01s)
1937:  �[32m✓�[0m Hosts list hosts DEP filters dep assign profile response= FAILED (0.01s)
1938:  �[32m✓�[0m Hosts list hosts DEP filters dep assign profile response= NOT ACCESSIBLE (0.01s)
1939:  �[32m✓�[0m Hosts list hosts DEP filters dep assign profile response= SUCCESS (0.01s)
1940:  �[32m✓�[0m Hosts list hosts DEP filters dep assign profile response= THROTTLED (0.01s)
1941:  �[32m✓�[0m Hosts list hosts DEP filters dep profile error=false returns all hosts without a DEP error (0.01s)
1942:  �[32m✓�[0m Hosts list hosts DEP filters dep profile error=true combined with team filter (0.02s)
1943:  �[32m✓�[0m Hosts list hosts DEP filters dep profile error=true returns only FAILED and THROTTLED hosts (0.01s)
1944:  �[32m✓�[0m Hosts list hosts DEP filters nil dep profile error returns all hosts (0.01s)
1945:  �[32m✓�[0m Hosts list hosts by profile UUID and status (1.59s)
...

1962:  �[32m✓�[0m Hosts list hosts lite by UUIDs team 1 maintainer team 2 observer sees team 1 (0.00s)
1963:  �[32m✓�[0m Hosts list hosts lite by UUIDs team 1 maintainer team 2 observer sees team 1 and 2 with observer (0.00s)
1964:  �[32m✓�[0m Hosts list hosts lite by UUIDs team 1 observer sees nothing (0.01s)
1965:  �[32m✓�[0m Hosts list hosts lite by UUIDs team 1 observer sees team 1 hosts with observer allowed (0.00s)
1966:  �[32m✓�[0m Hosts list hosts lite by UUIDs team 2 admin sees team 2 hosts (0.00s)
1967:  �[32m✓�[0m Hosts list hosts lite by UUIDs team 2 maintainer sees team 2 hosts (0.00s)
1968:  �[32m✓�[0m Hosts list hosts lite by UUIDs team 2 observer sees nothing (0.00s)
1969:  �[32m✓�[0m Hosts list hosts lite by UUIDs team 2 observer sees team 2 hosts with observer allowed (0.00s)
1970:  �[32m✓�[0m Hosts list hosts lite by UUIDs team 3 admin sees nothing even with observer (0.00s)
1971:  �[32m✓�[0m Hosts list hosts with pagination (1.42s)
1972:  �[32m✓�[0m Hosts list munki issue ID (3.98s)
1973:  �[32m✓�[0m Hosts list query (4.20s)
1974:  �[32m✓�[0m Hosts list status (3.70s)
1975:  �[32m✓�[0m Hosts list upcoming host maintenance windows (0.77s)
1976:  �[32m✓�[0m Hosts load host by device auth token (3.31s)
1977:  �[32m✓�[0m Hosts load host by device auth token fast fail (1.21s)
1978:  �[32m✓�[0m Hosts load host by node key (4.58s)
...

2062:  �[32m✓�[0m Invites create (4.02s)
2063:  �[32m✓�[0m Invites delete (4.04s)
2064:  �[32m✓�[0m Invites invite (4.30s)
2065:  �[32m✓�[0m Invites list (4.85s)
2066:  �[32m✓�[0m Invites update (4.51s)
2067:  �[32m✓�[0m Jobs (22.06s)
2068:  �[32m✓�[0m Jobs cleanup worker jobs (4.33s)
2069:  �[32m✓�[0m Jobs queue and process jobs (4.13s)
2070:  �[32m✓�[0m Jobs queue and process jobs# 01 (5.02s)
2071:  �[32m✓�[0m Labels (163.49s)
2072:  �[32m✓�[0m Labels IDs by name (4.06s)
2073:  �[32m✓�[0m Labels add all hosts deferred (4.33s)
2074:  �[32m✓�[0m Labels add all hosts not deferred (5.12s)
2075:  �[32m✓�[0m Labels add delete labels to from host (5.18s)
2076:  �[32m✓�[0m Labels apply label spec serial UUID (4.25s)
2077:  �[32m✓�[0m Labels apply label specs errors when label exists on another team (3.77s)
2078:  �[32m✓�[0m Labels apply label specs manual nil hosts (3.84s)
...

2146:  �[32m✓�[0m Labels record non existent query label execution (4.12s)
2147:  �[32m✓�[0m Labels save (4.43s)
2148:  �[32m✓�[0m Labels search (3.96s)
2149:  �[32m✓�[0m Labels set aside labels (4.25s)
2150:  �[32m✓�[0m Labels set aside labels cannot set aside global labels when applying to global (0.00s)
2151:  �[32m✓�[0m Labels set aside labels cannot set aside labels from the same team we're applying to (0.01s)
2152:  �[32m✓�[0m Labels set aside labels empty names list is a no-op (0.00s)
2153:  �[32m✓�[0m Labels set aside labels global admin can set aside global labels when applying to a team (0.02s)
2154:  �[32m✓�[0m Labels set aside labels global gitops can set aside global labels (0.02s)
2155:  �[32m✓�[0m Labels set aside labels global maintainer can set aside global labels (0.01s)
2156:  �[32m✓�[0m Labels set aside labels global observer cannot set aside global labels (0.00s)
2157:  �[32m✓�[0m Labels set aside labels global technician can set aside global labels (0.01s)
2158:  �[32m✓�[0m Labels set aside labels multi-team user can set aside labels from teams they have write access to (0.02s)
2159:  �[32m✓�[0m Labels set aside labels multi-team user cannot set aside labels from teams they don't have write access to (0.00s)
2160:  �[32m✓�[0m Labels set aside labels multiple labels can be set aside at once (0.01s)
2161:  �[32m✓�[0m Labels set aside labels non-existent label should fail (0.00s)
2162:  �[32m✓�[0m Labels set aside labels team admin can set aside their own team's labels if they can also edit the not-on team (0.01s)
...

2182:  �[32m✓�[0m List vulns by multiple OS versions empty input (5.11s)
2183:  �[32m✓�[0m List vulns by multiple OS versions mixed platforms (4.60s)
2184:  �[32m✓�[0m List vulns by multiple OS versions multiple linux OS with many kernels (4.94s)
2185:  �[32m✓�[0m List vulns by multiple OS versions non existent OS (4.52s)
2186:  �[32m✓�[0m List vulns by multiple OS versions with CVSS (4.08s)
2187:  �[32m✓�[0m List vulns by multiple OS versions with max vulnerabilities (4.95s)
2188:  �[32m✓�[0m List vulns by multiple OS versions with max vulnerabilities combined all OS types with max=2 (0.01s)
2189:  �[32m✓�[0m List vulns by multiple OS versions with max vulnerabilities combined os versions + linux OS with max=0 (0.01s)
2190:  �[32m✓�[0m List vulns by multiple OS versions with max vulnerabilities combined os versions + linux OS with max=1 (0.01s)
2191:  �[32m✓�[0m List vulns by multiple OS versions with max vulnerabilities max = 0 returns empty array with count (0.01s)
2192:  �[32m✓�[0m List vulns by multiple OS versions with max vulnerabilities max = 1 for linux kernel vulnerabilities (0.00s)
2193:  �[32m✓�[0m List vulns by multiple OS versions with max vulnerabilities max = 1 limits to 1 vulnerability (0.01s)
2194:  �[32m✓�[0m List vulns by multiple OS versions with max vulnerabilities max exceeds total returns all vulnerabilities (0.01s)
2195:  �[32m✓�[0m List vulns by multiple OS versions with max vulnerabilities multi-arch with max returns correct count (0.00s)
2196:  �[32m✓�[0m List vulns by multiple OS versions with max vulnerabilities multi-arch with max=0 returns correct count (0.00s)
2197:  �[32m✓�[0m List vulns by multiple OS versions with max vulnerabilities negative value returns error (0.00s)
2198:  �[32m✓�[0m List vulns by multiple OS versions with team filter (3.91s)
...

2229:  �[32m✓�[0m MDM apple apple MDM set batch async last seen at (2.28s)
2230:  �[32m✓�[0m MDM apple batch set MDM apple declarations case change (1.71s)
2231:  �[32m✓�[0m MDM apple batch set MDM apple declarations case change no team (0.16s)
2232:  �[32m✓�[0m MDM apple batch set MDM apple declarations case change no team host profile reflects name change (0.00s)
2233:  �[32m✓�[0m MDM apple batch set MDM apple declarations case change no team pending install not disrupted (0.01s)
2234:  �[32m✓�[0m MDM apple batch set MDM apple declarations case change no team preserves declaration UUID (0.01s)
2235:  �[32m✓�[0m MDM apple batch set MDM apple declarations case change no team verified install not disrupted (0.01s)
2236:  �[32m✓�[0m MDM apple batch set MDM apple declarations case change team (0.11s)
2237:  �[32m✓�[0m MDM apple batch set MDM apple declarations case change team host profile reflects name change (0.01s)
2238:  �[32m✓�[0m MDM apple batch set MDM apple declarations case change team pending install not disrupted (0.01s)
2239:  �[32m✓�[0m MDM apple batch set MDM apple declarations case change team preserves declaration UUID (0.01s)
2240:  �[32m✓�[0m MDM apple batch set MDM apple declarations case change team verified install not disrupted (0.01s)
2241:  �[32m✓�[0m MDM apple claim hosts for recovery lock clear (1.66s)
2242:  �[32m✓�[0m MDM apple claim hosts for recovery lock clear claims no-team host when appconfig disabled (0.08s)
2243:  �[32m✓�[0m MDM apple claim hosts for recovery lock clear claims verified host when config disabled (0.08s)
2244:  �[32m✓�[0m MDM apple claim hosts for recovery lock clear does not claim pending or failed hosts (0.07s)
2245:  �[32m✓�[0m MDM apple claim hosts for recovery lock clear get operation type (0.05s)
2246:  �[32m✓�[0m MDM apple claim hosts for recovery lock clear no hosts to clear returns empty (0.01s)
2247:  �[32m✓�[0m MDM apple claim hosts for recovery lock clear retries failed clear attempts (0.07s)
2248:  �[32m✓�[0m MDM apple claim hosts for recovery lock clear returns pending when operation type is install and status is NULL (0.01s)
...

2250:  �[32m✓�[0m MDM apple claim hosts for recovery lock clear soft delete marks password record as deleted (0.05s)
2251:  �[32m✓�[0m MDM apple cleanup orphaned nano refetch commands (1.62s)
2252:  �[32m✓�[0m MDM apple cleanup stale nano refetch commands (1.45s)
2253:  �[32m✓�[0m MDM apple delete MDM apple declaration by name cancels installs (1.88s)
2254:  �[32m✓�[0m MDM apple delete MDM apple declaration by name cancels installs no team (0.17s)
2255:  �[32m✓�[0m MDM apple delete MDM apple declaration by name cancels installs team (0.13s)
2256:  �[32m✓�[0m MDM apple delete host preserves recovery lock password (1.28s)
2257:  �[32m✓�[0m MDM apple device location (0.87s)
2258:  �[32m✓�[0m MDM apple file vault summary (5.21s)
2259:  �[32m✓�[0m MDM apple get MDM apple enrolled device deleted from fleet (1.59s)
2260:  �[32m✓�[0m MDM apple get and update ABM token (0.77s)
2261:  �[32m✓�[0m MDM apple get enrollment IDs with pending MDM apple commands (1.50s)
2262:  �[32m✓�[0m MDM apple get host recovery lock password status (1.89s)
2263:  �[32m✓�[0m MDM apple get host recovery lock password status returns enforcing status for pending install (0.02s)
2264:  �[32m✓�[0m MDM apple get host recovery lock password status returns enforcing status when status column is NULL (retry state) (0.02s)
2265:  �[32m✓�[0m MDM apple get host recovery lock password status returns failed status when operation type is remove and status is failed (0.03s)
2266:  �[32m✓�[0m MDM apple get host recovery lock password status returns failed status with error message (0.02s)
2267:  �[32m✓�[0m MDM apple get host recovery lock password status returns nil for host without recovery lock password (0.02s)
2268:  �[32m✓�[0m MDM apple get host recovery lock password status returns removing enforcement status for pending removal after populate status (0.02s)
2269:  �[32m✓�[0m MDM apple get host recovery lock password status returns verified status (0.02s)
2270:  �[32m✓�[0m MDM apple get host recovery lock password status returns verifying status (0.02s)
2271:  �[32m✓�[0m MDM apple get hosts for recovery lock action (2.18s)
2272:  �[32m✓�[0m MDM apple get nano MDM enrollment details (0.84s)
2273:  �[32m✓�[0m MDM apple get nano MDMUs er enrollment (0.75s)
2274:  �[32m✓�[0m MDM apple host MDM commands (1.15s)
2275:  �[32m✓�[0m MDM apple host recovery lock status matrix (1.20s)
2276:  �[32m✓�[0m MDM apple host recovery lock status matrix NULL status remove -> removing enforcement (clear retry) (0.01s)
2277:  �[32m✓�[0m MDM apple host recovery lock status matrix NULL status, install, password stored -> pending (0.02s)
2278:  �[32m✓�[0m MDM apple host recovery lock status matrix failed install + rotation in flight -> failed (0.01s)
2279:  �[32m✓�[0m MDM apple host recovery lock status matrix failed install -> failed with detail (0.01s)
2280:  �[32m✓�[0m MDM apple host recovery lock status matrix failed remove -> failed (0.01s)
2281:  �[32m✓�[0m MDM apple host recovery lock status matrix pending install + rotation in flight -> pending (0.02s)
2282:  �[32m✓�[0m MDM apple host recovery lock status matrix pending install, no rotation -> pending (0.01s)
2283:  �[32m✓�[0m MDM apple host recovery lock status matrix pending remove -> removing enforcement (0.02s)
2284:  �[32m✓�[0m MDM apple host recovery lock status matrix soft-deleted row is invisible to readers (0.02s)
2285:  �[32m✓�[0m MDM apple host recovery lock status matrix verified install -> verified (0.01s)
2286:  �[32m✓�[0m MDM apple ingest MDM apple device from OTA enrollment (1.30s)
2287:  �[32m✓�[0m MDM apple ingest MDM apple devices from DEP sync IOSI pad OS (1.24s)
2288:  �[32m✓�[0m MDM apple list IOS and i pad OS to refetch (1.84s)
2289:  �[32m✓�[0m MDM apple lock unlock wipe mac OS (2.71s)
2290:  �[32m✓�[0m MDM apple profile verification (5.62s)
2291:  �[32m✓�[0m MDM apple profile verification earliest install date (0.12s)
2292:  �[32m✓�[0m MDM apple profile verification expected profile (0.32s)
2293:  �[32m✓�[0m MDM apple profile verification expected profile failed then found expected (0.09s)
2294:  �[32m✓�[0m MDM apple profile verification expected profile pending then found expected (0.08s)
2295:  �[32m✓�[0m MDM apple profile verification expected profile verified then found expected (0.08s)
2296:  �[32m✓�[0m MDM apple profile verification expected profile verifying then found expected (0.07s)
2297:  �[32m✓�[0m MDM apple profile verification missing profile with retry (0.56s)
2298:  �[32m✓�[0m MDM apple profile verification outdated profile (0.28s)
2299:  �[32m✓�[0m MDM apple profile verification outdated profile failed then found outdated (0.07s)
2300:  �[32m✓�[0m MDM apple profile verification outdated profile pending then found outdated (0.08s)
2301:  �[32m✓�[0m MDM apple profile verification outdated profile verified then found outdated (0.07s)
2302:  �[32m✓�[0m MDM apple profile verification outdated profile verifying then found outdated (0.07s)
2303:  �[32m✓�[0m MDM apple profile verification unexpected profile (0.31s)
2304:  �[32m✓�[0m MDM apple profile verification unexpected profile failed then found expected and unexpected (0.08s)
2305:  �[32m✓�[0m MDM apple profile verification unexpected profile pending then found expected and unexpected (0.08s)
2306:  �[32m✓�[0m MDM apple profile verification unexpected profile verified then foun expected andd unexpected (0.07s)
2307:  �[32m✓�[0m MDM apple profile verification unexpected profile verifying then found expected and unexpected (0.08s)
2308:  �[32m✓�[0m MDM apple recovery lock auto rotation (1.53s)
2309:  �[32m✓�[0m MDM apple recovery lock auto rotation complete recovery lock rotation clears auto rotate at (0.03s)
2310:  �[32m✓�[0m MDM apple recovery lock auto rotation get host recovery lock password includes auto rotate at (0.02s)
2311:  �[32m✓�[0m MDM apple recovery lock auto rotation get hosts for auto rotation excludes future auto rotate at (0.02s)
2312:  �[32m✓�[0m MDM apple recovery lock auto rotation get hosts for auto rotation excludes hosts with pending rotation (0.02s)
2313:  �[32m✓�[0m MDM apple recovery lock auto rotation get hosts for auto rotation excludes non-verified hosts (0.01s)
2314:  �[32m✓�[0m MDM apple recovery lock auto rotation get hosts for auto rotation returns due hosts (0.02s)
2315:  �[32m✓�[0m MDM apple recovery lock auto rotation mark recovery lock password viewed fails for non-existent host (0.01s)
2316:  �[32m✓�[0m MDM apple recovery lock auto rotation mark recovery lock password viewed fails for remove operation (0.02s)
2317:  �[32m✓�[0m MDM apple recovery lock auto rotation mark recovery lock password viewed sets auto rotate at (0.03s)
...

2319:  �[32m✓�[0m MDM apple recovery lock password bulk set (1.28s)
2320:  �[32m✓�[0m MDM apple recovery lock password get not found (1.37s)
2321:  �[32m✓�[0m MDM apple recovery lock password set and get (1.17s)
2322:  �[32m✓�[0m MDM apple recovery lock password set overwrite (1.25s)
2323:  �[32m✓�[0m MDM apple recovery lock password updated at changes (2.27s)
2324:  �[32m✓�[0m MDM apple recovery lock readers return not found for soft deleted (1.12s)
2325:  �[32m✓�[0m MDM apple recovery lock reset on MDM re enrollment (1.54s)
2326:  �[32m✓�[0m MDM apple recovery lock reset on MDM re enrollment nulls auto rotate at on soft-delete (0.05s)
2327:  �[32m✓�[0m MDM apple recovery lock reset on MDM re enrollment nulls pending rotation fields on soft-delete (0.05s)
2328:  �[32m✓�[0m MDM apple recovery lock reset on MDM re enrollment preserves row during SCEP renewal (0.03s)
2329:  �[32m✓�[0m MDM apple recovery lock reset on MDM re enrollment re-animation after soft-delete yields clean state (0.04s)
2330:  �[32m✓�[0m MDM apple recovery lock reset on MDM re enrollment soft-deletes stuck-pending install row ( clear queue scenario) (0.04s)
2331:  �[32m✓�[0m MDM apple recovery lock reset on MDM re enrollment soft-deletes verified install row (0.04s)
2332:  �[32m✓�[0m MDM apple recovery lock rotation (1.38s)
2333:  �[32m✓�[0m MDM apple recovery lock rotation clear recovery lock rotation removes pending (0.02s)
2334:  �[32m✓�[0m MDM apple recovery lock rotation clear recovery lock rotation restores failed status (0.02s)
2335:  �[32m✓�[0m MDM apple recovery lock rotation complete recovery lock rotation success (0.03s)
2336:  �[32m✓�[0m MDM apple recovery lock rotation fail recovery lock rotation preserves pending password (0.03s)
2337:  �[32m✓�[0m MDM apple recovery lock rotation get recovery lock rotation status not found (0.00s)
2338:  �[32m✓�[0m MDM apple recovery lock rotation get recovery lock rotation status returns all fields (0.02s)
2339:  �[32m✓�[0m MDM apple recovery lock rotation has pending recovery lock rotation returns false for no record (0.00s)
2340:  �[32m✓�[0m MDM apple recovery lock rotation initiate recovery lock rotation allows failed status (0.03s)
2341:  �[32m✓�[0m MDM apple recovery lock rotation initiate recovery lock rotation rejects if already pending (0.02s)
2342:  �[32m✓�[0m MDM apple recovery lock rotation initiate recovery lock rotation rejects pending status (0.01s)
2343:  �[32m✓�[0m MDM apple recovery lock rotation initiate recovery lock rotation success (0.02s)
2344:  �[32m✓�[0m MDM apple recovery lock status methods (1.29s)
2345:  �[32m✓�[0m MDM apple recovery lock status methods clear recovery lock pending status (0.02s)
2346:  �[32m✓�[0m MDM apple recovery lock status methods clear recovery lock pending status only clears pending (0.02s)
2347:  �[32m✓�[0m MDM apple recovery lock status methods reset recovery lock for retry (0.02s)
2348:  �[32m✓�[0m MDM apple recovery lock status methods reset recovery lock for retry from failed state (0.02s)
2349:  �[32m✓�[0m MDM apple recovery lock status methods set hosts recovery lock passwords sets pending status atomically (0.02s)
2350:  �[32m✓�[0m MDM apple recovery lock status methods set recovery lock failed (0.02s)
2351:  �[32m✓�[0m MDM apple recovery lock status methods set recovery lock verified (0.01s)
...

2366:  �[32m✓�[0m MDM apple test MDM apple delete host DEPAs signments no matching serials (0.02s)
2367:  �[32m✓�[0m MDM apple test MDM apple delete host DEPAs signments no serials provided (0.02s)
2368:  �[32m✓�[0m MDM apple test MDM apple delete host DEPAs signments partial matches (0.15s)
2369:  �[32m✓�[0m MDM apple test MDM apple enrollment profile (2.93s)
2370:  �[32m✓�[0m MDM apple test MDM apple hosts disk encryption (4.61s)
2371:  �[32m✓�[0m MDM apple test MDM apple hosts profiles status (6.42s)
2372:  �[32m✓�[0m MDM apple test MDM apple id p account (4.70s)
2373:  �[32m✓�[0m MDM apple test MDM apple profile labels (1.35s)
2374:  �[32m✓�[0m MDM apple test MDM apple profile management (3.71s)
2375:  �[32m✓�[0m MDM apple test MDM apple profile management batch 2 (4.49s)
2376:  �[32m✓�[0m MDM apple test MDM apple profile management batch 3 (4.14s)
2377:  �[32m✓�[0m MDM apple test MDM apple reset enrollment (3.07s)
2378:  �[32m✓�[0m MDM apple test MDM apple reset on reenrollment (3.67s)
2379:  �[32m✓�[0m MDM apple test MDM apple reset on reenrollment clears expected tables and leaves other hosts untouched (0.10s)
2380:  �[32m✓�[0m MDM apple test MDM apple reset on reenrollment preserve host activities flag controls past activity history (0.21s)
2381:  �[32m✓�[0m MDM apple test MDM apple reset on reenrollment returns error and changes nothing when host UUID does not exist (0.04s)
2382:  �[32m✓�[0m MDM apple test MDM apple setup assistant (4.85s)
...

2385:  �[32m✓�[0m MDM apple test aggregate mac OS settings status with file vault (4.12s)
2386:  �[32m✓�[0m MDM apple test batch set MDM apple profiles (6.09s)
2387:  �[32m✓�[0m MDM apple test batch set MDM apple profiles clears stale broken labels (4.07s)
2388:  �[32m✓�[0m MDM apple test bulk upsert MDM apple config profiles (7.43s)
2389:  �[32m✓�[0m MDM apple test delete MDM apple config profile (4.52s)
2390:  �[32m✓�[0m MDM apple test delete MDM apple config profile by team and identifier (3.71s)
2391:  �[32m✓�[0m MDM apple test delete MDM apple config profile with pending installs (4.31s)
2392:  �[32m✓�[0m MDM apple test delete MDM apple declaration with pending installs (1.04s)
2393:  �[32m✓�[0m MDM apple test delete MDM apple profiles for host (3.31s)
2394:  �[32m✓�[0m MDM apple test get DEPAs sign profile expired cooldowns (2.86s)
2395:  �[32m✓�[0m MDM apple test get MDM apple command results (4.18s)
2396:  �[32m✓�[0m MDM apple test get MDM apple profiles contents (4.10s)
2397:  �[32m✓�[0m MDM apple test get latest apple MDM command of type (0.85s)
2398:  �[32m✓�[0m MDM apple test host details MDM profiles (4.48s)
2399:  �[32m✓�[0m MDM apple test host details MDM profiles IOSI pad OS (3.93s)
2400:  �[32m✓�[0m MDM apple test ignore MDM client error (4.78s)
2401:  �[32m✓�[0m MDM apple test list MDM apple commands (3.22s)
...

2471:  �[32m✓�[0m MDM shared test batch set profile label associations valid input windows (0.03s)
2472:  �[32m✓�[0m MDM shared test bulk set pending MDM host profiles (7.29s)
2473:  �[32m✓�[0m MDM shared test bulk set pending MDM host profiles batch 2 (7.14s)
2474:  �[32m✓�[0m MDM shared test bulk set pending MDM host profiles batch 3 (7.42s)
2475:  �[32m✓�[0m MDM shared test bulk set pending MDM host profiles exclude any (6.61s)
2476:  �[32m✓�[0m MDM shared test bulk set pending MDM host profiles lots of hosts (9.19s)
2477:  �[32m✓�[0m MDM shared test bulk set pending defers windows reconciliation (4.41s)
2478:  �[32m✓�[0m MDM shared test clean up MDM managed certificates (4.08s)
2479:  �[32m✓�[0m MDM shared test clean up MDM managed certificates non matching host profile record (0.02s)
2480:  �[32m✓�[0m MDM shared test clean up MDM managed certificates valid apple record stays (0.02s)
2481:  �[32m✓�[0m MDM shared test clean up MDM managed certificates valid windows record stays (0.03s)
2482:  �[32m✓�[0m MDM shared test delete MDM profiles cancels installs (5.26s)
2483:  �[32m✓�[0m MDM shared test delete team cancels windows profile installs (4.77s)
2484:  �[32m✓�[0m MDM shared test enqueue command with name (4.17s)
2485:  �[32m✓�[0m MDM shared test get MDM config profile status (5.18s)
2486:  �[32m✓�[0m MDM shared test get MDM config profile status android no team G1 profile failed (0.00s)
2487:  �[32m✓�[0m MDM shared test get MDM config profile status android team G2 profile failed + verified (0.01s)
2488:  �[32m✓�[0m MDM shared test get MDM config profile status android team G2 profile nil + pending = 2 pending (0.01s)
2489:  �[32m✓�[0m MDM shared test get MDM config profile status android team G2 profile nil=pending (0.00s)
2490:  �[32m✓�[0m MDM shared test get MDM config profile status android team G2 profile pending + verifying (0.01s)
2491:  �[32m✓�[0m MDM shared test get MDM config profile status android team G2 profile verified + verifying (0.01s)
2492:  �[32m✓�[0m MDM shared test get MDM config profile status mac OS no team decl D1 pending failed verified (0.01s)
2493:  �[32m✓�[0m MDM shared test get MDM config profile status mac OS no team profile A1 all pending NULL (0.01s)
2494:  �[32m✓�[0m MDM shared test get MDM config profile status mac OS team decl D2 all failed (0.01s)
2495:  �[32m✓�[0m MDM shared test get MDM config profile status mac OS team profile A2 verifying verified (0.01s)
2496:  �[32m✓�[0m MDM shared test get MDM config profile status windows no team profile W1 all pending NULL (0.01s)
2497:  �[32m✓�[0m MDM shared test get MDM config profile status windows no team profile W1 pending failed (0.01s)
2498:  �[32m✓�[0m MDM shared test get MDM config profile status windows team profile W2 pending (0.01s)
...

2569:  �[32m✓�[0m MDM shared test profile has ACME payload for command darwin host with ACME profile, no pending refetch (0.02s)
2570:  �[32m✓�[0m MDM shared test profile has ACME payload for command darwin host with non-a CME profile reports has acme payload=false (0.01s)
2571:  �[32m✓�[0m MDM shared test profile has ACME payload for command unknown command returns not found (0.00s)
2572:  �[32m✓�[0m MDM shared test profile has ACME payload for command unknown host returns not found (0.01s)
2573:  �[32m✓�[0m MDM shared test renew MDM managed certificates null type (4.16s)
2574:  �[32m✓�[0m MDM windows (215.67s)
2575:  �[32m✓�[0m MDM windows test MDM windows awaiting configuration CAS (4.67s)
2576:  �[32m✓�[0m MDM windows test MDM windows awaiting configuration CAS active row with expect from= pending is no-op (mismatched) (0.01s)
2577:  �[32m✓�[0m MDM windows test MDM windows awaiting configuration CAS active-> none matched (0.01s)
2578:  �[32m✓�[0m MDM windows test MDM windows awaiting configuration CAS awaiting configuration at preserved across transitions (0.02s)
2579:  �[32m✓�[0m MDM windows test MDM windows awaiting configuration CAS concurrent CAS yields exactly one winner (0.02s)
2580:  �[32m✓�[0m MDM windows test MDM windows awaiting configuration CAS none row with expect from= pending is no-op (idempotent retry) (0.01s)
2581:  �[32m✓�[0m MDM windows test MDM windows awaiting configuration CAS pending row with expect from= active is no-op (mismatched) (0.01s)
2582:  �[32m✓�[0m MDM windows test MDM windows awaiting configuration CAS pending-> active matched (0.02s)
2583:  �[32m✓�[0m MDM windows test MDM windows awaiting configuration CAS pending-> none matched (timeout-during-pending shortcut) (0.01s)
2584:  �[32m✓�[0m MDM windows test MDM windows awaiting configuration CAS unknown device id returns false without error (0.00s)
2585:  �[32m✓�[0m MDM windows test MDM windows awaiting configuration by host UUID (4.67s)
2586:  �[32m✓�[0m MDM windows test MDM windows awaiting configuration by host UUID does not cross-leak between hosts (0.04s)
2587:  �[32m✓�[0m MDM windows test MDM windows awaiting configuration by host UUID returns current state and reflects transitions (0.03s)
2588:  �[32m✓�[0m MDM windows test MDM windows awaiting configuration by host UUID returns most recent enrollment when host has multiple (0.04s)
2589:  �[32m✓�[0m MDM windows test MDM windows awaiting configuration by host UUID unknown host UUID returns not found (0.00s)
2590:  �[32m✓�[0m MDM windows test MDM windows command results (4.57s)
2591:  �[32m✓�[0m MDM windows test MDM windows command results with pending result (4.62s)
2592:  �[32m✓�[0m MDM windows test MDM windows config profiles (4.87s)
2593:  �[32m✓�[0m MDM windows test MDM windows config profiles with fleet vars (4.65s)
2594:  �[32m✓�[0m MDM windows test MDM windows disk encryption (13.86s)
2595:  �[32m✓�[0m MDM windows test MDM windows disk encryption disk encryption disabled (0.16s)
2596:  �[32m✓�[0m MDM windows test MDM windows disk encryption disk encryption enabled (8.78s)
2597:  �[32m✓�[0m MDM windows test MDM windows disk encryption disk encryption enabled OS settings filters include windows and mac OS hosts (0.04s)
2598:  �[32m✓�[0m MDM windows test MDM windows disk encryption disk encryption enabled bit locker failed status (1.79s)
2599:  �[32m✓�[0m MDM windows test MDM windows disk encryption disk encryption enabled bit locker host disks must update to transition from verifying to verified (0.67s)
...

2634:  �[32m✓�[0m MDM windows test MDM windows insert command and upsert host profiles for hosts batching works correctly (0.04s)
2635:  �[32m✓�[0m MDM windows test MDM windows insert command and upsert host profiles for hosts duplicate command uuid returns already exists (0.01s)
2636:  �[32m✓�[0m MDM windows test MDM windows insert command and upsert host profiles for hosts empty host list is a noop (0.00s)
2637:  �[32m✓�[0m MDM windows test MDM windows insert command and upsert host profiles for hosts inserts command and profiles for multiple hosts (0.03s)
2638:  �[32m✓�[0m MDM windows test MDM windows insert command and upsert host profiles for hosts upserts update existing host profiles (0.03s)
2639:  �[32m✓�[0m MDM windows test MDM windows insert command for hosts (5.34s)
2640:  �[32m✓�[0m MDM windows test MDM windows insert command skips unenrolled hosts (4.55s)
2641:  �[32m✓�[0m MDM windows test MDM windows profile labels (4.82s)
2642:  �[32m✓�[0m MDM windows test MDM windows profile management (4.73s)
2643:  �[32m✓�[0m MDM windows test MDM windows profiles summary (12.26s)
2644:  �[32m✓�[0m MDM windows test MDM windows profiles summary enumeration (35.52s)
2645:  �[32m✓�[0m MDM windows test MDM windows profiles summary profiles summary accounts for bitlocker status (5.06s)
2646:  �[32m✓�[0m MDM windows test MDM windows profiles summary profiles summary accounts for bitlocker status bitlocker disabled (0.58s)
2647:  �[32m✓�[0m MDM windows test MDM windows profiles summary profiles summary accounts for bitlocker status bitlocker enabled (4.48s)
2648:  �[32m✓�[0m MDM windows test MDM windows profiles summary profiles summary accounts for bitlocker status bitlocker enabled bit locker host disks must update to transition from verifying to verified (0.58s)
2649:  �[32m✓�[0m MDM windows test MDM windows profiles summary profiles summary accounts for bitlocker status bitlocker enabled bitlocker failed (1.28s)
2650:  �[32m✓�[0m MDM windows test MDM windows profiles summary profiles summary accounts for bitlocker status bitlocker enabled bitlocker pending (0.70s)
2651:  �[32m✓�[0m MDM windows test MDM windows profiles summary profiles summary accounts for bitlocker status bitlocker enabled bitlocker verified (0.91s)
2652:  �[32m✓�[0m MDM windows test MDM windows profiles summary profiles summary accounts for bitlocker status bitlocker enabled bitlocker verifying (0.98s)
2653:  �[32m✓�[0m MDM windows test MDM windows profiles summary profiles summary accounts for host profiles with mixed statuses (1.99s)
2654:  �[32m✓�[0m MDM windows test MDM windows profiles to remove skips orphaned hosts (4.63s)
2655:  �[32m✓�[0m MDM windows test MDM windows save response (4.86s)
2656:  �[32m✓�[0m MDM windows test MDM windows save response combined atomic and non-atomic profiles (0.13s)
2657:  �[32m✓�[0m MDM windows test MDM windows save response combined atomic and non-atomic profiles fails only for the failed profile (0.07s)
2658:  �[32m✓�[0m MDM windows test MDM windows save response combined atomic and non-atomic profiles saves correctly (0.06s)
2659:  �[32m✓�[0m MDM windows test MDM windows save response mixed install and remove in same batch (0.06s)
2660:  �[32m✓�[0m MDM windows test MDM windows save response non-atomic command saves and verifies correctly (0.04s)
2661:  �[32m✓�[0m MDM windows test MDM windows save response remove status outcomes (0.07s)
2662:  �[32m✓�[0m MDM windows test MDM windows save response remove status outcomes failed remove deletes row (0.03s)
2663:  �[32m✓�[0m MDM windows test MDM windows save response remove status outcomes verified remove deletes row (0.04s)
2664:  �[32m✓�[0m MDM windows test MDM windows save response remove then reinstall same profile (0.10s)
2665:  �[32m✓�[0m MDM windows test MDM windows save response wipe failure returns wipe failed result (0.12s)
2666:  �[32m✓�[0m MDM windows test MDM windows save response wipe failure returns wipe failed result...

@mostlikelee mostlikelee merged commit 430df5d into 40639-cert-renew May 12, 2026
46 of 48 checks passed
@mostlikelee mostlikelee deleted the pr-2.3-apple-validation-rename branch May 12, 2026 22:10
mostlikelee added a commit that referenced this pull request May 14, 2026
… 1.4)

Status sweep:
- PR 2.1 (#44344), PR 2.2 (#44345), PR 2.3 (#45043), PR 2.3b
  (#45364), PR 2.4 (#45237) all merged — task lists in §5/§6/§7/§7b/§8
  now reflect completion. Items deferred to PR 2.5 (release notes) stay
  open.
- §10.1 annotated with the 2026-05-14 local end-to-end validation
  result. Hydrant verification still required for customer readiness.

New refinement — Decision 1.4 and §7c (PR 2.3c):
- Surfaced during local validation: a user-installed Root CA ended up
  with origin=mdm because MDM CertificateList returned the entire
  keychain on InstallProfile ack, beating the next osquery sync.
- Decision: osquery sync downgrades existing origin=mdm rows to
  origin=osquery when it also observes them. One-way (mdm → osquery,
  never reverse).
- Rationale: osquery sees a strict superset of MDM CertificateList,
  so observation by both implies the cert is in the device keychain
  by some path — not evidence of MDM delivery. Origin should reflect
  "did Fleet deliver this?", not "who saw it first?"
- Renewal correctness is unaffected; matcher keys on the
  fleet-<profile_uuid> marker, not on origin.
mostlikelee added a commit that referenced this pull request May 15, 2026
…ision 2.6)

Removes the upload-time renewal-ID enforcement that PR 2.3 (#45043) and
PR 2.3b (#45364) initially shipped. Marker becomes purely advisory:
profiles that include it activate auto-renewal; profiles that don't
continue to work as in 4.85, unchanged.

Rationale (Decision 2.6 reversal):

- GitOps trap: `fleetctl gitops apply` on existing customer profiles
  (Conditional Access, Okta Verify, ACME) would fail after upgrade
  because pre-4.86 profiles lack the marker.
- UI-edit trap: opening an existing profile and saving an unrelated
  change would be rejected.
- Conditional Access: Fleet's own generated profile lacks the marker
  (#45580), so customers following the published guide can't deploy.

The cost of hard rejection — broken upgrades for customers who don't
even use auto-renewal — exceeded the benefit of catching the
misconfiguration that's only relevant to opt-in customers.

Spec updates:

- Decision 2.6: reversed entirely. Documents validator removal,
  GitOps-continuity rationale, and alternatives considered.
- Decision 2.7: collapsed the "net-new surfaces accept preferred-only,
  OU-only" enforcement rules. Variable rename + substitution back-compat
  still documented.
- Decision 1.2, 2.5: softened "customers must redeploy" to "customers
  may opt in by redeploying."
- §7 / §7b task lists: struck through reverted tasks, kept history.
- §7d added: revert tasks for removing the two Apple validators.
- §9 (docs): framing shifted from "required migration" to "opt-in
  capability." Existing customers continue unchanged unless they want
  auto-renewal.
- §10.6 / §10.7: validation-rejection QA replaced with opt-out path
  verification.

Code revert work (§7d) will follow on a separate PR branch.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Apple profile-upload validation for renewal-ID marker (sub-task of #40639)

3 participants