Skip to content

Add telemetry for databricks aitools install#5862

Open
parthban-db wants to merge 1 commit into
mainfrom
parthban-db/stack/telemetry
Open

Add telemetry for databricks aitools install#5862
parthban-db wants to merge 1 commit into
mainfrom
parthban-db/stack/telemetry

Conversation

@parthban-db

@parthban-db parthban-db commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

🥞 Stacked PR

Use this link to review incremental changes.


Changes

Emit a structured AitoolsInstallEvent telemetry event on every run of databricks aitools install (and its deprecated experimental aitools skills install alias, which delegates to the same command). The event records three closed-set, non-PII dimensions: the agents targeted (agents, from the fixed agent registry), the install scope (scope, project or global), and whether experimental skills were requested (experimental).

aitools install is otherwise a fully local command (skills are fetched from GitHub), so it never resolves auth on its own. tryConfigureAuth resolves auth config best-effort into the context so telemetry.Upload at command exit has somewhere to send. A user with no configured profile still installs offline and simply emits nothing.

Why

aitools install had no usage signal. This lets us measure adoption of Databricks skills across coding agents — how many installs, for which agents, at which scope, and how often the experimental skills are pulled in.

Tests

Unit tests.

@github-actions

github-actions Bot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Approval status: pending

/cmd/aitools/ - needs approval

5 files changed
Suggested: @simonfaltum
Also eligible: @fjakobs, @Shridhad, @atilafassina, @keugenek, @igrekun, @pkosiec, @MarioCadenas, @pffigueiredo, @ditadi, @calvarjorge, @renaudhartert-db, @hectorcast-db, @tanmay-db, @Divyansh-db, @tejaskochar-db, @mihaimitrea-db, @chrisst, @rauchy, @lennartkats-db

/libs/telemetry/ - needs approval

Files: libs/telemetry/protos/aitools_install.go, libs/telemetry/protos/frontend_log.go
Suggested: @simonfaltum
Also eligible: @renaudhartert-db, @hectorcast-db, @tanmay-db, @Divyansh-db, @tejaskochar-db, @mihaimitrea-db, @chrisst, @rauchy

Any maintainer (@andrewnester, @anton-107, @denik, @pietern, @shreyas-goenka, @simonfaltum, @renaudhartert-db, @janniklasrose) can approve all areas.
See OWNERS for ownership rules.

@eng-dev-ecosystem-bot

eng-dev-ecosystem-bot commented Jul 8, 2026

Copy link
Copy Markdown
Collaborator

Integration test report

Commit: b477ac6

Run: 28949299355

Env 🟨​KNOWN 💚​RECOVERED 🙈​SKIP ✅​pass 🙈​skip Time
💚​ aws linux 4 4 230 1069 4:05
💚​ aws windows 4 4 232 1067 3:56
💚​ aws-ucws linux 4 4 314 987 7:24
💚​ aws-ucws windows 4 4 316 985 6:08
💚​ azure linux 4 4 230 1068 4:15
💚​ azure windows 4 4 232 1066 4:04
💚​ azure-ucws linux 4 4 316 984 8:13
🟨​ azure-ucws windows 3 1 4 318 982 9:30
💚​ gcp linux 4 4 229 1070 3:56
💚​ gcp windows 4 4 231 1068 4:01
8 interesting tests: 4 SKIP, 3 KNOWN, 1 RECOVERED
Test Name aws linux aws windows aws-ucws linux aws-ucws windows azure linux azure windows azure-ucws linux azure-ucws windows gcp linux gcp windows
💚​ TestAccept 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
🙈​ TestAccept/bundle/invariant/no_drift 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/vector_search_endpoints/drift/recreated_same_name 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/vector_search_indexes/recreate/embedding_dimension 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/ssh/connection 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🟨​ TestFetchRepositoryInfoAPI_FromRepo 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 🟨​K 💚​R 💚​R
🟨​ TestFetchRepositoryInfoAPI_FromRepo/root 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 🟨​K 💚​R 💚​R
🟨​ TestFetchRepositoryInfoAPI_FromRepo/subdir 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 🟨​K 💚​R 💚​R
Top 7 slowest tests (at least 2 minutes):
duration env testname
3:33 azure-ucws windows TestAccept
3:06 gcp windows TestAccept
3:05 azure windows TestAccept
3:02 aws windows TestAccept
2:58 aws-ucws windows TestAccept
2:14 azure-ucws windows TestAccept/bundle/deploy/mlops-stacks/DATABRICKS_BUNDLE_ENGINE=terraform
2:10 aws-ucws windows TestAccept/bundle/deploy/mlops-stacks/DATABRICKS_BUNDLE_ENGINE=terraform

@parthban-db parthban-db force-pushed the parthban-db/stack/telemetry branch from cd71e38 to f2bf35b Compare July 8, 2026 14:08
@parthban-db parthban-db temporarily deployed to test-trigger-is July 8, 2026 14:08 — with GitHub Actions Inactive
@parthban-db parthban-db temporarily deployed to test-trigger-is July 8, 2026 14:08 — with GitHub Actions Inactive
@parthban-db parthban-db force-pushed the parthban-db/stack/telemetry branch from f2bf35b to b477ac6 Compare July 8, 2026 14:11
@parthban-db parthban-db temporarily deployed to test-trigger-is July 8, 2026 14:11 — with GitHub Actions Inactive
@parthban-db parthban-db temporarily deployed to test-trigger-is July 8, 2026 14:11 — with GitHub Actions Inactive
@parthban-db parthban-db changed the title Add Aitools install telemetry Add telemetry for databricks aitools install Jul 8, 2026
@@ -13,7 +14,7 @@ func TestLegacySkillsInstallDelegatesToInstall(t *testing.T) {
setupTestAgents(t)
calls := setupInstallMock(t)

ctx := cmdio.MockDiscard(t.Context())
ctx := telemetry.WithNewLogger(cmdio.MockDiscard(t.Context()))

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need the logger for the telemetry because otherwise, it will panic. A logger is always present in the production path through the root command.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants