Introduce minimal dependency analytics interface by metamben · Pull Request #72262 · metabase/metabase

metamben · 2026-04-09T16:00:47Z

Description

This PR introduces an interface namespace for emitting internal analytics events, aiming for

providing a common interface for FE and BE event emission, and
allowing metabase lib to use the analytics interface without introducing cyclic dependencies.

The event emission parts that are provided by the new interface are migrated from analytics.core and prometheus to the new interface namespace.

How to verify

The tests should pass.

Checklist

Tests have been added/updated to cover changes in this PR
~~If adding new Loki tests: they pass stress testing~~

…interface

metamben · 2026-04-09T22:31:37Z

    :refer [gsheets gsheets!]]
   [metabase-enterprise.harbormaster.client :as hm.client]
-   [metabase.analytics.core :as analytics]
+   [metabase.analytics.interface :as analytics]


is this too unorthodox?

Not to bikeshed, but yeah my expectation would be that the "public interface" namespace would be the top-level metabase.analytics.

the issue here is that public API is usually metabase.<modul-name>.core and is required as <modul-name>. there is no convention how to deal with a separate interface namespace different from the core. I hijacked the standard analytics alias for the interface namespace, because in most cases that's what users should import.

I think in this case we'd probably normally actually want to have a separate interface module, e.g. analytics-interface with an API namespace of metabase.analytics-interface.core

metamben · 2026-04-09T22:32:56Z

      (let [table-size (row-count pgvector gate-table-name)]
        (log/debugf "Setting `semantic-gate-size` metric to %d" table-size)
-        (analytics/set! :metabase-search/semantic-gate-size table-size)
+        (analytics/set-gauge! :metabase-search/semantic-gate-size table-size)


I didn't feel like fighting the set! special form, especially with CLJS added to the picture.

metamben · 2026-04-09T22:33:38Z

+   [metabase.analytics.core :as analytics.core]
+   [metabase.analytics.interface :as analytics]


here we have both namespaces.

metamben · 2026-04-09T23:08:11Z

@@ -0,0 +1,19 @@
+(ns metabase.analytics.impl


too generic?

I think impl is fine, but you could also call it metabase.analytics.reporter.

metamben · 2026-04-09T23:13:48Z

@@ -1,4 +1,5 @@
 (ns metabase.analytics.init
  (:require
+   [metabase.analytics.impl]


to register the reporter.

It would be better to add a comment in the code to this effect 🙃

…interface

technomancy

Can't really comment on the frontend stuff, but the backend side makes sense.

…interface

camsaul

I think this all looks good but it would be my preference to split out analytics-interface into its own zero-dependency module and then have a separate analytics or analytics-impl module with the Prometheus/Snowplow/whatever implementations in it

…interface

github-actions · 2026-04-24T16:03:46Z

e2e tests failed on `c780483e85de6aa5504d99b6fb2f28a5c448ac98-1`

e2e test run

File	Test Name
`model-actions.cy.spec.js`	Write actions on model detail page (postgres) > should respect impersonated permission
`source-replacement.cy.spec.ts`	scenarios > data-studio > source replacement > Native queries > replaces a table referenced in a native SQL question
`dependency-unreferenced-list.cy.spec.ts`	scenarios > dependencies > unreferenced list > analysis > should show unreferenced entities
`dependency-unreferenced-list.cy.spec.ts`	scenarios > dependencies > unreferenced list > filters > should filter entities by type
`dependency-unreferenced-list.cy.spec.ts`	scenarios > dependencies > unreferenced list > filters > should persist filter changes after page reload
`dependency-unreferenced-list.cy.spec.ts`	scenarios > dependencies > unreferenced list > filters > should filter by location
`dependency-unreferenced-list.cy.spec.ts`	scenarios > dependencies > unreferenced list > selecting entities > should show the sidebar for supported entities and trigger snowplow event

Onlinehead

Metrics LGTM

* Lighter prometheus analytics There's a couple things about prometheus that have gotten a bit unwieldy. This pain has recently been seen in Tamas's change that broke startup in a very "action at a distance" way. #72262 is _excellent_ work. But prometheus starts up so late that it is hazardous. Principally it requires app db support (for the setting system) and it loads essentially last after app db is initialized, token checks, etc. And this is the bad part. Many systems can log analytic events before the system is initialized. There are calls that when incrementing a a metric, if it's not already initialized, to initialize it. But in the course of initializing, it would want to increment a metric, causing an infinite loop. This was exacerbated in that the safety valves of the token checker kicked in and suppressed errors for 30 seconds, ending up with threads locked up trying to initialize the collector. Solution: just initialize it. It's just a few atomic longs and maybe a small webserver. One complication is from chris's work in #52834 where they try to seed initial values so that when something registers a number it is a jump from 0 to 8 and therefore alarming, rather than just 8 with no differential. So i added an edge that these can be set later on in the process. One thing to call out is that the initial value is mostly 0, but the search ones use a value of 1 as a bit field to indicate it's true: ``` prometheus=> (initial-labelled-metric-values) [ ,,, {:metric :metabase-search/engine-default, :labels {:engine "appdb"}, :value 1} ;; -> indicates appdb is the default engine {:metric :metabase-search/engine-default, :labels {:engine "semantic"}, :value 0} ,,, {:metric :metabase-search/engine-active, :labels {:engine "in-place"}, :value 1} {:metric :metabase-search/engine-active, :labels {:engine "appdb"}, :value 1} {:metric :metabase-search/engine-active, :labels {:engine "semantic"}, :value 0} ``` So when we call the initial observed values it's just setting an affirmative 0 on the value, or an affirmative 1. we won't clobber the number of emails sent or anything * initialize prometheus in test runner * remove tests that care about reentrant and auto start * default no-op reporter * remove setting up dynamic var. we don't setup for you any more

Lighter prometheus analytics (#73456) * Lighter prometheus analytics There's a couple things about prometheus that have gotten a bit unwieldy. This pain has recently been seen in Tamas's change that broke startup in a very "action at a distance" way. #72262 is _excellent_ work. But prometheus starts up so late that it is hazardous. Principally it requires app db support (for the setting system) and it loads essentially last after app db is initialized, token checks, etc. And this is the bad part. Many systems can log analytic events before the system is initialized. There are calls that when incrementing a a metric, if it's not already initialized, to initialize it. But in the course of initializing, it would want to increment a metric, causing an infinite loop. This was exacerbated in that the safety valves of the token checker kicked in and suppressed errors for 30 seconds, ending up with threads locked up trying to initialize the collector. Solution: just initialize it. It's just a few atomic longs and maybe a small webserver. One complication is from chris's work in #52834 where they try to seed initial values so that when something registers a number it is a jump from 0 to 8 and therefore alarming, rather than just 8 with no differential. So i added an edge that these can be set later on in the process. One thing to call out is that the initial value is mostly 0, but the search ones use a value of 1 as a bit field to indicate it's true: ``` prometheus=> (initial-labelled-metric-values) [ ,,, {:metric :metabase-search/engine-default, :labels {:engine "appdb"}, :value 1} ;; -> indicates appdb is the default engine {:metric :metabase-search/engine-default, :labels {:engine "semantic"}, :value 0} ,,, {:metric :metabase-search/engine-active, :labels {:engine "in-place"}, :value 1} {:metric :metabase-search/engine-active, :labels {:engine "appdb"}, :value 1} {:metric :metabase-search/engine-active, :labels {:engine "semantic"}, :value 0} ``` So when we call the initial observed values it's just setting an affirmative 0 on the value, or an affirmative 1. we won't clobber the number of emails sent or anything * initialize prometheus in test runner * remove tests that care about reentrant and auto start * default no-op reporter * remove setting up dynamic var. we don't setup for you any more Co-authored-by: dpsutton <dan@dpsutton.com>

Master moved Prometheus helper fns (inc!, set!/set-gauge!, observe!) out of metabase.analytics.core and into metabase.analytics-interface.core (see #72262), and the rename broke the namespace-decl check on this branch. Point all metabase.mq.* files (and their tests) at the new namespace, rename set! → set-gauge!, and add analytics-interface to the mq module's :uses set.

metamben requested a review from a team as a code owner April 9, 2026 16:00

metamben added the no-backport Do not backport this PR to any branch label Apr 9, 2026

metabase-bot Bot assigned metamben Apr 9, 2026

metabase-bot Bot added the .Team/QueryingPlatform Platform subteam for .Team/Querying. Please use in conjunction with .Team/Querying label Apr 9, 2026

Implement mostly dependency free analytics interface

12f088c

metamben force-pushed the que2-507-analytics-interface branch from e4b95dd to 1b52135 Compare April 9, 2026 16:07

Migrate to analytics interface

a75027d

metamben force-pushed the que2-507-analytics-interface branch from 1b52135 to a75027d Compare April 9, 2026 16:25

metamben added 3 commits April 9, 2026 21:26

Migrate to analytics interface

c37479e

Fix CLJS tests

0e862b0

Merge remote-tracking branch 'origin/master' into que2-507-analytics-…

b63aa21

…interface

metamben force-pushed the que2-507-analytics-interface branch from e11e13c to b63aa21 Compare April 9, 2026 18:47

metamben commented Apr 9, 2026

View reviewed changes

Comment thread src/metabase/analytics/impl.cljs Outdated

metamben commented Apr 9, 2026

View reviewed changes

metamben added 2 commits April 10, 2026 02:35

Clean up a bit

eb1e329

Merge remote-tracking branch 'origin/master' into que2-507-analytics-…

6178ec5

…interface

metamben requested a review from a team as a code owner April 9, 2026 23:36

Merge remote-tracking branch 'origin/master' into que2-507-analytics-…

22190f9

…interface

technomancy reviewed Apr 21, 2026

View reviewed changes

Comment thread src/metabase/analytics/api.clj

technomancy approved these changes Apr 21, 2026

View reviewed changes

metamben added 5 commits April 22, 2026 17:53

Merge remote-tracking branch 'origin/master' into que2-507-analytics-…

3f0d15d

…interface

Fix merge

039f5a5

Merge remote-tracking branch 'origin/master' into que2-507-analytics-…

e4f4394

…interface

Limit flush frequency and put up with losing events

7d530a7

Merge remote-tracking branch 'origin/master' into que2-507-analytics-…

2eae243

…interface

camsaul approved these changes Apr 23, 2026

View reviewed changes

metamben added 4 commits April 24, 2026 01:59

Merge remote-tracking branch 'origin/master' into que2-507-analytics-…

635bc90

…interface

Introduce analytics-interface module

c3c5323

Merge remote-tracking branch 'origin/master' into que2-507-analytics-…

6226aef

…interface

Fix module usage and configuration

92555ac

metamben enabled auto-merge (squash) April 24, 2026 15:36

metamben added 2 commits April 24, 2026 18:49

Add analytics-interface to driver affecting modules

60fbff3

Merge remote-tracking branch 'origin/master' into que2-507-analytics-…

c780483

…interface

Onlinehead approved these changes Apr 24, 2026

View reviewed changes

metamben merged commit c0927e9 into master Apr 24, 2026
229 checks passed

metamben deleted the que2-507-analytics-interface branch April 24, 2026 18:45

github-actions Bot added this to the 0.61 milestone Apr 30, 2026

dpsutton mentioned this pull request Apr 30, 2026

Lighter prometheus analytics #73456

Merged

		[metabase.analytics.core :as analytics.core]
		[metabase.analytics.interface :as analytics]

Conversation

metamben commented Apr 9, 2026

Description

How to verify

Checklist

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

technomancy left a comment

Choose a reason for hiding this comment

Uh oh!

camsaul left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions Bot commented Apr 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

e2e tests failed on c780483e85de6aa5504d99b6fb2f28a5c448ac98-1

Uh oh!

Onlinehead left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

github-actions Bot commented Apr 24, 2026 •

edited

Loading

e2e tests failed on `c780483e85de6aa5504d99b6fb2f28a5c448ac98-1`