Add caller-side Nexus Operation metrics. by S15 · Pull Request #10026 · temporalio/temporal

S15 · 2026-04-22T16:18:44Z

What changed?

New metrics:

Counters

Metric	Description
`nexus_operation_success_count`	Successfully completed operations
`nexus_operation_failed_count`	Failed operations
`nexus_operation_cancel_count`	Cancelled operations
`nexus_operation_terminate_count`	Operations terminated before completion
`nexus_operation_timeout_count`	Operations timed out before completion

Histograms

Metric	Description
`nexus_operation_schedule_to_close_latency`	Time between schedule and close for sync and async operations
`nexus_operation_schedule_to_start_latency`	Time between schedule and start for sync and async operations
`nexus_operation_start_to_close_latency`	Time between start and close for async operations only

Labels

All metrics use the following labels:

Label	Notes
`namespace`
`nexus_endpoint`
`nexus_service`	Controlled by Dynamic config
`nexus_operation`	Controlled by Dynamic config
`workflowType`

Latency metrics also include:

Label	Values
`outcome`	Succeeded, Failed, Canceled, Terminated, TimedOut

Timeout Count also includes:

Label	Values
`timeout_type`	StartToClose, ScheduleToStart, ScheduleToClose

How did you test it?

Potential risks

Metrics emitting isn't nil safe, and enrich metrics could panic if misconfigured.

Add API boilerplate for standalone Nexus Operations. - [x] built - [ ] run locally and tested manually - [x] covered by existing tests - [ ] added new unit test(s) - [ ] added new functional test(s)

Add Nexus Standalone feature flag. Tests will be added to respective API impl.

Add Nexus Standalone Describe and Start handlers. - [ ] built - [ ] run locally and tested manually - [ ] covered by existing tests - [x] added new unit test(s) - [x] added new functional test(s) --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Add Nexus Standalone List and Count handlers. - [ ] built - [ ] run locally and tested manually - [ ] covered by existing tests - [ ] added new unit test(s) - [x] added new functional test(s)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

## What changed? Wire up Nexus Standalone with CHASM. ## How did you test it? - [ ] built - [ ] run locally and tested manually - [x] covered by existing tests - [ ] added new unit test(s) - [ ] added new functional test(s) --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: Roey Berman <roey@temporal.io>

## What changed? Add identity to `TerminatedFailureInfo`. PS: `CanceledFailureInfo` will be a separate PR. ## Why? Allow clients to access identity via API. ## How did you test it? - [ ] built - [ ] run locally and tested manually - [ ] covered by existing tests - [x] added new unit test(s) - [x] added new functional test(s)

S15 · 2026-04-22T18:42:47Z

The currently failing tests are shared with the feature/nexus-standalone branch

bergundy

Overall looks great. I think we should be done in another round.

bergundy · 2026-04-23T05:21:51Z

+				}
+			},
+			GoCtx: context.WithValue(context.Background(), nexusoperation.OperationContextKey, &nexusoperation.OperationContext{
+				NamespaceRegistry: nsRegistry,


You don't need the namespace registry in the context, the current namespace entry is already on the CHASM context,.

bergundy · 2026-04-23T05:24:01Z

+var NexusOperationSuccessCount = metrics.NewCounterDef(
+	"nexus_operation_success_count",
+	metrics.WithDescription("Nexus Operations successfully completed."),
+)
+var NexusOperationFailedCount = metrics.NewCounterDef(
+	"nexus_operation_failed_count",
+	metrics.WithDescription("Nexus Operations failures."),
+)
+var NexusOperationCancelCount = metrics.NewCounterDef(
+	"nexus_operation_cancel_count",
+	metrics.WithDescription("Nexus Operations cancellations."),
+)
+var NexusOperationTerminateCount = metrics.NewCounterDef(
+	"nexus_operation_terminate_count",
+	metrics.WithDescription("Nexus Operations that were terminated before completion."),
+)
+var NexusOperationTimeoutCount = metrics.NewCounterDef(
+	"nexus_operation_timeout_count",
+	metrics.WithDescription("Nexus Operations that timed out before completion."),
+)


On the fence with the naming here. On the one hand it's consistent with activities, OTOH, I would prefer to use the statuses and use the same tense for all outcomes (failed is past tense and inconsistent with the rest)

bergundy · 2026-04-23T05:27:06Z

 	if store, ok := o.Store.TryGet(ctx); ok {
 		return store.OnNexusOperationCompleted(ctx, o, result, links)
 	}
+	metricsHandler, err := o.EnrichMetricsHandler(ctx)


Instead of passing in a metrics handler, you can obtain it in the transition function.

bergundy · 2026-04-23T05:27:49Z

+func (o *Operation) EnrichMetricsHandler(ctx chasm.Context) (metrics.Handler, error) {
+	// nolint:revive // unchecked-type-assertion: intentional panic on missing context value
+	opCtx := ctx.Value(OperationContextKey).(*OperationContext)
+	namespaceName, err := opCtx.NamespaceRegistry.GetNamespaceName(namespace.ID(ctx.ExecutionKey().NamespaceID))


ctx.NamespaceEntry().Name().String()

bergundy · 2026-04-23T05:28:29Z

+		metrics.NexusEndpointTag(o.GetEndpoint()),
+		metrics.WorkflowTypeTag(WorkflowTypeTag),
+	}
+	if opCtx.MetricTagConfig != nil {


Let's just ensure that this is never nil. It shouldn't be.

bergundy · 2026-04-23T05:29:41Z

+}
+
+func (o *Operation) emitOnFailedMetrics(handler metrics.Handler, closeTime time.Time) {
+	outcomeTag := metrics.OutcomeTag(nexusoperationpb.OPERATION_STATUS_FAILED.String())


Maybe lower case?

bergundy · 2026-04-23T05:32:50Z

+	startedTime := o.GetStartedTime()
+	if startedTime != nil {
+		// Async operation that was started.
+		NexusOperationScheduleToStartLatency.With(handler).Record(startedTime.AsTime().Sub(scheduledTime), outcomeTag)


Emit this on transition to start instead of only emitting it when the operation completes.

bergundy · 2026-04-23T05:35:12Z

+	tags := []metrics.Tag{
+		metrics.NamespaceTag(namespaceName.String()),
+		metrics.NexusEndpointTag(o.GetEndpoint()),
+		metrics.WorkflowTypeTag(WorkflowTypeTag),


If you're in a workflow, you'll need to emit the workflow's type.

You can add a method to the Store interface: WorkflowTypeTag() string.

bergundy · 2026-04-23T05:41:24Z

+func newComponentOnlyLibrary(namespaceRegistry namespace.Registry, dc *dynamicconfig.Collection) *componentOnlyLibrary {
+	return &componentOnlyLibrary{
+		namespaceRegistry: namespaceRegistry,
+		metricTagConfig:   MetricTagConfiguration.Get(dc),


Hmmm... it's okay to reuse this since I figure users will want to turn it on for both callers and handler but this config struct was meant for the frontend (handler side) originally. You'll want to document how this config is used now that it has other purposes and mention that HeaderTagMappings is not relevant on the caller side.

stephanos and others added 17 commits April 21, 2026 12:34

Standalone Nexus Operations API (temporalio#9458)

a6e42fd

Add API boilerplate for standalone Nexus Operations. - [x] built - [ ] run locally and tested manually - [x] covered by existing tests - [ ] added new unit test(s) - [ ] added new functional test(s)

Nexus Standalone: Feature flag (temporalio#9567)

be86d14

Add Nexus Standalone feature flag. Tests will be added to respective API impl.

Nexus Standalone: List + Count (temporalio#9559)

2494b51

Add Nexus Standalone List and Count handlers. - [ ] built - [ ] run locally and tested manually - [ ] covered by existing tests - [ ] added new unit test(s) - [x] added new functional test(s)

Nexus Standalone: Terminate + Cancel (temporalio#9624)

e604915

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Nexus Standalone: Delete (temporalio#9654)

f4ca4c8

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Nexus Standalone: Poll (temporalio#9780)

46e06df

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Nexus Standalone: translate already started err

5536042

Nexus CHASM async completion

f4fc5de

fix merge

c7346c1

fixup

ee86d40

Adds caller side operation state machine metrics

20093e0

Updates tests to check that metrics are emitted

61a7130

Fixup tests

44362b3

Fix resolve order

cde4818

S15 marked this pull request as ready for review April 22, 2026 18:25

S15 requested review from a team as code owners April 22, 2026 18:25

S15 marked this pull request as draft April 22, 2026 18:25

S15 changed the title ~~Add caller-side nexus operation metrics.~~ Add caller-side Nexus Operation metrics. Apr 22, 2026

S15 marked this pull request as ready for review April 22, 2026 18:42

S15 requested review from Quinn-With-Two-Ns, bergundy and stephanos April 22, 2026 18:46

Adds workflowtypetag and fixes outcomes

20d7bb6

bergundy reviewed Apr 23, 2026

View reviewed changes

stephanos force-pushed the feature/nexus-standalone branch from 38923fc to 340f40a Compare April 23, 2026 17:11

stephanos requested review from a team as code owners April 23, 2026 19:09

stephanos force-pushed the feature/nexus-standalone branch 3 times, most recently from ce242b7 to 397eca3 Compare April 23, 2026 20:21

S15 force-pushed the feature/nexus-standalone branch from 397eca3 to 7ae0421 Compare April 23, 2026 20:35

stephanos force-pushed the feature/nexus-standalone branch 21 times, most recently from 378e20c to 80b781d Compare April 24, 2026 04:25

stephanos deleted the branch temporalio:feature/nexus-standalone April 24, 2026 05:49

stephanos closed this Apr 24, 2026

S15 mentioned this pull request Apr 24, 2026

Add caller-side Nexus operation metrics #10071

Open

5 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add caller-side Nexus Operation metrics.#10026

Add caller-side Nexus Operation metrics.#10026
S15 wants to merge 18 commits intotemporalio:feature/nexus-standalonefrom
S15:nexus-metrics

S15 commented Apr 22, 2026 •

edited

Loading

Uh oh!

S15 commented Apr 22, 2026

Uh oh!

bergundy left a comment

Uh oh!

bergundy Apr 23, 2026

Uh oh!

bergundy Apr 23, 2026

Uh oh!

bergundy Apr 23, 2026

Uh oh!

bergundy Apr 23, 2026

Uh oh!

bergundy Apr 23, 2026

Uh oh!

bergundy Apr 23, 2026

Uh oh!

bergundy Apr 23, 2026

Uh oh!

bergundy Apr 23, 2026

Uh oh!

bergundy Apr 23, 2026

Uh oh!

bergundy Apr 23, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

S15 commented Apr 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What changed?

Counters

Histograms

Labels

How did you test it?

Potential risks

Uh oh!

S15 commented Apr 22, 2026

Uh oh!

bergundy left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

S15 commented Apr 22, 2026 •

edited

Loading