Skip to content

DSL debugger (OAL): emit source @ScopeDefaultColumn fields on output sample via codegen#13867

Merged
wu-sheng merged 1 commit intomasterfrom
oal-debug-fields-codegen
May 9, 2026
Merged

DSL debugger (OAL): emit source @ScopeDefaultColumn fields on output sample via codegen#13867
wu-sheng merged 1 commit intomasterfrom
oal-debug-fields-codegen

Conversation

@wu-sheng
Copy link
Copy Markdown
Member

@wu-sheng wu-sheng commented May 9, 2026

Fix missing attr0..5 / entityId columns in OAL debug capture

Bug. A debug session against any OAL metric (e.g.
oal/core.oal/service_cpm) shows the terminal output sample with only
the family parent's value columns:

{ "type": "ServiceCpmMetrics", "timeBucket": …, "id": …,
  "total": 1, "value": 0 }

The source-derived storage columns attr0..attr5 and entityId are
absent — even though the OAL-generated subclass DOES carry them on the
live instance, the entry function copied them from the Source, and the
storage row in BanyanDB DOES persist them. Operators verifying e.g. the
layer name written to attr0 by ServiceDecorator cannot see it in
the debugger; they have to cross-reference with a storage query.

Why the gap. For OAL-generated metrics, attr0..5 are NOT
inherited from a fixed parent — they are declared on the source class
via @ScopeDefaultColumn.DefinedByField, surfaced by
SourceColumnsFactory.getColumns(), and copied to the generated
metrics subclass at OAL boot by OALClassGeneratorV2.generateMetricsClass
(line 220, the for (CodeGenModel.SourceFieldV2 field : model.getFieldsFromSource()) metricsClass.addField(...) loop). The
family parent (CPMMetrics, SumMetrics, …) doesn't know the dynamic
fields exist, so its appendDebugFields(JsonObject) override only
emits its own value columns. Metrics.toJson() then renders just
{type, timeBucket, lastUpdateTimestamp, id} + family fields.

The fix. Codegen-side. The same OAL pass that materialises the
dynamic fields now also emits an appendDebugFields(JsonObject)
override on the generated subclass:

protected void appendDebugFields(com.google.gson.JsonObject obj) {
    super.appendDebugFields(obj);  // delegates to family override
                                   // (CPMMetrics → total + value, …)
    obj.addProperty("entity_id", entityId);
    obj.addProperty("attr0", attr0);
    // … each @ScopeDefaultColumn-declared field
}

Mechanically: a new FreeMarker template
oap-server/oal-rt/src/main/resources/code-templates-v2/metrics/appendDebugFields.ftl
iterates fieldsFromSource and emits the right typed
obj.addProperty(...) overload (Long / Integer / Double / Float /
Boolean wrappers for primitives so Javassist resolves the
addProperty(String, Number) overload unambiguously; null-safe
toString() for object types). The generator picks it up by listing
"appendDebugFields" in METRICS_CLASS_METHODS, the same array that
already drives id.ftl, serialize.ftl, etc.

Why codegen, not reflection / a Convert2Storage shim. The fields
are codegen-materialised — putting their debug projection in the same
codegen pass keeps the field list literally hard-coded in the bytecode
(zero reflection at runtime, exact parity with what the generator
declared). Recorder-side reflection or a JsonObject-backed
Convert2Storage would also work but adds a runtime walker per probed
emit; codegen is strictly cheaper.

  • Add a unit test to verify that the fix works.

    • All 98 existing oap-server/oal-rt unit tests pass against the new
      METRICS_CLASS_METHODS entry — the existing
      OALClassGeneratorV2Test and RuntimeOALGenerationTest exercise
      the full template-driven generator. The generated method is loaded
      onto every *Metrics subclass at OAP boot, so a generator failure
      would surface as a class-load error before a single source event
      is dispatched.
    • Live-verified post-deploy: an OAL session against
      oal/core.oal/service_cpm now shows the dynamic columns on the
      terminal sample in addition to the family parent's value fields.
  • Explain briefly why the bug exists and how to fix it.

    • See above. The bug is debug-output only; the storage row was
      always correct, only the wire payload missed the dynamic columns.
      The fix is debug-only and zero-cost when
      SW_DSL_DEBUGGING_INJECTION_ENABLED=false (the override is only
      invoked from Metrics.toJson(), which is only called by the
      gated dsl-debugging recorder).
  • If this pull request closes/resolves/fixes an existing issue, replace the issue number. Closes #.

  • Update the CHANGES log.

…sample via codegen

The terminal `output` sample of an OAL debug session was missing the
source-derived @column fields that the OAL-generated subclass carries
on its instance — entityId + attr0..attr5 for any Service-style scope.
Operators inspecting `service_cpm` saw only `total` / `value` from
CPMMetrics.appendDebugFields; the layer name written to attr0 by
ServiceDecorator (and any other source attributes) was on the live
object and in the storage row, but invisible in the debug capture.

Why the gap: `attr0..attr5` are NOT inherited from a fixed parent for
OAL — they're declared on the source class via
`@ScopeDefaultColumn.DefinedByField` and copied to the generated
metrics subclass at OAL boot by `OALClassGeneratorV2.generateMetricsClass`
(line 220, the per-source-field `addField` loop). The family parent
(CPMMetrics, SumMetrics, ...) doesn't know they exist, so its
`appendDebugFields` override can't surface them. Reflection-based or
`Convert2Storage` shim approaches were considered; codegen-side
generation is the right scope — the same pass that materialises the
fields also emits their debug projection.

Adds `appendDebugFields` to METRICS_CLASS_METHODS so the existing
template loop generates an override on every OAL-generated `*Metrics`
subclass. The override calls `super.appendDebugFields(obj)` first
(delegating the family's value columns — total / value / summation /
...) then iterates `fieldsFromSource` to emit each
@ScopeDefaultColumn-declared field with the right typed
`obj.addProperty(...)` overload (Long / Integer / Double / Float /
Boolean wrappers for primitives so Javassist resolves the
`addProperty(String, Number)` overload unambiguously; null-safe
`toString()` for object types).

Verification:
- `mvn test -pl oap-server/oal-rt`: 98/98 pass.
- After deploy, an OAL debug session against `oal/core.oal` /
  `service_cpm` will show `payload.attr0=GENERAL` (set by
  ServiceDecorator) on the terminal output sample, alongside the
  family's `total` and `value`.

Cost: zero when `SW_DSL_DEBUGGING_INJECTION_ENABLED=false` (codegen
emits an `appendDebugFields` override either way; it's only invoked
from `Metrics.toJson()` which is only called by the gated debug
recorder). When debug is on, one extra method call per probed emit
per debug session.
@wu-sheng wu-sheng requested a review from Copilot May 9, 2026 12:39
@wu-sheng wu-sheng added backend OAP backend related. enhancement Enhancement on performance or codes labels May 9, 2026
@wu-sheng wu-sheng added this to the 10.5.0 milestone May 9, 2026
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates the OAL (OAL v2) code generation pipeline so that OAL-generated *Metrics subclasses include @ScopeDefaultColumn-derived fields (e.g., entity_id, attr0..attr5) in DSL debugging JSON output by generating an appendDebugFields(JsonObject) override.

Changes:

  • Add a new FreeMarker template to generate appendDebugFields(JsonObject) for OAL-generated metrics classes, emitting fieldsFromSource into the JSON debug payload.
  • Register the new template/method in OALClassGeneratorV2.METRICS_CLASS_METHODS so it is generated for all OAL metrics subclasses.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File Description
oap-server/oal-rt/src/main/resources/code-templates-v2/metrics/appendDebugFields.ftl New template that emits source-derived columns into the metrics debug JSON via a generated appendDebugFields override.
oap-server/oal-rt/src/main/java/org/apache/skywalking/oal/v2/generator/OALClassGeneratorV2.java Adds appendDebugFields to the method-generation list so the template is applied to generated metrics classes.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +1 to +5
protected void appendDebugFields(com.google.gson.JsonObject obj) {
super.appendDebugFields(obj);
<#list fieldsFromSource as field>
<#if field.typeName == "long">
obj.addProperty("${field.columnName}", java.lang.Long.valueOf(${field.fieldGetter}()));
@wu-sheng wu-sheng merged commit e35cea3 into master May 9, 2026
445 of 448 checks passed
@wu-sheng wu-sheng deleted the oal-debug-fields-codegen branch May 9, 2026 13:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backend OAP backend related. enhancement Enhancement on performance or codes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants