Skip to content

Fix decode error for dictionary-encoded structs#354

Merged
tigrannajaryan merged 1 commit intomainfrom
tigran/fix-profile-bug
Feb 15, 2026
Merged

Fix decode error for dictionary-encoded structs#354
tigrannajaryan merged 1 commit intomainfrom
tigran/fix-profile-bug

Conversation

@tigrannajaryan
Copy link
Collaborator

The problem occurs when decoding dictionary-encoded structs and the refNum is 0, which corresponds to nil struct. Nil is not allowed for structs unless they are optional.

Decoding error is found by FuzzSampleReader and corresponding seeds are included in this PR.

The problem occurs when decoding dictionary-encoded structs and the refNum is 0,
which corresponds to nil struct. Nil is not allowed for structs unless they are optional.

Decoding error is found by FuzzSampleReader and corresponding seeds are included in this PR.
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Fixes a decode failure when dictionary-encoded struct references use refNum == 0 (nil) in contexts where nil structs are invalid, and adds fuzz-regression seeds capturing the issue.

Changes:

  • Add explicit nil checks after decoding dict-encoded struct pointers (reject nil for non-optional/invalid cases).
  • Add a generic pkg.ErrDecodeError sentinel used for “invalid decode state” returns.
  • Refactor many generated decoders to use inline if err := ...; err != nil style and adjust oneof decoders to surface buffer errors immediately after reading the variant type.

Reviewed changes

Copilot reviewed 51 out of 51 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
stefc/templates/go/struct.go.tmpl Add nil rejection for non-optional dict-encoded pointer fields after decode; simplify error handling.
stefc/templates/go/oneof.go.tmpl Check buffer error after reading oneof type; reject nil for dict-encoded pointer variants; return nil when no further buffer interaction is needed.
stefc/templates/go/array.go.tmpl Reject nil elements for dict-encoded struct arrays; simplify error handling in loops.
go/pkg/errors.go Introduce ErrDecodeError decode-error sentinel.
go/otel/otelstef/uint64array.go Inline decode error handling style cleanup.
go/otel/otelstef/summaryvalue.go Inline decode error handling style cleanup.
go/otel/otelstef/spanstatus.go Inline decode error handling style cleanup.
go/otel/otelstef/spans.go Inline decode error handling; reject nil for non-optional dict-decoded struct pointers.
go/otel/otelstef/span.go Inline decode error handling style cleanup.
go/otel/otelstef/scope.go Inline decode error handling style cleanup.
go/otel/otelstef/resource.go Inline decode error handling style cleanup.
go/otel/otelstef/quantilevaluearray.go Inline decode error handling style cleanup.
go/otel/otelstef/quantilevalue.go Inline decode error handling style cleanup.
go/otel/otelstef/pointvalue.go Oneof: check buffer error after reading type; return nil at end.
go/otel/otelstef/point.go Inline decode error handling style cleanup.
go/otel/otelstef/metrics.go Inline decode error handling; reject nil for non-optional dict-decoded struct pointers.
go/otel/otelstef/metric.go Inline decode error handling style cleanup.
go/otel/otelstef/linkarray.go Inline decode error handling style cleanup.
go/otel/otelstef/link.go Inline decode error handling style cleanup.
go/otel/otelstef/histogramvalue.go Inline decode error handling style cleanup.
go/otel/otelstef/float64array.go Inline decode error handling style cleanup.
go/otel/otelstef/exphistogramvalue.go Inline decode error handling style cleanup.
go/otel/otelstef/exphistogrambuckets.go Inline decode error handling style cleanup.
go/otel/otelstef/exemplarvalue.go Oneof: check buffer error after reading type; return nil at end.
go/otel/otelstef/exemplararray.go Inline decode error handling style cleanup.
go/otel/otelstef/exemplar.go Inline decode error handling style cleanup.
go/otel/otelstef/eventarray.go Inline decode error handling style cleanup.
go/otel/otelstef/event.go Inline decode error handling style cleanup.
go/otel/otelstef/envelope.go Inline decode error handling style cleanup.
go/otel/otelstef/anyvaluearray.go Inline decode error handling style cleanup.
go/otel/otelstef/anyvalue.go Oneof: check buffer error after reading type; return nil at end.
examples/profile/internal/profile/testdata/fuzz/FuzzSampleReader/d369e9adf349dff1 Add fuzz seed reproducing the decode issue.
examples/profile/internal/profile/testdata/fuzz/FuzzSampleReader/814b20d96326fa74 Add fuzz seed reproducing the decode issue.
examples/profile/internal/profile/stringarray.go Inline decode error handling style cleanup.
examples/profile/internal/profile/samplevaluetype.go Inline decode error handling style cleanup.
examples/profile/internal/profile/samplevaluearray.go Inline decode error handling style cleanup.
examples/profile/internal/profile/samplevalue.go Inline decode error handling; reject nil for non-optional dict-decoded struct pointers.
examples/profile/internal/profile/sample.go Inline decode error handling style cleanup.
examples/profile/internal/profile/profilemetadata.go Inline decode error handling; reject nil for non-optional dict-decoded struct pointers.
examples/profile/internal/profile/numvalue.go Inline decode error handling style cleanup.
examples/profile/internal/profile/mapping.go Inline decode error handling style cleanup.
examples/profile/internal/profile/locationarray.go Reject nil elements for dict-encoded struct arrays; inline decode error handling.
examples/profile/internal/profile/location.go Inline decode error handling; reject nil for non-optional dict-decoded struct pointers.
examples/profile/internal/profile/linearray.go Inline decode error handling style cleanup.
examples/profile/internal/profile/line.go Inline decode error handling; reject nil for non-optional dict-decoded struct pointers.
examples/profile/internal/profile/labelvalue.go Oneof: check buffer error after reading type; return nil at end.
examples/profile/internal/profile/function.go Inline decode error handling style cleanup.
examples/jsonl/internal/jsonstef/record.go Inline decode error handling style cleanup.
examples/jsonl/internal/jsonstef/jsonvaluearray.go Inline decode error handling style cleanup.
examples/jsonl/internal/jsonstef/jsonvalue.go Oneof: check buffer error after reading type; return nil at end.
examples/ints/internal/ints/record.go Inline decode error handling style cleanup.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@github-actions
Copy link

github-actions bot commented Feb 6, 2026

Benchmark Result

Benchmark diff with base branch
goos: linux
goarch: amd64
pkg: github.com/splunk/stef/benchmarks
cpu: AMD EPYC 7763 64-Core Processor                
                                                 │ bench-main.txt │           bench-new.txt            │
                                                 │     sec/op     │    sec/op     vs base              │
SerializeNative/STEF/serialize-4                     7.099m ±  2%   7.151m ±  6%       ~ (p=0.699 n=6)
SerializeNative/STEFU/serialize-4                    33.89m ±  2%   34.60m ±  2%  +2.10% (p=0.002 n=6)
DeserializeNative/STEF/deser-4                       2.428m ±  1%   2.411m ±  2%       ~ (p=0.065 n=6)
DeserializeNative/STEFU/deser-4                      7.206m ±  0%   7.209m ±  0%       ~ (p=0.310 n=6)
SerializeFromPdata/STEF/serialize-4                  137.3m ±  3%   137.7m ±  2%       ~ (p=0.937 n=6)
SerializeFromPdata/STEFU/serialize-4                 34.04m ±  1%   34.31m ±  1%       ~ (p=0.065 n=6)
DeserializeToPdata/STEF/deserialize-4                45.34m ±  1%   45.24m ±  1%       ~ (p=0.937 n=6)
DeserializeToPdata/STEFU/deserialize-4               62.34m ±  2%   62.30m ±  1%       ~ (p=0.818 n=6)
STEFReaderRead-4                                     2.501m ±  1%   2.509m ±  1%       ~ (p=0.589 n=6)
STEFSerializeMultipart/astronomy-otelmetrics-4        3.360 ± 21%    3.344 ± 21%       ~ (p=0.818 n=6)
STEFDeserializeMultipart/astronomy-otelmetrics-4     79.55m ± 13%   73.36m ± 11%       ~ (p=0.310 n=6)
ReadSTEF-4                                           2.554m ±  2%   2.537m ±  3%       ~ (p=0.485 n=6)
ReadSTEFZ-4                                          3.119m ±  0%   3.115m ±  0%       ~ (p=0.240 n=6)
ReadSTEFZWriteSTEF-4                                 7.287m ±  3%   7.072m ±  0%  -2.96% (p=0.002 n=6)
geomean                                              20.40m         20.27m        -0.65%

                                                 │ bench-main.txt │           bench-new.txt            │
                                                 │   sec/point    │  sec/point    vs base              │
SerializeNative/STEF/serialize-4                     106.2n ±  2%   107.0n ±  6%       ~ (p=0.699 n=6)
SerializeNative/STEFU/serialize-4                    506.9n ±  2%   517.5n ±  2%  +2.10% (p=0.002 n=6)
DeserializeNative/STEF/deser-4                       36.31n ±  1%   36.06n ±  1%  -0.69% (p=0.050 n=6)
DeserializeNative/STEFU/deser-4                      107.7n ±  0%   107.8n ±  0%       ~ (p=0.260 n=6)
SerializeFromPdata/STEF/serialize-4                  2.053µ ±  3%   2.060µ ±  2%       ~ (p=0.855 n=6)
SerializeFromPdata/STEFU/serialize-4                 509.1n ±  1%   513.3n ±  1%       ~ (p=0.065 n=6)
DeserializeToPdata/STEF/deserialize-4                678.2n ±  1%   676.8n ±  1%       ~ (p=0.937 n=6)
DeserializeToPdata/STEFU/deserialize-4               932.7n ±  2%   931.8n ±  1%       ~ (p=0.818 n=6)
STEFReaderRead-4                                     37.41n ±  1%   37.53n ±  1%       ~ (p=0.623 n=6)
STEFSerializeMultipart/astronomy-otelmetrics-4       4.271µ ± 21%   4.251µ ± 21%       ~ (p=0.732 n=6)
STEFDeserializeMultipart/astronomy-otelmetrics-4    101.10n ± 13%   93.23n ± 11%       ~ (p=0.310 n=6)
ReadSTEF-4                                           38.23n ±  2%   37.98n ±  2%       ~ (p=0.485 n=6)
ReadSTEFZ-4                                          46.69n ±  0%   46.62n ±  0%       ~ (p=0.240 n=6)
ReadSTEFZWriteSTEF-4                                 109.1n ±  3%   105.9n ±  0%  -2.93% (p=0.002 n=6)
geomean                                              214.5n         213.2n        -0.64%

                                                 │ bench-main.txt │            bench-new.txt             │
                                                 │      B/op      │     B/op      vs base                │
SerializeNative/STEF/serialize-4                     3.335Mi ± 0%   3.336Mi ± 0%       ~ (p=0.485 n=6)
SerializeNative/STEFU/serialize-4                    7.557Mi ± 0%   7.557Mi ± 0%       ~ (p=0.723 n=6)
DeserializeNative/STEF/deser-4                       951.4Ki ± 0%   951.4Ki ± 0%       ~ (p=1.000 n=6) ¹
DeserializeNative/STEFU/deser-4                      1.715Mi ± 0%   1.715Mi ± 0%       ~ (p=1.000 n=6)
SerializeFromPdata/STEF/serialize-4                  76.55Mi ± 0%   76.55Mi ± 0%       ~ (p=0.738 n=6)
SerializeFromPdata/STEFU/serialize-4                 7.557Mi ± 0%   7.557Mi ± 0%       ~ (p=0.905 n=6)
DeserializeToPdata/STEF/deserialize-4                31.99Mi ± 0%   31.99Mi ± 0%       ~ (p=0.662 n=6)
DeserializeToPdata/STEFU/deserialize-4               38.88Mi ± 0%   38.88Mi ± 0%       ~ (p=0.318 n=6)
STEFReaderRead-4                                     953.1Ki ± 0%   953.1Ki ± 0%       ~ (p=1.000 n=6) ¹
STEFSerializeMultipart/astronomy-otelmetrics-4       3.384Gi ± 0%   3.381Gi ± 0%       ~ (p=0.240 n=6)
STEFDeserializeMultipart/astronomy-otelmetrics-4     20.30Mi ± 0%   20.30Mi ± 0%       ~ (p=0.675 n=6)
ReadSTEF-4                                           953.2Ki ± 0%   953.2Ki ± 0%       ~ (p=0.784 n=6)
ReadSTEFZ-4                                          10.29Mi ± 0%   10.29Mi ± 0%       ~ (p=0.589 n=6)
ReadSTEFZWriteSTEF-4                                 13.44Mi ± 0%   13.44Mi ± 0%       ~ (p=0.937 n=6)
geomean                                              10.66Mi        10.66Mi       -0.00%
¹ all samples are equal

                                                 │ bench-main.txt │            bench-new.txt            │
                                                 │   allocs/op    │  allocs/op   vs base                │
SerializeNative/STEF/serialize-4                      2.646k ± 0%   2.647k ± 0%       ~ (p=0.368 n=6)
SerializeNative/STEFU/serialize-4                      884.5 ± 0%    884.5 ± 0%       ~ (p=1.000 n=6)
DeserializeNative/STEF/deser-4                         463.0 ± 0%    463.0 ± 0%       ~ (p=1.000 n=6) ¹
DeserializeNative/STEFU/deser-4                        496.0 ± 0%    496.0 ± 0%       ~ (p=1.000 n=6) ¹
SerializeFromPdata/STEF/serialize-4                   134.7k ± 0%   134.7k ± 0%       ~ (p=0.758 n=6)
SerializeFromPdata/STEFU/serialize-4                   886.0 ± 0%    886.0 ± 0%       ~ (p=1.000 n=6)
DeserializeToPdata/STEF/deserialize-4                 756.2k ± 0%   756.2k ± 0%       ~ (p=1.000 n=6)
DeserializeToPdata/STEFU/deserialize-4                944.9k ± 0%   944.9k ± 0%       ~ (p=1.000 n=6) ¹
STEFReaderRead-4                                       463.0 ± 0%    463.0 ± 0%       ~ (p=1.000 n=6) ¹
STEFSerializeMultipart/astronomy-otelmetrics-4        13.15M ± 0%   13.15M ± 0%       ~ (p=0.699 n=6)
STEFDeserializeMultipart/astronomy-otelmetrics-4      1.956k ± 0%   1.956k ± 0%       ~ (p=1.000 n=6)
ReadSTEF-4                                             464.0 ± 0%    464.0 ± 0%       ~ (p=1.000 n=6) ¹
ReadSTEFZ-4                                            501.0 ± 0%    501.0 ± 0%       ~ (p=1.000 n=6) ¹
ReadSTEFZWriteSTEF-4                                  1.231k ± 0%   1.231k ± 0%       ~ (p=1.000 n=6)
geomean                                               6.303k        6.303k       +0.01%
¹ all samples are equal
Benchmark result
benchstat bench-new.txt
goos: linux
goarch: amd64
pkg: github.com/splunk/stef/benchmarks
cpu: AMD EPYC 7763 64-Core Processor                
                                                 │ bench-new.txt │
                                                 │    sec/op     │
SerializeNative/STEF/serialize-4                    7.151m ±  6%
SerializeNative/STEFU/serialize-4                   34.60m ±  2%
DeserializeNative/STEF/deser-4                      2.411m ±  2%
DeserializeNative/STEFU/deser-4                     7.209m ±  0%
SerializeFromPdata/STEF/serialize-4                 137.7m ±  2%
SerializeFromPdata/STEFU/serialize-4                34.31m ±  1%
DeserializeToPdata/STEF/deserialize-4               45.24m ±  1%
DeserializeToPdata/STEFU/deserialize-4              62.30m ±  1%
STEFReaderRead-4                                    2.509m ±  1%
STEFSerializeMultipart/astronomy-otelmetrics-4       3.344 ± 21%
STEFDeserializeMultipart/astronomy-otelmetrics-4    73.36m ± 11%
ReadSTEF-4                                          2.537m ±  3%
ReadSTEFZ-4                                         3.115m ±  0%
ReadSTEFZWriteSTEF-4                                7.072m ±  0%
geomean                                             20.27m

                                                 │ bench-new.txt │
                                                 │   sec/point   │
SerializeNative/STEF/serialize-4                    107.0n ±  6%
SerializeNative/STEFU/serialize-4                   517.5n ±  2%
DeserializeNative/STEF/deser-4                      36.06n ±  1%
DeserializeNative/STEFU/deser-4                     107.8n ±  0%
SerializeFromPdata/STEF/serialize-4                 2.060µ ±  2%
SerializeFromPdata/STEFU/serialize-4                513.3n ±  1%
DeserializeToPdata/STEF/deserialize-4               676.8n ±  1%
DeserializeToPdata/STEFU/deserialize-4              931.8n ±  1%
STEFReaderRead-4                                    37.53n ±  1%
STEFSerializeMultipart/astronomy-otelmetrics-4      4.251µ ± 21%
STEFDeserializeMultipart/astronomy-otelmetrics-4    93.23n ± 11%
ReadSTEF-4                                          37.98n ±  2%
ReadSTEFZ-4                                         46.62n ±  0%
ReadSTEFZWriteSTEF-4                                105.9n ±  0%
geomean                                             213.2n

                                                 │ bench-new.txt │
                                                 │     B/op      │
SerializeNative/STEF/serialize-4                    3.336Mi ± 0%
SerializeNative/STEFU/serialize-4                   7.557Mi ± 0%
DeserializeNative/STEF/deser-4                      951.4Ki ± 0%
DeserializeNative/STEFU/deser-4                     1.715Mi ± 0%
SerializeFromPdata/STEF/serialize-4                 76.55Mi ± 0%
SerializeFromPdata/STEFU/serialize-4                7.557Mi ± 0%
DeserializeToPdata/STEF/deserialize-4               31.99Mi ± 0%
DeserializeToPdata/STEFU/deserialize-4              38.88Mi ± 0%
STEFReaderRead-4                                    953.1Ki ± 0%
STEFSerializeMultipart/astronomy-otelmetrics-4      3.381Gi ± 0%
STEFDeserializeMultipart/astronomy-otelmetrics-4    20.30Mi ± 0%
ReadSTEF-4                                          953.2Ki ± 0%
ReadSTEFZ-4                                         10.29Mi ± 0%
ReadSTEFZWriteSTEF-4                                13.44Mi ± 0%
geomean                                             10.66Mi

                                                 │ bench-new.txt │
                                                 │   allocs/op   │
SerializeNative/STEF/serialize-4                     2.647k ± 0%
SerializeNative/STEFU/serialize-4                     884.5 ± 0%
DeserializeNative/STEF/deser-4                        463.0 ± 0%
DeserializeNative/STEFU/deser-4                       496.0 ± 0%
SerializeFromPdata/STEF/serialize-4                  134.7k ± 0%
SerializeFromPdata/STEFU/serialize-4                  886.0 ± 0%
DeserializeToPdata/STEF/deserialize-4                756.2k ± 0%
DeserializeToPdata/STEFU/deserialize-4               944.9k ± 0%
STEFReaderRead-4                                      463.0 ± 0%
STEFSerializeMultipart/astronomy-otelmetrics-4       13.15M ± 0%
STEFDeserializeMultipart/astronomy-otelmetrics-4     1.956k ± 0%
ReadSTEF-4                                            464.0 ± 0%
ReadSTEFZ-4                                           501.0 ± 0%
ReadSTEFZWriteSTEF-4                                 1.231k ± 0%
geomean                                              6.303k

@tigrannajaryan tigrannajaryan marked this pull request as ready for review February 6, 2026 23:46
@tigrannajaryan
Copy link
Collaborator Author

Merging this one to continue fuzzing.

@tigrannajaryan tigrannajaryan merged commit 7c61aa6 into main Feb 15, 2026
15 checks passed
@tigrannajaryan tigrannajaryan deleted the tigran/fix-profile-bug branch February 15, 2026 23:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

Comments