tcp: mark ISN and timestamp-offset secrets nosave for checkpoint#13164
tcp: mark ISN and timestamp-offset secrets nosave for checkpoint#13164ibondarenko1 wants to merge 1 commit into
Conversation
The TCP protocol struct in pkg/tcpip/transport/tcp/protocol.go is
+stateify savable. seqnumSecret and tsOffsetSecret are 16-byte CSPRNG
secrets used for ISN and timestamp-offset generation (RFC 6528).
Neither field carried the state:"nosave" tag, so both 16-byte values
were written verbatim into the checkpoint state stream by the
generator. A reader of a checkpoint file could recover the secrets
and predict ISNs and timestamp offsets on the restored sandbox.
Sibling fields in the same struct already carry the tag (mu at
protocol.go:92, probe at protocol.go:115). secureRNG itself is
state:"nosave" at pkg/tcpip/stack/stack.go:159. The absence of the
tag on the two secret fields is an omission against the established
convention rather than a design choice.
Tag both fields state:"nosave" and add a protocol.afterLoad hook
that redraws fresh 16-byte secrets from stack.SecureRNG() on restore.
Save: the two fields are omitted from the checkpoint blob, which now
contains neither the bytes nor the field names. Load: stateify
restores the protocol with both fields zeroed, afterLoad reseeds.
Adds two regression tests in protocol_state_test.go: a reflection
check that the tag is present, and a runtime check that afterLoad
repopulates zero-valued fields with fresh non-zero bytes drawn from
the secure RNG.
Tested:
bazel build //pkg/tcpip/transport/tcp:tcp
bazel test //pkg/tcpip/transport/tcp:tcp_test \
--test_filter='TestProtocolSecretsHaveNosaveTag|TestProtocolAfterLoadRegeneratesSecrets'
Related: CVE-2024-10026 and the NDSS 2025 paper "You Can Rand but You
Can't Hide" (Kaplan, Even, Klein) improved the cryptographic quality
of these same secrets (commits 83f7508, e54bfde, f956b5a)
but did not address persistence into checkpoint state.
## Summary
`pkg/tcpip/transport/tcp/protocol.go` is annotated `+stateify savable`. Two of its fields are 16-byte CSPRNG secrets used for ISN and timestamp-offset generation (RFC 6528):
```go
// protocol.go:88-119 (before this PR)
// +stateify savable
type protocol struct {
...
// The following secrets are initialized once and stay unchanged after.
seqnumSecret [16]byte
tsOffsetSecret [16]byte
}
```
Neither field carries `state:"nosave"`. The stateify generator therefore writes both 16-byte values verbatim into the checkpoint state stream. A reader of a checkpoint file recovers the secrets and predicts ISNs and timestamp offsets on the restored sandbox.
## Background
Sibling fields in the same struct already carry the tag:
- `protocol.go:92`: `mu protocolRWMutex` with `state:"nosave"`
- `protocol.go:115`: `probe TCPProbeFunc` with `state:"nosave"`
The source of randomness itself, `secureRNG`, is also `state:"nosave"` at `pkg/tcpip/stack/stack.go:159`. The absence of the tag on `seqnumSecret` / `tsOffsetSecret` is an omission against the established codebase convention, not a deliberate choice.
## Linux / BSD reference
Linux stores its ISN secret (`net_secret` in `net/core/secure_seq.c`) in kernel memory per netns; FreeBSD and OpenBSD store theirs in `tcp_subr.c` kernel memory. None of those stacks have a checkpoint/restore equivalent. CRIU does not dump kernel memory, so the Linux ISN secret is not exposed through the closest analog. The checkpoint surface is unique to gVisor save-restore.
CVE-2024-10026 and the NDSS 2025 paper "You Can Rand but You Can't Hide" (Kaplan, Even, Klein; commits `83f75082e5`, `e54bfde792`, `f956b5ac17`, fixed in `v20231204.0.0`) improved the cryptographic quality of these same secrets. Persistence into checkpoint state was not addressed by that work.
## Change
After this PR, `protocol.go:114-120` reads:
```go
probe TCPProbeFunc `state:"nosave"`
// The following secrets are used for ISN and timestamp-offset
// generation. They are not serialized into checkpoint state and are
// freshly drawn from the secure RNG on restore (see afterLoad).
seqnumSecret [16]byte `state:"nosave"`
tsOffsetSecret [16]byte `state:"nosave"`
```
New file `pkg/tcpip/transport/tcp/protocol_state.go` adds an `afterLoad` hook that redraws both secrets from `p.stack.SecureRNG()` on restore.
Save: stateify omits both fields from the checkpoint blob, which now contains neither the bytes nor the field names.
Load: stateify restores the protocol with both fields zeroed; `afterLoad` reseeds from `stack.SecureRNG()`.
Existing live behavior at `newProtocol()` (initial seeding at construction) is unchanged.
## Tests
New file `pkg/tcpip/transport/tcp/protocol_state_test.go` adds two regression tests:
- `TestProtocolSecretsHaveNosaveTag` is a reflection-based structural check that both secret fields carry `state:"nosave"`. Removing the tag in a future change would fail this test.
- `TestProtocolAfterLoadRegeneratesSecrets` zeroes the two secrets (simulating the post-restore state for `nosave` fields), invokes `afterLoad`, and asserts both fields become non-zero and differ from the initial values.
```
$ bazel test //pkg/tcpip/transport/tcp:tcp_test --test_filter='TestProtocolSecretsHaveNosaveTag|TestProtocolAfterLoadRegeneratesSecrets'
--- PASS: TestProtocolSecretsHaveNosaveTag (0.00s)
--- PASS: TestProtocolAfterLoadRegeneratesSecrets (0.00s)
PASS
```
`bazel build //pkg/tcpip/transport/tcp:tcp` builds clean against `master`.
## Framing
This is hardening, not an advisory. Disclosure requires read access to a checkpoint file, which is host-side and orchestrator-controlled (`g3doc/user_guide/checkpoint_restore.md`). The fix is minimal, the convention already exists in the same file, and the patch closes a remaining persistence surface adjacent to the CVE-2024-10026 / NDSS 2025 secret-quality work.
## References
- RFC 6528 (Defending against Sequence Number Attacks, Gont/Bellovin, 2012)
- CVE-2024-10026 (NVD)
- NDSS 2025: Kaplan, Even, Klein. "You Can Rand but You Can't Hide."
- gVisor `g3doc/user_guide/checkpoint_restore.md`
- Sibling `state:"nosave"`: `pkg/tcpip/stack/stack.go:159`, `pkg/tcpip/transport/tcp/protocol.go:92,115`.
FUTURE_COPYBARA_INTEGRATE_REVIEW=#13164 from ibondarenko1:hardening/tcp-isn-secrets-nosave b027f39
PiperOrigin-RevId: 918043127
## Summary
`pkg/tcpip/transport/tcp/protocol.go` is annotated `+stateify savable`. Two of its fields are 16-byte CSPRNG secrets used for ISN and timestamp-offset generation (RFC 6528):
```go
// protocol.go:88-119 (before this PR)
// +stateify savable
type protocol struct {
...
// The following secrets are initialized once and stay unchanged after.
seqnumSecret [16]byte
tsOffsetSecret [16]byte
}
```
Neither field carries `state:"nosave"`. The stateify generator therefore writes both 16-byte values verbatim into the checkpoint state stream. A reader of a checkpoint file recovers the secrets and predicts ISNs and timestamp offsets on the restored sandbox.
## Background
Sibling fields in the same struct already carry the tag:
- `protocol.go:92`: `mu protocolRWMutex` with `state:"nosave"`
- `protocol.go:115`: `probe TCPProbeFunc` with `state:"nosave"`
The source of randomness itself, `secureRNG`, is also `state:"nosave"` at `pkg/tcpip/stack/stack.go:159`. The absence of the tag on `seqnumSecret` / `tsOffsetSecret` is an omission against the established codebase convention, not a deliberate choice.
## Linux / BSD reference
Linux stores its ISN secret (`net_secret` in `net/core/secure_seq.c`) in kernel memory per netns; FreeBSD and OpenBSD store theirs in `tcp_subr.c` kernel memory. None of those stacks have a checkpoint/restore equivalent. CRIU does not dump kernel memory, so the Linux ISN secret is not exposed through the closest analog. The checkpoint surface is unique to gVisor save-restore.
CVE-2024-10026 and the NDSS 2025 paper "You Can Rand but You Can't Hide" (Kaplan, Even, Klein; commits `83f75082e5`, `e54bfde792`, `f956b5ac17`, fixed in `v20231204.0.0`) improved the cryptographic quality of these same secrets. Persistence into checkpoint state was not addressed by that work.
## Change
After this PR, `protocol.go:114-120` reads:
```go
probe TCPProbeFunc `state:"nosave"`
// The following secrets are used for ISN and timestamp-offset
// generation. They are not serialized into checkpoint state and are
// freshly drawn from the secure RNG on restore (see afterLoad).
seqnumSecret [16]byte `state:"nosave"`
tsOffsetSecret [16]byte `state:"nosave"`
```
New file `pkg/tcpip/transport/tcp/protocol_state.go` adds an `afterLoad` hook that redraws both secrets from `p.stack.SecureRNG()` on restore.
Save: stateify omits both fields from the checkpoint blob, which now contains neither the bytes nor the field names.
Load: stateify restores the protocol with both fields zeroed; `afterLoad` reseeds from `stack.SecureRNG()`.
Existing live behavior at `newProtocol()` (initial seeding at construction) is unchanged.
## Tests
New file `pkg/tcpip/transport/tcp/protocol_state_test.go` adds two regression tests:
- `TestProtocolSecretsHaveNosaveTag` is a reflection-based structural check that both secret fields carry `state:"nosave"`. Removing the tag in a future change would fail this test.
- `TestProtocolAfterLoadRegeneratesSecrets` zeroes the two secrets (simulating the post-restore state for `nosave` fields), invokes `afterLoad`, and asserts both fields become non-zero and differ from the initial values.
```
$ bazel test //pkg/tcpip/transport/tcp:tcp_test --test_filter='TestProtocolSecretsHaveNosaveTag|TestProtocolAfterLoadRegeneratesSecrets'
--- PASS: TestProtocolSecretsHaveNosaveTag (0.00s)
--- PASS: TestProtocolAfterLoadRegeneratesSecrets (0.00s)
PASS
```
`bazel build //pkg/tcpip/transport/tcp:tcp` builds clean against `master`.
## Framing
This is hardening, not an advisory. Disclosure requires read access to a checkpoint file, which is host-side and orchestrator-controlled (`g3doc/user_guide/checkpoint_restore.md`). The fix is minimal, the convention already exists in the same file, and the patch closes a remaining persistence surface adjacent to the CVE-2024-10026 / NDSS 2025 secret-quality work.
## References
- RFC 6528 (Defending against Sequence Number Attacks, Gont/Bellovin, 2012)
- CVE-2024-10026 (NVD)
- NDSS 2025: Kaplan, Even, Klein. "You Can Rand but You Can't Hide."
- gVisor `g3doc/user_guide/checkpoint_restore.md`
- Sibling `state:"nosave"`: `pkg/tcpip/stack/stack.go:159`, `pkg/tcpip/transport/tcp/protocol.go:92,115`.
FUTURE_COPYBARA_INTEGRATE_REVIEW=#13164 from ibondarenko1:hardening/tcp-isn-secrets-nosave b027f39
PiperOrigin-RevId: 918043127
## Summary
`pkg/tcpip/transport/tcp/protocol.go` is annotated `+stateify savable`. Two of its fields are 16-byte CSPRNG secrets used for ISN and timestamp-offset generation (RFC 6528):
```go
// protocol.go:88-119 (before this PR)
// +stateify savable
type protocol struct {
...
// The following secrets are initialized once and stay unchanged after.
seqnumSecret [16]byte
tsOffsetSecret [16]byte
}
```
Neither field carries `state:"nosave"`. The stateify generator therefore writes both 16-byte values verbatim into the checkpoint state stream. A reader of a checkpoint file recovers the secrets and predicts ISNs and timestamp offsets on the restored sandbox.
## Background
Sibling fields in the same struct already carry the tag:
- `protocol.go:92`: `mu protocolRWMutex` with `state:"nosave"`
- `protocol.go:115`: `probe TCPProbeFunc` with `state:"nosave"`
The source of randomness itself, `secureRNG`, is also `state:"nosave"` at `pkg/tcpip/stack/stack.go:159`. The absence of the tag on `seqnumSecret` / `tsOffsetSecret` is an omission against the established codebase convention, not a deliberate choice.
## Linux / BSD reference
Linux stores its ISN secret (`net_secret` in `net/core/secure_seq.c`) in kernel memory per netns; FreeBSD and OpenBSD store theirs in `tcp_subr.c` kernel memory. None of those stacks have a checkpoint/restore equivalent. CRIU does not dump kernel memory, so the Linux ISN secret is not exposed through the closest analog. The checkpoint surface is unique to gVisor save-restore.
CVE-2024-10026 and the NDSS 2025 paper "You Can Rand but You Can't Hide" (Kaplan, Even, Klein; commits `83f75082e5`, `e54bfde792`, `f956b5ac17`, fixed in `v20231204.0.0`) improved the cryptographic quality of these same secrets. Persistence into checkpoint state was not addressed by that work.
## Change
After this PR, `protocol.go:114-120` reads:
```go
probe TCPProbeFunc `state:"nosave"`
// The following secrets are used for ISN and timestamp-offset
// generation. They are not serialized into checkpoint state and are
// freshly drawn from the secure RNG on restore (see afterLoad).
seqnumSecret [16]byte `state:"nosave"`
tsOffsetSecret [16]byte `state:"nosave"`
```
New file `pkg/tcpip/transport/tcp/protocol_state.go` adds an `afterLoad` hook that redraws both secrets from `p.stack.SecureRNG()` on restore.
Save: stateify omits both fields from the checkpoint blob, which now contains neither the bytes nor the field names.
Load: stateify restores the protocol with both fields zeroed; `afterLoad` reseeds from `stack.SecureRNG()`.
Existing live behavior at `newProtocol()` (initial seeding at construction) is unchanged.
## Tests
New file `pkg/tcpip/transport/tcp/protocol_state_test.go` adds two regression tests:
- `TestProtocolSecretsHaveNosaveTag` is a reflection-based structural check that both secret fields carry `state:"nosave"`. Removing the tag in a future change would fail this test.
- `TestProtocolAfterLoadRegeneratesSecrets` zeroes the two secrets (simulating the post-restore state for `nosave` fields), invokes `afterLoad`, and asserts both fields become non-zero and differ from the initial values.
```
$ bazel test //pkg/tcpip/transport/tcp:tcp_test --test_filter='TestProtocolSecretsHaveNosaveTag|TestProtocolAfterLoadRegeneratesSecrets'
--- PASS: TestProtocolSecretsHaveNosaveTag (0.00s)
--- PASS: TestProtocolAfterLoadRegeneratesSecrets (0.00s)
PASS
```
`bazel build //pkg/tcpip/transport/tcp:tcp` builds clean against `master`.
## Framing
This is hardening, not an advisory. Disclosure requires read access to a checkpoint file, which is host-side and orchestrator-controlled (`g3doc/user_guide/checkpoint_restore.md`). The fix is minimal, the convention already exists in the same file, and the patch closes a remaining persistence surface adjacent to the CVE-2024-10026 / NDSS 2025 secret-quality work.
## References
- RFC 6528 (Defending against Sequence Number Attacks, Gont/Bellovin, 2012)
- CVE-2024-10026 (NVD)
- NDSS 2025: Kaplan, Even, Klein. "You Can Rand but You Can't Hide."
- gVisor `g3doc/user_guide/checkpoint_restore.md`
- Sibling `state:"nosave"`: `pkg/tcpip/stack/stack.go:159`, `pkg/tcpip/transport/tcp/protocol.go:92,115`.
FUTURE_COPYBARA_INTEGRATE_REVIEW=#13164 from ibondarenko1:hardening/tcp-isn-secrets-nosave b027f39
PiperOrigin-RevId: 918043127
## Summary
`pkg/tcpip/transport/tcp/protocol.go` is annotated `+stateify savable`. Two of its fields are 16-byte CSPRNG secrets used for ISN and timestamp-offset generation (RFC 6528):
```go
// protocol.go:88-119 (before this PR)
// +stateify savable
type protocol struct {
...
// The following secrets are initialized once and stay unchanged after.
seqnumSecret [16]byte
tsOffsetSecret [16]byte
}
```
Neither field carries `state:"nosave"`. The stateify generator therefore writes both 16-byte values verbatim into the checkpoint state stream. A reader of a checkpoint file recovers the secrets and predicts ISNs and timestamp offsets on the restored sandbox.
## Background
Sibling fields in the same struct already carry the tag:
- `protocol.go:92`: `mu protocolRWMutex` with `state:"nosave"`
- `protocol.go:115`: `probe TCPProbeFunc` with `state:"nosave"`
The source of randomness itself, `secureRNG`, is also `state:"nosave"` at `pkg/tcpip/stack/stack.go:159`. The absence of the tag on `seqnumSecret` / `tsOffsetSecret` is an omission against the established codebase convention, not a deliberate choice.
## Linux / BSD reference
Linux stores its ISN secret (`net_secret` in `net/core/secure_seq.c`) in kernel memory per netns; FreeBSD and OpenBSD store theirs in `tcp_subr.c` kernel memory. None of those stacks have a checkpoint/restore equivalent. CRIU does not dump kernel memory, so the Linux ISN secret is not exposed through the closest analog. The checkpoint surface is unique to gVisor save-restore.
CVE-2024-10026 and the NDSS 2025 paper "You Can Rand but You Can't Hide" (Kaplan, Even, Klein; commits `83f75082e5`, `e54bfde792`, `f956b5ac17`, fixed in `v20231204.0.0`) improved the cryptographic quality of these same secrets. Persistence into checkpoint state was not addressed by that work.
## Change
After this PR, `protocol.go:114-120` reads:
```go
probe TCPProbeFunc `state:"nosave"`
// The following secrets are used for ISN and timestamp-offset
// generation. They are not serialized into checkpoint state and are
// freshly drawn from the secure RNG on restore (see afterLoad).
seqnumSecret [16]byte `state:"nosave"`
tsOffsetSecret [16]byte `state:"nosave"`
```
New file `pkg/tcpip/transport/tcp/protocol_state.go` adds an `afterLoad` hook that redraws both secrets from `p.stack.SecureRNG()` on restore.
Save: stateify omits both fields from the checkpoint blob, which now contains neither the bytes nor the field names.
Load: stateify restores the protocol with both fields zeroed; `afterLoad` reseeds from `stack.SecureRNG()`.
Existing live behavior at `newProtocol()` (initial seeding at construction) is unchanged.
## Tests
New file `pkg/tcpip/transport/tcp/protocol_state_test.go` adds two regression tests:
- `TestProtocolSecretsHaveNosaveTag` is a reflection-based structural check that both secret fields carry `state:"nosave"`. Removing the tag in a future change would fail this test.
- `TestProtocolAfterLoadRegeneratesSecrets` zeroes the two secrets (simulating the post-restore state for `nosave` fields), invokes `afterLoad`, and asserts both fields become non-zero and differ from the initial values.
```
$ bazel test //pkg/tcpip/transport/tcp:tcp_test --test_filter='TestProtocolSecretsHaveNosaveTag|TestProtocolAfterLoadRegeneratesSecrets'
--- PASS: TestProtocolSecretsHaveNosaveTag (0.00s)
--- PASS: TestProtocolAfterLoadRegeneratesSecrets (0.00s)
PASS
```
`bazel build //pkg/tcpip/transport/tcp:tcp` builds clean against `master`.
## Framing
This is hardening, not an advisory. Disclosure requires read access to a checkpoint file, which is host-side and orchestrator-controlled (`g3doc/user_guide/checkpoint_restore.md`). The fix is minimal, the convention already exists in the same file, and the patch closes a remaining persistence surface adjacent to the CVE-2024-10026 / NDSS 2025 secret-quality work.
## References
- RFC 6528 (Defending against Sequence Number Attacks, Gont/Bellovin, 2012)
- CVE-2024-10026 (NVD)
- NDSS 2025: Kaplan, Even, Klein. "You Can Rand but You Can't Hide."
- gVisor `g3doc/user_guide/checkpoint_restore.md`
- Sibling `state:"nosave"`: `pkg/tcpip/stack/stack.go:159`, `pkg/tcpip/transport/tcp/protocol.go:92,115`.
FUTURE_COPYBARA_INTEGRATE_REVIEW=#13164 from ibondarenko1:hardening/tcp-isn-secrets-nosave b027f39
PiperOrigin-RevId: 918574087
|
If that is the failure, I can rework the hook: guard against an unready RNG, or move the reseed to a point where the stack RNG is guaranteed available. If a maintainer can share what the internal check reported, I'll turn a fix around quickly. |
Summary
pkg/tcpip/transport/tcp/protocol.gois annotated+stateify savable. Two of its fields are 16-byte CSPRNG secrets used for ISN and timestamp-offset generation (RFC 6528):Neither field carries
state:"nosave". The stateify generator therefore writes both 16-byte values verbatim into the checkpoint state stream. A reader of a checkpoint file recovers the secrets and predicts ISNs and timestamp offsets on the restored sandbox.Background
Sibling fields in the same struct already carry the tag:
protocol.go:92:mu protocolRWMutexwithstate:"nosave"protocol.go:115:probe TCPProbeFuncwithstate:"nosave"The source of randomness itself,
secureRNG, is alsostate:"nosave"atpkg/tcpip/stack/stack.go:159. The absence of the tag onseqnumSecret/tsOffsetSecretis an omission against the established codebase convention, not a deliberate choice.Linux / BSD reference
Linux stores its ISN secret (
net_secretinnet/core/secure_seq.c) in kernel memory per netns; FreeBSD and OpenBSD store theirs intcp_subr.ckernel memory. None of those stacks have a checkpoint/restore equivalent. CRIU does not dump kernel memory, so the Linux ISN secret is not exposed through the closest analog. The checkpoint surface is unique to gVisor save-restore.CVE-2024-10026 and the NDSS 2025 paper "You Can Rand but You Can't Hide" (Kaplan, Even, Klein; commits
83f75082e5,e54bfde792,f956b5ac17, fixed inv20231204.0.0) improved the cryptographic quality of these same secrets. Persistence into checkpoint state was not addressed by that work.Change
After this PR,
protocol.go:114-120reads:New file
pkg/tcpip/transport/tcp/protocol_state.goadds anafterLoadhook that redraws both secrets fromp.stack.SecureRNG()on restore.Save: stateify omits both fields from the checkpoint blob, which now contains neither the bytes nor the field names.
Load: stateify restores the protocol with both fields zeroed;
afterLoadreseeds fromstack.SecureRNG().Existing live behavior at
newProtocol()(initial seeding at construction) is unchanged.Tests
New file
pkg/tcpip/transport/tcp/protocol_state_test.goadds two regression tests:TestProtocolSecretsHaveNosaveTagis a reflection-based structural check that both secret fields carrystate:"nosave". Removing the tag in a future change would fail this test.TestProtocolAfterLoadRegeneratesSecretszeroes the two secrets (simulating the post-restore state fornosavefields), invokesafterLoad, and asserts both fields become non-zero and differ from the initial values.bazel build //pkg/tcpip/transport/tcp:tcpbuilds clean againstmaster.Framing
This is hardening, not an advisory. Disclosure requires read access to a checkpoint file, which is host-side and orchestrator-controlled (
g3doc/user_guide/checkpoint_restore.md). The fix is minimal, the convention already exists in the same file, and the patch closes a remaining persistence surface adjacent to the CVE-2024-10026 / NDSS 2025 secret-quality work.References
g3doc/user_guide/checkpoint_restore.mdstate:"nosave":pkg/tcpip/stack/stack.go:159,pkg/tcpip/transport/tcp/protocol.go:92,115.