You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Date and BigInt are the friction people will actually hit. Today the log is
JSON, so a Date comes back as a string and a BigInt throws ERR_ENTRY_NOT_SERIALIZABLE. That is documented, but "serialise deliberately"
only works when the payload shape is yours — it is no help when you are
forwarding a webhook body you did not author.
Why not a pluggable codec
The obvious answer is a codec seam so people can drop in superjson or CBOR. I do
not think that is a feature; I think it is a v2, because it breaks things the
package currently guarantees:
Framing. Recovery depends on records being newline-terminated. CBOR has no
newline framing, so healTail — which scans backwards for the last \n —
stops working.
The repair procedure.docs/durability.md tells you that every line is
independent JSON and gives you a node -e one-liner to find the bad ones. A
binary codec makes that page wrong.
Validation.scanAccounting and writeSurvivors parse JSON lines to
check that sequence numbers increase and to decide what survives compaction.
That is the on-disk format, the recovery path and the operational story, not an
extension point.
The narrower version that fits
replacer and reviver options, passed through to JSON.stringify and JSON.parse:
Framing is untouched — still one JSON object per line — so healing, compaction,
sequence validation and the repair procedure all keep working unchanged. No
runtime dependency either: the caller brings the transform.
Three sharp edges, verified rather than assumed
1. The replacer never sees a Date.toJSON() runs first, so the replacer
receives the ISO string. Detecting a Date requires this[key], which means the
replacer cannot be an arrow function:
valor recibido: string "1970-01-01T00:00:00.000Z" | this[k]: Date real
2. A replacer does fix BigInt. Confirmed — without one JSON.stringify
throws TypeError, with one it produces {"n":{"$bigint":"1"}}.
3. The reviver sees the envelope, not just the value. Parsing a record calls
it with the keys ["seq","d","value",""]. A careless reviver can corrupt seq.
Options: apply it to the whole line and let the existing validation reject a
mangled seq, or double-parse to isolate the value, which costs a round trip on
every read.
Open questions
Does a mismatched replacer/reviver pair across a restart need guarding? Today
the on-disk format is fixed; this would make it caller-dependent, with no
version marker in the file to catch a changed transform against old records.
Is edge 1 acceptable, or does the ergonomic answer need a different shape than
raw JSON.stringify semantics?
Is this worth it at all, given wal.append({ at: date.toISOString() }) already
works for payloads you control?
Not scheduled. Opening it here so the reasoning exists before anyone asks, and
so the codec version gets rejected for a stated reason rather than silently.
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
Uh oh!
There was an error while loading. Please reload this page.
The problem is real
DateandBigIntare the friction people will actually hit. Today the log isJSON, so a
Datecomes back as a string and aBigIntthrowsERR_ENTRY_NOT_SERIALIZABLE. That is documented, but "serialise deliberately"only works when the payload shape is yours — it is no help when you are
forwarding a webhook body you did not author.
Why not a pluggable codec
The obvious answer is a codec seam so people can drop in superjson or CBOR. I do
not think that is a feature; I think it is a v2, because it breaks things the
package currently guarantees:
newline framing, so
healTail— which scans backwards for the last\n—stops working.
docs/durability.mdtells you that every line isindependent JSON and gives you a
node -eone-liner to find the bad ones. Abinary codec makes that page wrong.
scanAccountingandwriteSurvivorsparse JSON lines tocheck that sequence numbers increase and to decide what survives compaction.
That is the on-disk format, the recovery path and the operational story, not an
extension point.
The narrower version that fits
replacerandreviveroptions, passed through toJSON.stringifyandJSON.parse:Framing is untouched — still one JSON object per line — so healing, compaction,
sequence validation and the repair procedure all keep working unchanged. No
runtime dependency either: the caller brings the transform.
Three sharp edges, verified rather than assumed
1. The replacer never sees a
Date.toJSON()runs first, so the replacerreceives the ISO string. Detecting a
Daterequiresthis[key], which means thereplacer cannot be an arrow function:
2. A replacer does fix
BigInt. Confirmed — without oneJSON.stringifythrows
TypeError, with one it produces{"n":{"$bigint":"1"}}.3. The reviver sees the envelope, not just the value. Parsing a record calls
it with the keys
["seq","d","value",""]. A careless reviver can corruptseq.Options: apply it to the whole line and let the existing validation reject a
mangled
seq, or double-parse to isolate the value, which costs a round trip onevery read.
Open questions
the on-disk format is fixed; this would make it caller-dependent, with no
version marker in the file to catch a changed transform against old records.
raw
JSON.stringifysemantics?wal.append({ at: date.toISOString() })alreadyworks for payloads you control?
Not scheduled. Opening it here so the reasoning exists before anyone asks, and
so the codec version gets rejected for a stated reason rather than silently.
All reactions