Semantic field malformation

[morehouse]

Problem

Currently there are several message fields that the IR can only produce (mostly) valid values for:

Message	Field	Why it's always valid
funding_created	funding_txid	derived from the constructed funding outpoint
funding_created	funding_output_index	derived from the constructed funding outpoint (always 0)
funding_created	signature	computed
channel_announcement	all 4 signatures	computed
node_announcement	signature	computed
announcement_signatures	both signatures	computed

For these fields, smite is never able to produce invalid values such as all zeroes, all 0xff bytes, or random data.

This currently prevents smite from finding bugs like the CLN funding_txid==0 assertion crash.

Solution

We can add the ability to selectively malform the bytes of messages before we send them to the target.

Design

New module smite-ir/src/malform.rs.

/// Overwrites one field of an encoded message with the given bytes.
pub struct Malformation {
    /// Byte offset into the encoded message, including the 2-byte type prefix.
    pub offset: u16,
    /// Replacement bytes.
    pub bytes: Vec<u8>,
}

impl Malformation {
    /// Applies the edit in place. Returns `true` if the bytes changed.
    pub fn apply(&self, msg: &mut Vec<u8>) -> bool;
}

The relevant Send* operations (i.e. SendFundingCreated, SendMessage) get a new Option<Malformation> parameter, and the executor applies the malformation if present before sending the corresponding message.

Choosing Malformations

Malformations are limited to the fields listed above since other fields can already be freely modified by mutators. In theory, generators could apply malformations themselves, but in general I think it's better to leave it up to OperationParamMutator.

OperationParamMutator can randomly pick an allowlisted field for the current message type and then generate a sequence of bytes to replace that field with -- either random bytes or a repeated byte biased toward the existing INTERESTING_U8 values. Smaller field replacements could also draw directly from the INTERESTING_U* lists (encoding them in big endian).

Handling Oracles

Currently the executor maintains various oracle state to validate target responses against. If we malform the messages we send to the target, our oracle state can drift from the target's state and lead to false positive Violations.

I think in general the right solution is to update our oracle state based on the messages we actually send rather than the messages we originally constructed. And as long as the malformed messages are still decodable, we should be able to extract the needed values to do this. If the messages do not decode, or if a round trip produces a different byte sequence, then we can set a state_desynced flag in the executor that tells it to make all future Violations log-only and non-fatal. Crashes and hangs remain detectable.

[Chand-ra]

Choosing Malformations

Malformations are limited to the fields listed above since other fields can already be freely modified by mutators. In theory, generators could apply malformations themselves, but in general I think it's better to leave it up to OperationParamMutator.

OperationParamMutator can randomly pick an allowlisted field for the current message type and then generate a sequence of bytes to replace that field with -- either random bytes or a repeated byte biased toward the existing INTERESTING_U8 values. Smaller field replacements could also draw directly from the INTERESTING_U* lists (encoding them in big endian).

I don't think it's right to overload the OperationParamMutator like this. OperationParamMutator is stateless and straightforward: it matches on an Operation and applies a simple semantic tweak (like changing an HTLC amount from 500 to 546 sats). From what I can understand, malforming requires the following steps:

1. Toggling Option<Malformation> from None to Some(...).
2. Looking up the allowlisted field offsets for a specific message type (e.g., funding_txid is at offset 34).
3. Generating corruption payloads.
4. Clearing a malformation (Some -> None) to let the program recover back to a happy path.

If someone adds a new Send operation, OperationParamMutator shouldn't need to know the binary wire offsets of that message type. A dedicated MalformationMutator can own the field allowlists and wire offsets instead.

This should also give the fuzzer stack (and a future mutation scheduler) granular control over how often wire corruption is attempted versus semantic parameter tweaking.

Handling Oracles

Currently the executor maintains various oracle state to validate target responses against. If we malform the messages we send to the target, our oracle state can drift from the target's state and lead to false positive Violations.

I think in general the right solution is to update our oracle state based on the messages we actually send rather than the messages we originally constructed. And as long as the malformed messages are still decodable, we should be able to extract the needed values to do this. If the messages do not decode, or if a round trip produces a different byte sequence, then we can set a state_desynced flag in the executor that tells it to make all future Violations log-only and non-fatal. Crashes and hangs remain detectable.

I believe a big benefit of a separate mutator would be the ability to detect false positives: the target accepted a message that was supposed to fail.

[morehouse]

With the current design, malformations are parameters, so I think it's reasonable for OperationParamMutator to handle the mutations. Especially since much of OperationParamMutator's byte-vector mutation logic should be reusable for malformations. But I also think it can make sense to encapsulate the field offsets somewhere outside of OperationParamMutator, perhaps with the codecs themselves.

I believe a big benefit of a separate mutator would be the ability to detect false positives: the target accepted a message that was supposed to fail.

How will this work? The only way we currently can tell if a message should be rejected is with our existing oracle state and logic. I think we need to rely on that oracle state either way (regardless of whether the mutator is separate from the param mutator or not).

[NishantBansal2003]

| Message | Field | Why it's always valid |
|---|---|---|
| funding_created | funding_txid | derived from the constructed funding outpoint |
| funding_created | funding_output_index | derived from the constructed funding outpoint (always 0) |
| funding_created | signature | computed |
| channel_announcement | all 4 signatures | computed |
| node_announcement | signature | computed |
| announcement_signatures | both signatures | computed |

I think one solution for all of these would be to separate them into distinct operations and variable types. Then, when building a message like funding_created, we can pick from a pool of those operations as inputs. The drawback is that the probability of choosing the correct input would decrease significantly, so we'd have a much lower chance of completing the full flow.

So I think your approach makes sense.

Q: are we going to malform specific fields of the message, with the offsets (of only the above mentioned fields) correctly specified by us and selected by the mutator from a correct pool of offsets?

And IIUC, in the current case, we're only storing funding_txid and funding_output_index, which shouldn't be too hard to update once we build the message, encode it, malform it, decode it, and then extract the updated outpoint. We’ll just need to add a few more checks going forward, for eg, if the txid or vout is invalid, we shouldn't query the chain, etc.

[morehouse]

Q: are we going to malform specific fields of the message, with the offsets (of only the above mentioned fields) correctly specified by us and selected by the mutator from a correct pool of offsets?

Yes, this was my plan to keep malformations scoped to only the fields we can't mutate other ways. In theory we could allow arbitrary offsets and sizes for malformations, but I don't see much benefit to that currently -- we probably end up exercising the message parser more, but we already have the encrypted_bytes scenario for that.

[Chand-ra]

How will this work? The only way we currently can tell if a message should be rejected is with our existing oracle state and logic. I think we need to rely on that oracle state either way (regardless of whether the mutator is separate from the param mutator or not).

Now that I think about it, you're right: the oracle I was thinking of is a result of Malformation being a separate parameter, and has no load bearing on which mutator deals with it.

As for how we detect the false positives with the oracle, I was thinking something along these lines:

if instr.malformation.is_some() {
    // We sent garbage.
    if target_accepted_message_and_advanced() {
        return Err(Violation::AcceptedInvalidSignatureOrTxid);
    } else {
        // Target rejected it or closed the connection -> Expected behavior.
        self.state_desynced = true; 
    }
}

Because the fields we are allowlisting for malformation (funding_txid, signature, etc.) are tied to cryptographic validation, any wire-level corruption of them should always result in a target error.

[morehouse]

It's a little more complicated than that. The target doesn't always immediately know that we sent them garbage. For example, they have no way of knowing whether the txid or funding outpoint is valid until they see the actual tx on-chain and can validate its construction. So it's expected that the target still respond with funding_signed in this case.

But I think the signature idea is a good improvement -- we can record some oracle state that says "target should reject" since we sent an invalid signature. If they don't reject, that's a pretty bad bug.