sysio.msig: chunked storage for large proposals + read-only getproposal

[heifner]

Summary

Wire's per-row KV value limit is 256 KiB but a direct setcode transaction can carry roughly 524287 bytes (max_transaction_net_usage), so post-#291 sysio.msig setcode regressed to a smaller ceiling than non-msig setcode. This PR restores parity by chunking the serialized inner trx across rows of a new propchunks table, reassembling it on exec, and exposing a read-only getproposal action so external tooling can fetch the assembled proposal regardless of how the bytes are physically laid out on disk.

No chain-config changes are required: each individual dispatched action is still bounded by max_inline_action_size, and propose still has to fit inside one outer transaction. This PR purely lifts the storage ceiling so msig setcode matches direct setcode at current chain limits.

Depends on:

• Migrate production contracts from multi_index/singleton to kv::table #291 — the KV-table msig migration this builds on
• KV table_id namespace isolation wire-cdt#49 — [[sysio::action, sysio::read_only]] works on non-trivial bodies (false-positive validator fix)

Storage layout

The proposal row gains three appended binary_extension fields. Old shape stays bit-for-bit identical at the head, so external tooling that reads the row directly via get_table_rows keeps working for small proposals.

struct proposal {
   name                                                proposal_name;
   std::vector<char>                                   packed_transaction; // empty when chunked
   binary_extension<std::optional<time_point>>         earliest_exec_time;
   binary_extension<uint32_t>                          chunk_count;        // 0 / absent => not chunked
   binary_extension<uint32_t>                          total_size;
   binary_extension<sysio::checksum256>                trx_hash;           // sha256 of assembled blob
};

struct propchunk {                                                        // new scoped table
   name              proposal_name;
   uint32_t          chunk_index;
   std::vector<char> data;
};

• Inner trx ≤ proposal_chunk_size (200 KiB): legacy single-row layout, packed_transaction populated, chunk_count = 0. Same wire format as today plus three trailing extension fields. msig.app / cleos / bloks.io render unchanged.
• Inner trx > 200 KiB: packed_transaction is empty, bytes split across N propchunks rows keyed by (proposal_name, chunk_index). Tooling has to call getproposal to retrieve the assembled blob.

approve --proposal-hash keeps working for both shapes: the inline path uses assert_sha256 against packed_transaction (the historical semantic), and the chunked path checks against the precomputed trx_hash stored at propose time, so chunks don't have to be reassembled on every approve.

On-chain changes

• propose stores sysio::sha256(trx_pos, size) in trx_hash and either keeps the bytes inline (≤ 200 KiB) or splits across N propchunks rows.
• exec reassembles via assemble_packed_trx once into WASM linear memory, deserializes, dispatches each action via act.send() exactly as before. After successful exec, erases the chunk rows.
• approve / unapprove read the trx header from chunk 0 alone (not the full blob) so the common case stays cheap. The full reassembly only fires once per proposal — when it transitions to "fully approved" and earliest_exec_time is being set.
• cancel erases all chunk rows after erasing the parent proposal. Together with exec's cleanup this guarantees no orphaned RAM.
• New get_proposal action: [[sysio::action(\"getproposal\"), sysio::read_only]] proposal get_proposal(name proposer, name proposal_name);. Returns the assembled struct with packed_transaction populated. CDT auto-generates set_action_return_value for non-void actions, and the chain skips the max_action_return_value_size cap in read-only context, so the full ~512 KiB blob comes back via the trace's action_traces[0].return_value.

clio changes ⭐ (please pick up for clio docs)

Two clio updates land in this PR — both worth documenting and both generic enough to benefit any future contract / use case, not just msig.

1. multisig review now uses getproposal via send_read_only_transaction

Previously clio multisig review reached into the proposal table directly via /v1/chain/get_table_rows and decoded the packed_transaction field locally. After this PR it builds a one-action transaction calling sysio.msig::getproposal(proposer, proposal_name), posts it to /v1/chain/send_read_only_transaction, and pulls the assembled proposal struct from processed.action_traces[0].return_value_data (ABI-decoded by chain_plugin via the new action_results entry CDT generates).

Why it matters for docs:

• Storage-layout-agnostic. Clients depend on a stable action surface, not on table internals. If sysio.msig refactors its tables again — as it just did from multi_index to kv::scoped_table and now to chunked storage — multisig review keeps working as long as the getproposal signature is preserved. get_table_rows clients have to relearn the schema every time.
• Avoids the field-name brittleness from Migrate production contracts from multi_index/singleton to kv::table #291. The legacy review code matched ABI field names like primary_key / account / proposal_name literally; that's exactly the brittleness that bit us in Migrate production contracts from multi_index/singleton to kv::table #291 when the underlying ABI shape changed. The new path is shape-stable.
• This is the recommended pattern for new clio commands going forward. Read-only contract actions are the supported cross-fork retrieval API. Wire's design principle: contracts expose read-only "view" actions; clients call those instead of touching tables.
• Operator note: the API node must have --read-only-threads N set (with N >= 1); otherwise multisig review fails with unsupported_feature: read-only transactions execution not enabled on API node. The producer node rejects this flag, so the standard topology is producer + non-producer API node, with review traffic routed at the API node.
• --show-approvals is unchanged: it still uses get_table_rows for approvals2 / approvals / invals because those tables don't have any storage indirection.

2. multisig propose_trx accepts structured action.data

Previously multisig propose_trx only accepted a transaction JSON whose action.data fields were already hex-encoded (because the propose action's ABI types trx as transaction, and the chain transaction ABI types action.data as bytes). Passing a structured object — the natural shape, the same one clio push transaction already accepts — produced Bad Cast (7) Invalid cast from type 'object_type' to string.

This PR adds the same try/catch fallback that push transaction already had at main.cpp:3145:

try {
   (void)trx_var.as<transaction>();              // legacy hex-data path
} catch (const std::exception&) {
   transaction trx;
   abi_serializer::from_variant(trx_var, trx,    // ABI-resolver-aware decode
                                abi_serializer_resolver,
                                abi_serializer::create_yield_function(...));
   trx_var = trx;                                // re-serialize with hex action data
}

Why it matters for docs:

• Strict superset of the old behavior. Old hex-data callers keep working unchanged (the literal cast succeeds, the catch never runs). New callers get to write transactions naturally, with structured action data, and clio recursively encodes via each contract's ABI on the fly.
• No more clio convert pack_action_data dance. Users who today pre-encode each inner action's data via clio convert pack_action_data sysio setcode '{...}' and splice the resulting hex into the trx JSON can drop that step entirely.
• Documentation should show the natural shape:

  {
    \"actions\": [{
      \"account\": \"sysio\",
      \"name\": \"setcode\",
      \"authorization\": [{\"actor\": \"alice\", \"permission\": \"active\"}],
      \"data\": { \"account\": \"alice\", \"vmtype\": 0, \"vmversion\": 0, \"code\": \"<hex>\" }
    }]
  }

• Generic improvement. This benefits anyone using multisig propose_trx with any contract, not just msig setcode.

Tests

contracts/tests/sysio.msig_tests.cpp

Three new cases under sysio_msig_tests, all running against default chain config (no setparams bumps):

• big_transaction_chunked — builds a >200 KiB inner trx (two setcode actions of sysio.system.wasm to alice and bob), proposes, exercises the chunked-path --proposal-hash check (correct hash succeeds, wrong hash rejected against the stored trx_hash), approves with both signers, execs end-to-end. Verifies the trace contains the dispatched setcode actions in order.
• getproposal_read_only_returns_assembled — pushes a read-only trx invoking getproposal, ABI-decodes the action_trace return value via the cached msig abi_serializer, asserts that the reassembled packed_transaction byte-equals the original fc::raw::pack(trx), and that chunk_count > 0 and total_size matches.
• cancel_chunked_proposal_cleans_chunks — proposes a chunked proposal, cancels (as proposer, no expiration wait), re-proposes under the same name, approves and execs. Would fail with "key already exists" on the second propose if cancel didn't erase the chunk rows.

tests/multisig_review_test.py

New Python integration test (add_np_test multisig_review_test) that exercises the full clio path end-to-end:

1. Stands up a producer + non-producer API node cluster with --read-only-threads 1 on the API node.
2. Creates alice / bob, deploys the freshly built sysio.msig wasm.
3. Submits a chunked propose via clio multisig propose_trx with structured (non-hex) action data — exercises the new clio fallback.
4. Waits for the propose trx to be in a block on the API node.
5. Runs clio multisig review against the API node (so it actually uses send_read_only_transaction → getproposal).
6. Validates the JSON output: chunk_count > 0, total_size > 200 KiB, trx_hash populated, two setcode actions present, and packed_transaction non-empty in the review output (which proves clio is going through the read-only path rather than echoing the empty inline row).
7. Runs multisig review --show-approvals, verifies both signers show as unapproved and the approvals/invals fetch path is intact.

clio review output shape (non-breaking addition)

clio multisig review now exposes three additional top-level fields in its JSON output, copied through from the proposal struct that sysio.msig::getproposal returns:

• trx_hash — the precomputed sha256(packed_transaction) stored by the contract at propose time. Most useful: users who pass --proposal-hash to clio multisig approve can now read this directly from review output instead of computing sha256(packed_transaction) themselves.
• chunk_count — 0 for inline proposals, > 0 for chunked. Tells the user at a glance whether this proposal is stored inline or split across the propchunks table.
• total_size — size in bytes of the assembled serialized inner transaction.

Purely additive — no fields removed, no types changed, no renames. Scrapers using lenient JSON parsers (jq, Python's json.load) keep working. Only strict-schema parsers that reject unknown keys are affected, and those would already need to adapt to the fork's additions to the proposal struct (like earliest_exec_time before this PR).

Review follow-up commit

A review pass after the initial commit produced a follow-up (sysio.msig review fixes) addressing:

• assemble_packed_trx now consumes proposal&& — moves the inline blob out via std::move(prop.packed_transaction) instead of copying. NRVO doesn't apply to returning a const-lvalue member, so the previous shape paid a ~200 KiB heap copy on every inline approve/unapprove/exec auth recheck. New small helper read_proposal_chunks holds the chunked-reading logic and is called directly by get_proposal (which preserves prop for return, avoiding moved-from-then-write subtleties).
• erase_proposal_chunks takes primitives — (uint32_t chunk_count, name proposal_name, name self, name proposer) instead of a const proposal&, so exec and cancel can call it after the proposal has been moved into assemble_packed_trx.
• static_assert(proposal_chunk_size >= 128) — defense against anyone reducing the constant below the size of a serialized transaction_header, which would silently break read_trx_header's chunk-0 fast path.
• clio exception handling tightened — replaced catch(...) { "Proposal not found"; } with catch(const fc::exception&) that only friendly-prints the literal contract assertion and rethrows everything else. Network errors, server errors, parse failures, and unrelated contract assertions now reach clio's main handler with their real messages. The missing-RPC error message now mentions --read-only-threads N and the producer-node restriction so operators don't have to grep nodeop's source.
• clio JSON envelope walk wrapped in try/catch — catches fc::exception (covers both bad_cast_exception and key_not_found_exception) and prints the raw response JSON on failure so users can diagnose what the node actually returned. proposal_object is carried out of the try via a const fc::variant_object* pointer so the downstream code stays outside the try block and no ~200 KiB copy of the packed_transaction hex is made.
• Three new tests — getproposal_read_only_inline (the inline branch of get_proposal), unapprove_chunked_past_threshold (the chunked-trx-recheck path in unapprove), cancel_chunked_by_non_proposer_past_expiration (the chunk-0 read_trx_header fast path from cancel's non-proposer branch, computed against a near-future expiration and advanced past it with produce_blocks(20)).
• Misc contract cleanups — dropped unnecessary (char*) casts on memcpy(...), replaced a ternary chunk-length calc with std::min, moved the get_proposal action declaration and getproposal_action wrapper from between propchunks and old_approval_key down to after invalidations so all tables group above the single read-only action at the bottom. sysio.msig.abi is unchanged by the reordering.

Pre-commit sweep: contracts_unit_test 120/120, plugin_test clean, unit_test --run_test=read_only_trx_tests 14/14, multisig_review_test.py end-to-end integration test passes.