Emissions: configurable cadence + epoch-frequency-agnostic gate#281
Open
heifner wants to merge 76 commits into
Open
Emissions: configurable cadence + epoch-frequency-agnostic gate#281heifner wants to merge 76 commits into
heifner wants to merge 76 commits into
Conversation
Port emissions from develop branch with key improvement: all emission parameters are configurable via setemitcfg action instead of hard-coded constants. Includes node owner tiered distributions, epoch-based T5 treasury emissions with decay, performance-based producer pay, and category splits (compute, capital, capex, governance). New actions: setemitcfg, setinittime, addnodeowner, claimnodedis, viewnodedist, initt5, processepoch, viewepoch, viewemitcfg. Producer round tracking added to onblock for performance-based rewards. ROA contract updated to call addnodeowner on node registration. 75 emission tests plus ROA test updates for emission config setup.
The emission config singleton must exist before regnodeowner can call addnodeowner. Insert setemitcfg with default parameters into the Cluster.bootstrap() sequence before activateroa/forcereg.
undo_index::post_modify handles AVL tree rebalancing when composite index key fields change. This avoids node deallocation, fresh node allocation, and reinsertion into all 3 AVL trees per secondary index update. Added test proving modify correctly rekeys across indices.
92-commit merge bringing in the kv migration of sysio.system tables (PR #291) and the OPP contract family -- sysio.opreg, sysio.epoch, sysio.msgch, sysio.chalg, sysio.uwrit (PR #303). Conflicts resolved: - contracts/sysio.system/src/producer_pay.cpp: layer this branch's eligible_rounds tracking onto master's kv::table API (producer_key_t, contains/modify(same_payer, key, ...)) - contracts/sysio.system/sysio.system.abi, contracts/sysio.system/sysio.system.wasm, contracts/sysio.roa/sysio.roa.wasm: take master's versions; will be regenerated by CDT after subsequent commits migrate emissions to kv. emissions.hpp/emissions.cpp/emissions_tests.cpp retained on this branch and will be rewritten on kv in the following commits.
Rewrite contracts/sysio.system/include/sysio.system/emissions.hpp and contracts/sysio.system/src/emissions.cpp to use sysio::kv::table / sysio::kv::global in place of multi_index / singleton.
Behavior changes:
- processepoch is permissionless and decoupled from sysio.epoch::advance. It reads epoch_state.current_epoch_index and distributes exactly one epoch per call (target_epoch = last_epoch_index + 1). Safe gap-catch-up by repeated invocation; idempotent via the last_epoch_index guard.
- Recipients filtered by sysio.opreg status (skip non-OPERATOR_STATUS_ACTIVE); slashed / terminated share stays in treasury -- total_distributed is only incremented for amounts actually paid.
- Batch-op pay routed to the current rotation group of 7 from epoch_state.batch_op_groups[current_batch_op_group]; fixed 1/7 slice per member; unfilled slices (slashed / terminated / unknown) stay in treasury.
- Producer pay proportional to eligible_rounds with the existing rank-weight formula; reset to 0 after distribution. Rank resolved at processepoch time. Standby producers (rank 22..cfg.standby_end_rank) paid via the existing rank weight with opreg filter, no eligible_rounds required.
- Verify sysio's WIRE balance is sufficient before transferring (via sysio::token::accounts kv::scoped_table).
Node-owner distributions (addnodeowner, claimnodedis, viewnodedist) ported straight to the kv API with unchanged semantics. Direct sysio.token::transfer remains the payment mechanism; opreg stake deposits intentionally not used -- those represent slashable collateral and must not conflate with rewards.
Cross-contract state reads use the authex_readonly pattern: opreg::operators and epoch::epochstate are mirrored inside emissions.cpp (implementation detail, not public API) so that emissions.hpp stays free of opp / protobuf dependencies and sysio.roa can include it to guard its inline addnodeowner call.
CMakeLists.txt adds ${CMAKE_BINARY_DIR}/libraries/opp/generated-cdt and contracts/sysio.opp.common/include to sysio.system's include path so emissions.cpp can pull the generated protobuf types and shared DataStream operator overloads.
sysio.roa::regnodeowner calls sysio.system::addnodeowner inline so the emissions node-owner distribution row is created in the same transaction as ROA registration. Guarded on sysiosystem::emissions::emitcfg_t::exists("sysio"_n) so bootstrap paths that do not deploy sysio.system (loadSystemContract=False in Cluster.py) or that run forcereg before setemitcfg remain functional -- the inline action is simply skipped when emissions is not configured.
Include sysio.system/emissions.hpp; the header is intentionally free of opp / protobuf deps so this does not widen sysio.roa's dependency surface.
Fixture rewrite: - Deploy sysio.opreg and sysio.epoch as real contracts (not mocks) so processepoch's cross-contract reads exercise the same kv::table code paths as production. - Add ROA RAM policies for sysio.opreg / sysio.epoch / sysio.chalg / sysio.msgch so set_code succeeds on the ~40 KB contract WASMs. - bootstrap_epoch() in the constructor configures sysio.epoch with a short 5-second epoch duration and advances genesis to index 1, leaving the fixture in a state where a single processepoch call can consume one epoch. - setup_producers() now auto-registers each producer as a bootstrapped opreg operator (OPERATOR_STATUS_ACTIVE) so existing producer-pay tests pass through the new opreg filter without test-level churn. - New helpers: register_operator, slash_operator, init_epoch_state, advance_epoch_state, get_opreg_operator, get_epoch_state_row. - Swap <test_contracts.hpp> for "contracts.hpp" and contracts:: accessors for consistency with opreg/epoch tests. - get_t5_state / get_emission_state keyed on the table name (kv::global primary-key convention) instead of the legacy 0 sentinel. Test semantic updates: - Update 75 ported test cases for the new processepoch behavior: recipients filtered by opreg status, batch-op share goes to the current rotation group (or stays in treasury when empty), total_distributed reflects only amounts actually paid. Tests using the "no operators" baseline now assert against compute_undistributed_if_no_operators (producer_pool + batch_pool) instead of producer_pool alone. - Multi-epoch tests call advance_epoch_state between processepoch calls so sysio.epoch's current_epoch_index stays ahead of t5state.last_epoch_index. - processepoch_fails_when_caught_up_to_epoch replaces processepoch_fails_before_epoch_elapsed (the failure condition changed from wall-clock to index comparison). - t5_epoch_info FC_REFLECT gains last_epoch_index, matching the new t5_state field. New integration tests: - opreg_slashed_producer_excluded_from_pay: slashed producer gets 0, remaining producers keep their share (no redistribution). - opreg_unregistered_producer_excluded_from_pay: sysio.system-registered but not opreg-registered producers are filtered out. - processepoch_gap_catchup_one_epoch_per_call: three sysio.epoch advances require three separate processepoch calls (no catch-up batching). - processepoch_fails_on_insufficient_treasury_balance: balance check fires before the first transfer when sysio is underfunded. - roa_forcereg_skips_addnodeowner_when_emitcfg_absent: happy-path coverage of the guard landing an inline sysio::addnodeowner when emitcfg is set.
Artifacts produced by wire-cdt-master (matches wire-sysio master's kv host function signatures). Contains the kv-migrated emissions tables plus the sysio.roa regnodeowner guard that inline-calls sysio::addnodeowner only when the emissions config singleton exists.
Cross-contract mirrors - Move opp/epoch read-only mirrors out of emissions.cpp into public headers owned by sysio.opreg and sysio.epoch. Co-locating the mirrors with the canonical structs makes the invariant visible to maintainers of those contracts. setemitcfg hardening - Cap standby_end_rank at 100 to bound processepoch inline-action count. - After initt5, reject setemitcfg changes that would brick emissions: t5_distributable < floor + total_distributed, or epoch_min_emission above remaining distributable. - Reject node_rewards_start = epoch 0 in setinittime; it silently blocks all future claims via compute_node_claim's guard. - Drop unreachable default case in addnodeowner switch. - Extract BPS_DENOMINATOR constant; no more bare 10000 literals. processepoch - Read operators_per_epoch from sysio.epoch::epochcfg at distribution time instead of hardcoding 7; tracks any upstream group-size change. - Fuse the two prod_by_rank passes into a single walk. - Rename block_seq -> prod_counter_stamp in onblock with invariant comment; the counter is NOT a monotonic block height, the gap check only works because processepoch also resets last_block_num. Audit log and types - Rename CAPEX_ACCOUNT -> CAPEX_OPERATIONS_ACCOUNT with a clarifying comment (capex bucket lives on sysio.ops). - Key epoch_log on sysio.epoch's current_epoch_index so forensics speak the protocol's language; keep the internal epoch_count field alongside. - Memo strings constexpr auto -> constexpr std::string_view; update send_wire_transfer to accept string_view. Read-only view actions - Mark viewnodedist / viewepoch / viewemitcfg [[sysio::read_only]]. sysio.roa finalizereg - Add require_auth(get_self()). - Fix status check to 2 || 3 (was 3 || 4, with 2 as dead code in the confirm branch). - Confirm branch now actually calls regnodeowner. Misc cleanup - Replace Unicode em-dashes with ASCII double-hyphen in three doc comments. - Remove unused <unordered_set> include from sysio.system.hpp. - Rename roa_forcereg_skips_addnodeowner_when_emitcfg_absent to roa_forcereg_inlines_addnodeowner_happy_path, matching actual coverage. Tests - Add setemitcfg_rejects_standby_rank_over_cap. - Add setinittime_rejects_epoch_zero. - Add setemitcfg_post_initt5_rejects_brick_reduce and setemitcfg_post_initt5_rejects_unreachable_min_emission. - Update epoch_log_records_all_fields for new schema. Build - Fix nested-project OPP include path: CMAKE_BINARY_DIR doesn't refer to the host build root inside the contracts ExternalProject; use CMAKE_CURRENT_BINARY_DIR navigation matching sysio.opreg/sysio.epoch. - Add sysio.opreg/include and sysio.epoch/include to sysio.system's include dirs for the new readonly mirror headers.
Security fix: processepoch previously read sysio.epoch's CURRENT batch-op group on every call, so when processepoch lagged sysio.epoch by one or more advances, the current group collected emissions owed to the epochs it processed -- even though those epochs had different groups active. Since processepoch is permissionless, any batch operator could time calls to land during their group's rotation window and harvest catch-up emissions belonging to other groups. Fix uses a per-epoch snapshot written at advance() time and consumed (then pruned) by sysio.system at payout time: sysio.epoch - New batch_snapshot row keyed by epoch_index, capturing the active rotation group's members as they stood when that epoch began. - advance() writes the snapshot after rotating current_batch_op_group. - New prunesnap(epoch_index) action, auth-gated to sysio, erases a consumed row. sysio.system emissions - processepoch reads the snapshot for target_epoch (not sysio.epoch's live state) to build the batch-op recipient list. - After payout, inline-calls sysio.epoch::prunesnap(target_epoch). A contains() guard tolerates missing snapshots (defensive for replays of epochs that predate the snapshot table). readonly mirror - sysio.epoch_readonly.hpp adds mirrors of batchsnap_key / batch_snapshot so sysio.system can read the table. Tests - batch_snapshot_written_on_advance: verifies each advance writes a row. - processepoch_prunes_only_consumed_snapshot: N pending snapshots, each processepoch prunes exactly its target epoch's row. - batch_snapshot_captures_rotation_index: multiple advances capture distinct active_group_index values; epoch 1 and epoch 4 (same rotation slot) are SEPARATE snapshot rows. - prunesnap_requires_sysio_system_auth: non-sysio callers rejected. - prunesnap_rejects_unknown_epoch: unknown index fails cleanly.
operator+/-/*// for fc::variant were declared and defined but never called from anywhere in the tree (verified by grep + a clean full build of unit_test, plugin_test, contracts_unit_test, nodeop, and test_fc with the definitions removed and `= delete` declarations in their place). operator- additionally contained an unreachable bug: the array-walk loop counter decremented (`--i`) instead of incrementing, mirroring the otherwise-identical operator+. Rather than fix dead code, the four operators are marked `= delete` so any future caller fails at compile time with a clear message instead of silently invoking surprising multi-type coercion.
Adds libraries/libfc/benchmark/variant_bench.cpp, modeled on libraries/chain/benchmark/abi_serializer_bench.cpp -- plain chrono timing, no external benchmark dependency, EXCLUDE_FROM_ALL standalone target run manually. Each scenario warms up, then takes the median of 10 runs of N iterations and prints median/min/max ns/op. Scenario coverage targets the paths the fc::variant performance follow-on series will touch: ctor / copy / find / as_* / json round- trip on small and 50-key workloads. The 50-key shape mirrors an ABI-decoded get_table_rows row so deltas reflect a realistic caller. BASELINES.md documents how to build (Release is required) and which scenarios watch which catalogued perf items (lazy variant_object alloc, find_or, from_chars, hash side-table). The baseline-numbers row is a placeholder; the next commit captures it from a Release build. Build / run: ninja -C cmake-build-release -j8 variant_bench ./cmake-build-release/libraries/libfc/benchmark/variant_bench
variant_bench numbers from cmake-build-relwithdebinfo (-O2 -g -DNDEBUG) on 12th Gen Intel Core i9-12900K, clang 18.1.8. Headline observations that motivate Phase A: - as_enum_string_invalid: ~4 us per call. All of it is stoll throwing and the catch(...) unwinding. Replacing with from_chars (Phase A item 3) should drop this by 1-2 orders of magnitude. - find_hit_50key_last: 51 ns vs find_hit_50key_first 2.9 ns -- a 17x ratio. That's the linear scan over 50 entries. Phase B item 4 watches this; find_hit_4key at 4.1 ns is the regression watch (a hash index that hurts small objects is not a win). - contains_then_op_50key: 36.6 ns -- two scans of the same vector. Phase A item 2 (find_or) collapses this to one. - ctor_empty_mvo / ctor_empty_vo: 7.5 / 8.6 ns -- the make_shared<vector<entry>> allocation cost on every default ctor. Phase A item 1 (lazy-allocate) targets this. The series is pinned to RelWithDebInfo so deltas remain comparable and the binary stays debuggable; a re-baseline at -O3 can happen once at the end if absolute numbers need to leave the project.
Adds seven new test files under libraries/libfc/test/variant/ to lock
down current behaviour before the perf series starts touching it.
102 new test cases across:
- test_variant_ctor.cpp: every ctor (primitives, int128/256, char*,
wchar_t*, string, blob, variant_object,
mutable_variant_object, variants, copy,
move, optional<T>) plus deep-copy and
move-leaves-source-null invariants.
- test_variant_assign.cpp: self-assign (copy + move), cross-type
and same-type assignment, template
assignment.
- test_variant_as.cpp: conversion matrix for as_int64 /
as_uint64 / as_int128 / as_uint128 /
as_int256 / as_uint256 / as_double /
as_bool / as_string / as_blob, including
bad-cast paths.
- test_variant_enum.cpp: as_enum_value<E> for int / uint / bool /
double / valid string / invalid string /
object / array / blob sources. Watches
the upcoming stoll -> from_chars switch
(Phase A item 3) for behaviour parity.
- test_variant_visitor.cpp: visit() dispatch coverage of all 13 type
tags. Documents that int128 / uint128 /
int256 / uint256 dispatch through
handle(string), not the typed handle()
overloads -- those are unreachable in
the current visit() implementation.
- test_variant_object_misc.cpp: insertion-order preservation,
contains/find/operator[] consistency,
mvo set vs operator() (variant overload
appends, T&& template overload dedups
via set), erase, merge, op[] insert-
default-on-miss.
- test_variant_operators.cpp: ==, !=, <, > for primitives + arrays;
cross-type string coercion behaviour;
operator! truthiness; documents that
arithmetic operators are = delete.
Suite names use distinct prefixes (variant_ctor_suite,
variant_assign_suite, ...) to coexist with the pre-existing
variant_test_suite / variant_estimated_size_suite /
json_variant_test_suite without filter ambiguity.
The string-source path of variant::as_enum_value<E> previously used
std::stoll wrapped in `try { ... } catch (...) {}`. When the input
was non-numeric, stoll threw std::invalid_argument; the inner catch
swallowed it and the function then threw std::runtime_error to the
caller. Two stack unwinds per bad input.
std::from_chars is non-throwing and parses the same numeric grammar
(leading minus accepted, leading whitespace and leading '+' rejected,
suffix garbage silently ignored), so the swap is observably
equivalent for the cases the ABI serializer can produce -- the
existing test_variant_enum.cpp suite (added in the prior commit)
covers all of them.
Bench delta (libraries/libfc/benchmark/variant_bench, RelWithDebInfo,
12th Gen i9-12900K, ns/op median):
as_enum_string_valid 11.6 -> 4.6 (-60%, 2.5x)
as_enum_string_invalid 3976.4 -> 2965.0 (-25%, the outer
throw still dominates
the bad-input path)
as_enum_int 1.8 -> 1.5 (noise)
Other scenarios are within run-to-run noise (no shared code path).
Default-constructed variant_object and mutable_variant_object no
longer call make_shared<vector<entry>> / make_unique<...> at
construction time. _key_value stays null until the first mutating
operation (op[] / op() / set / reserve, or assignment from a
populated source). Read paths (begin/end/find/contains/op[]/size/
estimated_size) route through a non-const empty-vector singleton in
the null case so iterators are well-formed and comparable.
The mutable singleton is never written to: every write path
allocates a fresh vector first. As a side effect this fixes a
latent aliasing bug in `variant_object::operator=(const
mutable_variant_object&)` where the old `*_key_value = *obj._key_value`
write-through path would mutate any sibling variant_object that had
been copy-shared from the assignee. The new path always
make_shared<vector<entry>>(*obj._key_value), detaching cleanly.
find() in both vo and mvo bypasses the public begin()/end() to keep
the hot loop free of per-iteration null branches: one upfront
null-check, then iterate directly on the vector.
Bench delta (libraries/libfc/benchmark/variant_bench, RelWithDebInfo,
12th Gen i9-12900K, ns/op median):
ctor_empty_mvo 7.5 -> 1.4 (-81%, 5.4x)
ctor_empty_vo 8.6 -> 1.2 (-86%, 7.2x)
json_parse_50key 9760.6 -> ~9500 (~3%, low signal)
find_hit_50key_last 51.0 -> 55.1 (+8%, within run-to-run noise
band of +/-10% measured
across 3 runs)
walk_50key_by_name 997.4 -> 972.2 (within noise)
Adds variant_object::find_or(key, default_value) -> const variant& (plus a string-key overload). Returns the value for `key` if present, otherwise a reference to `default_value`. Caller is responsible for the default's lifetime. Replaces the common `obj.contains(k) ? obj[k] : default_v` pattern, which scans the entry vector twice on hit and throws+catches a key_not_found_exception on miss when the alternate branch is taken via op[]. Bench delta (libraries/libfc/benchmark/variant_bench, RelWithDebInfo, 12th Gen i9-12900K, ns/op median): contains_then_op_50key 38.9 ns (existing double-scan) find_or_50key_hit 19.9 ns (49% faster, single scan) find_or_50key_miss 15.1 ns (no throw on miss) Tests in test_variant_object_misc.cpp cover the hit / miss / empty- object / string-key-overload paths and assert that the returned reference is to the matching entry on hit and to the supplied default on miss.
…allers Step 1 of small-string-optimisation prep: change the public string accessor so it can return a view of inline bytes without requiring a heap std::string object. Pure API + caller migration in this commit; no SSO storage yet, all variant strings still go through `new std::string`. Migrated APIs (signatures changed; existing callers either keep working via implicit conversion or were updated): - fc::variant::get_string() now returns std::string_view (was const std::string&). as_string() (returns std::string by value) is unchanged for callers that need an owning copy. - fc::variant gains a std::string_view ctor. - fc::throw_bad_enum_cast(k, e) takes std::string_view k. - fc::reflector<E>::from_string collapsed into one std::string_view overload (was const char* + const std::string&). FC_REFLECT_ENUM_FROM_STRING uses string_view operator==; the WITH_STRIP variant replaces the `str = b + s` allocation with a starts_with + substr check on the elem name (allocation only happens when strip_base_enum=true and only for the prefix `b`). - fc::from_hex(const std::string&, ...) takes std::string_view (the body only walked iterators). - sysio::chain::asset::from_string and symbol::from_string take std::string_view; both also use std::string_view internally end- to-end via boost::algorithm::trim_copy(string_view) which returns string_view. to_int64 / sysio::chain::to_string still need owning strings, materialised at the call site. - sysio::chain::symbol's `string_to_symbol` / `string_to_symbol_c` helpers and the `symbol(uint8_t, ...)` ctor take std::string_view. - chain_plugin's `string_to_symbol` callers drop now-redundant `.c_str()` calls. Caller migrations: - fc::reflect/variant.hpp's enum from_variant adapter passes v.get_string() through to fc::reflector<T>::from_string without materialising. - bitset's from_variant passes v.get_string() into fc::bitset (which already had a string_view ctor). - variant.cpp's as_blob / from_variant<vector<char>> use string_view locally; base64_decode still gets a materialised std::string because its signature is out of scope. - sysio::chain::symbol_code's from_variant adapter constructs the symbol directly from v.get_string(). The next commit adds the actual SSO storage and reaps the construction-cost win on short strings.
Strings up to 14 bytes are now stored inline in the variant's 16-byte buffer instead of going through `new std::string`. Layout: bytes 0..13 : string content byte 14 : length (0..14) byte 15 : type tag = string_sso_type (13) Heap-allocated strings keep the existing layout (pointer in bytes 0..7, type tag string_type in byte 15) for any input longer than 14 bytes; the storage choice is made by every string-constructing ctor (const char*, char*, wchar_t*, const wchar_t*, std::string, std::string_view) based on the source length. Affected paths: - variant ctors for string-shaped sources route through a single make_string_inline_or_heap helper. std::string ctor short-circuits the move when the source fits inline, so the source's heap buffer (if any) drops on the way out. - clear() / copy ctor / copy assignment / move-related paths handle the new tag: SSO bytes are byte-copied via the existing _data array assignment with no heap involvement. - get_string() returns a view of inline bytes when the tag is string_sso_type, of the heap-resident string when string_type. - as_string() materialises a fresh std::string from inline bytes for SSO; existing heap path unchanged. - as_int64 / as_uint64 / as_int128 / as_uint128 / as_int256 / as_uint256 / as_double / as_bool / as_blob each gained an SSO branch parallel to the existing string_type branch. fc::to_int64 / to_uint64 / to_double now take std::string_view so the SSO branch can pass the view through without materialising. - is_string() returns true for both encodings. - visitor::handle(const std::string&) is now handle(std::string_view) -- visit() dispatches with a view of either inline or heap bytes for string_type / string_sso_type, and the visitor's similarly- unreachable handle(string_view) path is also used for the int128 / uint128 / int256 / uint256 cases (they have always packed a decimal-string view, not the typed handler). raw::variant_packer is the only production override; it now writes the bytes directly rather than calling raw::pack(string_view) (overload resolution picks the generic pack<Stream, T> over a string_view-typed pack via partial-ordering rules; inlining sidesteps that). - raw::pack(variant) translates string_sso_type to string_type at the wire boundary so existing peers and chainbase-resident buffers deserialize unchanged. Payload bytes are identical. - estimated_size() routes string_sso_type through the same formula as string_type (allowed to over-report; keeps tests stable). - json::to_stream handles the new tag alongside string_type. The single non-libfc caller checking `get_type() == string_type` (tests/trx_generator/trx_generator.cpp) is migrated to is_string(). Bench delta (libraries/libfc/benchmark/variant_bench, RelWithDebInfo, 12th Gen i9-12900K, ns/op median): ctor_short_string 14.2 -> 3.6 (-75%, 3.9x) ctor_sso_boundary_14 -- -> 3.4 (new row, max inline) ctor_just_over_sso_15 -- -> 14.3 (new row, just heap) ctor_long_string 19.6 -> 19.7 (unchanged, heap path) copy_short_string 11.0 -> 2.2 (-80%, 5.0x) copy_long_string 16.5 -> 18.8 (unchanged within noise) Workload-shaped scenarios (json_parse / walk) are within run-to-run noise: short-string allocation savings exist but are diluted by non-string parser cost in the 50-key fixture. New tests in test_variant_ctor.cpp cover the SSO/heap boundary, both encoding paths, copy independence, move-leaves-source-null for SSO, and SSO/heap equality.
variant::operator=(const variant&) previously did clear() (which delete's the existing heap object for heap-backed variants) followed by a fresh `new` of the rhs's underlying type. When the lhs already holds a heap object of the same type as the rhs, the dealloc+alloc pair is wasted -- the existing object can absorb the rhs's value directly via its own assignment operator. This commit checks for the same-type case at the top of op=(const&) and routes through the existing heap object for object_type, array_type, blob_type, string_type, and the std::string-backed multi-precision integer encodings (int128/uint128/int256/uint256). Cross-type assignment, and assignment between inline encodings (null/int/uint/double/bool/string_sso), continue to use the existing clear+memcpy path. Bench delta (libraries/libfc/benchmark/variant_bench, RelWithDebInfo, 12th Gen i9-12900K, ns/op median; pre/post captured by stashing the variant.cpp change and rerunning): assign_long_string_to_long 17.0 -> 3.3 (-81%, 5.2x) assign_object_to_object 10.3 -> 2.0 (-81%, 5.1x) assign_array_to_array 32.9 -> 12.1 (-63%, 2.7x) The non-assign rows in the log are unchanged within run-to-run noise (B5 only affects the same-type op=(const&) path).
The per-commit log in BASELINES.md tracks RelWithDebInfo (-O2 -g) so commits stay directly comparable. This commit captures the same post-B5 scenarios at Release (-O3 -DNDEBUG) on the same host so an external reader can see the absolute numbers Wire-Sysio hits in a deployment build. -O3 is 2-15% faster than -O2 on the inlinable paths (find loops, array copies, find_or, contains_then_op). A handful of rows (copy_short_string, copy_object_50key, as_enum_string_invalid) are slower at -O3, all within run-to-run variance for those particular scenarios. The relative ordering across scenarios is preserved, so the per-commit deltas remain meaningful. No code change in this commit -- BASELINES.md only.
`read_table_rows` posts onto the executor's read_only queue and blocks the caller on the future. From a main-thread context (e.g. a plugin's `plugin_startup`), the main thread is the only drain for that queue during write window, so the post sits until the deadline and returns empty rows. This silently broke `batch_operator_plugin::refresh_outposts` at startup -- the cron pool came up sized for zero OPP jobs. When `std::this_thread::get_id() == executor().get_main_thread_id()`, run the scan inline; the read-window discipline is already satisfied on the main thread. Off-thread callers keep the post + wait path so cron threads still iterate chainbase during the read window. Scan body is factored into a `run_scan` lambda shared by both paths.
…42hrs+`) with integrated pruning, cross-chain data consistency & chain-agnostic orchestration layer
## Overview
Achieved memory & on-chain storage stability (`>2400` epochs over `42hrs+`) with integrated pruning, cross-chain data consistency & chain-agnostic orchestration layer.
This commit implements a comprehensive architectural refactoring to achieve cross-chain data-state consistency between WIRE and external blockchains. It introduces a chain-agnostic orchestration layer (`depot_ops` + `outpost_opp_job`) that cleanly separates WIRE table operations from chain-specific client implementations, fixing critical bugs in secondary index queries, row unwrapping, and EIP-1559 RLP signature encoding that previously broke batch operator envelope delivery.
Additionally, the commit removes obsolete auto-generated contract files, adds devcontainer awareness to build tooling, eliminates deprecated macOS signing infrastructure, and standardizes CMake module naming conventions.
---
## Detailed Summary of Changes
### **Removed Deprecated Auto-Generated Contract Files**
Cleaned up stale build artifacts that should never have been committed to version control:
- **Deleted** `contracts/sysio.chalg/sysio.chalg.cpp.actions.cpp` (138 lines of codegen action wrappers)
- **Deleted** `contracts/sysio.chalg/sysio.chalg.dispatch.cpp` (40 lines of codegen dispatch handlers)
- **Deleted** `contracts/sysio.chalg/sysio.chalg.sysio.chalg.cpp.desc` (264 lines of ABI metadata)
These files are regenerated during the build process and do not belong in the repository.
---
### **Chain-Agnostic Orchestration Layer**
Introduced two new abstractions to decouple WIRE operations from outpost-specific client logic:
#### **New Interface: `depot_ops` (`batch_operator_plugin/depot_ops.hpp`)**
Provides chain-agnostic WIRE table/action operations:
- `read_pending_outbound()`: Queries `sysio.msgch::outenvelopes` for pending delivery
- `has_delivered_envelope()`: Checks `sysio.msgch::envelopes` secondary index for delivery status
- `deliver_to_depot()`: Pushes inbound messages via `sysio.msgch::deliver` action
- `emit_debug_envelope()`: Publishes debugging events to `external_debugging_plugin`
- `within_epoch_window()`, `is_elected()`, `current_epoch()`: Epoch state accessors
Concrete implementation (`depot_ops_impl_t`) translates high-level requests into `read_table()` / `push_action()` calls with row unwrapping and batch operator authorization checks.
#### **New Orchestrator: `outpost_opp_job` (`batch_operator_plugin/outpost_opp_job.hpp`)**
Per-outpost job lifecycle manager:
- Owns a single `outpost_client` instance (Ethereum or Solana)
- Runs on dedicated cron thread to avoid blocking other outposts
- **Outbound Flow (WIRE → L1):**
Queries pending envelope → submits to outpost → waits for confirmation → emits debug event
- **Inbound Flow (L1 → WIRE):**
Polls L1 events → checks delivery status → submits to WIRE → prevents duplicates
#### **Refactored `batch_operator_plugin::impl`**
Replaced monolithic `run_epoch_cycle()` with job-based architecture:
- Removed direct outpost client management (`opp_client`, `sol_outpost_client`, transient state tracking)
- Introduced `std::map<uint64_t, std::shared_ptr<outpost_opp_job>> opp_jobs` (one job per outpost)
- Added `build_opp_jobs()` factory using chain plugin `make_outpost_client()` APIs
- Centralized WIRE contract constants (`msgch` and `epoch` namespaces) to prevent string literal duplication
---
### **Critical Bug Fixes**
#### **1. Secondary Index Query Fix (`chain_plugin.cpp`)**
**Problem:** Secondary indexes on `multi_index` tables store keys as `[scope:8B][value:N]`, but `json=true` queries only provided the value portion in bounds, causing incorrect row lookups.
**Solution:** Prepend `scope_prefix_bytes` to lower/upper bounds when:
- Query uses secondary index (`!resolved_index_name.empty()`)
- Scope is set (`!scope_prefix_bytes.empty()`)
- JSON mode is enabled (`p.json == true`)
**Test Coverage:**
- `(sec-10)`: `find` on `multi_index` secondary index returns exact match
- `(sec-11)`: `find` miss returns zero rows (not lexicographically-next row)
#### **2. Row Unwrapping Fix (`batch_operator_plugin.cpp`)**
**Problem:** `chain_plugin::get_table_rows` wraps rows as `{"key", "value", "payer"}`, breaking direct field access in plugin code.
**Solution:** Post-process `combined.rows` vector to extract `"value"` object using safe pattern:
```cpp
fc::variant value{row_obj["value"]}; // Copy to temporary
row = std::move(value); // Move-assign to avoid self-destruction
```
**Critical Pattern:** Direct assignment `row = row["value"]` causes undefined behavior because `variant::operator=` calls `clear()` before reading the source.
#### **3. EIP-1559 RLP Signature Encoding Fix (`ethereum_rlp_encoder.cpp`)**
**Problem:** Signature components `r` and `s` were encoded as fixed 32-byte strings. When the MSB was `0x00`, strict RLP decoders (alloy-rs in reth/anvil) rejected transactions with "leading zero" errors (~1/256 signatures affected).
**Solution:** Implemented `encode_sig_scalar()` to strip leading zeros and emit minimal big-endian integers per EIP-2718.
**Test Coverage:**
- `signed_rlp_strips_leading_zeros_in_r_and_s`: Verifies 31-byte encoding when MSB is `0x00`
- `eip1559_signed_rlp_strips_leading_zero_in_s`: Regression test for production failure case (612-byte payload, s=`0x00 9b bd d7 ...`)
---
### **Plugin API Enhancements**
#### **Outpost Client Factory Methods**
**Ethereum:** `outpost_ethereum_client_plugin::make_outpost_client()`
Creates `outpost_ethereum_client` from ETH client ID, outpost ID, chain ID, contract addresses. Validates client existence and contract address configuration.
**Solana:** `outpost_solana_client_plugin::make_outpost_client()`
Creates `outpost_solana_client` from SOL client ID, outpost ID, chain ID, program ID. Filters loaded IDL set to match `OPP_SOLANA_OUTPOST_PROGRAM_NAME` and asserts IDL availability.
#### **Debug Event Type Relocation**
Moved `DebugEnvelopeEvent` to dedicated header `batch_operator_plugin/debug_envelope_event.hpp` to break circular dependency between `batch_operator_plugin.hpp` and `depot_ops.hpp`. Remains in `sysio::opp::debugging` namespace.
---
### **Build System Improvements**
#### **CMake Module Standardization**
- Renamed `VersionUtils.cmake` → `version-tools.cmake`
- Renamed `AddTestHelpers.cmake` → `test-helpers.cmake`
- Follows lowercase-hyphenated naming convention
#### **Devcontainer Detection in Version Generation**
Enhanced `version-tools.cmake` to gracefully handle non-git environments:
```cmake
if(IS_DIRECTORY ${CMAKE_SOURCE_DIR}/.git AND NOT "$ENV{IN_DEVCONTAINER}" STREQUAL "1")
```
Sets `V_HASH="unknown"` and `V_DIRTY="true"` in devcontainers/tarballs to prevent build failures.
#### **Removed Deprecated MAS Signing Infrastructure**
Deleted `cmake/MASSigning.cmake` (22 lines of unused App Store code signing macros) and removed `mas_sign(${KEY_STORE_EXECUTABLE_NAME})` invocation from `programs/kiod/CMakeLists.txt`.
#### **OPP Bundle Generation Enhancements**
**`generate-opp-bundles.fish`:**
- Added `
Avoid materialising a std::string in variant::as_blob's base64 fallback path, where the source is already a string_view from variant::get_string(). The string_view overload is added alongside the existing const std::string& one - keeping both is necessary because the template fc::base64_decode is also visible at call sites and would silently win for std::string callers (returning std::string instead of std::vector<char>) if only string_view were left. detail::decode's remove_linebreaks branch is also routed through std::string regardless of the input String type - it needs erase(), which std::string_view does not provide.
…nto `wire-sysio` removing a potential circular dep with `wire-libraries-ts` ## Overview Moved `protoc-gen-<solana|solidity>` plugins and protobuf-bundler into `wire-sysio` removing a potential circular dep with `wire-libraries-ts` Additionally, it adds a build options `BUILD_OPP_BUNDLES` (default is `ON`), which allows gating of the OPP bundler The output by default is `<wire-sysio>/build/opp/<solidity|typescript|solana>` and can be link with either `pnpm` or `npm` package managers from there in the case you'd like to use them for local development.
- get_string() doc: add explicit lifetime contract - find_or() doc: add on-hit reference invalidation note - mutable_variant_object(variant) / (T&&): drop wasted entry-vector alloc (operator= replaces _key_value, so the initializer was overwritten) - mutable_variant_object: add const contains() overloads to keep the empty-state lookup allocation-free for default-constructed mvos - estimated_size: use [] instead of at() inside size-bounded loop - write_sso: static_assert(sso_max_length < 128) so a future bump can't silently corrupt the signed-byte length round-trip - test: variant_sso_wire_roundtrip pins the pack/unpack normalisation invariant (string_sso_type tag 13 must wire as legacy string_type 9) - test: self_assign_aliased_subvariant_same_type locks the spicy `v = v.get_object()["k"]` path under the same-type op= heap reuse
… assignment ASAN heap-use-after-free in self_assign_aliased_subvariant_same_type: when v aliases storage owned by *this (e.g. v = v.get_object()["k"] where lhs is object_type and rhs is string_type), the different-type path called clear() before reading from v in the new switch -- and clear() destroys the heap object that v references. Deep-copy v into a temporary BEFORE clear(), then take the temp's data and disarm its destructor. Same observable effect on lhs as the inline new-per-type switch the path used to do (the copy ctor does the same per-type allocation), just safe under aliasing. This is a latent pre-existing bug surfaced by the new test, not a regression introduced by the same-type op= heap reuse refactor.
… storage The int128/uint128/int256/uint256 variant types store their value via new std::string(...) (variant.cpp:134-155), but neither clear() nor the copy ctor handled these type tags: - clear(): leaks the heap std::string for these types on every destructor / assignment that should free them. - copy ctor: falls through to the inline _data byte-copy default arm, which copies the std::string* pointer rather than deep-copying. Two variants then share the same pointer; whichever clear() runs first (after the matched fix below) double-frees. Fix both atomically: - clear() adds the four type tags alongside string_type, all of which use the same std::string* storage shape. - copy ctor adds the same fan-out, deep-copying via new std::string(*v.string_ptr). These are pre-existing latent bugs (the leak was silent, the share was masked by the missing clear() handler so no double-free fired). Surfaced while reviewing the same-type op= refactor.
The merge from master in cd02d6d brought commit 30b1697, which collapsed kv_index_object's (table, index_id) into a single uint16_t table_id and reduced kv_iterator_pool::{allocate_secondary, invalidate_secondary_cache} to 2/3 args. apply_context.cpp::kv_idx_update merged correctly; the two test cases added in 4fe6dfd and 1146c38 still referenced the old fields and signatures. - kv_index_modify_rekeys_correctly: rewritten against table_id and by_code_table_id_seckey. Replaces the by_code_table_idx_prikey lookup (no longer a separate index) with a composite find on (sec_key + pri_key) plus an explicit "old key no longer resolves" check. - kv_iterator_pool_invalidate_secondary_cache: 2-arg allocate_secondary, 3-arg invalidate_secondary_cache. Old (different table) vs (different index) slots collapse to "different table_id"; kept as two distinct table_ids so the loop preserves more than one non-matching slot.
…ions Optimize kv_idx_update: use modify instead of remove+create
At a 6-minute epoch the per-epoch payepoch was firing 100+ inline
send_wire_transfer calls (~21 active producers + ~79 standbys + 3
batch ops) every 6 minutes -- ~1000 transfers/hour for tiny per-
recipient amounts. The frequency is the cost driver, not the math.
Add a single `uint16_t pay_cadence_epochs` knob on emit_cfg
controlling how many epochs accumulate before payepoch fires.
Recommended production value 100; setemitcfg rejects 0.
Schema:
- emit_cfg gains pay_cadence_epochs.
- t5_state gains pending_emission_amount (period accumulator),
period_start_epoch (period anchor), and batch_group_epochs
(per-batch-op-group active-epoch counter).
Gate (sysio.epoch::check_emissions_ready):
- Decides is_pay_epoch via `target >= period_start + cadence - 1`.
- TREASURY_EXHAUSTED still gates every epoch so the chain cannot
roll forward into a depleted treasury.
- Balance-coverage check fires only on pay-epochs against the
period total (pending + this-epoch's per-epoch share).
Advance (sysio.epoch::advance):
- Always queues accrueepoch.
- Conditionally queues payepoch on pay-epochs.
- Inline FIFO ordering means payepoch reads the post-accrue state.
accrueepoch (new sysio.system action, auth sysio.epoch):
- Owns per-epoch state advancement (last_epoch_index /
last_epoch_time / last_epoch_emission, decay continuity).
- Increments pending_emission_amount and bumps
batch_group_epochs[batch_group_index].
payepoch (rewritten):
- Takes batch_op_groups and period_emission instead of single-
epoch active_batch_group + emission_amount.
- Producer pool: expected_rounds scales by pay_cadence_epochs;
eligible_rounds accumulates across non-pay epochs and resets
only here.
- Batch pool: each group's slice is weighted by its active-epoch
count (state.batch_group_epochs[g] / pay_cadence_epochs);
groups active in zero epochs are skipped (only possible when
cadence < batch_op_groups.size()), their slice stays in
treasury, same convention as slashed members.
- On completion drains pending_emission_amount, zeros
batch_group_epochs, advances period_start_epoch.
- epochlog gains one row per pay-epoch; non-pay epochs add none.
Tests (t5_emissions_tests):
- pay_cadence_2_pays_every_other_epoch
- pay_cadence_pending_accumulates_then_drains (cadence=3)
- pay_cadence_epochlog_only_on_pay_epoch
- pay_cadence_treasury_exhausted_gates_non_pay_epoch
- pay_cadence_change_via_setemitcfg_takes_effect
setemitcfg_defaults stays at cadence=1 (preserves existing test
semantics bit-for-bit). New setemitcfg_with_cadence helper is used
by the cadence>1 cases. tests/TestHarness/Cluster.py bootstraps
local soak clusters at cadence=2.
Regenerates contracts/sysio.{system,epoch}/sysio.{system,epoch}.{wasm,abi}.
heifner
changed the title
…d names Wire CDT emits `first`/`second` for `std::pair` and `std::map` fields in generated ABIs (see tests/toolchain/abigen-pass/nested_container.abi in wire-cdt). cc1dde4 previously swapped in a Leap-derived ABI using `key`/`value` as a workaround, but it regresses every time the WASM is regenerated and diverges from Wire CDT's own toolchain tests. Remove the workaround. The contract ABI now uses `first`/`second` matching CDT output, and the Python test inputs and assertions are updated to match. The `pvo` case (`pair<uint16_t, vec_op_uint16>`) was already on `first`/`second` and is unchanged. `transaction_json[...]['value']` row accessors are untouched - that is the clio RPC envelope field, not a pair field name. WASM is unchanged: kv_multi_index binary serialization is positional, so the field rename is purely a JSON-side concern.
…st-cdt-abi-names Test: align nested_container_multi_index with CDT pair/map field names
…ools
Pre-split, apply_context kept a single 1024-slot kv_iterator_pool with
each slot a union of every field used by either iterator kind (four
std::vector<char> buffers + an is_primary flag). Hot paths
(kv_it_next, kv_idx_next) loaded cache lines holding data the
operation never touched, and a contract that iterated both kinds
concurrently competed for the same 1024-slot budget.
Pool split
----------
* kv_iterator_slot_common holds the fields every iterator needs
(in_use, status, code, table_id, cached_id).
* kv_primary_slot adds primary-only byte buffers (prefix,
current_key).
* kv_secondary_slot adds secondary-only fields (current_sec_key,
current_pri_key).
* kv_primary_iterator_pool and kv_secondary_iterator_pool each
expose config::max_kv_iterators (1024) slots independently. A
contract may now hold up to 1024 primary and 1024 secondary
iterators simultaneously.
* Both pools lazily resize on first allocate(). Actions that never
touch KV iterators (e.g. sysio.token transfer, which routes
through kv_get/kv_set only) pay zero heap for the pools. First
allocate pays the same ~82 KB this code paid up front before;
no realloc, no reference invalidation, identical get() path.
Handle encoding (CONSENSUS-OBSERVABLE)
--------------------------------------
Handle values are contract-visible -- they are the return value of
kv_it_create / kv_idx_find_secondary / kv_idx_lower_bound, and
contracts may read, store, and branch on them. The encoding is
part of the consensus surface.
[ 0.. 9] slot index (covers max_kv_iterators = 1024)
[10..15] RESERVED -- must be zero
[16 ] secondary-pool tag (1 = secondary, 0 = primary)
[17..30] RESERVED -- must be zero
[31 ] always zero -- keeps handles positive when read as
int32_t (negative is reserved for "not found")
Bit 16 was chosen over a high bit so handle values stay small and
readable in logs/error messages (secondary handle 0x00010005 vs
0x40000005). Reserved bits give future protocol features room to
encode e.g. iterator generation counters or additional pool
discriminators without disturbing deployed contract code. Any
change to this layout is a protocol change.
Reserved-bit guard
------------------
kv_handle_check_reserved_zero() is called at every host-intrinsic
entry point that consumes a handle. A contract that fabricates a
handle by setting reserved bits fails with a clean
kv_invalid_iterator exception instead of silently aliasing a real
slot through the truncated slot-index mask.
Prefix-size bound on kv_it_create
---------------------------------
Cap prefix_size at config::max_kv_key_size_limit (constexpr absolute
ceiling, 1024 B). The prefix bytes are memcpy'd into the iterator
slot's std::vector<char>, so an unbounded prefix_size would let a
contract allocate arbitrary host-side storage per iterator (up to
max_kv_iterators slots per action). The absolute ceiling is used
instead of the dynamic max_kv_key_size because the cap exists to
limit host-side slot memory, not to match the current on-chain
stored-key limit, and the constexpr compile-time compare has zero
runtime cost. Legitimate CDT usage passes 0 or kv_scope_size
(8 bytes), far below the cap.
Capacity change
---------------
No contract that worked pre-split gets a new error. Some that
exhausted the shared 1024-slot pool mid-mixed use now succeed
(primary+secondary sum up to 2048).
Host ABI / chainbase
--------------------
No change. Host intrinsic signatures, kv_object/kv_index_object
layout, and serialization are all untouched.
…re tests
- kv_context.hpp: static_assert that config::max_kv_iterators fits in
kv_handle_slot_index_mask + 1 (catches mask-narrowing footgun if the
iterator budget grows).
- kv_context.hpp: validate_primary_handle / validate_secondary_handle
and kv_check_prefix_size moved out of apply_context.cpp into the
header as inline helpers. Lets the destroy paths share the same
validation as the other entry points and makes the checks unit-
testable without spinning up an apply_context.
- apply_context.cpp: kv_it_destroy / kv_idx_destroy now go through
validate_primary_handle / validate_secondary_handle, matching the
pattern of every other kv_it_* / kv_idx_* entry. kv_it_create's
prefix-size cap moves to kv_check_prefix_size.
- kv_tests.cpp:
* kv_validate_handle_dispatch: end-to-end coverage of the
validators (well-formed handle, wrong-pool tag, each
reserved-bit class).
* kv_check_prefix_size_bounds: at-limit OK, over-limit throws.
* kv_iterator_pool_basic: drop the implicit assumption that
primary handles equal slot indices -- compare against
kv_handle_slot_index(h1) so future encoding changes do not
break the test.
Reverse the on-disk byte order of the snapshot v1 header magic so a hex dump of a snapshot file reads 'W','I','R','E' instead of 'E','R','I','W'. Stored little-endian as 0x45524957 -> bytes on disk 57 49 52 45. Pre-launch; no backward compatibility. Snapshots written with the old magic cannot be read after this change.
The snapshot magic flip changes the on-disk header bytes. Regenerated
via:
unit_test --run_test='snapshot_part2_tests/*' -- --sys-vm \
--save-snapshot --generate-snapshot-log
head_block_id in sysio_util_snapshot_info_test.py updated to match the
new fixture; blocks.log / blocks.index regenerated as a byproduct of
--generate-snapshot-log.
The snapshot regen run on this branch also drifted the action_mroot
that savanna_misc_tests/verify_block_compatibitity compares against.
Regenerated via:
unit_test -t "savanna_misc_tests/verify_block_compatibitity" \
-- --sys-vm --save-blockchain
jglanz
requested changes
May 12, 2026
Collaborator
jglanz
left a comment
jglanz
left a comment
There was a problem hiding this comment.
I left a few questions - but I'm happy with the PR once you go thru them
Comment on lines
+54
to
+55
| // whether emissions can pay this epoch. If not, advance emits an | ||
| // EmissionsBlocked attestation per outpost (deduped by the local blocklog |
Collaborator
There was a problem hiding this comment.
why would an emissions crank fail?
Contributor
Author
There was a problem hiding this comment.
The gate returns a non-ready result for four reasons, enumerated by EmissionsBlockReason and visible cross-chain via the EmissionsBlocked attestation:
CONFIG_MISSING--sysio.system::emitcfghas never been set.STATE_UNINITIALIZED--sysio.system::t5statehas not been initialized (setinittimenever ran).TREASURY_EXHAUSTED--t5_distributable - t5_floor - total_distributed <= 0. Computed every advance (pay or non-pay) so we never roll forward into a depleted treasury.BALANCE_INSUFFICIENT-- on a pay-epoch only,sysio.tokenWIRE balance can't coverpending + this-epoch's share. Non-pay epochs don't transfer, so no balance check fires.
All four should not fail in practice -- the gate is defense-in-depth so we report a clean cross-chain signal and hold the epoch index, rather than silently corrupting state.
Collaborator
There was a problem hiding this comment.
Personally i'd have an emissions config thats a oneshot call when everything is bootstrapped which would remove the first 2 reasons?
| }; | ||
|
|
||
| void roa::finalizereg(const name& owner,const uint8_t& status) { | ||
| void roa::finalizereg(const name& owner, const uint8_t& status) { |
Contributor
Author
There was a problem hiding this comment.
This was created before enum support. We could change it but I'm not really sure how many clients that would affect.
Comment on lines
+283
to
+287
| int64_t total_emission = 0; | ||
| int64_t compute_amount = 0; | ||
| int64_t capital_amount = 0; | ||
| int64_t capex_amount = 0; | ||
| int64_t governance_amount = 0; |
Collaborator
There was a problem hiding this comment.
is there any value in using std::vector<TokenAmount> to potentially support other token emissions considering our place in the market?
Contributor
Author
There was a problem hiding this comment.
I'm not sure I'm following -- the four fields here (compute/capital/capex/governance_amount) are category splits of a single WIRE emission per the bps fields on emit_cfg, not different tokens. Did you mean the treasury could emit in tokens other than WIRE (BTC/USDC/etc.), or something else?
…nfigurable Conflict resolution: - types.proto: opp-part3 added 60953-60956 (UNDERWRITE_INTENT_COMMIT, UNDERWRITE_INTENT_REJECT, SWAP_REVERT, DEPOSIT_REVERT). Renumbered ATTESTATION_TYPE_EMISSIONS_BLOCKED 60953 -> 60957 to avoid collision. EmissionsBlockReason enum preserved unchanged. - contracts/sysio.epoch/sysio.epoch.wasm: ours (emissions branch has the contract source changes). - contracts/sysio.msgch/sysio.msgch.abi: theirs (emissions does not touch msgch; opp-part3 added new attestation lifecycle). - contracts/sysio.uwrit/sysio.uwrit.abi: theirs (same). WASM/ABI artifacts will be regenerated in a follow-up commit so all contracts pick up the renumbered EMISSIONS_BLOCKED.
Picks up: - types.proto renumber (EMISSIONS_BLOCKED 60953 -> 60957) - opp-part3 lifecycle additions (UNDERWRITE_INTENT_COMMIT/REJECT, SWAP_REVERT, DEPOSIT_REVERT) - new sysio.reserv contract from opp-part3
heifner
changed the base branch from
master
to
feature/opp-part3-operator-management
Fc: variant performance improvements
…swap Flip WIRE snapshot magic for hex-dump readability
…plit Kv: split iterator slot pool into independent primary and secondary pools
Regenerate unittests/deep-mind/deep-mind.log -- reference log went stale after the master merge brought in emissions / opp / serialization changes that affect transaction trace output. Regenerate unittests/test-data/consensus_blockchain/snapshot -- snapshot bytes drifted by one byte after the merge; blocks.log / index / id are unchanged. Register ATTESTATION_TYPE_EMISSIONS_BLOCKED in opp.hpp's FC_REFLECT_ENUM so the host-side variant pipeline can decode AttestationType values read out of msgch.attestations. Without this any test that scans attestations after an emissions-gate-block hits bad_enum_cast. Wire up the sysio.system + sysio.token bootstrap in sysio_epoch_flushwtdw_tester. The emissions readiness gate (added in 2d9f053) requires emitcfg + t5state to exist for sysio.epoch::advance to actually advance; without that the three advance-dependent wtdw tests stalled at current_epoch_index=0. Bump EPOCH_DURATION_SEC from 1 to 60 to clear the MIN_EPOCH_DURATION_SEC floor introduced in 0b3d178, and create the sysio.cap / sysio.gov / sysio.ops payepoch destination accounts so the first pay-epoch transfer doesn't fail.
…nt build Prior regen at 734872d used a stale unit_test binary (May 6); recent chain-library rebuilds had not relinked it, so the captured trace bytes reflected an outdated code path. CI's clean build surfaced the mismatch. Rebuilt unit_test, re-ran --save-dmlog and --save-blockchain. Verified the new reference data on cmake-build-release (matching CI's -DCMAKE_BUILD_TYPE=Release) across sys-vm / sys-vm-jit / sys-vm-oc.
…ssions-configurable opp-part3's v6 data-model refactor moves the outpost/chain registry out of sysio.epoch into the new sysio.chains contract and rewrites epoch::advance with the sliding-window batch-op schedule, per-op delivery evaluation, and the sysio.uwrit::chklocks sweep. emissions-configurable contributes the emissions readiness gate, the blocklog table, and the accrueepoch/payepoch inline actions. Conflict resolutions: - AttestationType (types.proto, opp.hpp): take opp-part3's enum refactor; ATTESTATION_TYPE_EMISSIONS_BLOCKED moves 60957 -> 60962 since opp-part3 claimed 60957 for ATTESTATION_TYPE_SWAP_REJECTED. The EmissionsBlockReason enum is retained. - sysio.epoch.hpp: drop the local outposts table (the registry now lives on sysio.chains); keep the new blocklog table. - sysio.epoch.cpp: advance() runs the emissions readiness gate first, then opp-part3's per-op delivery evaluation, a single epoch increment, the sliding-window schedule, and finally both the chklocks sweep and the accrueepoch/payepoch actions. emit_emissions_block_attestation fans out over sysio.chains::chains_t (is_active_outpost filter) instead of the removed local outposts table. - sysio.epoch_flushwtdw_tests.cpp: union both fixtures; EPOCH_DURATION_SEC stays 60 to satisfy the merged contract's MIN_EPOCH_DURATION_SEC. emissions_tests.cpp deploys sysio.uwrit because the merged epoch::advance unconditionally inlines sysio.uwrit::chklocks; the codeless CHALG/MSGCH placeholder accounts take smaller RAM policies so the added allocation fits nodedaddy's tier-1 pool. Contract .wasm/.abi regenerated against the merged source.
heifner
changed the base branch from
feature/opp-part3-operator-management
to
feature/opp-part3-token-chain-reserve-refactor
jglanz
requested changes
May 22, 2026
Collaborator
jglanz
left a comment
jglanz
left a comment
There was a problem hiding this comment.
Generally looks good, just a few CMake questions/concerns and a couple of code/contract call outs
Comment on lines
+36
to
+38
| # cdt-cpp ignores -isystem; add vcpkg root explicitly as -I so magic_enum resolves. | ||
| PRIVATE | ||
| ${CDT_CONTRACT_INCLUDE_PATH} |
Collaborator
There was a problem hiding this comment.
I had automated the magic_enum issue in contract-tools.cmake? what changed
Contributor
Author
There was a problem hiding this comment.
You did automate it -- CDTWasmToolchain.cmake already adds ${CDT_CONTRACT_INCLUDE_PATH} globally via include_directories(BEFORE ...), so the WASM build resolves magic_enum without any per-contract -I. These additions were a redundant workaround and are reverted in 52e89af. The real gap was the native-module build -- add_native_contract sets up the CDT includes but never linked magic_enum; fixed in Wire-Network/wire-cdt#65.
| $<BUILD_INTERFACE:${CMAKE_CURRENT_SOURCE_DIR}/../sysio.epoch/include> | ||
| $<BUILD_INTERFACE:${CMAKE_CURRENT_SOURCE_DIR}/../sysio.system/include> | ||
| # cdt-cpp ignores -isystem; add vcpkg root explicitly as -I so magic_enum resolves. | ||
| PRIVATE |
Collaborator
There was a problem hiding this comment.
same as above, appplies to all the CMake contract changes
Contributor
Author
There was a problem hiding this comment.
Same -- the per-contract magic_enum -I additions are reverted across all the contract CMakeLists in 52e89af. See the authex thread for details.
Comment on lines
+97
to
+119
| struct blocklog_key { | ||
| uint64_t epoch_index; | ||
| uint64_t primary_key() const { return epoch_index; } | ||
| SYSLIB_SERIALIZE(blocklog_key, (epoch_index)) | ||
| }; | ||
|
|
||
| struct [[sysio::table("blocklog")]] blocklog_entry { | ||
| uint32_t epoch_index = 0; | ||
| sysio::opp::types::EmissionsBlockReason reason = | ||
| sysio::opp::types::EMISSIONS_BLOCK_REASON_UNSPECIFIED; | ||
| int64_t attempted_emission = 0; | ||
| int64_t treasury_remaining = 0; | ||
| int64_t sysio_balance = 0; | ||
| uint32_t first_blocked_at = 0; // unix seconds | ||
| uint32_t last_retry_at = 0; // unix seconds | ||
| uint32_t retry_count = 0; | ||
|
|
||
| SYSLIB_SERIALIZE(blocklog_entry, | ||
| (epoch_index)(reason)(attempted_emission)(treasury_remaining) | ||
| (sysio_balance)(first_blocked_at)(last_retry_at)(retry_count)) | ||
| }; | ||
|
|
||
| using blocklog_t = sysio::kv::table<"blocklog"_n, blocklog_key, blocklog_entry>; |
Contributor
Author
There was a problem hiding this comment.
Yes. blocklog only holds rows for epochs currently failing the emissions gate: record_gate_block inserts on a block, clear_gate_block erases on the next successful gate pass. advance() only ever targets current_epoch_index + 1, so at most the single in-flight blocked epoch is recorded and the table is empty in steady state -- no separate sweep needed.
| #include <sysio.system/emissions.hpp> | ||
| #include <sysio.chains/sysio.chains.hpp> | ||
| #include <sysio/opp/attestations/attestations.pb.hpp> | ||
| #include <zpp_bits.h> |
Collaborator
There was a problem hiding this comment.
i think the zpp include comes with the actual proto includes (i could be wrong)
Contributor
Author
There was a problem hiding this comment.
Confirmed -- attestations.pb.hpp includes <zpp_bits.h> directly, so the explicit include was redundant. Removed in c151993.
Comment on lines
+54
to
+55
| // whether emissions can pay this epoch. If not, advance emits an | ||
| // EmissionsBlocked attestation per outpost (deduped by the local blocklog |
Collaborator
There was a problem hiding this comment.
Personally i'd have an emissions config thats a oneshot call when everything is bootstrapped which would remove the first 2 reasons?
| "epoch_duration_sec exceeds 30-day ceiling"); | ||
| check(operators_per_epoch > 0, "operators_per_epoch must be positive"); | ||
| check(batch_op_groups > 0, "batch_op_groups must be positive"); | ||
| check(batch_operator_minimum_active == operators_per_epoch * batch_op_groups, |
Collaborator
There was a problem hiding this comment.
this particular attestation could cause mega problems in the case that somehow 1 operator hits a performance termination and the replacement has not been activated yet. All the others look appropriate/correct
Contributor
Author
There was a problem hiding this comment.
Fixed in c151993. advance() no longer aborts when the new tail group can't be filled -- it pushes the (possibly empty) group so the sliding window keeps its N slots and advancement always completes. The shortage is reported via print() and stays visible cross-chain in the BatchOperatorGroups attestation, so it's observable without halting OPP. Recovery is operator-roster repair (activating replacements).
Comment on lines
+40
to
+42
| # sysio.token has a flat header layout; expose contracts/ so <sysio.token/sysio.token.hpp> resolves. | ||
| $<BUILD_INTERFACE:${CMAKE_CURRENT_SOURCE_DIR}/..> | ||
| # cdt-cpp ignores -isystem; add vcpkg root explicitly as -I so magic_enum resolves. |
Collaborator
There was a problem hiding this comment.
fix sysio.token layour instead of customizing the CMake config
Contributor
Author
There was a problem hiding this comment.
Done in 52e89af -- sysio.token moved from its flat layout to the standard contracts/sysio.token/include/sysio.token/ layout. epoch and sysio.system now point at sysio.token/include like every other contract, so the contracts/-on-path customization is gone.
| $<BUILD_INTERFACE:${CMAKE_CURRENT_SOURCE_DIR}/..> | ||
| # cdt-cpp ignores -isystem; add vcpkg root explicitly as -I so magic_enum resolves. | ||
| PRIVATE | ||
| ${CDT_CONTRACT_INCLUDE_PATH} |
Contributor
Author
There was a problem hiding this comment.
This line was the magic_enum ${CDT_CONTRACT_INCLUDE_PATH} workaround -- reverted in 52e89af, same as the authex/chalg CMakeLists.
PR #281 review feedback. The per-contract `PRIVATE ${CDT_CONTRACT_INCLUDE_PATH}` include additions were a workaround for magic_enum not resolving in WASM contract builds. They are redundant: the CDT WASM toolchain already adds that path globally via include_directories(BEFORE ...). Revert them in authex, chalg, epoch, msgch, opreg, system and uwrit. The real gap -- magic_enum missing from native contract builds -- is fixed separately in CDT's add_native_contract. sysio.token used a flat header layout, which forced epoch and sysio.system to put contracts/ on their include path so that <sysio.token/sysio.token.hpp> would resolve. Move the header to the standard contracts/sysio.token/include/sysio.token/ layout and point consumers at sysio.token/include like every other contract.
PR #281 review feedback. advance() aborted with a hard check() when no eligible batch operators remained for the new sliding-window tail group. If operators are terminated for performance faster than replacements activate, that abort halts OPP epoch advancement for the whole network, leaving manual roster repair as the only recovery. Push the (possibly empty) tail group instead, so the window keeps its N groups and advance() always completes. The shortfall is reported via print() and stays visible cross-chain in the BatchOperatorGroups attestation. Also drop a redundant <zpp_bits.h> include already provided by attestations.pb.hpp.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
T5 treasury emissions, triggered inline from
sysio.epoch::advancevia a cross-contract readiness gate. Three composable pieces:sysio.epoch::check_emissions_readyreadssysio.system::emitcfg,sysio.system::t5state, andsysio.token's WIRE balance to compute the next epoch's emission. If emissions cannot pay, the gate writes ablocklogrow onsysio.epochand emits anEmissionsBlockedOPP attestation per registered outpost. The chain stays at the prior epoch index until conditions allow a successful pass; the wall-clock duration of the current epoch effectively extends.emit_cfg; per-epoch values are derived insidecompute_epoch_emissionfrom the canonicalepoch_duration_seconsysio.epoch::epochcfg. Governance can retune the epoch frequency without distorting the wall-clock emission curve.emit_cfg.pay_cadence_epochscontrols how many epochs accumulate before payepoch fires. At a 6-min epoch the per-epoch payepoch was firing ~31 inlinesend_wire_transfercalls (21 active + 7 standbys at defaultstandby_end_rank=28+ 3 batch ops) every 6 min, ~310/hr -- the frequency was the cost driver, not the math. Recommended production value is 100 (≈10 h between pays at 6-min epochs);setemitcfgrejects 0.Cadence-aware lifecycle
Every successful
advancequeues two emission-side inline actions in FIFO order:accrueepoch(epoch_index, batch_group_index, per_epoch_emission)-- always. Owns per-epoch state advancement (last_epoch_index,last_epoch_time,last_epoch_emissionfor decay continuity) plus the period accumulators (pending_emission_amount,batch_group_epochs[]).payepoch(epoch_index, batch_op_groups, period_emission)-- pay-epochs only. Drainspending_emission_amountandbatch_group_epochs, distributes the period total across producer / batch / capital / capex / governance pools, resets the accumulator, and advancesperiod_start_epoch.Pay-epoch decision:
target_epoch >= period_start_epoch + pay_cadence_epochs - 1. Withpay_cadence_epochs=1this collapses to "every advance is a pay-epoch" -- the legacy per-epoch behavior, which existing tests cover bit-for-bit.TREASURY_EXHAUSTEDgates every epoch (pay or non-pay) so the chain cannot silently roll forward into a depleted treasury. The balance-coverage check fires only on pay-epochs against the period total (pending + this-epoch's per-epoch share), so a non-pay epoch never readssysio.token::accounts.Producer / batch pool math under cadence > 1
eligible_rounds / expected_rounds_period, whereexpected_rounds_period = epoch_duration_sec * pay_cadence_epochs * 2 / TOTAL_BLOCKS_PER_ROUND. Counters accumulate across non-pay epochs (accrueepochdoes not reset them); onlypayepochclears them. Below the truncate-to-zero floor (~126 s effective period duration, e.g. 60 s epoch with cadence=1),expected_roundsclamps to 1 and pay collapses to "elig_rounds==0 ? 0 : full_share" -- same fallback as before.cfg.standby_end_rank). Paid every pay-epoch unconditionally by rank-decreasing weight; noeligible_roundsrequirement. Defaultstandby_end_rank=28gives 7 standbys; safety cap is 100.batch_pooldivides into per-group slices weighted by each group's active-epoch count over the period:group_pool = batch_pool * batch_group_epochs[g] / pay_cadence_epochs. Members not registered as ACTIVE insysio.opreg(slashed / terminated / unknown) are skipped; their slice stays in the treasury. Groups that were active in zero epochs (only possible whenpay_cadence_epochs < batch_op_groups.size()) are skipped the same way.epochloggains one row per pay-epoch; non-pay epochs add nothing. Retention pruning is unchanged (head-first, capped atcfg.epoch_log_retention_count).emission_config (governance-tunable via
setemitcfg)Node-owner allocations
t1_allocation,t2_allocation,t3_allocation(int64, WIRE subunits) -- total vesting amount per tier registration.t1_duration,t2_duration,t3_duration(uint32, seconds) -- linear vesting period per tier.min_claimable(int64) -- minimum claim amount; smaller claims are deferred until vested or threshold is met.T5 treasury (annual semantics)
t5_distributable(int64) -- total amount available for T5 emissions across the schedule.t5_floor(int64) -- treasury reserve. Emissions stop oncet5_distributable - t5_floor - total_distributed <= 0.target_annual_decay_bps(uint16,(0, 10000]) -- surviving fraction per year;compute_per_epoch_decayderives the per-epoch factor from this andepoch_duration_sec.annual_initial_emission(int64) -- E_0 expressed per year.annual_max_emission,annual_min_emission(int64) -- per-year ceiling / floor; per-epoch values derive viascale_annual_to_epoch.Pay cadence
pay_cadence_epochs(uint16,> 0) -- how many epochs accumulate before payepoch fires. Default helper sets 1; recommended production value 100; soak cluster bootstrap uses 2.Audit-log retention
epoch_log_retention_count(uint32) -- capsepochlogrows. payepoch prunes head-first up to two rows per call to drain after a retention-cap shrink.Category splits (basis points; must sum to 10000)
compute_bps,capital_bps,capex_bps,governance_bps.Compute sub-split (basis points; must sum to 10000)
producer_bps,batch_op_bps.Producer config
standby_end_rank(uint32,[22, 100]) -- last standby rank that receives pay.Hardcoded economic constants (in
emissions.hpp)T1_MAX_NODE_OWNERS = 21,T2_MAX_NODE_OWNERS = 84,T3_MAX_NODE_OWNERS = 1000-- per-tier caps; same constants used bysysio.roa::activateroafor pool seeding.ACTIVE_PRODUCER_COUNT = 21,STANDBY_START_RANK = 22,MAX_STANDBY_END_RANK = 100.BPS_DENOMINATOR = 10000.TOTAL_BLOCKS_PER_ROUND = 21 * 12 = 252(per-rotation block count).New / changed actions
payepoch(epoch_index, batch_op_groups, period_emission)(sysio.system,require_auth(sysio.epoch)) -- pays one pay period; called inline from advance. Signature changed from the per-epoch shape: takes the fullbatch_op_groupsvector (so it can pay any group that was active in the period) and the period total.accrueepoch(epoch_index, batch_group_index, per_epoch_emission)(sysio.system,require_auth(sysio.epoch)) -- new. Per-epoch state advancement + accumulator update; runs every successful advance.setemitcfg,setinittime,addnodeowner,claimnodedis,viewnodedist,initt5,viewepoch,viewemitcfg-- emissions configuration and queries.sysio.epoch::advance-- runs the readiness gate before any state mutation; on block, writesblocklogrow and queuesEmissionsBlockedattestations. On gate-pass, queuesaccrueepoch(always) pluspayepoch(pay-epochs only).sysio.roa::finalizereg-- gainsrequire_auth(get_self())(was missing in master, allowing any caller to bypass registration); corrects status validation so confirmation is reachable.New protobuf types
ATTESTATION_TYPE_EMISSIONS_BLOCKED = 60953.EmissionsBlockedmessage (epoch_index,reason,attempted_emission,treasury_remaining,sysio_balance,first_blocked_at).EmissionsBlockReasonenum (UNSPECIFIED, CONFIG_MISSING, STATE_UNINITIALIZED, TREASURY_EXHAUSTED, BALANCE_INSUFFICIENT).Removed
processepochaction (cranker entry; no caller exists in this design).prunesnapaction andbatchsnaptable on sysio.epoch.[[sysio::contract(\"...\")]]attribute on each table prevents ABI pollution).