Multiple stable memory regions (and a new primitive type `Region`). #3768

matthewhammer · 2023-02-02T19:17:49Z

Runtime-system implementation of the API described in dfinity/motoko-base#516

Checklist:

TODO separately:

add stable-regions.md to portal doc (sidebars.js)
gracefully fail compilation of stablemem/region ops by generating trapping functions
protect access to Region.new

See also:

Forum post, for community discussion.
StableBTree edits needed to upgrade to this system:
- Upgrade to Regions (WIP) sardariuss/MotokoStableBTree#4
- Upgrade to Regions (WIP) sardariuss/MotokoStableBTreeTest#1

src/prelude/prim.mo

github-actions · 2023-02-04T01:51:27Z

Comparing from bbe8c25 to 7765630:
In terms of gas, 5 tests regressed and the mean change is +52.4%.
In terms of size, 5 tests regressed and the mean change is +4.7%.

rts/motoko-rts/src/types.rs

crusso

On the other hand, maybe it's not that important to worry about the cost of maintaining stable memory data, since the replica team is working on optimizing that at the moment anyway.

design/StableRegions.md

design/StableRegions-20230209.md

matthewhammer · 2023-02-13T22:10:27Z

On the other hand, maybe it's not that important to worry about the cost of maintaining stable memory data, since the replica team is working on optimizing that at the moment anyway.

I think we want to claim that our costs (in terms of stable accesses) are on par with the Rust implementation, to claim a kind of "feature parity", so I think it's worth reducing the accesses per operation to the minimum, if possible.

design/StableRegions-20230209.md

crusso

Makes sense to me. Would be good if @luc-blaeser could take another pass before we commit.

For recovery, I imagine we could have a function that only the actor, but not libraries, could call to recover an array of regions from stable memory using a SM metadata scan.

design/StableRegions-20230209.md

ulan · 2023-02-20T16:36:10Z

design/StableRegions-20230209.md

+
+## Questions and answers
+
+### Q: What determines the 8MB non-empty region minimum?


The 8MB default block size turned out to be a footgun for OpenChat that has thousands of canisters. The developers were not aware that stable-structures are allocating at least 8MB even if the actual usage is a few KB.

It might become an issue for Motoko canisters too. Ideally, the developer provides this number themselves such that they are aware of the implications. How difficult it is to make this a parameter that's configurable at runtime?

In Rust we do expose an API that allows users to change the region/bucket size from 8MiB to something else. It does bring additional complexity to developers though as they now need to understand that regions are pre-allocated with some minimum size, but maybe that's an unavoidable detail.

On a related note, I'm currently benchmarking smaller region sizes in the Rust implementation to better understand the performance trade-offs. I'll share those results once I have them.

Thanks @ulan and @ielashi

I liked how the 8MB minimum simplified some other decisions for the implementor, but I also wondered how to guide people to use these the right way without running up a budget of unused scratch space in their apps.

Since we are trying to "catch up" to the Rust stable memory manager at the moment, and not surpass its features, I'll stick with the current 8MB minimum for now. But I'm very curious to hear about the future test results that @ielashi may have about other options.

I'm also curious about a more layered approach, where initially-small regions can use a more fine-grained setting, and then use coarser blocks as they grow. That seems potentially too complex, though.

@ulan asks

How difficult it is to make this a parameter that's configurable at runtime?

That's a good question. At "runtime" still means fixed at some point, right?

When would that parameter be given during the compilation and deployment phases, I wonder? Would it be a parameter to the actor, given when it's first installed?

rts/motoko-rts/src/types.rs

test/run-drun/stable-region-blob.mo

rts/motoko-rts/src/types.rs

test/run-drun/stable-region-blob.mo

matthewhammer · 2023-03-02T17:16:33Z

So a region objhect is Region header followed by one word for the blob as a Motoko value.

Oh I see. That's not what I envisioned originally, but it makes sense now.
So there are two blobs for each region:

One fixed-sized one (for the region meta data fields)

The first one is not a Blob value per se, just part of the header for that type of Heap Obj with tag REGION (or whatever its called). Each tag has its own header format.

One variable-sized one (growing as needed to hold the block IDs)

Right, that's an actual motoko Blob value, referenced from the payload of the Region object and managed by GC (eg. reclaiming the old one after a resize).

And the header points to the first one, which has a field pointing to the second one?
The object will look something like this in memory
 | TAG_REGION | Region struct   ...           |   Value  |
   1 word           _n_   words                   1-word (skewed pointer to Blob)
                                                ^ payload points here, I think.
Similar to ASCII art for layout in module Variant of compile.ml, I think.

Ok, thanks for the repeated explanations. It's taken a while for the all of the details to sink into my brain. :)

After changing the file and getting the definition that you expected (finally), it also made sense to me too.

* lazy migration; still need to reqwrite stable mem ops * dynamically choose stablemem operations, based on current version - uses ifs consuming multiple values that trip up wasmopt (sigh) * format region.rs * update bench numbers * update heap-32 test output * workaround wasmopt bug * remove dead code * update test output * refactor (use constants for versions) * comment * refactor stablememory api and prims to simplify; delete most of region0.rs * Update test/run-drun/regions-pay-as-you-go.mo * Update test/run-drun/stable-mem-pay-as-you-go.mo * comment compile.ml/rename StableMemory->StableMemoryInterface * remove dangerous stableMemoryRegion prim (that breaks aliasing) * adjust test for removal of Regions.region0 * nuke region0.rs; change type of use_stable_regions arg to u32 (bool, really); change assert in get_region0 to debug assert for perf

* add missing write barrier; export and reuse Region.rs allocation function in compile.ml; add Region.init_region * add field accessors to Region.rs, rmove them from compile.ml, for better abstraction, less code * fix bug in Region.sanity_check * remove obsolete Region.sanity_check * Claudio/msm missing barrier debug (#4173) * repro for bug * test output (broken) * don't check tag on region field access for serialization * delete repro, its just regions-pay-as-you-go.mo * delete def of TAG_STABLE_SEEN * Update rts/motoko-rts/src/types.rs

* 64-bit id field * adjust idl::skip * extend block table with 64-bit region id and internal pagecount; WIP (see buggy repro.mo) * fix bug in block_page_count * remove region table; max out region limit to u64::MAX - 1; modify tests and update test output; reduce overhead from 9 to 6 pages * update bench numbers * Apply suggestions from code review Co-authored-by: Luc Blaeser <112870813+luc-blaeser@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: Luc Blaeser <112870813+luc-blaeser@users.noreply.github.com> * remove unused Region::payload_addr; add static asserts; update numbers * appease Rust formatter * Update rts/motoko-rts/src/region.rs Co-authored-by: Luc Blaeser <112870813+luc-blaeser@users.noreply.github.com> * apply Luc's suggestions --------- Co-authored-by: Luc Blaeser <112870813+luc-blaeser@users.noreply.github.com>

ZenVoich · 2023-08-29T08:02:49Z

Is it possible to remove a region and release memory?

crusso · 2023-08-29T20:08:47Z

Is it possible to remove a region and release memory?

Not with this PR, but the intention is that we will, in future, garbage collect the memory blocks of regions objects that become unreachable, for re-use by other regions that are still reachable. The PR is designed to be compatible with that.

…om 8MB to 1MB (16 pages) (#4187) * reduce overhead of pure regions to 1MB; dynamic choice and use of BLOCK_BASE; record BLOCK_SIZE and BLOCK_BASE * fix bug * rust formatting * add tests showing low and high regions base offsets

* up EXTRA_BATCHES from 1 to 32 * Update test/drun-wrapper.sh * Update test/drun-wrapper.sh

* documentation and changelog for regions types * typo

…ry (#580) Note this is intended for branch next-moc. CI expected to fail for base branch next-moc. Builds fine against moc regions branch Replaces original PR #516 (that I forgot existed but also targeted master, not next-moc as required) - [ ] Update moc PR dfinity/motoko#3768, sources.nix and udpdate base doc there. - [ ] revise overheads # Region Byte-level access to isolated, (virtual) stable memory _regions_. This is a moderately lightweight abstraction over IC _stable memory_ and supports persisting regions of binary data across Motoko upgrades. Use of this module is fully compatible with Motoko's use of _stable variables_, whose persistence mechanism also uses (real) IC stable memory internally, but does not interfere with this API. It is also fully compatible with existing uses of the `ExperimentalStableMemory` library, which has a similar interface, but, only supported a single memory region, without isolation between different applications. Memory is allocated, using `grow(region, pages)`, sequentially and on demand, in units of 64KiB logical pages, starting with 0 allocated pages. New pages are zero initialized. Growth is capped by a soft limit on physical page count controlled by compile-time flag `--max-stable-pages <n>` (the default is 65536, or 4GiB). Each `load` operation loads from region relative byte address `offset` in little-endian format using the natural bit-width of the type in question. The operation traps if attempting to read beyond the current region size. Each `store` operation stores to region relative byte address `offset` in little-endian format using the natural bit-width of the type in question. The operation traps if attempting to write beyond the current region size. Text values can be handled by using `Text.decodeUtf8` and `Text.encodeUtf8`, in conjunction with `loadBlob` and `storeBlob`. The current region allocation and region contents are preserved across upgrades. NB: The IC's actual stable memory size (`ic0.stable_size`) may exceed the total page size reported by summing all regions sizes. This (and the cap on growth) are to accommodate Motoko's stable variables and bookkeeping for regions. Applications that plan to use Motoko stable variables sparingly or not at all can increase `--max-stable-pages` as desired, approaching the IC maximum (initially 8GiB, then 32Gib, currently 64Gib). All applications should reserve at least one page for stable variable data, even when no stable variables are used. Usage: ```motoko no-repl import Region "mo:base/Region"; ``` ## Type `Region` ``` motoko no-repl type Region = Prim.Types.Region ``` A stateful handle to an isolated region of IC stable memory. `Region` is a stable type and regions can be stored in stable variables. ## Value `new` ``` motoko no-repl let new : () -> Region ``` Allocate a new, isolated Region of size 0. Example: ```motoko no-repl let region = Region.new(); assert Region.size(region) == 0; ``` ## Value `id` ``` motoko no-repl let id : Region -> Nat ``` Return a Nat identifying the given region. Maybe be used for equality, comparison and hashing. NB: Regions returned by `new()` are numbered from 16 (regions 0..15 are currently reserved for internal use). Allocate a new, isolated Region of size 0. Example: ```motoko no-repl let region = Region.new(); assert Region.id(region) == 16; ``` ## Value `size` ``` motoko no-repl let size : (region : Region) -> (pages : Nat64) ``` Current size of `region`, in pages. Each page is 64KiB (65536 bytes). Initially `0`. Preserved across upgrades, together with contents of allocated stable memory. Example: ```motoko no-repl let region = Region.new(); let beforeSize = Region.size(region); ignore Region.grow(region, 10); let afterSize = Region.size(region); afterSize - beforeSize // => 10 ``` ## Value `grow` ``` motoko no-repl let grow : (region : Region, newPages : Nat64) -> (oldPages : Nat64) ``` Grow current `size` of `region` by the given number of pages. Each page is 64KiB (65536 bytes). Returns the previous `size` when able to grow. Returns `0xFFFF_FFFF_FFFF_FFFF` if remaining pages insufficient. Every new page is zero-initialized, containing byte 0x00 at every offset. Function `grow` is capped by a soft limit on `size` controlled by compile-time flag `--max-stable-pages <n>` (the default is 65536, or 4GiB). Example: ```motoko no-repl import Error "mo:base/Error"; let region = Region.new(); let beforeSize = Region.grow(region, 10); if (beforeSize == 0xFFFF_FFFF_FFFF_FFFF) { throw Error.reject("Out of memory"); }; let afterSize = Region.size(region); afterSize - beforeSize // => 10 ``` ## Value `loadNat8` ``` motoko no-repl let loadNat8 : (region : Region, offset : Nat64) -> Nat8 ``` Within `region`, load a `Nat8` value from `offset`. Traps on an out-of-bounds access. Example: ```motoko no-repl let region = Region.new(); let offset = 0; let value = 123; Region.storeNat8(region, offset, value); Region.loadNat8(region, offset) // => 123 ``` ## Value `storeNat8` ``` motoko no-repl let storeNat8 : (region : Region, offset : Nat64, value : Nat8) -> () ``` Within `region`, store a `Nat8` value at `offset`. Traps on an out-of-bounds access. Example: ```motoko no-repl let region = Region.new(); let offset = 0; let value = 123; Region.storeNat8(region, offset, value); Region.loadNat8(region, offset) // => 123 ``` ## Value `loadNat16` ``` motoko no-repl let loadNat16 : (region : Region, offset : Nat64) -> Nat16 ``` Within `region`, load a `Nat16` value from `offset`. Traps on an out-of-bounds access. Example: ```motoko no-repl let region = Region.new(); let offset = 0; let value = 123; Region.storeNat16(region, offset, value); Region.loadNat16(region, offset) // => 123 ``` ## Value `storeNat16` ``` motoko no-repl let storeNat16 : (region : Region, offset : Nat64, value : Nat16) -> () ``` Within `region`, store a `Nat16` value at `offset`. Traps on an out-of-bounds access. Example: ```motoko no-repl let region = Region.new(); let offset = 0; let value = 123; Region.storeNat16(region, offset, value); Region.loadNat16(region, offset) // => 123 ``` ## Value `loadNat32` ``` motoko no-repl let loadNat32 : (region : Region, offset : Nat64) -> Nat32 ``` Within `region`, load a `Nat32` value from `offset`. Traps on an out-of-bounds access. Example: ```motoko no-repl let region = Region.new(); let offset = 0; let value = 123; Region.storeNat32(region, offset, value); Region.loadNat32(region, offset) // => 123 ``` ## Value `storeNat32` ``` motoko no-repl let storeNat32 : (region : Region, offset : Nat64, value : Nat32) -> () ``` Within `region`, store a `Nat32` value at `offset`. Traps on an out-of-bounds access. Example: ```motoko no-repl let region = Region.new(); let offset = 0; let value = 123; Region.storeNat32(region, offset, value); Region.loadNat32(region, offset) // => 123 ``` ## Value `loadNat64` ``` motoko no-repl let loadNat64 : (region : Region, offset : Nat64) -> Nat64 ``` Within `region`, load a `Nat64` value from `offset`. Traps on an out-of-bounds access. Example: ```motoko no-repl let region = Region.new(); let offset = 0; let value = 123; Region.storeNat64(region, offset, value); Region.loadNat64(region, offset) // => 123 ``` ## Value `storeNat64` ``` motoko no-repl let storeNat64 : (region : Region, offset : Nat64, value : Nat64) -> () ``` Within `region`, store a `Nat64` value at `offset`. Traps on an out-of-bounds access. Example: ```motoko no-repl let region = Region.new(); let offset = 0; let value = 123; Region.storeNat64(region, offset, value); Region.loadNat64(region, offset) // => 123 ``` ## Value `loadInt8` ``` motoko no-repl let loadInt8 : (region : Region, offset : Nat64) -> Int8 ``` Within `region`, load a `Int8` value from `offset`. Traps on an out-of-bounds access. Example: ```motoko no-repl let region = Region.new(); let offset = 0; let value = 123; Region.storeInt8(region, offset, value); Region.loadInt8(region, offset) // => 123 ``` ## Value `storeInt8` ``` motoko no-repl let storeInt8 : (region : Region, offset : Nat64, value : Int8) -> () ``` Within `region`, store a `Int8` value at `offset`. Traps on an out-of-bounds access. Example: ```motoko no-repl let region = Region.new(); let offset = 0; let value = 123; Region.storeInt8(region, offset, value); Region.loadInt8(region, offset) // => 123 ``` ## Value `loadInt16` ``` motoko no-repl let loadInt16 : (region : Region, offset : Nat64) -> Int16 ``` Within `region`, load a `Int16` value from `offset`. Traps on an out-of-bounds access. Example: ```motoko no-repl let region = Region.new(); let offset = 0; let value = 123; Region.storeInt16(region, offset, value); Region.loadInt16(region, offset) // => 123 ``` ## Value `storeInt16` ``` motoko no-repl let storeInt16 : (region : Region, offset : Nat64, value : Int16) -> () ``` Within `region`, store a `Int16` value at `offset`. Traps on an out-of-bounds access. Example: ```motoko no-repl let region = Region.new(); let offset = 0; let value = 123; Region.storeInt16(region, offset, value); Region.loadInt16(region, offset) // => 123 ``` ## Value `loadInt32` ``` motoko no-repl let loadInt32 : (region : Region, offset : Nat64) -> Int32 ``` Within `region`, load a `Int32` value from `offset`. Traps on an out-of-bounds access. Example: ```motoko no-repl let region = Region.new(); let offset = 0; let value = 123; Region.storeInt32(region, offset, value); Region.loadInt32(region, offset) // => 123 ``` ## Value `storeInt32` ``` motoko no-repl let storeInt32 : (region : Region, offset : Nat64, value : Int32) -> () ``` Within `region`, store a `Int32` value at `offset`. Traps on an out-of-bounds access. Example: ```motoko no-repl let region = Region.new(); let offset = 0; let value = 123; Region.storeInt32(region, offset, value); Region.loadInt32(region, offset) // => 123 ``` ## Value `loadInt64` ``` motoko no-repl let loadInt64 : (region : Region, offset : Nat64) -> Int64 ``` Within `region`, load a `Int64` value from `offset`. Traps on an out-of-bounds access. Example: ```motoko no-repl let region = Region.new(); let offset = 0; let value = 123; Region.storeInt64(region, offset, value); Region.loadInt64(region, offset) // => 123 ``` ## Value `storeInt64` ``` motoko no-repl let storeInt64 : (region : Region, offset : Nat64, value : Int64) -> () ``` Within `region`, store a `Int64` value at `offset`. Traps on an out-of-bounds access. Example: ```motoko no-repl let region = Region.new(); let offset = 0; let value = 123; Region.storeInt64(region, offset, value); Region.loadInt64(region, offset) // => 123 ``` ## Value `loadFloat` ``` motoko no-repl let loadFloat : (region : Region, offset : Nat64) -> Float ``` Within `region`, loads a `Float` value from the given `offset`. Traps on an out-of-bounds access. Example: ```motoko no-repl let region = Region.new(); let offset = 0; let value = 1.25; Region.storeFloat(region, offset, value); Region.loadFloat(region, offset) // => 1.25 ``` ## Value `storeFloat` ``` motoko no-repl let storeFloat : (region : Region, offset : Nat64, value : Float) -> () ``` Within `region`, store float `value` at the given `offset`. Traps on an out-of-bounds access. Example: ```motoko no-repl let region = Region.new(); let offset = 0; let value = 1.25; Region.storeFloat(region, offset, value); Region.loadFloat(region, offset) // => 1.25 ``` ## Value `loadBlob` ``` motoko no-repl let loadBlob : (region : Region, offset : Nat64, size : Nat) -> Blob ``` Within `region,` load `size` bytes starting from `offset` as a `Blob`. Traps on an out-of-bounds access. Example: ```motoko no-repl import Blob "mo:base/Blob"; let region = Region.new(); let offset = 0; let value = Blob.fromArray([1, 2, 3]); let size = value.size(); Region.storeBlob(region, offset, value); Blob.toArray(Region.loadBlob(region, offset, size)) // => [1, 2, 3] ``` ## Value `storeBlob` ``` motoko no-repl let storeBlob : (region : Region, offset : Nat64, value : Blob) -> () ``` Within `region, write `blob.size()` bytes of `blob` beginning at `offset`. Traps on an out-of-bounds access. Example: ```motoko no-repl import Blob "mo:base/Blob"; let region = Region.new(); let offset = 0; let value = Blob.fromArray([1, 2, 3]); let size = value.size(); Region.storeBlob(region, offset, value); Blob.toArray(Region.loadBlob(region, offset, size)) // => [1, 2, 3] ```

crusso

Looks splendid

…4200) The regions PR #3768 bumped up drun-wrapper extra batches from 1 to 128, affecting the perf of perf/dao due to a longer running heartbeat. This restores the extra batches back to 1.

matthewhammer changed the title ~~Multiple-stable memories, as a new primitive type Memory.~~ Multiple stable memories, as a new primitive type Memory. Feb 2, 2023

matthewhammer mentioned this pull request Feb 2, 2023

WIP: Region module (Non-experimental stable memory API) dfinity/motoko-base#516

Closed

This comment was marked as resolved.

Sign in to view

crusso reviewed Feb 3, 2023

View reviewed changes

src/prelude/prim.mo Outdated Show resolved Hide resolved

crusso reviewed Feb 3, 2023

View reviewed changes

src/prelude/prim.mo Outdated Show resolved Hide resolved

matthewhammer changed the title ~~Multiple stable memories, as a new primitive type Memory.~~ Multiple stable memory regions (and a new primitive type Region). Feb 3, 2023

luc-blaeser reviewed Feb 7, 2023

View reviewed changes

rts/motoko-rts/src/types.rs Outdated Show resolved Hide resolved

crusso reviewed Feb 10, 2023

View reviewed changes

matthewhammer commented Feb 14, 2023

View reviewed changes