Add async serialize option via coroutines #4

oratory · 2021-04-12T04:27:57Z

This adds an async=true config option to coroutine.yield() in the serialize function. This removes or reduces framerate loss while processing large tables.

rossnichols

This is a great idea, but it needs some more work before I can merge it.

LibSerialize.lua

tests.lua

rossnichols · 2021-04-12T05:21:58Z

LibSerialize.lua

@@ -1265,10 +1279,13 @@ end

 function LibSerialize:SerializeEx(opts, ...)
    self:_ClearReferences()


Since the library stores state on itself during serialization, you're going to have a bad time if someone (perhaps another addon) tries to serialize something while another serialization is happening asynchronously. You'll need to update all such state to be managed at the function call level. Also add a test case for this (make sure it fails in your current version).

You can probably rename combinedOpts and use it as the state for the current serialization, since it's already plumbed everywhere. Just need to make sure the loop that writes user-specified keys is restricted to valid options.

Do not store any global states in the library.

The state can be a function local variable, because the state is cleared at the beginning and at the end of Serialization/Deserialization,

Change the prototype of self:_ClearReferences() to self:_ClearReferences(state), where "state" is initialized as a table.
Store "refsDirty", "stringRefs" and "tableRefs" in the variable "state" instead.

Yep, I probably shouldn't have done it that way to begin with, but it didn't make a difference until now :) unless people tried to dig into the library's internals, I suppose, but that was never a support scenario and I'm not concerned about breaking them.

LibSerialize.lua

SafeteeWoW · 2021-04-12T15:02:40Z

LibDeflate got a similar PR.

I suggest a DeserializeBegin/Continue/End API, instead of coroutine, so the serialization can be done by chunk.
Need to add a state machine in the code.
Much more complicated than coroutine though

rossnichols · 2021-04-12T18:01:58Z

LibDeflate got a similar PR.

I suggest a DeserializeBegin/Continue/End API, instead of coroutine, so the serialization can be done by chunk.
Need to add a state machine in the code.
Much more complicated than coroutine though

Yeah I was thinking about that, but as you said it's much more disruptive to the entire serialization process, and given that the current code uses recursion and iterators as implicit state while serializing, it'd be quite a large undertaking. We do have regression tests, but a change of that magnitude still worries me.

Given that the coroutine is opt-in and quite undisruptive to the code, I think it's preferable - you're taking on the overhead of the coroutine, sure, but it's an opt-in feature to avoid mini-hangs so I think the tradeoff is reasonable. But in case we did want to refactor to remove it in the future, I'd still want the coroutine management to be moved into the library, as I suggested above.

For reference, the LibDeflate PR: SafeteeWoW/LibDeflate#6

rossnichols · 2021-04-13T14:57:43Z

@oratory i haven't heard from you since you first opened the PR - are you planning on making these changes?

oratory · 2021-04-13T15:18:18Z

Hey yes I'm hoping to make these changes today or tomorrow.

oratory · 2021-04-14T17:44:43Z

Coroutines are now all managed within the lib, and cleaned up a bunch. SerializeAsync() and DeserializeAsync() are separate functions rather than having an async option.

One thing I'd still like to add is a check if an async process is currently running since it will break if a 2nd is started.

rossnichols · 2021-04-15T02:25:14Z

One thing I'd still like to add is a check if an async process is currently running since it will break if a 2nd is started.

This needs to be supported - this is a global library and may be used by any number of addons, which should not have to handle the case of "I want to serialize something but I'm told to come back later".

rossnichols

The current iteration still has issues:

Introduces non-WoW dependencies
Doesn't support multiple simultaneous serialization/deserialization
Insufficient test coverage

The API shape as implemented isn't possible to implement in pure Lua, so it'll need updating. I'd recommend adding a section to the readme to describe how to incorporate it into WoW's environment, though, using an OnUpdate frame script to drive the API.

rossnichols · 2021-04-15T02:27:52Z

LibSerialize.lua

+    Callback arguments:
+    * `...`: the deserialized value(s)


What callback is this referring to?

rossnichols · 2021-04-15T02:31:19Z

LibSerialize.lua

+
+    Arguments:
+    * `callback`: a function called when serialization is complete
+      `callback` is called with the serialized string as its argument, or nil on failure


Remove this, you have the explicit section below.

The serialization API didn't have a concept of a non-error()ing failure mode before - can you propagate any failure from within the coroutine and "rethrow" it?

rossnichols · 2021-04-15T02:31:54Z

LibSerialize.lua

+    Arguments:
+    * `input`: a string previously returned from `LibSerialize:Serialize()`
+    * `callback`: a function called when deserialization is complete
+      `callback` is called with `success` (Boolean) as the first parameter,


Same as above - remove it since you have the specific section below.

rossnichols · 2021-04-15T02:32:29Z

LibSerialize.lua

+
+    Arguments:
+    * `input`: a string previously returned from `LibSerialize:Serialize()`
+    * `callback`: a function called when deserialization is complete


Same as above - remove it since you have the specific section below. But this time you actually don't have the section, so you need to add it :)

rossnichols · 2021-04-15T02:33:20Z

LibSerialize.lua

@@ -182,6 +230,14 @@ The following serialization options are supported:
    table encountered during serialization. The function must return true for
    the pair to be serialized. It may be called multiple times on a table for
    the same key/value pair. See notes on reeentrancy and table modification.
+* `asyncYieldSize`: `number` (default 50000)


This should be documented and enforced as only being valid from SerializeAsync().

LibSerialize.lua

rossnichols · 2021-04-15T02:58:55Z

LibSerialize.lua

 }

 local canSerializeFnOptions = {
    errorOnUnserializableType = false
 }

+local CreateFrame = CreateFrame


The library is designed to be runnable on Lua 5.1.4+ with no modifications - you can't introduce WoW-specific dependencies. Unfortunately this will complicate the API, since you can't drive the async loop yourself, but it'll fix the other issue of not supporting multiple simultaneous callers.

For reference, this is what I do on Windows:

Get Lua from https://sourceforge.net/projects/luabinaries/files/5.1.4/Tools%20Executables/lua5_1_4_Win64_bin.zip/download

lua.exe tests.lua

All tests should pass.

rossnichols · 2021-04-15T03:02:37Z

LibSerialize.lua

    return FlushWriter()
 end

+function LibSerialize:SerializeAsyncEx(callback, opts, ...)
+    if not CreateFrame then


Yeah, this isn't sufficient of a deterrent, since the tests still need to run outside the WoW environment.

I have an idea here that if the lib is run outside of WoW (or without a callback) then the async function will instead return some simple handlers to operate the coroutine. I'll work on that and update.

That won't be sufficient - the async version needs to be able to be tested outside WoW.

rossnichols · 2021-04-15T03:05:13Z

LibSerialize.lua

    self:_ClearReferences()
    local ReadBytes, ReaderBytesLeft = CreateReader(input)

    self._readBytes = ReadBytes
+  opts = opts or {}


Here and elsewhere - inconsistent indentation (you're using 2 instead of 4?)

Do not modify function arguments. Make a copy

rossnichols · 2021-04-15T03:08:48Z

tests.lua

@@ -66,6 +66,21 @@ function LibSerialize:RunTests()
        assert(tab.nested.c == nil)
    end

+    do


You'll need more test cases for the other scenarios we've brought up, e.g. actually triggering a yield, multiple simultaneous async serializations/deserializations (including mixing those two categories), propagating an error from a failed serialization/deserialization.

Try to remove all global states that may share between API calls.
Test for multiple simultaneous async serializations/deserializations is still needed.

SafeteeWoW · 2021-04-15T15:48:23Z

LibSerialize.lua

@@ -1265,10 +1279,13 @@ end

 function LibSerialize:SerializeEx(opts, ...)
    self:_ClearReferences()
-    local WriteString, FlushWriter = CreateWriter()
+    local WriteString, FlushWriter, DoYield = CreateWriter()

    self._writeString = WriteString


Do not modify "self" in API. This introduces global states

SafeteeWoW · 2021-04-15T15:48:58Z

LibSerialize.lua


    self._writeString = WriteString
    self:_WriteByte(SERIALIZATION_VERSION)
+    if opts.async then
+        self._doYield = DoYield


Do not modify "self" in API. This introduces global states

SafeteeWoW · 2021-04-15T15:51:48Z

Currently, this library uses lots of global states, so coroutine will have lots of problems.

Do not modify "self" in API.
Do not modify file-scope variable in API.

Store the states in a local table variable, and pass it through function

SafeteeWoW · 2021-04-15T15:59:36Z

LibSerialize.lua

@@ -1346,6 +1464,26 @@ function LibSerialize:DeserializeValue(input)
    return unpack(output, 1, outputSize)
 end

+function LibSerialize:DeserializeValueAsync(input, callback, opts)


I don't think a WoW specific API is needed here.
Better to put it in a example code

SafeteeWoW · 2021-04-15T16:11:02Z

LibSerialize.lua

+function LibSerialize:DeserializeAsync(input, callback, opts)
+  opts = opts or {}


Do not modify user arguments. Make a copy

rossnichols · 2021-04-15T17:10:34Z

@oratory as mentioned in several comments, adding this functionality is difficult due to the library storing state effectively globally. Would you like me to make a separate PR fixing that, so you don't have to worry about it in this one? You'd just have to rebase onto it.

oratory · 2021-04-15T22:07:04Z

If you want to make that change anyway, then sure. Then we won't need to worry about multiple serializing processes running at once.

rossnichols · 2021-04-15T22:20:59Z

If you want to make that change anyway, then sure. Then we won't need to worry about multiple serializing processes running at once.

Well this PR won't be merged until that's supported, so the work has to be done by one of us :). It's a hard requirement that calling into the library to serialize/deserialize doesn't fail because some random other addon happens to be doing the same thing. Especially since it impacts synchronous users of the API as well.

oratory · 2021-04-15T22:39:11Z

My original plan was to cheat and just add a isWorking boolean to the state and make new requests wait, but this method seems better. :)

rossnichols · 2021-04-15T22:42:05Z

My original plan was to cheat and just add a isWorking boolean to the state and make new requests wait, but this method seems better. :)

Your way wouldn't work either, since as I said even synchronous API requests are impacted if there's an ongoing async request. There's no way to block a synchronous request unless you decide to immediately/synchronously finish the work for the async one.

I'll take a look at updating state tracking over the weekend.

rossnichols · 2021-04-16T05:50:38Z

@oratory I've just merged the update to encapsulate state - see #5. Please rebase your branch before your next round of changes.

rossnichols · 2021-04-16T21:31:49Z

I wonder if the yield time should be expressed in microseconds, rather than number of items?

oratory · 2021-04-16T21:36:07Z

I was thinking that - then on the WoW side it could be set by the current FPS.

rossnichols · 2021-04-16T21:37:47Z

I was thinking that - then on the WoW side it could be set by the current FPS.

I don't think the library itself would do that, but rather you'd include an end-to-end example in the readme that covers the WoW-specific integration (using a frame's OnUpdate handler, calculating the yield time based on current FPS, etc.

rossnichols · 2021-04-22T03:08:55Z

@oratory any update?

oratory · 2021-04-22T05:17:43Z

Ya I've got some updates in the works, just a bit swamped lately. Will update this week.

oratory · 2021-04-26T03:02:27Z

I changed things around a bit. Instead of being a typical async function, I'm calling it Chunks Mode which returns a handler which gets called on repeat until processing is finished.

Some rough benchmarks

Seralize & Compress

Small table: 115 Bytes

Standard	Chunks Mode
Serialize: 0.21 ms	Serialize: 1.19 ms
Compress: 1.30 ms	Compress: 10.59 ms
Total: 1.51 ms	Total: 11.78 ms

Medium Table: 163584 Bytes

Standard	Chunks Mode
Serialize: 10 ms	Serialize: 12 ms
Compress: 579 ms	Compress: 674 ms
Total: 589 ms	Total: 686 ms

Large Table: 1750073 Bytes

Standard	Chunks Mode
Serialize: 560 ms	Serialize: 609 ms
Compress: 7578 ms	Compress: 8323 ms
Total: 8128 ms	Total: 8932 ms

X-Large Table: 3511073 Bytes

Standard	Chunks Mode
Serialize: 1130 ms	Serialize: 1223 ms
Compress: 16489 ms	Compress: 16700 ms
Total: 17589 ms	Total: 17923 ms

Decompress and Deserialize

Small table: 115 Bytes

Standard	Chunks Mode
Decompress: 0.24 ms	Decompress: 9 ms
Deserialize: 0.10 ms	Deserialize: 11 ms
Total: 0.34 ms	Total: 20 ms

Medium Table: 163584 Bytes

Standard	Chunks Mode
Decompress: 192 ms	Decompress: 264 ms
Deserialize: 16 ms	Deserialize: 27 ms
Total: 208 ms	Total: 291 ms

Large Table: 1750073 Bytes

Standard	Chunks Mode
Decompress: 3094 ms	Decompress: 3668 ms
Deserialize: 574 ms	Deserialize: 772 ms
Total: 3668 ms	Total: 4440 ms

X-Large Table: 3511073 Bytes

Standard	Chunks Mode
Decompress: 6308 ms	Decompress: 7465 ms
Deserialize: 1074 ms	Deserialize: 1536 ms
Total: 7382 ms	Total: 9001 ms

Chunks Mode does have a small increase in processing time but has the benefit (in WoW and likely other environments) of not locking up the game and making the OS think it's non-responsive when processing large data.

rossnichols · 2021-04-26T16:16:34Z

Please rebase on top of latest master commit instead of merging, so that the PR doesn't have my recent changes in it.

oratory · 2021-04-26T23:57:46Z

Alrighty I think that did it.

rossnichols

There's still a lot to be done before I can approve this. If you'd like me to work on this feature instead of iterating on this PR, let me know - I could probably spend some time on it in the next week or so.

rossnichols · 2021-04-27T01:16:55Z

LibSerialize.lua

@@ -479,7 +558,7 @@ local function CreateWriter()
        bufferSize = 0
        return flushed
    end
-
+    


Remove this extra space (and reconfigure your editor to trim trailing spaces on save imo!)

rossnichols · 2021-04-27T01:18:04Z

LibSerialize.lua

@@ -633,6 +712,7 @@ end
 local LibSerializeInt = {}

 local function CreateSerializer(opts)
+    opts = opts or {}


Not necessary - remove.

rossnichols · 2021-04-27T01:20:44Z

LibSerialize.lua

@@ -700,6 +790,15 @@ end
 function LibSerializeInt:_ReadObject()
    local value = self:_ReadByte()

+    self.currentSize = (self.currentSize or 0) + value/8


Change to _currentSize, initialize in CreateDeserializer().

rossnichols · 2021-04-27T01:22:59Z

LibSerialize.lua

@@ -700,6 +790,15 @@ end
 function LibSerializeInt:_ReadObject()
    local value = self:_ReadByte()

+    self.currentSize = (self.currentSize or 0) + value/8


value has no relationship with the size.

rossnichols · 2021-04-27T01:23:48Z

LibSerialize.lua

+    if self._opts.chunksMode and (
+        (self.currentTime and self._opts.timeFn() - self.currentTime > self._opts.yieldOnChunkTime)
+        or self.currentSize > self._opts.yieldOnChunkSize) then
+        if self._opts.timeFn then self.currentTime = self._opts.timeFn() end


timeFn isn't included in the default options table (even as a nil entry, just for code documentation)

rossnichols · 2021-04-27T02:17:19Z

LibSerialize.lua

@@ -182,6 +209,24 @@ The following serialization options are supported:
    table encountered during serialization. The function must return true for
    the pair to be serialized. It may be called multiple times on a table for
    the same key/value pair. See notes on reeentrancy and table modification.
+* `chunksMode`: `boolean` (default false)


Rename to async

rossnichols · 2021-04-27T02:17:42Z

LibSerialize.lua

+* `chunksMode`: `boolean` (default false)
+  * `true`: the serialize function will return a coroutine handler to process
+    the serialization. If true, the following additional options are considered:
+    * `yieldOnChunkSize`: `number` How large to allow the buffer before yielding


Rename to yieldOnObjectCount

rossnichols · 2021-04-27T02:18:09Z

LibSerialize.lua

+  * `true`: the serialize function will return a coroutine handler to process
+    the serialization. If true, the following additional options are considered:
+    * `yieldOnChunkSize`: `number` How large to allow the buffer before yielding
+    * `yieldOnChunkTime`: `number` Max duration between yields


rename to yieldOnElapsedTime

rossnichols · 2021-04-27T02:18:35Z

LibSerialize.lua

+    * `yieldOnChunkTime`: `number` Max duration between yields
+    * `timeFn`: `function` To return the time in `number` format relevant to the
+      environment.
+  * `false`: the serialize function will return a the serialized string directly.


rossnichols · 2021-04-27T02:19:56Z

tests.lua

@@ -145,6 +145,30 @@ function LibSerialize:RunTests()
    end


+    --[[---------------------------------------------------------------------------


Testing is still insufficient.

oratory · 2021-04-30T06:33:30Z

Tests 1 (nil) and 35 (0/0) are currently failing on async, not sure why yet. Otherwise the other requested updates are completed.

LibSerialize.lua

README.md

tests.lua

rossnichols · 2021-05-15T22:12:09Z

@oratory any update?

oratory · 2021-05-16T06:34:47Z

Ya I've been working on it here and there, should have something to post tomorrow.

rossnichols · 2021-05-17T16:24:47Z

LibSerialize.lua

-    * `timeFn`: `function` To return the time in `number` format relevant to the
-      environment.
-  * `false`: the serialize function will return the serialized string directly.
+When using `SerializeAsyncEx()`, these additional options are supported:


Per my comments in the LibDeflate PR, I think these can be collapsed into a single option, where you supply either:

yieldCheckFn: function(objectCount: number) => boolean - a yield will occur if this function returns true, or
yieldOnObjectCount: number - a yield will occur after this number of objects

And then if neither are provided, yield after objects. With this approach, the library code just calls yieldCheckFn and does bookkeeping on the count of objects since yield, and the caller (or default args) handles the specifics. A time-based yield is then up to the caller to implement, which is easy (they just need to reset their own "elapsed" when returning true).

The yieldCheckFn works well, makes yieldOnObjectCount redundant even since it's now all self contained.

LibSerialize.lua

tests.lua

LibSerialize.lua

tests.lua

rossnichols · 2021-05-25T04:15:09Z

Can you please retarget your PR to be into the new async-staging branch I just created?

Stanzilla · 2024-02-29T14:48:51Z

@rossnichols hey we are revisiting this and were wondering if you can take a look at this again?

rossnichols · 2024-04-27T02:02:03Z

@Stanzilla sorry for the delay here. I've stopped playing WoW and would be happy to let others from the community help make changes here. Would you like to be added?

Stanzilla · 2024-04-27T20:49:10Z

@Stanzilla sorry for the delay here. I've stopped playing WoW and would be happy to let others from the community help make changes here. Would you like to be added?

Hey there, I wouldn't say "want" but I think it might be a good idea. We were considering a fork for Wago related content but I really don't want more on my/our plate so rather avoid that. So far Async is all we need so If we can get this PR into a good shape and merge (Can do that If you don't have time) that would be great.

cc @oratory @Nnoggie

rossnichols · 2024-04-27T20:56:09Z

Sounds good - if anyone from the community wants to be added to the repo I'm happy to do so. For this PR specifically, IIRC I was mostly wanting to ensure adequate test coverage and validating that existing tests are unaffected. I'd have to spend some time going back through it to refresh my memory on the actual code changes. It'd be great if you can take a pass and try to make it ready, I'll spend some time looking at it as well.

LibSerialize.lua

Nnoggie · 2024-05-01T18:59:57Z

Hello! I tested this PR against our use cases at Wago.
We aim to serialize (and compress) lots of big tables at once, hence the need for async versions of all the libraries that do that.
In the specific case for LibSerialize we want to export a lot of WeakAuras to export strings.

When accounting for the above mentioned issue it looks to be working nicely for our purposes and has similar performance compared to a seperate implementation that i temporarily used.

As Stan mentioned we are intested in this PR getting merged. I am more than happy to assist with the tests or other work.
Let me know what you think!

rossnichols · 2024-05-11T19:45:38Z

Alright well since this is targeting my async-staging branch, I'll go ahead and merge this and then take a look at merging into master. Thanks!

* Add chunks mode for coroutine processing * Requested updates * Split sync/async functions, others * Further updates

* Add async serialize option via coroutines (#4) * Add chunks mode for coroutine processing * Requested updates * Split sync/async functions, others * Further updates * Implement support for custom readers and writers (#12) * Implement generic support for custom readers This permits a new options table to be supplied to the Deserialize and DeserializeValue functions on the library which may contain two keys that control how the library reads bytes from the stream for deserialization, and how it detects the end-of-stream state. As-is this implementation is fully backwards compatible and allows users of the library to throw processing into a coroutine for large inputs if required, meeting some use cases that have prompted similar PRs whilst also being more flexible and much simpler. The new signatures for Deserialize and DeserializeValue are as follows: function LibSerialize:DeserializeValue(input[, options]) function LibSerialize:Deserialize(input[, options]) The options parameter, if supplied, is expected to be a table that may contain the following optional keys: - readBytes - atEnd The "readBytes" key if present is expected to point to a function of the signature "readBytes(input, i, j)" and must return a string whose length is equal to (j - i) + 1). The "input" parameter is the same as-supplied to Deserialize/DeserializeValue verbatim. The "i" and "j" parameter represent an inclusive range of indices ([i, j]) typically as byte offsets within the input value. This convention is identical to that provided by many Lua functions, such as string.sub, table.concat, and unpack. The "atEnd" key if present is expected to point to a function of the signature "atEnd(input, i)". As with "readBytes", the "input" parameter is whatever value was supplied to Deserialize/DeserializeValue, and the "i" parameter represents the offset of the stream. By default if neither of these keys are supplied, or no options table is given, then the implementation will just default to using string.sub for "readBytes" and for "atEnd" will test if the offset exceeds the length of the input as-provided by the length operator. Examples are provided in the test cases, where we test the following scenarios: - A case where "input" may have additional data after the payload that we don't want deserializing. - A case where the "input" is a table of character bytes. - A case where we yield from within "readBytes" and feed data from the main thread, simulating receiving data with a potentially unknown bound. * Implement support for custom writers This mirrors recent changes submitted for the deserialization API and provides a generic system for custom writer support to control how and where the byte stream generated during serialization is written. The change as-is is fully backwards compatible and passes the existing and expanded test cases. Primary use cases would align with existing async serialization attempts but don't put the complexity of thread management or throttling logic into the library, instead leaving the minutia of how those aspects are handled to callers. It also enables some other advanced use cases such as streaming the data to a destination without buffering it fully beforehand, which could enable efficiencies elsewhere. * Convert generic reader/writer to use objects instead of closures * Hoist async into reader/writer layer instead of serialize/deserialize * Rename function to GenericReader_AtEnd * Expand tests, add example for reader/writer protocol * Update readme * Fix formatting * Globals cleanup --------- Co-authored-by: Mark Bosley <mark.bosley@gmail.com> Co-authored-by: Daniel Yates <me@meorawr.io>

rossnichols requested changes Apr 12, 2021

View reviewed changes

rossnichols requested changes Apr 15, 2021

View reviewed changes

SafeteeWoW reviewed Apr 15, 2021

View reviewed changes

oratory mentioned this pull request Apr 26, 2021

Add asynchronous processing option via coroutines SafeteeWoW/LibDeflate#6

Closed

Add chunks mode for coroutine processing

3392253

oratory force-pushed the master branch from 8eeb716 to 3392253 Compare April 26, 2021 23:55

rossnichols requested changes Apr 27, 2021

View reviewed changes

Requested updates

b38bdc2

rossnichols requested changes May 1, 2021

View reviewed changes

This was referenced May 11, 2021

new shit emptyrivers/Archivist#6

Merged

Async support emptyrivers/Archivist#7

Open

Split sync/async functions, others

d5501fa

rossnichols requested changes May 17, 2021

View reviewed changes

Further updates

18ff122

rossnichols requested changes May 25, 2021

View reviewed changes

oratory changed the base branch from master to async-staging May 25, 2021 04:22

emptyrivers mentioned this pull request Jul 15, 2021

Suggestion: Distribute display to all raid members at the same time via caching and group-channels WeakAuras/WeakAuras2#3209

Closed

This was referenced Jan 15, 2022

Generic Reader Support #8

Closed

Generic Writer Support #10

Closed

Nnoggie reviewed May 1, 2024

View reviewed changes

LibSerialize.lua Show resolved Hide resolved

rossnichols merged commit b871125 into rossnichols:async-staging May 11, 2024

rossnichols pushed a commit that referenced this pull request May 16, 2024

Add async serialize option via coroutines (#4)

98eac0f

* Add chunks mode for coroutine processing * Requested updates * Split sync/async functions, others * Further updates

rossnichols pushed a commit that referenced this pull request May 16, 2024

Add async serialize option via coroutines (#4)

e5b17b5

* Add chunks mode for coroutine processing * Requested updates * Split sync/async functions, others * Further updates

rossnichols mentioned this pull request May 16, 2024

Merge async APIs and generic reader/writer support #11

Merged

		@@ -1265,10 +1279,13 @@ end

		function LibSerialize:SerializeEx(opts, ...)
		self:_ClearReferences()

		function LibSerialize:DeserializeAsync(input, callback, opts)
		opts = opts or {}

		@@ -145,6 +145,30 @@ function LibSerialize:RunTests()
		end


		--[[---------------------------------------------------------------------------

Add async serialize option via coroutines #4

Add async serialize option via coroutines #4

Conversation

oratory commented Apr 12, 2021

rossnichols left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

SafeteeWoW commented Apr 12, 2021

rossnichols commented Apr 12, 2021 • edited Loading

rossnichols commented Apr 13, 2021

oratory commented Apr 13, 2021

oratory commented Apr 14, 2021

rossnichols commented Apr 15, 2021

rossnichols left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

SafeteeWoW commented Apr 15, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rossnichols commented Apr 15, 2021

oratory commented Apr 15, 2021

rossnichols commented Apr 15, 2021

oratory commented Apr 15, 2021

rossnichols commented Apr 15, 2021

rossnichols commented Apr 16, 2021

rossnichols commented Apr 16, 2021

oratory commented Apr 16, 2021

rossnichols commented Apr 16, 2021

rossnichols commented Apr 22, 2021

oratory commented Apr 22, 2021

oratory commented Apr 26, 2021

Seralize & Compress

Decompress and Deserialize

rossnichols commented Apr 26, 2021

oratory commented Apr 26, 2021

rossnichols left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

oratory commented Apr 30, 2021

rossnichols commented May 15, 2021

oratory commented May 16, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rossnichols commented May 25, 2021

Stanzilla commented Feb 29, 2024

rossnichols commented Apr 27, 2024

Stanzilla commented Apr 27, 2024

rossnichols commented Apr 27, 2024

Nnoggie commented May 1, 2024

rossnichols commented May 11, 2024

rossnichols commented Apr 12, 2021 •

edited

Loading