Enhanced DPR #521
Merged
Enhanced DPR #521
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* added sample * cleaned up async stress * Fix memory leak + add serialized wrapper sample * added testcases for concurrency within session in async. * clean up LMD
* Initial checkin of test suite for remote * updated configs. * fix sln * Update azure-pipelines.yml * Update azure-pipelines.yml * Update 50-remote-basics.md * Update 50-remote-basics.md
* Initial checkin of test suite for remote * updated configs. * fix sln * Update azure-pipelines.yml * Update azure-pipelines.yml * Ensure correct disposals of server and client. * rename testcase * Update 50-remote-basics.md * Update 50-remote-basics.md
* [C#] Fix double dispose in CompletePendingWithOutputs * Update MultiReadSpanByteKey.cs * Cleaned up testcase, merged with other SpanByte test. Co-authored-by: Tim Coleman <timothy.coleman@gmail.com>
…pulating) (microsoft#455) * don't pre-create 120 read and 120 write file handles. * fix asyncpool semaphore sizing. allow control over LocalStorageNamedDeviceFactory's Device.ThrottleLimit * dispose off the semaphore in AsyncPool.Dispose() * minor fixes to AsyncPool. * Speed up async pool. * Fix ordering of GetOrAdd and wait. Co-authored-by: Badrish Chandramouli <badrishc@microsoft.com>
…t#462) * Updated SpanByte sample with RMW for ASCII number sums * Sample refactoring as class per sub-sample
…ly by default commit manager. (microsoft#494)
* Add <LangVersion>latest to csprojs to pick up C#9; remove "is {}" in favor of "is not null"; use "Type x = new()", which is particularly useful for field definitions
* Add net5.0 framework to core component builds; add "|| NET" to "#if NETSTANDARD" where necessary; switch samples to net5.0; add Debug builds to Linux
* Update azure-pipelines.yml
* Move C# Windows tasks to dotnet; add build/test tasks for Remote to C# pipelines
* search for all *.test.csproj in build/test tasks
* Restrict Linux to netcoreapp3.1 and net5.0
* remove duplicate langversion
Co-authored-by: Badrish Chandramouli <badrishc@microsoft.com>
* Add BackendProvider concept to support backend other than FasterKV. Add protocol byte to binary protocol to enable support for more protocols in the future. * Fix some typos and add some comments * add .idea to gitignore * Add constructor overload to avoid API change, tweaks to serialization performance. * remove unused import * Cleanup & minor perf improvements Co-authored-by: Tianyu Li <t-litianyu@microsoft.com> Co-authored-by: Badrish Chandramouli <badrishc@microsoft.com>
* Refactor remote provider * Cleanup and speedup * cleanup and updates
Co-authored-by: Tianyu Li <t-litianyu@microsoft.com>
* make error more detailed * try sorting by creation time for linux compat * change test to use specific token, as workaround for linux timestamp sorting granularity issue. * Ignore socket disposal exception in RecvEventArg_Completed * Restore full Azure pipeline
* Add Output to RMW: - ** BREAKING CHANGE ** I(Advanced)Functions.CopyUpdater, InitialUpdater, InPlaceUpdater, and RMWCompletionCallback now have a ref Output parameter - ** BREAKING CHANGE ** RmwAsyncResult.Complete now returns a (Status, Output) tuple * Fix Remote IFunctions implementations to new RMW signatures * Change RMW callback signature (move Output to end) * Update Remote to new RMW Output signature; add $ReadPercentages to run_benchmark * Add Output to NeedCopyUpdate as well * Propagate RMW Output throughout Remote (ICallbackFunctions etc.); update docs for RMW Output
* Added SpanByteFasterKVProvider, and changed SpanByteSerializer to SpanByteServerSerializer and SpanByteClientSerializer * Removed unnecessary functions / debugs * fixed nit
…t#519) * Support log commit path specification to FasterLog. * updates * Convert serialized SpanByte into its non-serialized form (safe to heap-copy as long as underlying payload is fixed)
…anced-dpr # Conflicts: # .gitignore
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
New and extended DPR model to support low latency DPR commits.
Added EnhancedDprFinder and associated implementations. StateObjects can now choose to persist DPR dependency information along with checkpoint data in one I/O roundtrip. Given this, EnhancedDprFinder can eliminate the need for additional persistence on the DPR commit critical path.
New solution is backward compatible for the original DPR finder. This is useful for cases where it is not feasible to write DPR header to checkpoints.