Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

On disk cache serialization #6094

Merged

Conversation

@rokonec
Copy link
Contributor

@rokonec rokonec commented Jan 27, 2021

Context

BinaryFormater serialization of on-disk RAR cache is slow and unsecure.
Related issue: #6057

Changes Made

Serialization changed to use custom binary format by using existing ITranslatable

Testing

Unit testing
Manual testing

  • Roslyn repo rebuild + incremental build

Notes

@Forgind
Copy link
Member

@Forgind Forgind commented Jan 28, 2021

Looks roughly good to me, and as I said over Teams—don't worry about the mono failures.

@Forgind Forgind mentioned this pull request Feb 2, 2021
@rokonec rokonec self-assigned this Feb 2, 2021
@rokonec rokonec marked this pull request as ready for review Feb 2, 2021
@rokonec rokonec requested a review from Forgind Feb 2, 2021
@Forgind
Forgind approved these changes Feb 3, 2021
Copy link
Member

@Forgind Forgind left a comment

Looks good! Just a couple questions and some nits.

src/Build.UnitTests/BackEnd/BinaryTranslator_Tests.cs Outdated Show resolved Hide resolved
src/Build.UnitTests/BackEnd/BinaryTranslator_Tests.cs Outdated Show resolved Hide resolved
src/Shared/TranslatorHelpers.cs Outdated Show resolved Hide resolved
src/Tasks/SystemState.cs Outdated Show resolved Hide resolved
src/Tasks/SystemState.cs Outdated Show resolved Hide resolved
src/Tasks/SystemState.cs Outdated Show resolved Hide resolved
src/Tasks/SystemState.cs Outdated Show resolved Hide resolved
src/Tasks/TaskTranslatorHelpers.cs Show resolved Hide resolved
@cdmihai
Copy link
Contributor

@cdmihai cdmihai commented Feb 4, 2021

Very curious to see what the time differences are between this and master. Please post numbers once you have them :)

@cdmihai

This comment has been hidden.

@cdmihai
cdmihai approved these changes Feb 4, 2021
Copy link
Contributor

@cdmihai cdmihai left a comment

General approach looks good. Curios to see perf data comparisons.

@rokonec
Copy link
Contributor Author

@rokonec rokonec commented Feb 8, 2021

@cdmihai

General approach looks good. Curios to see perf data comparisons.

Perf data:

  • On rotational DISK incremental build of Roslyn is about 8% shorter at RAR wall clock time.
  • On SSD incremental build of OrchardCore is about 2% shorter at RAR wall clock time.
  • RAR cache on-disk size is about 5x smaller (20% of original size).
  • Memory allocated by RAR in incremental Roslyn build is about 15% smaller.
@ladipro ladipro merged commit 6819f7a into dotnet:master Feb 25, 2021
7 checks passed
7 checks passed
license/cla All CLA requirements met.
Details
@azure-pipelines
msbuild-pr Build #20210204.4 succeeded
Details
@azure-pipelines
msbuild-pr (Linux Core) Linux Core succeeded
Details
@azure-pipelines
msbuild-pr (Windows Core) Windows Core succeeded
Details
@azure-pipelines
msbuild-pr (Windows Full Release (no bootstrap)) Windows Full Release (no bootstrap) succeeded
Details
@azure-pipelines
msbuild-pr (Windows Full) Windows Full succeeded
Details
@azure-pipelines
msbuild-pr (macOS Core) macOS Core succeeded
Details
@rokonec rokonec deleted the rokonec:rokonec/6057-on-disk-cache-serialization branch Apr 1, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

4 participants