internal/zstd: optimize window reset using make for buffer allocation #64221

aimuz · 2023-11-17T13:46:40Z

Optimize the zstd window's reset function to use make for buffer
allocation when needed. This ensures efficient memory usage by
conditionally reallocating the buffer based on its capacity, improving
performance in scenarios where the existing buffer is insufficient.

goos: darwin
goarch: amd64
pkg: internal/zstd
cpu: Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz
         │   old.out   │            new.out            │
         │   sec/op    │   sec/op     vs base          │
Large-12   7.299m ± 5%   7.235m ± 1%  ~ (p=0.353 n=10)

         │   old.out    │            new.out             │
         │     B/s      │     B/s       vs base          │
Large-12   35.56Mi ± 4%   35.87Mi ± 1%  ~ (p=0.353 n=10)

         │   old.out    │               new.out                │
         │     B/op     │     B/op      vs base                │
Large-12   68.86Ki ± 4%   15.87Ki ± 1%  -76.95% (p=0.000 n=10)

         │  old.out   │            new.out             │
         │ allocs/op  │ allocs/op   vs base            │
Large-12   0.000 ± 0%   0.000 ± 0%  ~ (p=1.000 n=10) ¹
¹ all samples are equal

gopherbot · 2023-11-17T13:52:43Z

This PR (HEAD: aca6717) has been imported to Gerrit for code review.

Please visit Gerrit at https://go-review.googlesource.com/c/go/+/543375.

Important tips:

Don't comment on this PR. All discussion takes place in Gerrit.
You need a Gmail or other Google account to log in to Gerrit.
To change your code in response to feedback:
- Push a new commit to the branch used by your GitHub PR.
- A new "patch set" will then appear in Gerrit.
- Respond to each comment by marking as Done in Gerrit if implemented as suggested. You can alternatively write a reply.
- Critical: you must click the blue Reply button near the top to publish your Gerrit responses.
- Multiple commits in the PR will be squashed by GerritBot.
The title and description of the GitHub PR are used to construct the final commit message.
- Edit these as needed via the GitHub web interface (not via Gerrit or git).
- You should word wrap the PR description at ~76 characters unless you need longer lines (e.g., for tables or URLs).
See the Sending a change via GitHub and Reviews sections of the Contribution Guide as well as the FAQ for details.

gopherbot · 2023-11-17T14:00:06Z

Message from Gopher Robot:

Patch Set 1:

(1 comment)

Please don’t reply on this GitHub thread. Visit golang.org/cl/543375.
After addressing review feedback, remember to publish your drafts!

gopherbot · 2023-11-17T14:07:40Z

Message from qiulaidongfeng:

Patch Set 1: Run-TryBot+1

Please don’t reply on this GitHub thread. Visit golang.org/cl/543375.
After addressing review feedback, remember to publish your drafts!

gopherbot · 2023-11-17T14:15:05Z

Message from Gopher Robot:

Patch Set 1:

(1 comment)

Please don’t reply on this GitHub thread. Visit golang.org/cl/543375.
After addressing review feedback, remember to publish your drafts!

gopherbot · 2023-11-17T14:22:38Z

Message from Gopher Robot:

Patch Set 1: TryBot-Result+1

(1 comment)

Please don’t reply on this GitHub thread. Visit golang.org/cl/543375.
After addressing review feedback, remember to publish your drafts!

gopherbot · 2023-11-18T02:21:08Z

Message from Aimuz me:

Patch Set 1:

(1 comment)

Please don’t reply on this GitHub thread. Visit golang.org/cl/543375.
After addressing review feedback, remember to publish your drafts!

gopherbot · 2023-11-18T06:04:21Z

Message from Ingo Oeser:

Patch Set 1:

(2 comments)

Please don’t reply on this GitHub thread. Visit golang.org/cl/543375.
After addressing review feedback, remember to publish your drafts!

gopherbot · 2023-11-18T11:59:06Z

Message from Aimuz me:

Patch Set 1:

(1 comment)

Please don’t reply on this GitHub thread. Visit golang.org/cl/543375.
After addressing review feedback, remember to publish your drafts!

gopherbot · 2023-11-18T12:06:06Z

This PR (HEAD: 8cde50e) has been imported to Gerrit for code review.

Please visit Gerrit at https://go-review.googlesource.com/c/go/+/543375.

Important tips:

Don't comment on this PR. All discussion takes place in Gerrit.
You need a Gmail or other Google account to log in to Gerrit.
To change your code in response to feedback:
- Push a new commit to the branch used by your GitHub PR.
- A new "patch set" will then appear in Gerrit.
- Respond to each comment by marking as Done in Gerrit if implemented as suggested. You can alternatively write a reply.
- Critical: you must click the blue Reply button near the top to publish your Gerrit responses.
- Multiple commits in the PR will be squashed by GerritBot.
The title and description of the GitHub PR are used to construct the final commit message.
- Edit these as needed via the GitHub web interface (not via Gerrit or git).
- You should word wrap the PR description at ~76 characters unless you need longer lines (e.g., for tables or URLs).
See the Sending a change via GitHub and Reviews sections of the Contribution Guide as well as the FAQ for details.

gopherbot · 2023-11-21T10:45:16Z

Message from Aimuz me:

Patch Set 2:

(1 comment)

Please don’t reply on this GitHub thread. Visit golang.org/cl/543375.
After addressing review feedback, remember to publish your drafts!

gopherbot · 2023-11-22T05:31:21Z

Message from Jorropo:

Patch Set 2:

(1 comment)

Please don’t reply on this GitHub thread. Visit golang.org/cl/543375.
After addressing review feedback, remember to publish your drafts!

gopherbot · 2023-11-22T07:42:39Z

Message from Aimuz me:

Patch Set 2:

(1 comment)

Please don’t reply on this GitHub thread. Visit golang.org/cl/543375.
After addressing review feedback, remember to publish your drafts!

gopherbot · 2023-11-22T08:10:22Z

Message from Ingo Oeser:

Patch Set 2:

(2 comments)

Please don’t reply on this GitHub thread. Visit golang.org/cl/543375.
After addressing review feedback, remember to publish your drafts!

gopherbot · 2023-11-22T08:17:19Z

Message from Aimuz me:

Patch Set 2:

(1 comment)

Please don’t reply on this GitHub thread. Visit golang.org/cl/543375.
After addressing review feedback, remember to publish your drafts!

gopherbot · 2023-11-22T08:24:26Z

Message from Aimuz me:

Patch Set 2:

(1 comment)

Please don’t reply on this GitHub thread. Visit golang.org/cl/543375.
After addressing review feedback, remember to publish your drafts!

gopherbot · 2023-11-22T08:24:35Z

This PR (HEAD: 2a13fed) has been imported to Gerrit for code review.

Please visit Gerrit at https://go-review.googlesource.com/c/go/+/543375.

Important tips:

Don't comment on this PR. All discussion takes place in Gerrit.
You need a Gmail or other Google account to log in to Gerrit.
To change your code in response to feedback:
- Push a new commit to the branch used by your GitHub PR.
- A new "patch set" will then appear in Gerrit.
- Respond to each comment by marking as Done in Gerrit if implemented as suggested. You can alternatively write a reply.
- Critical: you must click the blue Reply button near the top to publish your Gerrit responses.
- Multiple commits in the PR will be squashed by GerritBot.
The title and description of the GitHub PR are used to construct the final commit message.
- Edit these as needed via the GitHub web interface (not via Gerrit or git).
- You should word wrap the PR description at ~76 characters unless you need longer lines (e.g., for tables or URLs).
See the Sending a change via GitHub and Reviews sections of the Contribution Guide as well as the FAQ for details.

gopherbot · 2023-11-22T08:38:39Z

Message from Jorropo:

Patch Set 3:

(1 comment)

Please don’t reply on this GitHub thread. Visit golang.org/cl/543375.
After addressing review feedback, remember to publish your drafts!

gopherbot · 2023-11-23T11:40:04Z

Message from Aimuz me:

Patch Set 3:

(1 comment)

Please don’t reply on this GitHub thread. Visit golang.org/cl/543375.
After addressing review feedback, remember to publish your drafts!

gopherbot · 2023-11-23T11:40:13Z

This PR (HEAD: 3221685) has been imported to Gerrit for code review.

Please visit Gerrit at https://go-review.googlesource.com/c/go/+/543375.

Important tips:

Don't comment on this PR. All discussion takes place in Gerrit.
You need a Gmail or other Google account to log in to Gerrit.
To change your code in response to feedback:
- Push a new commit to the branch used by your GitHub PR.
- A new "patch set" will then appear in Gerrit.
- Respond to each comment by marking as Done in Gerrit if implemented as suggested. You can alternatively write a reply.
- Critical: you must click the blue Reply button near the top to publish your Gerrit responses.
- Multiple commits in the PR will be squashed by GerritBot.
The title and description of the GitHub PR are used to construct the final commit message.
- Edit these as needed via the GitHub web interface (not via Gerrit or git).
- You should word wrap the PR description at ~76 characters unless you need longer lines (e.g., for tables or URLs).
See the Sending a change via GitHub and Reviews sections of the Contribution Guide as well as the FAQ for details.

gopherbot · 2023-11-23T12:56:19Z

Message from Jorropo:

Patch Set 4:

(1 comment)

Please don’t reply on this GitHub thread. Visit golang.org/cl/543375.
After addressing review feedback, remember to publish your drafts!

gopherbot · 2023-11-24T04:26:02Z

Message from Aimuz me:

Patch Set 4:

(1 comment)

Please don’t reply on this GitHub thread. Visit golang.org/cl/543375.
After addressing review feedback, remember to publish your drafts!

gopherbot · 2023-12-16T08:59:08Z

Message from qiulaidongfeng:

Patch Set 4: Run-TryBot+1

Please don’t reply on this GitHub thread. Visit golang.org/cl/543375.
After addressing review feedback, remember to publish your drafts!

gopherbot · 2023-12-16T08:59:09Z

Message from Gopher Robot:

Patch Set 4:

(1 comment)

Please don’t reply on this GitHub thread. Visit golang.org/cl/543375.
After addressing review feedback, remember to publish your drafts!

gopherbot · 2023-12-16T09:05:44Z

Message from Gopher Robot:

Patch Set 4:

(1 comment)

Please don’t reply on this GitHub thread. Visit golang.org/cl/543375.
After addressing review feedback, remember to publish your drafts!

gopherbot · 2023-12-16T09:05:45Z

Message from Gopher Robot:

Patch Set 4: TryBot-Result-1

(1 comment)

Please don’t reply on this GitHub thread. Visit golang.org/cl/543375.
After addressing review feedback, remember to publish your drafts!

gopherbot · 2023-12-19T04:44:15Z

Message from Aimuz me:

Patch Set 4:

(1 comment)

Please don’t reply on this GitHub thread. Visit golang.org/cl/543375.
After addressing review feedback, remember to publish your drafts!

gopherbot · 2024-01-17T08:26:36Z

Message from qiulaidongfeng:

Patch Set 5: Run-TryBot+1

Please don’t reply on this GitHub thread. Visit golang.org/cl/543375.
After addressing review feedback, remember to publish your drafts!

gopherbot · 2024-01-17T08:34:00Z

Message from Gopher Robot:

Patch Set 5:

(1 comment)

Please don’t reply on this GitHub thread. Visit golang.org/cl/543375.
After addressing review feedback, remember to publish your drafts!

gopherbot · 2024-01-17T08:48:17Z

Message from Gopher Robot:

Patch Set 5: TryBot-Result+1

(1 comment)

Please don’t reply on this GitHub thread. Visit golang.org/cl/543375.
After addressing review feedback, remember to publish your drafts!

gopherbot · 2024-03-04T02:29:33Z

Message from Ian Lance Taylor:

Patch Set 5:

(2 comments)

Please don’t reply on this GitHub thread. Visit golang.org/cl/543375.
After addressing review feedback, remember to publish your drafts!

gopherbot · 2024-03-04T12:52:24Z

This PR (HEAD: f2b1223) has been imported to Gerrit for code review.

Please visit Gerrit at https://go-review.googlesource.com/c/go/+/543375.

Important tips:

Don't comment on this PR. All discussion takes place in Gerrit.
You need a Gmail or other Google account to log in to Gerrit.
To change your code in response to feedback:
- Push a new commit to the branch used by your GitHub PR.
- A new "patch set" will then appear in Gerrit.
- Respond to each comment by marking as Done in Gerrit if implemented as suggested. You can alternatively write a reply.
- Critical: you must click the blue Reply button near the top to publish your Gerrit responses.
- Multiple commits in the PR will be squashed by GerritBot.
The title and description of the GitHub PR are used to construct the final commit message.
- Edit these as needed via the GitHub web interface (not via Gerrit or git).
- You should word wrap the PR description at ~76 characters unless you need longer lines (e.g., for tables or URLs).
See the Sending a change via GitHub and Reviews sections of the Contribution Guide as well as the FAQ for details.

gopherbot · 2024-03-04T12:59:15Z

Message from Zhen Li:

Patch Set 5:

(3 comments)

Please don’t reply on this GitHub thread. Visit golang.org/cl/543375.
After addressing review feedback, remember to publish your drafts!

gopherbot · 2024-03-04T14:08:22Z

Message from Ian Lance Taylor:

Patch Set 6: Commit-Queue+1

Please don’t reply on this GitHub thread. Visit golang.org/cl/543375.
After addressing review feedback, remember to publish your drafts!

gopherbot · 2024-03-04T14:08:23Z

Message from Go LUCI:

Patch Set 6:

Dry run: CV is trying the patch.

Bot data: {"action":"start","triggered_at":"2024-03-04T14:01:58Z","revision":"3c8b3d6cea992017a3ac00a981efd6d44d1af75b"}

Please don’t reply on this GitHub thread. Visit golang.org/cl/543375.
After addressing review feedback, remember to publish your drafts!

gopherbot · 2024-03-04T14:15:18Z

Message from Ian Lance Taylor:

Patch Set 6: -Commit-Queue

Please don’t reply on this GitHub thread. Visit golang.org/cl/543375.
After addressing review feedback, remember to publish your drafts!

gopherbot · 2024-03-04T14:15:19Z

Message from Go LUCI:

Patch Set 6:

This CL has failed the run. Reason:

Tryjob golang/try/x_tools-gotip-linux-amd64 has failed with summary (view all results):

FAILURE

go on master (change 543375)
tools on master (commit 5bf7d00)

Error:
failed to run "make.bash": exit status 2

Links:

"make.bash" (combined output)

Please don’t reply on this GitHub thread. Visit golang.org/cl/543375.
After addressing review feedback, remember to publish your drafts!

gopherbot · 2024-03-04T14:15:20Z

Message from Go LUCI:

Patch Set 6: LUCI-TryBot-Result-1

Please don’t reply on this GitHub thread. Visit golang.org/cl/543375.
After addressing review feedback, remember to publish your drafts!

Optimize the zstd window's reset function to use `make` for buffer allocation when needed. This ensures efficient memory usage by conditionally reallocating the buffer based on its capacity, improving performance in scenarios where the existing buffer is insufficient.

gopherbot · 2024-03-05T03:03:02Z

This PR (HEAD: c00344c) has been imported to Gerrit for code review.

Please visit Gerrit at https://go-review.googlesource.com/c/go/+/543375.

Important tips:

Don't comment on this PR. All discussion takes place in Gerrit.
You need a Gmail or other Google account to log in to Gerrit.
To change your code in response to feedback:
- Push a new commit to the branch used by your GitHub PR.
- A new "patch set" will then appear in Gerrit.
- Respond to each comment by marking as Done in Gerrit if implemented as suggested. You can alternatively write a reply.
- Critical: you must click the blue Reply button near the top to publish your Gerrit responses.
- Multiple commits in the PR will be squashed by GerritBot.
The title and description of the GitHub PR are used to construct the final commit message.
- Edit these as needed via the GitHub web interface (not via Gerrit or git).
- You should word wrap the PR description at ~76 characters unless you need longer lines (e.g., for tables or URLs).
See the Sending a change via GitHub and Reviews sections of the Contribution Guide as well as the FAQ for details.

gopherbot · 2024-03-05T12:53:54Z

Message from qiulaidongfeng:

Patch Set 8: Commit-Queue+1

Please don’t reply on this GitHub thread. Visit golang.org/cl/543375.
After addressing review feedback, remember to publish your drafts!

gopherbot · 2024-03-05T12:53:55Z

Message from Go LUCI:

Patch Set 8:

Dry run: CV is trying the patch.

Bot data: {"action":"start","triggered_at":"2024-03-05T12:41:52Z","revision":"5587853420a83d085cdaf1404ff47ebe3d68c8c9"}

Please don’t reply on this GitHub thread. Visit golang.org/cl/543375.
After addressing review feedback, remember to publish your drafts!

gopherbot · 2024-03-05T13:14:50Z

Message from qiulaidongfeng:

Patch Set 8: -Commit-Queue

Please don’t reply on this GitHub thread. Visit golang.org/cl/543375.
After addressing review feedback, remember to publish your drafts!

gopherbot · 2024-03-05T13:14:51Z

Message from Go LUCI:

Patch Set 8:

This CL has passed the run

Please don’t reply on this GitHub thread. Visit golang.org/cl/543375.
After addressing review feedback, remember to publish your drafts!

gopherbot · 2024-03-05T13:14:52Z

Message from Go LUCI:

Patch Set 8: LUCI-TryBot-Result+1

Please don’t reply on this GitHub thread. Visit golang.org/cl/543375.
After addressing review feedback, remember to publish your drafts!

gopherbot · 2024-03-30T03:39:41Z

Message from qiulaidongfeng:

Patch Set 8: Code-Review+1

Please don’t reply on this GitHub thread. Visit golang.org/cl/543375.
After addressing review feedback, remember to publish your drafts!

gopherbot · 2024-04-01T18:06:59Z

Message from Ian Lance Taylor:

Patch Set 8: Auto-Submit+1 Code-Review+2 Commit-Queue+1

(1 comment)

Please don’t reply on this GitHub thread. Visit golang.org/cl/543375.
After addressing review feedback, remember to publish your drafts!

gopherbot · 2024-04-01T18:13:18Z

Message from Go LUCI:

Patch Set 8:

Dry run: CV is trying the patch.

Bot data: {"action":"start","triggered_at":"2024-04-01T17:59:59Z","revision":"5587853420a83d085cdaf1404ff47ebe3d68c8c9"}

Please don’t reply on this GitHub thread. Visit golang.org/cl/543375.
After addressing review feedback, remember to publish your drafts!

gopherbot · 2024-04-01T18:19:37Z

Message from Ian Lance Taylor:

Patch Set 8: -Commit-Queue

Please don’t reply on this GitHub thread. Visit golang.org/cl/543375.
After addressing review feedback, remember to publish your drafts!

gopherbot · 2024-04-01T18:19:38Z

Message from Go LUCI:

Patch Set 8:

This CL has passed the run

Please don’t reply on this GitHub thread. Visit golang.org/cl/543375.
After addressing review feedback, remember to publish your drafts!

Optimize the zstd window's reset function to use `make` for buffer allocation when needed. This ensures efficient memory usage by conditionally reallocating the buffer based on its capacity, improving performance in scenarios where the existing buffer is insufficient. ``` goos: darwin goarch: amd64 pkg: internal/zstd cpu: Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz │ old.out │ new.out │ │ sec/op │ sec/op vs base │ Large-12 7.299m ± 5% 7.235m ± 1% ~ (p=0.353 n=10) │ old.out │ new.out │ │ B/s │ B/s vs base │ Large-12 35.56Mi ± 4% 35.87Mi ± 1% ~ (p=0.353 n=10) │ old.out │ new.out │ │ B/op │ B/op vs base │ Large-12 68.86Ki ± 4% 15.87Ki ± 1% -76.95% (p=0.000 n=10) │ old.out │ new.out │ │ allocs/op │ allocs/op vs base │ Large-12 0.000 ± 0% 0.000 ± 0% ~ (p=1.000 n=10) ¹ ¹ all samples are equal ``` Change-Id: Icd9c278b42ea149107a9861d006f40129c59831b GitHub-Last-Rev: c00344c GitHub-Pull-Request: #64221 Reviewed-on: https://go-review.googlesource.com/c/go/+/543375 Reviewed-by: qiulaidongfeng <2645477756@qq.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Reviewed-by: Ian Lance Taylor <iant@google.com> Auto-Submit: Ian Lance Taylor <iant@google.com>

gopherbot · 2024-04-02T13:52:53Z

This PR is being closed because golang.org/cl/543375 has been merged.

aimuz force-pushed the fix-optimize branch from aca6717 to 8cde50e Compare November 18, 2023 12:00

aimuz force-pushed the fix-optimize branch from 8cde50e to 2a13fed Compare November 22, 2023 08:17

aimuz force-pushed the fix-optimize branch from 2a13fed to 3221685 Compare November 23, 2023 11:36

aimuz force-pushed the fix-optimize branch from 3221685 to 4401fbc Compare December 19, 2023 06:25

aimuz force-pushed the fix-optimize branch from 4401fbc to f2b1223 Compare March 4, 2024 12:47

aimuz changed the title ~~internal/zstd: optimize buffer grow to avoid multiple allocations~~ internal/zstd: optimize window reset with bytealg Mar 4, 2024

aimuz changed the title ~~internal/zstd: optimize window reset with bytealg~~ internal/zstd: optimize window reset using make for buffer allocation Mar 5, 2024

aimuz force-pushed the fix-optimize branch from f2b1223 to c00344c Compare March 5, 2024 02:56

gopherbot closed this Apr 2, 2024

internal/zstd: optimize window reset using make for buffer allocation #64221

internal/zstd: optimize window reset using make for buffer allocation #64221

Conversation

aimuz commented Nov 17, 2023 • edited

gopherbot commented Nov 17, 2023

gopherbot commented Nov 17, 2023

gopherbot commented Nov 17, 2023

gopherbot commented Nov 17, 2023

gopherbot commented Nov 17, 2023

gopherbot commented Nov 18, 2023

gopherbot commented Nov 18, 2023

gopherbot commented Nov 18, 2023

gopherbot commented Nov 18, 2023

gopherbot commented Nov 21, 2023

gopherbot commented Nov 22, 2023

gopherbot commented Nov 22, 2023

gopherbot commented Nov 22, 2023

gopherbot commented Nov 22, 2023

gopherbot commented Nov 22, 2023

gopherbot commented Nov 22, 2023

gopherbot commented Nov 22, 2023

gopherbot commented Nov 23, 2023

gopherbot commented Nov 23, 2023

gopherbot commented Nov 23, 2023

gopherbot commented Nov 24, 2023

gopherbot commented Dec 16, 2023

gopherbot commented Dec 16, 2023

gopherbot commented Dec 16, 2023

gopherbot commented Dec 16, 2023

gopherbot commented Dec 19, 2023

gopherbot commented Jan 17, 2024

gopherbot commented Jan 17, 2024

gopherbot commented Jan 17, 2024

gopherbot commented Mar 4, 2024

gopherbot commented Mar 4, 2024

gopherbot commented Mar 4, 2024

gopherbot commented Mar 4, 2024

gopherbot commented Mar 4, 2024

gopherbot commented Mar 4, 2024

gopherbot commented Mar 4, 2024

gopherbot commented Mar 4, 2024

gopherbot commented Mar 5, 2024

gopherbot commented Mar 5, 2024

gopherbot commented Mar 5, 2024

gopherbot commented Mar 5, 2024

gopherbot commented Mar 5, 2024

gopherbot commented Mar 5, 2024

gopherbot commented Mar 30, 2024

gopherbot commented Apr 1, 2024

gopherbot commented Apr 1, 2024

gopherbot commented Apr 1, 2024

gopherbot commented Apr 1, 2024

gopherbot commented Apr 2, 2024

aimuz commented Nov 17, 2023 •

edited