lib/model: Optimize jobQueue performance and memory use #8023
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
DeepSource wants crypto/rand to be used even in tests. That seems kind of extreme. |
By truncating time.Time to an int64 nanosecond count, we lose the ability to precisely order timestamps before 1678 or after 2262, but we gain (linux/amd64, Go 1.17.1): name old time/op new time/op delta JobQueuePushPopDone10k-8 2.85ms ± 5% 2.29ms ± 2% -19.80% (p=0.000 n=20+18) JobQueueBump-8 34.0µs ± 1% 29.8µs ± 1% -12.35% (p=0.000 n=19+19) name old alloc/op new alloc/op delta JobQueuePushPopDone10k-8 2.56MB ± 0% 1.76MB ± 0% -31.31% (p=0.000 n=18+13) name old allocs/op new allocs/op delta JobQueuePushPopDone10k-8 23.0 ± 0% 23.0 ± 0% ~ (all equal) Results for BenchmarkJobQueueBump are with the fixed version, which no longer depends on b.N for the amount of work performed. rand.Rand.Intn is cheap at ~10ns per iteration.
|
Yeah deepsource can go pound sand in this case. |
calmh
approved these changes
Oct 29, 2021
calmh
added a commit
to calmh/syncthing
that referenced
this pull request
Nov 22, 2021
* main: (32 commits) cmd/syncthing: Implement generate as a subcommand with optional API credential setting (fixes syncthing#8021) (syncthing#8043) lib/model: Correct "reverting folder" log entry lib/model: Correct handling of fakefs cache gui, lib: Fix tracking deleted locally-changed on encrypted (fixes syncthing#7715) (syncthing#7726) lib/config: Move the bcrypt password hashing to GUIConfiguration (syncthing#8028) lib/syncthing: Clean up / refactor LoadOrGenerateCertificate() utility function. (syncthing#8025) lib/api: http.Request.BasicAuth instead of custom code (syncthing#8039) Normalize CLI options to always use two dashes. (syncthing#8037) gui: Display identicons for discovered device IDs. (syncthing#8022) cmd/syncthing/cli: indexDumpSize doesn't need a heap (syncthing#8024) lib/model: Optimize jobQueue performance and memory use (syncthing#8023) lib/model: Limit the number of default hashers on Android (ref syncthing#2220) lib/model: Set mod. time after writing trailer in shortcut (ref syncthing#7992) lib/protocol: Simplify codeToError, errorToCode lib/protocol: Eliminate nativeModel on Unix gui: Add direct link to Ignore Patterns from folder panel (fixes syncthing#4293) (syncthing#7993) gui: Translate theme names in settings (syncthing#8006) lib/model: Pull when a new connection is established (fixes syncthing#8012) (syncthing#8013) gui, man, authors: Update docs, translations, and contributors gui, man, authors: Update docs, translations, and contributors ...
calmh
added a commit
to calmh/syncthing
that referenced
this pull request
Nov 22, 2021
* main: (32 commits) cmd/syncthing: Implement generate as a subcommand with optional API credential setting (fixes syncthing#8021) (syncthing#8043) lib/model: Correct "reverting folder" log entry lib/model: Correct handling of fakefs cache gui, lib: Fix tracking deleted locally-changed on encrypted (fixes syncthing#7715) (syncthing#7726) lib/config: Move the bcrypt password hashing to GUIConfiguration (syncthing#8028) lib/syncthing: Clean up / refactor LoadOrGenerateCertificate() utility function. (syncthing#8025) lib/api: http.Request.BasicAuth instead of custom code (syncthing#8039) Normalize CLI options to always use two dashes. (syncthing#8037) gui: Display identicons for discovered device IDs. (syncthing#8022) cmd/syncthing/cli: indexDumpSize doesn't need a heap (syncthing#8024) lib/model: Optimize jobQueue performance and memory use (syncthing#8023) lib/model: Limit the number of default hashers on Android (ref syncthing#2220) lib/model: Set mod. time after writing trailer in shortcut (ref syncthing#7992) lib/protocol: Simplify codeToError, errorToCode lib/protocol: Eliminate nativeModel on Unix gui: Add direct link to Ignore Patterns from folder panel (fixes syncthing#4293) (syncthing#7993) gui: Translate theme names in settings (syncthing#8006) lib/model: Pull when a new connection is established (fixes syncthing#8012) (syncthing#8013) gui, man, authors: Update docs, translations, and contributors gui, man, authors: Update docs, translations, and contributors ...
st-review
added
the
frozen-due-to-age
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
By truncating time.Time to an int64 nanosecond count, we lose the ability to precisely order timestamps before 1678 or after 2262, but we gain (linux/amd64, Go 1.17.1):
Results for BenchmarkJobQueueBump are with the fixed version, which no longer depends on b.N for the amount of work performed. rand.Rand.Intn is cheap at ~10ns per iteration.
The choice in Go 1.16, the current minimum required version, is between seconds and nanoseconds. When Go 1.17 becomes the minimum, the time.Time.UnixMicro or UnixMilli methods can be used instead to strike a different balance between timescale and precision.