plumbing: Optimise memory consumption for filesystem storage #799
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
CI is currently broken, once #802 is merged this needs rebasing. |
pjbgf
force-pushed
the
perf2
branch
3 times, most recently
from
47cf139
to
60ef5df
Compare
|
Rebased and fixed issue with |
hiddeco
reviewed
Jul 11, 2023
|
CI is broken, fix will be handled on a separate PR. |
|
@pjbgf please fix your branch so we all enjoy your performance boost :) |
The changes aim to make that specific benchmark more reliable for setting a baseline which can later be use to compare against future changes on the most basic feature of go-git: plain cloning repositories.
Previously, as part of building the index representation, the resolveObject func would create an interim plumbing.MemoryObject, which would then be saved into storage via storage.SetEncodedObject. This meant that objects would be unnecessarily loaded into memory, to then be saved into disk. The changes streamlines this process by: - Introducing the LazyObjectWriter interface which enables the write operation to take places directly against the filesystem-based storage. - Leverage multi-writers to process the input data once, while targeting multiple writers (e.g. hasher and storage). An additional change relates to the caching of object info children within Parser.get. The cache is now skipped when a seekable filesystem is being used. The impact of the changes can be observed when using seekable filesystem storages, especially when cloning large repositories. The stats below were captured by adapting the BenchmarkPlainClone test to clone https://github.com/torvalds/linux.git: pkg: github.com/go-git/go-git/v5 cpu: Intel(R) Core(TM) i9-10885H CPU @ 2.40GHz │ /tmp/old │ /tmp/new │ │ sec/op │ sec/op vs base │ PlainClone-16 41.68 ± 17% 48.04 ± 9% +15.27% (p=0.015 n=6) │ /tmp/old │ /tmp/new │ │ B/op │ B/op vs base │ PlainClone-16 1127.8Mi ± 7% 256.7Mi ± 50% -77.23% (p=0.002 n=6) │ /tmp/old │ /tmp/new │ │ allocs/op │ allocs/op vs base │ PlainClone-16 3.125M ± 0% 3.800M ± 0% +21.60% (p=0.002 n=6) Notice that on average the memory consumption per operation is over 75% smaller. The time per operation increased by 15%, which may actual be less on long running applications, due to the decreased GC pressure and the garbage collection costs. Signed-off-by: Paulo Gomes <pjbgf@linux.com>
|
PR is now rebased. |
hiddeco
reviewed
Nov 6, 2023
| return 0, plumbing.ZeroHash, err | ||
| } | ||
|
|
||
| if isCopyFromSrc(cmd) { |
There was a problem hiding this comment.
Seems like a switch may be more fitting here as there is non-conditional else behavior which could be captured with a default.
There was a problem hiding this comment.
Good shout. I will add this change on a follow-up PR. 👍
renovate bot
added a commit
to anoriqq/qpm
that referenced
this pull request
Nov 27, 2023
[](https://renovatebot.com) This PR contains the following updates: | Package | Type | Update | Change | |---|---|---|---| | [github.com/go-git/go-git/v5](https://togithub.com/go-git/go-git) | require | patch | `v5.10.0` -> `v5.10.1` | --- ### Release Notes <details> <summary>go-git/go-git (github.com/go-git/go-git/v5)</summary> ### [`v5.10.1`](https://togithub.com/go-git/go-git/releases/tag/v5.10.1) [Compare Source](https://togithub.com/go-git/go-git/compare/v5.10.0...v5.10.1) #### What's Changed - Worktree, ignore ModeSocket files by [@​steiler](https://togithub.com/steiler) in [go-git/go-git#930 - git: add tracer package by [@​aymanbagabas](https://togithub.com/aymanbagabas) in [go-git/go-git#916 - remote: Flip clause for fast-forward only check by [@​adityasaky](https://togithub.com/adityasaky) in [go-git/go-git#875 - plumbing: transport/ssh, Fix nil pointer dereference caused when an unreachable proxy server is set. Fixes [#​900](https://togithub.com/go-git/go-git/issues/900) by [@​anandf](https://togithub.com/anandf) in [go-git/go-git#901 - plumbing: uppload-server-info, implement upload-server-info by [@​aymanbagabas](https://togithub.com/aymanbagabas) in [go-git/go-git#896 - plumbing: optimise memory consumption for filesystem storage by [@​pjbgf](https://togithub.com/pjbgf) in [go-git/go-git#799 - plumbing: format/packfile, Refactor patch delta by [@​pjbgf](https://togithub.com/pjbgf) in [go-git/go-git#908 - plumbing: fix empty uploadpack request error by [@​aymanbagabas](https://togithub.com/aymanbagabas) in [go-git/go-git#932 - plumbing: transport/git, Improve tests error message by [@​pjbgf](https://togithub.com/pjbgf) in [go-git/go-git#752 - plumbing: format/pktline, Respect pktline error-line errors by [@​aymanbagabas](https://togithub.com/aymanbagabas) in [go-git/go-git#936 - utils: remove ioutil.Pipe and use std library io.Pipe by [@​aymanbagabas](https://togithub.com/aymanbagabas) in [go-git/go-git#922 - utils: move trace to utils by [@​aymanbagabas](https://togithub.com/aymanbagabas) in [go-git/go-git#931 - cli: separate go module for cli by [@​aymanbagabas](https://togithub.com/aymanbagabas) in [go-git/go-git#914 - build: bump github.com/google/go-cmp from 0.5.9 to 0.6.0 by [@​dependabot](https://togithub.com/dependabot) in [go-git/go-git#887 - build: bump actions/setup-go from 3 to 4 by [@​dependabot](https://togithub.com/dependabot) in [go-git/go-git#891 - build: bump github.com/skeema/knownhosts from 1.2.0 to 1.2.1 by [@​dependabot](https://togithub.com/dependabot) in [go-git/go-git#888 - build: bump actions/checkout from 3 to 4 by [@​dependabot](https://togithub.com/dependabot) in [go-git/go-git#890 - build: bump golang.org/x/sys from 0.13.0 to 0.14.0 by [@​dependabot](https://togithub.com/dependabot) in [go-git/go-git#907 - build: bump golang.org/x/text from 0.13.0 to 0.14.0 by [@​dependabot](https://togithub.com/dependabot) in [go-git/go-git#906 - build: bump golang.org/x/crypto from 0.14.0 to 0.15.0 by [@​dependabot](https://togithub.com/dependabot) in [go-git/go-git#917 - build: bump golang.org/x/net from 0.17.0 to 0.18.0 by [@​dependabot](https://togithub.com/dependabot) in [go-git/go-git#918 #### New Contributors - [@​anandf](https://togithub.com/anandf) made their first contribution in [go-git/go-git#901 - [@​steiler](https://togithub.com/steiler) made their first contribution in [go-git/go-git#930 **Full Changelog**: go-git/go-git@v5.10.0...v5.10.1 </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Enabled. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://developer.mend.io/github/anoriqq/qpm). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy41OS44IiwidXBkYXRlZEluVmVyIjoiMzcuNTkuOCIsInRhcmdldEJyYW5jaCI6Im1haW4ifQ==--> Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
another-rex
pushed a commit
to google/osv-scanner
that referenced
this pull request
Dec 3, 2023
[](https://renovatebot.com) This PR contains the following updates: | Package | Type | Update | Change | |---|---|---|---| | [github.com/go-git/go-git/v5](https://togithub.com/go-git/go-git) | require | patch | `v5.10.0` -> `v5.10.1` | | [github.com/urfave/cli/v2](https://togithub.com/urfave/cli) | require | minor | `v2.25.7` -> `v2.26.0` | | golang.org/x/exp | require | digest | `9a3e603` -> `6522937` | | golang.org/x/term | require | minor | `v0.14.0` -> `v0.15.0` | --- ### Release Notes <details> <summary>go-git/go-git (github.com/go-git/go-git/v5)</summary> ### [`v5.10.1`](https://togithub.com/go-git/go-git/releases/tag/v5.10.1) [Compare Source](https://togithub.com/go-git/go-git/compare/v5.10.0...v5.10.1) #### What's Changed - Worktree, ignore ModeSocket files by [@​steiler](https://togithub.com/steiler) in [go-git/go-git#930 - git: add tracer package by [@​aymanbagabas](https://togithub.com/aymanbagabas) in [go-git/go-git#916 - remote: Flip clause for fast-forward only check by [@​adityasaky](https://togithub.com/adityasaky) in [go-git/go-git#875 - plumbing: transport/ssh, Fix nil pointer dereference caused when an unreachable proxy server is set. Fixes [#​900](https://togithub.com/go-git/go-git/issues/900) by [@​anandf](https://togithub.com/anandf) in [go-git/go-git#901 - plumbing: uppload-server-info, implement upload-server-info by [@​aymanbagabas](https://togithub.com/aymanbagabas) in [go-git/go-git#896 - plumbing: optimise memory consumption for filesystem storage by [@​pjbgf](https://togithub.com/pjbgf) in [go-git/go-git#799 - plumbing: format/packfile, Refactor patch delta by [@​pjbgf](https://togithub.com/pjbgf) in [go-git/go-git#908 - plumbing: fix empty uploadpack request error by [@​aymanbagabas](https://togithub.com/aymanbagabas) in [go-git/go-git#932 - plumbing: transport/git, Improve tests error message by [@​pjbgf](https://togithub.com/pjbgf) in [go-git/go-git#752 - plumbing: format/pktline, Respect pktline error-line errors by [@​aymanbagabas](https://togithub.com/aymanbagabas) in [go-git/go-git#936 - utils: remove ioutil.Pipe and use std library io.Pipe by [@​aymanbagabas](https://togithub.com/aymanbagabas) in [go-git/go-git#922 - utils: move trace to utils by [@​aymanbagabas](https://togithub.com/aymanbagabas) in [go-git/go-git#931 - cli: separate go module for cli by [@​aymanbagabas](https://togithub.com/aymanbagabas) in [go-git/go-git#914 - build: bump github.com/google/go-cmp from 0.5.9 to 0.6.0 by [@​dependabot](https://togithub.com/dependabot) in [go-git/go-git#887 - build: bump actions/setup-go from 3 to 4 by [@​dependabot](https://togithub.com/dependabot) in [go-git/go-git#891 - build: bump github.com/skeema/knownhosts from 1.2.0 to 1.2.1 by [@​dependabot](https://togithub.com/dependabot) in [go-git/go-git#888 - build: bump actions/checkout from 3 to 4 by [@​dependabot](https://togithub.com/dependabot) in [go-git/go-git#890 - build: bump golang.org/x/sys from 0.13.0 to 0.14.0 by [@​dependabot](https://togithub.com/dependabot) in [go-git/go-git#907 - build: bump golang.org/x/text from 0.13.0 to 0.14.0 by [@​dependabot](https://togithub.com/dependabot) in [go-git/go-git#906 - build: bump golang.org/x/crypto from 0.14.0 to 0.15.0 by [@​dependabot](https://togithub.com/dependabot) in [go-git/go-git#917 - build: bump golang.org/x/net from 0.17.0 to 0.18.0 by [@​dependabot](https://togithub.com/dependabot) in [go-git/go-git#918 #### New Contributors - [@​anandf](https://togithub.com/anandf) made their first contribution in [go-git/go-git#901 - [@​steiler](https://togithub.com/steiler) made their first contribution in [go-git/go-git#930 **Full Changelog**: go-git/go-git@v5.10.0...v5.10.1 </details> <details> <summary>urfave/cli (github.com/urfave/cli/v2)</summary> ### [`v2.26.0`](https://togithub.com/urfave/cli/releases/tag/v2.26.0) [Compare Source](https://togithub.com/urfave/cli/compare/v2.25.7...v2.26.0) #### What's Changed - Bash completion nits by [@​meatballhat](https://togithub.com/meatballhat) in [urfave/cli#1762 - Chore: Rename mkdocs requirements file name by [@​dearchap](https://togithub.com/dearchap) in [urfave/cli#1776 - Fix:(issue\_1787) Add fix for commands not listed when hide help comma… by [@​dearchap](https://togithub.com/dearchap) in [urfave/cli#1788 - Fix nil HelpFlag panic (v2) by [@​wxiaoguang](https://togithub.com/wxiaoguang) in [urfave/cli#1795 - Always get 0 for a nested int64 value in v2.25.7 by [@​stephenfire](https://togithub.com/stephenfire) in [urfave/cli#1799 - Helper messages for documenting build process by [@​abitrolly](https://togithub.com/abitrolly) in [urfave/cli#1800 - fix: check duplicated sub command name and alias by [@​linrl3](https://togithub.com/linrl3) in [urfave/cli#1805 - Fix:(issue\_1689) Have consistent behavior for default text in man and… by [@​dearchap](https://togithub.com/dearchap) in [urfave/cli#1825 - Fix linting issues by [@​skelouse](https://togithub.com/skelouse) in [urfave/cli#1696 #### New Contributors - [@​stephenfire](https://togithub.com/stephenfire) made their first contribution in [urfave/cli#1799 - [@​linrl3](https://togithub.com/linrl3) made their first contribution in [urfave/cli#1805 **Full Changelog**: urfave/cli@v2.25.7...v2.26.0 </details> --- ### Configuration 📅 **Schedule**: Branch creation - "before 6am on monday" in timezone Australia/Sydney, Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 👻 **Immortal**: This PR will be recreated if closed unmerged. Get [config help](https://togithub.com/renovatebot/renovate/discussions) if that's undesired. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://developer.mend.io/github/google/osv-scanner). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy44MS4zIiwidXBkYXRlZEluVmVyIjoiMzcuODEuMyIsInRhcmdldEJyYW5jaCI6Im1haW4ifQ==-->
hogo6002
pushed a commit
to google/osv.dev
that referenced
this pull request
Dec 4, 2023
[](https://renovatebot.com) This PR contains the following updates: | Package | Type | Update | Change | |---|---|---|---| | | | lockFileMaintenance | All locks refreshed | | [cloud.google.com/go/secretmanager](https://togithub.com/googleapis/google-cloud-go) | require | patch | `v1.11.1` -> `v1.11.4` | | [github.com/go-git/go-git/v5](https://togithub.com/go-git/go-git) | require | minor | `v5.9.0` -> `v5.10.1` | | [github.com/google/go-cmp](https://togithub.com/google/go-cmp) | require | minor | `v0.5.9` -> `v0.6.0` | | [github.com/google/osv-scanner](https://togithub.com/google/osv-scanner) | require | patch | `v1.4.0` -> `v1.4.3` | | golang | stage | digest | `f475434` -> `70afe55` | | golang.org/x/exp | require | digest | `7918f67` -> `6522937` | 🔧 This Pull Request updates lock files to use the latest dependency versions. --- ### Release Notes <details> <summary>go-git/go-git (github.com/go-git/go-git/v5)</summary> ### [`v5.10.1`](https://togithub.com/go-git/go-git/releases/tag/v5.10.1) [Compare Source](https://togithub.com/go-git/go-git/compare/v5.10.0...v5.10.1) #### What's Changed - Worktree, ignore ModeSocket files by [@​steiler](https://togithub.com/steiler) in [go-git/go-git#930 - git: add tracer package by [@​aymanbagabas](https://togithub.com/aymanbagabas) in [go-git/go-git#916 - remote: Flip clause for fast-forward only check by [@​adityasaky](https://togithub.com/adityasaky) in [go-git/go-git#875 - plumbing: transport/ssh, Fix nil pointer dereference caused when an unreachable proxy server is set. Fixes [#​900](https://togithub.com/go-git/go-git/issues/900) by [@​anandf](https://togithub.com/anandf) in [go-git/go-git#901 - plumbing: uppload-server-info, implement upload-server-info by [@​aymanbagabas](https://togithub.com/aymanbagabas) in [go-git/go-git#896 - plumbing: optimise memory consumption for filesystem storage by [@​pjbgf](https://togithub.com/pjbgf) in [go-git/go-git#799 - plumbing: format/packfile, Refactor patch delta by [@​pjbgf](https://togithub.com/pjbgf) in [go-git/go-git#908 - plumbing: fix empty uploadpack request error by [@​aymanbagabas](https://togithub.com/aymanbagabas) in [go-git/go-git#932 - plumbing: transport/git, Improve tests error message by [@​pjbgf](https://togithub.com/pjbgf) in [go-git/go-git#752 - plumbing: format/pktline, Respect pktline error-line errors by [@​aymanbagabas](https://togithub.com/aymanbagabas) in [go-git/go-git#936 - utils: remove ioutil.Pipe and use std library io.Pipe by [@​aymanbagabas](https://togithub.com/aymanbagabas) in [go-git/go-git#922 - utils: move trace to utils by [@​aymanbagabas](https://togithub.com/aymanbagabas) in [go-git/go-git#931 - cli: separate go module for cli by [@​aymanbagabas](https://togithub.com/aymanbagabas) in [go-git/go-git#914 - build: bump github.com/google/go-cmp from 0.5.9 to 0.6.0 by [@​dependabot](https://togithub.com/dependabot) in [go-git/go-git#887 - build: bump actions/setup-go from 3 to 4 by [@​dependabot](https://togithub.com/dependabot) in [go-git/go-git#891 - build: bump github.com/skeema/knownhosts from 1.2.0 to 1.2.1 by [@​dependabot](https://togithub.com/dependabot) in [go-git/go-git#888 - build: bump actions/checkout from 3 to 4 by [@​dependabot](https://togithub.com/dependabot) in [go-git/go-git#890 - build: bump golang.org/x/sys from 0.13.0 to 0.14.0 by [@​dependabot](https://togithub.com/dependabot) in [go-git/go-git#907 - build: bump golang.org/x/text from 0.13.0 to 0.14.0 by [@​dependabot](https://togithub.com/dependabot) in [go-git/go-git#906 - build: bump golang.org/x/crypto from 0.14.0 to 0.15.0 by [@​dependabot](https://togithub.com/dependabot) in [go-git/go-git#917 - build: bump golang.org/x/net from 0.17.0 to 0.18.0 by [@​dependabot](https://togithub.com/dependabot) in [go-git/go-git#918 #### New Contributors - [@​anandf](https://togithub.com/anandf) made their first contribution in [go-git/go-git#901 - [@​steiler](https://togithub.com/steiler) made their first contribution in [go-git/go-git#930 **Full Changelog**: go-git/go-git@v5.10.0...v5.10.1 ### [`v5.10.0`](https://togithub.com/go-git/go-git/releases/tag/v5.10.0) [Compare Source](https://togithub.com/go-git/go-git/compare/v5.9.0...v5.10.0) #### What's Changed - PlainInitOptions.Bare and allow using InitOptions with PlainInitWithOptions by [@​ThinkChaos](https://togithub.com/ThinkChaos) in [go-git/go-git#782 - Worktree, apply ProxyOption on Pull by [@​nodivbyzero](https://togithub.com/nodivbyzero) in [go-git/go-git#840 - Repository: add clone --shared feature by [@​enverbisevac](https://togithub.com/enverbisevac) in [go-git/go-git#860 - build: Add github workflow to check commit message format by [@​pjbgf](https://togithub.com/pjbgf) in [go-git/go-git#867 - Improve handling of remote errors by [@​makkes](https://togithub.com/makkes) in [go-git/go-git#866 - build(deps): bump golang.org/x/net from 0.15.0 to 0.17.0 by [@​dependabot](https://togithub.com/dependabot) in [go-git/go-git#873 - plumbing: commitgraph, Add generation v2 support by [@​zeripath](https://togithub.com/zeripath) in [go-git/go-git#869 - plumbing: protocol/packp, Add validation for decodeLine by [@​pjbgf](https://togithub.com/pjbgf) in [go-git/go-git#868 - plumbing: parse the encoding header of the commit object by [@​liwenqiu](https://togithub.com/liwenqiu) in [go-git/go-git#761 - plumbing: commitgraph, allow SHA256 commit-graphs by [@​zeripath](https://togithub.com/zeripath) in [go-git/go-git#853 - plumbing: commitgraph, Allow reading commit-graph chains by [@​zeripath](https://togithub.com/zeripath) in [go-git/go-git#854 - plumbing/object: Support mergetag in merge commits by [@​adityasaky](https://togithub.com/adityasaky) in [go-git/go-git#847 #### New Contributors - [@​nodivbyzero](https://togithub.com/nodivbyzero) made their first contribution in [go-git/go-git#840 - [@​adityasaky](https://togithub.com/adityasaky) made their first contribution in [go-git/go-git#847 - [@​hezhizhen](https://togithub.com/hezhizhen) made their first contribution in [go-git/go-git#836 - [@​0x34d](https://togithub.com/0x34d) made their first contribution in [go-git/go-git#855 - [@​liwenqiu](https://togithub.com/liwenqiu) made their first contribution in [go-git/go-git#761 - [@​enverbisevac](https://togithub.com/enverbisevac) made their first contribution in [go-git/go-git#860 - [@​makkes](https://togithub.com/makkes) made their first contribution in [go-git/go-git#866 **Full Changelog**: go-git/go-git@v5.9.0...v5.10.0 </details> <details> <summary>google/go-cmp (github.com/google/go-cmp)</summary> ### [`v0.6.0`](https://togithub.com/google/go-cmp/releases/tag/v0.6.0) [Compare Source](https://togithub.com/google/go-cmp/compare/v0.5.9...v0.6.0) New API: - ([#​340](https://togithub.com/google/go-cmp/issues/340)) Add `cmpopts.EquateComparable` Documentation changes: - ([#​337](https://togithub.com/google/go-cmp/issues/337)) Use of hotlinking of Go identifiers Build changes: - ([#​325](https://togithub.com/google/go-cmp/issues/325)) Remove purego fallbacks Testing changes: - ([#​322](https://togithub.com/google/go-cmp/issues/322)) Run tests for Go 1.20 version - ([#​332](https://togithub.com/google/go-cmp/issues/332)) Pin GitHub action versions - ([#​327](https://togithub.com/google/go-cmp/issues/327)) set workflow permission to read-only </details> <details> <summary>google/osv-scanner (github.com/google/osv-scanner)</summary> ### [`v1.4.3`](https://togithub.com/google/osv-scanner/blob/HEAD/CHANGELOG.md#v143) [Compare Source](https://togithub.com/google/osv-scanner/compare/v1.4.2...v1.4.3) ##### Features - [Feature #​621](https://togithub.com/google/osv-scanner/pull/621) Add support for scanning vendored C/C++ files. - [Feature #​581](https://togithub.com/google/osv-scanner/pull/581) Scan submodules commit hashes. ##### Fixes - [Bug #​626](https://togithub.com/google/osv-scanner/issues/626) Fix gitignore matching for root directory - [Bug #​622](https://togithub.com/google/osv-scanner/issues/622) Go binary not found should not be an error - [Bug #​588](https://togithub.com/google/osv-scanner/issues/588) handle npm/yarn aliased packages - [Bug #​607](https://togithub.com/google/osv-scanner/pull/607) fix: remove some extra newlines in sarif report ### [`v1.4.2`](https://togithub.com/google/osv-scanner/blob/HEAD/CHANGELOG.md#v142) [Compare Source](https://togithub.com/google/osv-scanner/compare/v1.4.1...v1.4.2) ##### Fixes - [Bug #​574](https://togithub.com/google/osv-scanner/issues/574) Support versions with build metadata in `yarn.lock` files - [Bug #​599](https://togithub.com/google/osv-scanner/issues/599) Add name field to sarif rule output ### [`v1.4.1`](https://togithub.com/google/osv-scanner/blob/HEAD/CHANGELOG.md#v141) [Compare Source](https://togithub.com/google/osv-scanner/compare/v1.4.0...v1.4.1) ##### Features - [Feature #​534](https://togithub.com/google/osv-scanner/pull/534) New SARIF format that separates out individual vulnerabilities, see https://github.com/google/osv-scanner/issue/216 - [Experimental Feature #​57](https://togithub.com/google/osv-scanner/issues/57) Experimental Github Action! Have a look at https://google.github.io/osv-scanner/experimental/ for how to use the new Github Action in your repo. Experimental, so might change with only a minor update. ##### API Features - [Feature #​557](https://togithub.com/google/osv-scanner/pull/557) Add new ecosystems, and a slice containing all of them. </details> --- ### Configuration 📅 **Schedule**: Branch creation - "before 4am on monday" in timezone Australia/Sydney, Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 👻 **Immortal**: This PR will be recreated if closed unmerged. Get [config help](https://togithub.com/renovatebot/renovate/discussions) if that's undesired. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://developer.mend.io/github/google/osv.dev). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy41OS44IiwidXBkYXRlZEluVmVyIjoiMzcuNTkuOCIsInRhcmdldEJyYW5jaCI6Im1hc3RlciJ9-->
amenowanna
added a commit
to mergestat/mergestat
that referenced
this pull request
Mar 13, 2024
](https://renovatebot.com){kind=link}
…rity] (#1148) [](https://renovatebot.com) This PR contains the following updates: | Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | [github.com/go-git/go-git/v5](https://togithub.com/go-git/go-git) | `v5.5.2` -> `v5.11.0` | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | ### GitHub Vulnerability Alerts #### [CVE-2023-49568](https://togithub.com/go-git/go-git/security/advisories/GHSA-mw99-9chc-xw7r) ### Impact A denial of service (DoS) vulnerability was discovered in go-git versions prior to `v5.11`. This vulnerability allows an attacker to perform denial of service attacks by providing specially crafted responses from a Git server which triggers resource exhaustion in `go-git` clients. Applications using only the in-memory filesystem supported by `go-git` are not affected by this vulnerability. This is a `go-git` implementation issue and does not affect the upstream `git` cli. ### Patches Users running versions of `go-git` from `v4` and above are recommended to upgrade to `v5.11` in order to mitigate this vulnerability. ### Workarounds In cases where a bump to the latest version of `go-git` is not possible, we recommend limiting its use to only trust-worthy Git servers. ## Credit Thanks to Ionut Lalu for responsibly disclosing this vulnerability to us. ### References - [GHSA-mw99-9chc-xw7r](https://togithub.com/go-git/go-git/security/advisories/GHSA-mw99-9chc-xw7r) #### [CVE-2023-49569](https://togithub.com/go-git/go-git/security/advisories/GHSA-449p-3h89-pw88) ### Impact A path traversal vulnerability was discovered in go-git versions prior to `v5.11`. This vulnerability allows an attacker to create and amend files across the filesystem. In the worse case scenario, remote code execution could be achieved. Applications are only affected if they are using the [ChrootOS](https://pkg.go.dev/github.com/go-git/go-billy/v5/osfs#ChrootOS), which is the default when using "Plain" versions of Open and Clone funcs (e.g. PlainClone). Applications using [BoundOS](https://pkg.go.dev/github.com/go-git/go-billy/v5/osfs#BoundOS) or in-memory filesystems are not affected by this issue. This is a `go-git` implementation issue and does not affect the upstream `git` cli. ### Patches Users running versions of `go-git` from `v4` and above are recommended to upgrade to `v5.11` in order to mitigate this vulnerability. ### Workarounds In cases where a bump to the latest version of `go-git` is not possible in a timely manner, we recommend limiting its use to only trust-worthy Git servers. ## Credit Thanks to Ionut Lalu for responsibly disclosing this vulnerability to us. --- ### Release Notes <details> <summary>go-git/go-git (github.com/go-git/go-git/v5)</summary> ### [`v5.11.0`](https://togithub.com/go-git/go-git/releases/tag/v5.11.0) [Compare Source](https://togithub.com/go-git/go-git/compare/v5.10.1...v5.11.0) #### What's Changed - git: validate reference names ([#​929](https://togithub.com/go-git/go-git/issues/929)) by [@​aymanbagabas](https://togithub.com/aymanbagabas) in [go-git/go-git#950 - git: stop iterating at oldest shallow when pulling. Fixes [#​305](https://togithub.com/go-git/go-git/issues/305) by [@​dhoizner](https://togithub.com/dhoizner) in [go-git/go-git#939 - plumbing: object, enable renames in getFileStatsFromFilePatches by [@​djmoch](https://togithub.com/djmoch) in [go-git/go-git#941 - storage: filesystem, Add option to set a specific FS for alternates by [@​pjbgf](https://togithub.com/pjbgf) in [go-git/go-git#953 - Align worktree validation with upstream and remove build warnings by [@​pjbgf](https://togithub.com/pjbgf) in [go-git/go-git#958 #### New Contributors - [@​dhoizner](https://togithub.com/dhoizner) made their first contribution in [go-git/go-git#939 - [@​djmoch](https://togithub.com/djmoch) made their first contribution in [go-git/go-git#941 **Full Changelog**: go-git/go-git@v5.10.1...v5.11.0 ### [`v5.10.1`](https://togithub.com/go-git/go-git/releases/tag/v5.10.1) [Compare Source](https://togithub.com/go-git/go-git/compare/v5.10.0...v5.10.1) #### What's Changed - Worktree, ignore ModeSocket files by [@​steiler](https://togithub.com/steiler) in [go-git/go-git#930 - git: add tracer package by [@​aymanbagabas](https://togithub.com/aymanbagabas) in [go-git/go-git#916 - remote: Flip clause for fast-forward only check by [@​adityasaky](https://togithub.com/adityasaky) in [go-git/go-git#875 - plumbing: transport/ssh, Fix nil pointer dereference caused when an unreachable proxy server is set. Fixes [#​900](https://togithub.com/go-git/go-git/issues/900) by [@​anandf](https://togithub.com/anandf) in [go-git/go-git#901 - plumbing: uppload-server-info, implement upload-server-info by [@​aymanbagabas](https://togithub.com/aymanbagabas) in [go-git/go-git#896 - plumbing: optimise memory consumption for filesystem storage by [@​pjbgf](https://togithub.com/pjbgf) in [go-git/go-git#799 - plumbing: format/packfile, Refactor patch delta by [@​pjbgf](https://togithub.com/pjbgf) in [go-git/go-git#908 - plumbing: fix empty uploadpack request error by [@​aymanbagabas](https://togithub.com/aymanbagabas) in [go-git/go-git#932 - plumbing: transport/git, Improve tests error message by [@​pjbgf](https://togithub.com/pjbgf) in [go-git/go-git#752 - plumbing: format/pktline, Respect pktline error-line errors by [@​aymanbagabas](https://togithub.com/aymanbagabas) in [go-git/go-git#936 - utils: remove ioutil.Pipe and use std library io.Pipe by [@​aymanbagabas](https://togithub.com/aymanbagabas) in [go-git/go-git#922 - utils: move trace to utils by [@​aymanbagabas](https://togithub.com/aymanbagabas) in [go-git/go-git#931 - cli: separate go module for cli by [@​aymanbagabas](https://togithub.com/aymanbagabas) in [go-git/go-git#914 - build: bump github.com/google/go-cmp from 0.5.9 to 0.6.0 by [@​dependabot](https://togithub.com/dependabot) in [go-git/go-git#887 - build: bump actions/setup-go from 3 to 4 by [@​dependabot](https://togithub.com/dependabot) in [go-git/go-git#891 - build: bump github.com/skeema/knownhosts from 1.2.0 to 1.2.1 by [@​dependabot](https://togithub.com/dependabot) in [go-git/go-git#888 - build: bump actions/checkout from 3 to 4 by [@​dependabot](https://togithub.com/dependabot) in [go-git/go-git#890 - build: bump golang.org/x/sys from 0.13.0 to 0.14.0 by [@​dependabot](https://togithub.com/dependabot) in [go-git/go-git#907 - build: bump golang.org/x/text from 0.13.0 to 0.14.0 by [@​dependabot](https://togithub.com/dependabot) in [go-git/go-git#906 - build: bump golang.org/x/crypto from 0.14.0 to 0.15.0 by [@​dependabot](https://togithub.com/dependabot) in [go-git/go-git#917 - build: bump golang.org/x/net from 0.17.0 to 0.18.0 by [@​dependabot](https://togithub.com/dependabot) in [go-git/go-git#918 #### New Contributors - [@​anandf](https://togithub.com/anandf) made their first contribution in [go-git/go-git#901 - [@​steiler](https://togithub.com/steiler) made their first contribution in [go-git/go-git#930 **Full Changelog**: go-git/go-git@v5.10.0...v5.10.1 ### [`v5.10.0`](https://togithub.com/go-git/go-git/releases/tag/v5.10.0) [Compare Source](https://togithub.com/go-git/go-git/compare/v5.9.0...v5.10.0) #### What's Changed - PlainInitOptions.Bare and allow using InitOptions with PlainInitWithOptions by [@​ThinkChaos](https://togithub.com/ThinkChaos) in [go-git/go-git#782 - Worktree, apply ProxyOption on Pull by [@​nodivbyzero](https://togithub.com/nodivbyzero) in [go-git/go-git#840 - Repository: add clone --shared feature by [@​enverbisevac](https://togithub.com/enverbisevac) in [go-git/go-git#860 - build: Add github workflow to check commit message format by [@​pjbgf](https://togithub.com/pjbgf) in [go-git/go-git#867 - Improve handling of remote errors by [@​makkes](https://togithub.com/makkes) in [go-git/go-git#866 - build(deps): bump golang.org/x/net from 0.15.0 to 0.17.0 by [@​dependabot](https://togithub.com/dependabot) in [go-git/go-git#873 - plumbing: commitgraph, Add generation v2 support by [@​zeripath](https://togithub.com/zeripath) in [go-git/go-git#869 - plumbing: protocol/packp, Add validation for decodeLine by [@​pjbgf](https://togithub.com/pjbgf) in [go-git/go-git#868 - plumbing: parse the encoding header of the commit object by [@​liwenqiu](https://togithub.com/liwenqiu) in [go-git/go-git#761 - plumbing: commitgraph, allow SHA256 commit-graphs by [@​zeripath](https://togithub.com/zeripath) in [go-git/go-git#853 - plumbing: commitgraph, Allow reading commit-graph chains by [@​zeripath](https://togithub.com/zeripath) in [go-git/go-git#854 - plumbing/object: Support mergetag in merge commits by [@​adityasaky](https://togithub.com/adityasaky) in [go-git/go-git#847 #### New Contributors - [@​nodivbyzero](https://togithub.com/nodivbyzero) made their first contribution in [go-git/go-git#840 - [@​adityasaky](https://togithub.com/adityasaky) made their first contribution in [go-git/go-git#847 - [@​hezhizhen](https://togithub.com/hezhizhen) made their first contribution in [go-git/go-git#836 - [@​0x34d](https://togithub.com/0x34d) made their first contribution in [go-git/go-git#855 - [@​liwenqiu](https://togithub.com/liwenqiu) made their first contribution in [go-git/go-git#761 - [@​enverbisevac](https://togithub.com/enverbisevac) made their first contribution in [go-git/go-git#860 - [@​makkes](https://togithub.com/makkes) made their first contribution in [go-git/go-git#866 **Full Changelog**: go-git/go-git@v5.9.0...v5.10.0 ### [`v5.9.0`](https://togithub.com/go-git/go-git/releases/tag/v5.9.0) [Compare Source](https://togithub.com/go-git/go-git/compare/v5.8.1...v5.9.0) #### What's Changed - git: worktree: add Amend option to CommitOptions by [@​john-cai](https://togithub.com/john-cai) in [go-git/go-git#438 - git: worktree, reset ignored files that are part of the worktree: Fixes [#​819](https://togithub.com/go-git/go-git/issues/819) by [@​daolis](https://togithub.com/daolis) in [go-git/go-git#821 - plumbing: Do not swallow http message coming from VCS providers by [@​matejrisek](https://togithub.com/matejrisek) in [go-git/go-git#835 - plumbing: transport, handle IPv6 while parsing endpoint. Fixes [#​740](https://togithub.com/go-git/go-git/issues/740) by [@​ninedraft](https://togithub.com/ninedraft) in [go-git/go-git#820 - \*: update goproxy dependency to fix CVE-2023-37788 vulnerability by [@​svghadi](https://togithub.com/svghadi) in [go-git/go-git#832 - \*: bump dependencies and Go to 1.19 by [@​pjbgf](https://togithub.com/pjbgf) in [go-git/go-git#837 #### New Contributors - [@​svghadi](https://togithub.com/svghadi) made their first contribution in [go-git/go-git#832 - [@​daolis](https://togithub.com/daolis) made their first contribution in [go-git/go-git#821 **Full Changelog**: go-git/go-git@v5.8.1...v5.9.0 ### [`v5.8.1`](https://togithub.com/go-git/go-git/releases/tag/v5.8.1) [Compare Source](https://togithub.com/go-git/go-git/compare/v5.8.0...v5.8.1) #### What's Changed - \*: Bump dependencies by [@​pjbgf](https://togithub.com/pjbgf) in [go-git/go-git#815 **Full Changelog**: go-git/go-git@v5.8.0...v5.8.1 ### [`v5.8.0`](https://togithub.com/go-git/go-git/releases/tag/v5.8.0) [Compare Source](https://togithub.com/go-git/go-git/compare/v5.7.0...v5.8.0) #### What's Changed - git: Fix fetching after shallow clone. Fixes [#​305](https://togithub.com/go-git/go-git/issues/305) by [@​AriehSchneier](https://togithub.com/AriehSchneier) in [go-git/go-git#778 - git: enable fetch with unqualified references by [@​AriehSchneier](https://togithub.com/AriehSchneier) in [go-git/go-git#762 - git: don't add to want if exists, shallow and depth 1 by [@​AriehSchneier](https://togithub.com/AriehSchneier) in [go-git/go-git#763 - git: Clone HEAD should not force master. Fixes [#​363](https://togithub.com/go-git/go-git/issues/363) by [@​AriehSchneier](https://togithub.com/AriehSchneier) in [go-git/go-git#758 - git: fix the issue with submodules having the SCP style URL fail due to the wrong URL parsing by [@​matejrisek](https://togithub.com/matejrisek) in [go-git/go-git#756 - git: add a clone option to allow for shallow cloning of submodules by [@​matejrisek](https://togithub.com/matejrisek) in [go-git/go-git#765 - worktree: minor speedup for `doAddDirectory` by [@​ThinkChaos](https://togithub.com/ThinkChaos) in [go-git/go-git#702 - \_examples: Remove wrong comment by [@​pascal-hofmann](https://togithub.com/pascal-hofmann) in [go-git/go-git#357 - \*: Handle paths starting with tilde by [@​ricci2511](https://togithub.com/ricci2511) in [go-git/go-git#808 - \*: Handle paths starting with ~Username by [@​AriehSchneier](https://togithub.com/AriehSchneier) in [go-git/go-git#809 - storage: filesystem/dotgit, add support for tmp_objdir prefix by [@​L11R](https://togithub.com/L11R) in [go-git/go-git#812 - plumbing: gitignore, replace user dir in path by [@​Jleagle](https://togithub.com/Jleagle) in [go-git/go-git#772 - plumbing: gitignore, fix incorrect parsing. Fixes [#​500](https://togithub.com/go-git/go-git/issues/500) by [@​AriehSchneier](https://togithub.com/AriehSchneier) in [go-git/go-git#781 - plumbing: http, Fix empty repos on Git v2.41+ by [@​pjbgf](https://togithub.com/pjbgf) in [go-git/go-git#802 - plumbing: packp, A request is not empty if it contains shallows. Fixes [#​328](https://togithub.com/go-git/go-git/issues/328) by [@​AriehSchneier](https://togithub.com/AriehSchneier) in [go-git/go-git#792 - plumbing: blame, Complete rewrite. Fixes [#​603](https://togithub.com/go-git/go-git/issues/603) by [@​AriehSchneier](https://togithub.com/AriehSchneier) in [go-git/go-git#789 - plumbing: gitignore, Allow gitconfig to contain a gitignore relative to any user home. Fixes [#​578](https://togithub.com/go-git/go-git/issues/578) by [@​AriehSchneier](https://togithub.com/AriehSchneier) in [go-git/go-git#785 #### New Contributors - [@​Jleagle](https://togithub.com/Jleagle) made their first contribution in [go-git/go-git#772 - [@​pascal-hofmann](https://togithub.com/pascal-hofmann) made their first contribution in [go-git/go-git#357 - [@​ricci2511](https://togithub.com/ricci2511) made their first contribution in [go-git/go-git#808 - [@​L11R](https://togithub.com/L11R) made their first contribution in [go-git/go-git#812 **Full Changelog**: go-git/go-git@v5.7.0...v5.7.1 ### [`v5.7.0`](https://togithub.com/go-git/go-git/releases/tag/v5.7.0) [Compare Source](https://togithub.com/go-git/go-git/compare/v5.6.1...v5.7.0) #### What's Changed - \*: Add support for initializing SHA256 repositories by [@​pjbgf](https://togithub.com/pjbgf) in [go-git/go-git#707 - git: add mirror clone option by [@​aymanbagabas](https://togithub.com/aymanbagabas) in [go-git/go-git#735 - git: Add support to ls-remote with peeled references. Fixes [#​749](https://togithub.com/go-git/go-git/issues/749) by [@​pjbgf](https://togithub.com/pjbgf) in [go-git/go-git#750 - git: fix cloning with branch name by [@​AriehSchneier](https://togithub.com/AriehSchneier) in [go-git/go-git#755 - git: Worktree, add check to see if file already checked in. Fixes [#​718](https://togithub.com/go-git/go-git/issues/718) by [@​cbbm142](https://togithub.com/cbbm142) in [go-git/go-git#719 - git: Worktree, git grep bare repositories by [@​aymanbagabas](https://togithub.com/aymanbagabas) in [go-git/go-git#728 - git: Add Depth to SubmoduleUpdateOptions by [@​matejrisek](https://togithub.com/matejrisek) in [go-git/go-git#754 - git: Testing, Fix tests not cleaning temp folders by [@​AriehSchneier](https://togithub.com/AriehSchneier) in [go-git/go-git#769 - git: remote, add support for a configurable timeout. by [@​andrewpollock](https://togithub.com/andrewpollock) in [go-git/go-git#753 - git: Allow Initial Branch to be configurable by [@​techknowlogick](https://togithub.com/techknowlogick) in [go-git/go-git#764 - storage: filesystem/dotgit, Improve load packed-refs by [@​fcharlie](https://togithub.com/fcharlie) in [go-git/go-git#743 - storage: filesystem, Populate index before use. Fixes [#​148](https://togithub.com/go-git/go-git/issues/148) by [@​AriehSchneier](https://togithub.com/AriehSchneier) in [go-git/go-git#722 - plumbing: resolve non-external delta references by [@​ZauberNerd](https://togithub.com/ZauberNerd) in [go-git/go-git#485 - plumbing/transport: fix regression in scp-like match by [@​jotadrilo](https://togithub.com/jotadrilo) in [go-git/go-git#715 - plumbing/transport: Add support for custom proxy settings by [@​aryan9600](https://togithub.com/aryan9600) in [go-git/go-git#744 - \*: small fixes across the codebase by [@​pjbgf](https://togithub.com/pjbgf) in [go-git/go-git#770 - \*: bump github.com/cloudflare/circl from 1.1.0 to 1.3.3 by [@​dependabot](https://togithub.com/dependabot) in [go-git/go-git#776 - \*: bump dependencies by [@​pjbgf](https://togithub.com/pjbgf) in [go-git/go-git#748 - \*: bump Go version to 1.18 on go.mod by [@​pjbgf](https://togithub.com/pjbgf) in [go-git/go-git#774 - \*: add Codeql workflow and bump dependencies by [@​pjbgf](https://togithub.com/pjbgf) in [go-git/go-git#775 - ci: fix upstream git build for master branch by [@​pjbgf](https://togithub.com/pjbgf) in [go-git/go-git#739 #### New Contributors - [@​ZauberNerd](https://togithub.com/ZauberNerd) made their first contribution in [go-git/go-git#485 - [@​jotadrilo](https://togithub.com/jotadrilo) made their first contribution in [go-git/go-git#715 - [@​fcharlie](https://togithub.com/fcharlie) made their first contribution in [go-git/go-git#743 - [@​AriehSchneier](https://togithub.com/AriehSchneier) made their first contribution in [go-git/go-git#755 - [@​cbbm142](https://togithub.com/cbbm142) made their first contribution in [go-git/go-git#719 - [@​aryan9600](https://togithub.com/aryan9600) made their first contribution in [go-git/go-git#744 - [@​matejrisek](https://togithub.com/matejrisek) made their first contribution in [go-git/go-git#754 - [@​andrewpollock](https://togithub.com/andrewpollock) made their first contribution in [go-git/go-git#753 - [@​techknowlogick](https://togithub.com/techknowlogick) made their first contribution in [go-git/go-git#764 **Full Changelog**: go-git/go-git@v5.6.1...v5.7.0 ### [`v5.6.1`](https://togithub.com/go-git/go-git/releases/tag/v5.6.1) [Compare Source](https://togithub.com/go-git/go-git/compare/v5.6.0...v5.6.1) #### What's Changed - plumbing/transport: don't use the `firstErrLine` when it is empty by [@​ThinkChaos](https://togithub.com/ThinkChaos) in [go-git/go-git#682 - plumbing/transport: ssh, unable to pass a custom HostKeyCallback func by [@​aymanbagabas](https://togithub.com/aymanbagabas) in [go-git/go-git#655 - storage/filesystem: dotgit: fix a filesystem race in Refs/walkReferencesTree by [@​MichaelMure](https://togithub.com/MichaelMure) in [go-git/go-git#659 - \*: bump golang.org/x/net from 0.2.0 to 0.7.0 by [@​dependabot](https://togithub.com/dependabot) in [go-git/go-git#684 - \*: bump dependencies by [@​pjbgf](https://togithub.com/pjbgf) in [go-git/go-git#697 - \*: fix panic for empty revisions by [@​pjbgf](https://togithub.com/pjbgf) in [go-git/go-git#696 - ci: bump GitHub actions, enable go test race detection and stop using developer's GPG keys during test execution by [@​pjbgf](https://togithub.com/pjbgf) in [go-git/go-git#701 **Full Changelog**: go-git/go-git@v5.6.0...v5.6.1 ### [`v5.6.0`](https://togithub.com/go-git/go-git/releases/tag/v5.6.0) [Compare Source](https://togithub.com/go-git/go-git/compare/v5.5.2...v5.6.0) #### What's Changed - Worktree, check for empty parent dirs during Reset (Fixes [#​670](https://togithub.com/go-git/go-git/issues/670)) by [@​mbohy](https://togithub.com/mbohy) in [go-git/go-git#671 - \*: remove need to build with CGO by [@​pjbgf](https://togithub.com/pjbgf) in [go-git/go-git#688 - plumbing: support SSH/X509 signed tags by [@​hiddeco](https://togithub.com/hiddeco) in [go-git/go-git#690 **Full Changelog**: go-git/go-git@v5.5.2...v5.6.0 </details> --- ### Configuration 📅 **Schedule**: Branch creation - "" (UTC), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://developer.mend.io/github/mergestat/mergestat). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy4xMDMuMSIsInVwZGF0ZWRJblZlciI6IjM3LjEyNy4wIiwidGFyZ2V0QnJhbmNoIjoibWFpbiJ9-->
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Previously, as part of building the index representation, the
resolveObjectfunc would create an interim
plumbing.MemoryObject, which would then besaved into storage via
storage.SetEncodedObject. This meant that objectswould be unnecessarily loaded into memory, to then be saved into disk.
The changes streamlines this process by:
LazyObjectWriterinterface which enables the writeoperation to take place directly against the filesystem-based storage.
multiple writers (e.g. hasher and storage).
An additional change relates to the caching of object info children within
Parser.get. The cache is now skipped when a seekable filesystem is beingused.
The impact of the changes can be observed when using seekable filesystem
storages, especially when cloning large repositories.
The stats below were captured by adapting the
BenchmarkPlainClonetestto clone https://github.com/torvalds/linux.git:
Notice that on average the memory consumption per operation is over
75%smaller. The time per operation was increased by
15%, which may actual be lesson long running applications, due to the decreased GC pressure and the
garbage collection costs.
Relates to: #251 #315
pprofref using previous stats as base: