Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

g.extension: sparse checkout with history for fast commit log access #3051

Merged
merged 2 commits into from
Jun 15, 2023

Conversation

ninsbl
Copy link
Member

@ninsbl ninsbl commented Jun 13, 2023

git -log -1 is slow with sparse checkout if --filter=tree:0 is used because history is not checked out. Cloning is similarly fast with --filter=blob:none but then the history is checked out too and git log -1 should be fast again.

Please try e.g.:

dirs="src/raster/r.centroids src/imagery/i.pysptools.unmix"
cd /tmp
time (git clone -q --no-checkout --branch=grass8 --filter=blob:none https://github.com/osgeo/grass-addons
cd grass-addons
git sparse-checkout set $dirs
git checkout grass8)

cd grass-addons
for dir in $dirs
do
  time git log --first-parent -1 $dir
  
done
cd ..
rm -rf grass-addons

Fixes #2979

I marked this as blocker as 30 seconds wait time per addon is a pretty bad user experience. Feel free to remove the label if you disagree...

@ninsbl ninsbl added bug Something isn't working blocker Blocking a release Python Related code is in Python backport to 8.3 PR needs to be backported to release branch 8.3 backport to 8.2 PR needs to be backported to release branch 8.2 backport to 7.8 PR needs to be backported to release branch 7.8 labels Jun 13, 2023
@ninsbl ninsbl added this to the 8.3.0 milestone Jun 13, 2023
@ninsbl ninsbl requested a review from tmszi June 13, 2023 20:53
Copy link
Member

@tmszi tmszi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested and works as expected.

This code line required changes too.

"--filter=tree:0",

Small note:

With blob:none change size of downloaded data from 2.6M to 6.5M.

@ninsbl
Copy link
Member Author

ninsbl commented Jun 14, 2023

Tested and works as expected.

This code line required changes too.

"--filter=tree:0",

Thanks for testing and pointing out the missing change in mkhtml.py... Addressed in 48a86df.

@ninsbl ninsbl changed the title g.extension: sparse checkout with history fast commit log access g.extension: sparse checkout with history for fast commit log access Jun 15, 2023
@ninsbl ninsbl merged commit 0d6124b into OSGeo:main Jun 15, 2023
20 checks passed
@landam landam modified the milestones: 8.3.0, 8.4.0 Jun 21, 2023
ninsbl added a commit to ninsbl/grass that referenced this pull request Jun 21, 2023
@ninsbl
Copy link
Member Author

ninsbl commented Jun 21, 2023

Cannot be backported to 8.2 directly, should be added to a bunch of g.extension-fixes that should be backported...

ninsbl added a commit that referenced this pull request Jun 21, 2023
@ninsbl ninsbl removed the backport to 8.3 PR needs to be backported to release branch 8.3 label Jun 21, 2023
@petrasovaa petrasovaa removed the backport to 7.8 PR needs to be backported to release branch 7.8 label Aug 11, 2023
landam pushed a commit to landam/grass that referenced this pull request Oct 25, 2023
neteler pushed a commit to nilason/grass that referenced this pull request Nov 7, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport to 8.2 PR needs to be backported to release branch 8.2 blocker Blocking a release bug Something isn't working Python Related code is in Python
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Bug] g.extension test often times out in CI
4 participants