Add scripts for sparse checkout of some contribs #48183
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -550,6 +550,13 @@ jobs: | |
| with: | ||
| clear-repository: true | ||
| submodules: true | ||
| - name: Apply sparse checkout for contrib # in order to check that it doesn't break build | ||
|
There was a problem hiding this comment. Please, don't forget to roll it back. Unfortunately, it doesn't increase the checkout speed, it only decreases the allocated disk space. There was a problem hiding this comment. I had no intention of rolling it back, I'm going to keep it. See the comment below There was a problem hiding this comment.
I see. In this case, the job is done very well. We have all submodules cloned in the Maybe, if we do it in
There was a problem hiding this comment.
I think that having it in Also it makes sense to add an option to use There was a problem hiding this comment. @Felixoid, I added the same change to other workflows P.S. It would be great to refactor these yamls somehow and reduce the amount of copy-paste |
||
| run: | | ||
| rm -rf "$GITHUB_WORKSPACE/contrib" && echo 'removed' | ||
| git -C "$GITHUB_WORKSPACE" checkout . && echo 'restored' | ||
| "$GITHUB_WORKSPACE/contrib/update-submodules.sh" && echo 'OK' | ||
| du -hs "$GITHUB_WORKSPACE/contrib" ||: | ||
| find "$GITHUB_WORKSPACE/contrib" -type f | wc -l ||: | ||
| - name: Build | ||
| run: | | ||
| sudo rm -fr "$TEMP_PATH" | ||
|
|
||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,19 @@ | ||
| #!/bin/sh | ||
|
|
||
| set -e | ||
|
|
||
| git config submodule."contrib/llvm-project".update '!../sparse-checkout/update-llvm-project.sh' | ||
| git config submodule."contrib/croaring".update '!../sparse-checkout/update-croaring.sh' | ||
| git config submodule."contrib/aws".update '!../sparse-checkout/update-aws.sh' | ||
| git config submodule."contrib/openssl".update '!../sparse-checkout/update-openssl.sh' | ||
| git config submodule."contrib/boringssl".update '!../sparse-checkout/update-boringssl.sh' | ||
| git config submodule."contrib/arrow".update '!../sparse-checkout/update-arrow.sh' | ||
| git config submodule."contrib/grpc".update '!../sparse-checkout/update-grpc.sh' | ||
| git config submodule."contrib/orc".update '!../sparse-checkout/update-orc.sh' | ||
| git config submodule."contrib/h3".update '!../sparse-checkout/update-h3.sh' | ||
| git config submodule."contrib/icu".update '!../sparse-checkout/update-icu.sh' | ||
| git config submodule."contrib/boost".update '!../sparse-checkout/update-boost.sh' | ||
| git config submodule."contrib/aws-s2n-tls".update '!../sparse-checkout/update-aws-s2n-tls.sh' | ||
| git config submodule."contrib/protobuf".update '!../sparse-checkout/update-protobuf.sh' | ||
| git config submodule."contrib/libxml2".update '!../sparse-checkout/update-libxml2.sh' | ||
| git config submodule."contrib/brotli".update '!../sparse-checkout/update-brotli.sh' |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,12 @@ | ||
| #!/bin/sh | ||
|
|
||
| echo "Using sparse checkout for arrow" | ||
|
|
||
| FILES_TO_CHECKOUT=$(git rev-parse --git-dir)/info/sparse-checkout | ||
| echo '/*' > $FILES_TO_CHECKOUT | ||
| echo '!/*/*' >> $FILES_TO_CHECKOUT | ||
| echo '/cpp/*' >> $FILES_TO_CHECKOUT | ||
|
|
||
| git config core.sparsecheckout true | ||
| git checkout $1 | ||
| git read-tree -mu HEAD |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,13 @@ | ||
| #!/bin/sh | ||
|
|
||
| echo "Using sparse checkout for aws-s2n-tls" | ||
|
|
||
| FILES_TO_CHECKOUT=$(git rev-parse --git-dir)/info/sparse-checkout | ||
| echo '/*' > $FILES_TO_CHECKOUT | ||
| echo '!/test/*' >> $FILES_TO_CHECKOUT | ||
| echo '!/docs/*' >> $FILES_TO_CHECKOUT | ||
| echo '!/compliance/*' >> $FILES_TO_CHECKOUT | ||
|
|
||
| git config core.sparsecheckout true | ||
| git checkout $1 | ||
| git read-tree -mu HEAD |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,13 @@ | ||
| #!/bin/sh | ||
|
|
||
| echo "Using sparse checkout for aws" | ||
|
|
||
| FILES_TO_CHECKOUT=$(git rev-parse --git-dir)/info/sparse-checkout | ||
| echo '/*' > $FILES_TO_CHECKOUT | ||
| echo '!/*/*' >> $FILES_TO_CHECKOUT | ||
| echo '/aws-cpp-sdk-core/*' >> $FILES_TO_CHECKOUT | ||
| echo '/aws-cpp-sdk-s3/*' >> $FILES_TO_CHECKOUT | ||
|
|
||
| git config core.sparsecheckout true | ||
| git checkout $1 | ||
| git read-tree -mu HEAD |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,85 @@ | ||
| #!/bin/sh | ||
|
|
||
| echo "Using sparse checkout for boost" | ||
|
|
||
| FILES_TO_CHECKOUT=$(git rev-parse --git-dir)/info/sparse-checkout | ||
| echo '/*' > $FILES_TO_CHECKOUT | ||
| echo '!/*/*' >> $FILES_TO_CHECKOUT | ||
| echo '/boost/*' > $FILES_TO_CHECKOUT | ||
| echo '!/boost/*/*' >> $FILES_TO_CHECKOUT | ||
| echo '/boost/algorithm/*' >> $FILES_TO_CHECKOUT | ||
| echo '/boost/any/*' >> $FILES_TO_CHECKOUT | ||
| echo '/boost/atomic/*' >> $FILES_TO_CHECKOUT | ||
| echo '/boost/assert/*' >> $FILES_TO_CHECKOUT | ||
| echo '/boost/bind/*' >> $FILES_TO_CHECKOUT | ||
| echo '/boost/concept/*' >> $FILES_TO_CHECKOUT | ||
| echo '/boost/config/*' >> $FILES_TO_CHECKOUT | ||
| echo '/boost/container/*' >> $FILES_TO_CHECKOUT | ||
| echo '/boost/container_hash/*' >> $FILES_TO_CHECKOUT | ||
| echo '/boost/context/*' >> $FILES_TO_CHECKOUT | ||
| echo '/boost/convert/*' >> $FILES_TO_CHECKOUT | ||
| echo '/boost/coroutine/*' >> $FILES_TO_CHECKOUT | ||
| echo '/boost/core/*' >> $FILES_TO_CHECKOUT | ||
| echo '/boost/detail/*' >> $FILES_TO_CHECKOUT | ||
| echo '/boost/dynamic_bitset/*' >> $FILES_TO_CHECKOUT | ||
| echo '/boost/exception/*' >> $FILES_TO_CHECKOUT | ||
| echo '/boost/filesystem/*' >> $FILES_TO_CHECKOUT | ||
| echo '/boost/functional/*' >> $FILES_TO_CHECKOUT | ||
| echo '/boost/function/*' >> $FILES_TO_CHECKOUT | ||
| echo '/boost/geometry/*' >> $FILES_TO_CHECKOUT | ||
| echo '/boost/graph/*' >> $FILES_TO_CHECKOUT | ||
| echo '/boost/heap/*' >> $FILES_TO_CHECKOUT | ||
| echo '/boost/integer/*' >> $FILES_TO_CHECKOUT | ||
| echo '/boost/intrusive/*' >> $FILES_TO_CHECKOUT | ||
| echo '/boost/iostreams/*' >> $FILES_TO_CHECKOUT | ||
| echo '/boost/io/*' >> $FILES_TO_CHECKOUT | ||
| echo '/boost/iterator/*' >> $FILES_TO_CHECKOUT | ||
| echo '/boost/math/*' >> $FILES_TO_CHECKOUT | ||
| echo '/boost/move/*' >> $FILES_TO_CHECKOUT | ||
| echo '/boost/mpl/*' >> $FILES_TO_CHECKOUT | ||
| echo '/boost/multi_index/*' >> $FILES_TO_CHECKOUT | ||
| echo '/boost/multiprecision/*' >> $FILES_TO_CHECKOUT | ||
| echo '/boost/numeric/*' >> $FILES_TO_CHECKOUT | ||
| echo '/boost/predef/*' >> $FILES_TO_CHECKOUT | ||
| echo '/boost/preprocessor/*' >> $FILES_TO_CHECKOUT | ||
| echo '/boost/program_options/*' >> $FILES_TO_CHECKOUT | ||
| echo '/boost/range/*' >> $FILES_TO_CHECKOUT | ||
| echo '/boost/regex/*' >> $FILES_TO_CHECKOUT | ||
| echo '/boost/smart_ptr/*' >> $FILES_TO_CHECKOUT | ||
| echo '/boost/type_index/*' >> $FILES_TO_CHECKOUT | ||
| echo '/boost/type_traits/*' >> $FILES_TO_CHECKOUT | ||
| echo '/boost/system/*' >> $FILES_TO_CHECKOUT | ||
| echo '/boost/tti/*' >> $FILES_TO_CHECKOUT | ||
| echo '/boost/utility/*' >> $FILES_TO_CHECKOUT | ||
| echo '/boost/lexical_cast/*' >> $FILES_TO_CHECKOUT | ||
| echo '/boost/optional/*' >> $FILES_TO_CHECKOUT | ||
| echo '/boost/property_map/*' >> $FILES_TO_CHECKOUT | ||
| echo '/boost/pending/*' >> $FILES_TO_CHECKOUT | ||
| echo '/boost/multi_array/*' >> $FILES_TO_CHECKOUT | ||
| echo '/boost/tuple/*' >> $FILES_TO_CHECKOUT | ||
| echo '/boost/icl/*' >> $FILES_TO_CHECKOUT | ||
| echo '/boost/unordered/*' >> $FILES_TO_CHECKOUT | ||
| echo '/boost/typeof/*' >> $FILES_TO_CHECKOUT | ||
| echo '/boost/parameter/*' >> $FILES_TO_CHECKOUT | ||
| echo '/boost/mp11/*' >> $FILES_TO_CHECKOUT | ||
| echo '/boost/archive/*' >> $FILES_TO_CHECKOUT | ||
| echo '/boost/function_types/*' >> $FILES_TO_CHECKOUT | ||
| echo '/boost/serialization/*' >> $FILES_TO_CHECKOUT | ||
| echo '/boost/fusion/*' >> $FILES_TO_CHECKOUT | ||
| echo '/boost/variant/*' >> $FILES_TO_CHECKOUT | ||
| echo '/boost/format/*' >> $FILES_TO_CHECKOUT | ||
| echo '/boost/locale/*' >> $FILES_TO_CHECKOUT | ||
| echo '/boost/random/*' >> $FILES_TO_CHECKOUT | ||
| echo '/boost/spirit/*' >> $FILES_TO_CHECKOUT | ||
| echo '/boost/uuid/*' >> $FILES_TO_CHECKOUT | ||
| echo '/boost/xpressive/*' >> $FILES_TO_CHECKOUT | ||
| echo '/boost/asio/*' >> $FILES_TO_CHECKOUT | ||
| echo '/boost/circular_buffer/*' >> $FILES_TO_CHECKOUT | ||
| echo '/boost/proto/*' >> $FILES_TO_CHECKOUT | ||
| echo '/boost/qvm/*' >> $FILES_TO_CHECKOUT | ||
| echo '/boost/property_tree/*' >> $FILES_TO_CHECKOUT | ||
| echo '/libs/*' >> $FILES_TO_CHECKOUT | ||
|
|
||
| git config core.sparsecheckout true | ||
| git checkout $1 | ||
| git read-tree -mu HEAD |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,14 @@ | ||
| #!/bin/sh | ||
|
|
||
| echo "Using sparse checkout for boringsll" | ||
|
|
||
| FILES_TO_CHECKOUT=$(git rev-parse --git-dir)/info/sparse-checkout | ||
| echo '/*' > $FILES_TO_CHECKOUT | ||
| echo '!/fuzz/*' >> $FILES_TO_CHECKOUT | ||
| echo '!/crypto/cipher_extra/test/*' >> $FILES_TO_CHECKOUT | ||
| echo '!/third_party/wycheproof_testvectors/*' >> $FILES_TO_CHECKOUT | ||
| echo '!/third_party/googletest/*' >> $FILES_TO_CHECKOUT | ||
|
|
||
| git config core.sparsecheckout true | ||
| git checkout $1 | ||
| git read-tree -mu HEAD |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,12 @@ | ||
| #!/bin/sh | ||
|
|
||
| echo "Using sparse checkout for brotli" | ||
|
|
||
| FILES_TO_CHECKOUT=$(git rev-parse --git-dir)/info/sparse-checkout | ||
| echo '/*' > $FILES_TO_CHECKOUT | ||
| echo '!/*/*' >> $FILES_TO_CHECKOUT | ||
| echo '/c/*' >> $FILES_TO_CHECKOUT | ||
|
|
||
| git config core.sparsecheckout true | ||
| git checkout $1 | ||
| git read-tree -mu HEAD |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,12 @@ | ||
| #!/bin/sh | ||
|
|
||
| echo "Using sparse checkout for croaring" | ||
|
|
||
| FILES_TO_CHECKOUT=$(git rev-parse --git-dir)/info/sparse-checkout | ||
| echo '/*' > $FILES_TO_CHECKOUT | ||
| echo '!/benchmarks/*' >> $FILES_TO_CHECKOUT | ||
| echo '!/tests/*' >> $FILES_TO_CHECKOUT | ||
|
|
||
| git config core.sparsecheckout true | ||
| git checkout $1 | ||
| git read-tree -mu HEAD |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,22 @@ | ||
| #!/bin/sh | ||
|
|
||
| echo "Using sparse checkout for grpc" | ||
|
|
||
| FILES_TO_CHECKOUT=$(git rev-parse --git-dir)/info/sparse-checkout | ||
| echo '/*' > $FILES_TO_CHECKOUT | ||
| echo '!/test/*' >> $FILES_TO_CHECKOUT | ||
| echo '/test/build/*' >> $FILES_TO_CHECKOUT | ||
| echo '!/tools/*' >> $FILES_TO_CHECKOUT | ||
| echo '/tools/codegen/*' >> $FILES_TO_CHECKOUT | ||
| echo '!/examples/*' >> $FILES_TO_CHECKOUT | ||
| echo '!/doc/*' >> $FILES_TO_CHECKOUT | ||
| # FIXME why do we need csharp? | ||
| #echo '!/src/csharp/*' >> $FILES_TO_CHECKOUT | ||
| echo '!/src/python/*' >> $FILES_TO_CHECKOUT | ||
| echo '!/src/objective-c/*' >> $FILES_TO_CHECKOUT | ||
| echo '!/src/php/*' >> $FILES_TO_CHECKOUT | ||
| echo '!/src/ruby/*' >> $FILES_TO_CHECKOUT | ||
|
|
||
| git config core.sparsecheckout true | ||
| git checkout $1 | ||
| git read-tree -mu HEAD |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,12 @@ | ||
| #!/bin/sh | ||
|
|
||
| echo "Using sparse checkout for h3" | ||
|
|
||
| FILES_TO_CHECKOUT=$(git rev-parse --git-dir)/info/sparse-checkout | ||
| echo '/*' > $FILES_TO_CHECKOUT | ||
| echo '!/tests/*' >> $FILES_TO_CHECKOUT | ||
| echo '!/website/*' >> $FILES_TO_CHECKOUT | ||
|
|
||
| git config core.sparsecheckout true | ||
| git checkout $1 | ||
| git read-tree -mu HEAD |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,12 @@ | ||
| #!/bin/sh | ||
|
|
||
| echo "Using sparse checkout for icu" | ||
|
|
||
| FILES_TO_CHECKOUT=$(git rev-parse --git-dir)/info/sparse-checkout | ||
| echo '/*' > $FILES_TO_CHECKOUT | ||
| echo '!/*/*' >> $FILES_TO_CHECKOUT | ||
| echo '/icu4c/*' >> $FILES_TO_CHECKOUT | ||
|
|
||
| git config core.sparsecheckout true | ||
| git checkout $1 | ||
| git read-tree -mu HEAD |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,16 @@ | ||
| #!/bin/sh | ||
|
|
||
| echo "Using sparse checkout for libxml2" | ||
|
|
||
| FILES_TO_CHECKOUT=$(git rev-parse --git-dir)/info/sparse-checkout | ||
| echo '/*' > $FILES_TO_CHECKOUT | ||
| echo '!/result/*' >> $FILES_TO_CHECKOUT | ||
| echo '!/test/*' >> $FILES_TO_CHECKOUT | ||
| echo '!/doc/*' >> $FILES_TO_CHECKOUT | ||
| echo '!/os400/*' >> $FILES_TO_CHECKOUT | ||
| echo '!/fuzz/*' >> $FILES_TO_CHECKOUT | ||
| echo '!/python/*' >> $FILES_TO_CHECKOUT | ||
|
|
||
| git config core.sparsecheckout true | ||
| git checkout $1 | ||
| git read-tree -mu HEAD |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,27 @@ | ||
| #!/bin/sh | ||
|
|
||
| echo "Using sparse checkout for llvm-project" | ||
|
|
||
| FILES_TO_CHECKOUT=$(git rev-parse --git-dir)/info/sparse-checkout | ||
| echo '/*' > $FILES_TO_CHECKOUT | ||
| echo '!/*/*' >> $FILES_TO_CHECKOUT | ||
| echo '/llvm/*' >> $FILES_TO_CHECKOUT | ||
| echo '!/llvm/*/*' >> $FILES_TO_CHECKOUT | ||
| echo '/llvm/cmake/*' >> $FILES_TO_CHECKOUT | ||
| echo '/llvm/projects/*' >> $FILES_TO_CHECKOUT | ||
| echo '/llvm/include/*' >> $FILES_TO_CHECKOUT | ||
| echo '/llvm/lib/*' >> $FILES_TO_CHECKOUT | ||
| echo '/llvm/utils/TableGen/*' >> $FILES_TO_CHECKOUT | ||
| echo '/libcxxabi/*' >> $FILES_TO_CHECKOUT | ||
| echo '!/libcxxabi/test/*' >> $FILES_TO_CHECKOUT | ||
| echo '/libcxx/*' >> $FILES_TO_CHECKOUT | ||
| echo '!/libcxx/test/*' >> $FILES_TO_CHECKOUT | ||
| echo '/libunwind/*' >> $FILES_TO_CHECKOUT | ||
| echo '!/libunwind/test/*' >> $FILES_TO_CHECKOUT | ||
| echo '/compiler-rt/*' >> $FILES_TO_CHECKOUT | ||
| echo '!/compiler-rt/test/*' >> $FILES_TO_CHECKOUT | ||
| echo '/cmake/*' >> $FILES_TO_CHECKOUT | ||
|
|
||
| git config core.sparsecheckout true | ||
| git checkout $1 | ||
| git read-tree -mu HEAD |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,15 @@ | ||
| #!/bin/sh | ||
|
|
||
| echo "Using sparse checkout for openssl" | ||
|
|
||
| FILES_TO_CHECKOUT=$(git rev-parse --git-dir)/info/sparse-checkout | ||
| echo '/*' > $FILES_TO_CHECKOUT | ||
| echo '!/fuzz/*' >> $FILES_TO_CHECKOUT | ||
| echo '!/test/*' >> $FILES_TO_CHECKOUT | ||
| echo '!/doc/*' >> $FILES_TO_CHECKOUT | ||
| echo '!/providers/*' >> $FILES_TO_CHECKOUT | ||
| echo '!/apps/*' >> $FILES_TO_CHECKOUT | ||
|
|
||
| git config core.sparsecheckout true | ||
| git checkout $1 | ||
| git read-tree -mu HEAD |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,13 @@ | ||
| #!/bin/sh | ||
|
|
||
| echo "Using sparse checkout for orc" | ||
|
|
||
| FILES_TO_CHECKOUT=$(git rev-parse --git-dir)/info/sparse-checkout | ||
| echo '/*' > $FILES_TO_CHECKOUT | ||
| echo '!/*/*' >> $FILES_TO_CHECKOUT | ||
| echo '/c++/*' >> $FILES_TO_CHECKOUT | ||
| echo '/proto/*' >> $FILES_TO_CHECKOUT | ||
|
|
||
| git config core.sparsecheckout true | ||
| git checkout $1 | ||
| git read-tree -mu HEAD |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,13 @@ | ||
| #!/bin/sh | ||
|
|
||
| echo "Using sparse checkout for protobuf" | ||
|
|
||
| FILES_TO_CHECKOUT=$(git rev-parse --git-dir)/info/sparse-checkout | ||
| echo '!/*' > $FILES_TO_CHECKOUT | ||
| echo '/*/*' >> $FILES_TO_CHECKOUT | ||
| echo '/src/*' >> $FILES_TO_CHECKOUT | ||
| echo '/cmake/*' >> $FILES_TO_CHECKOUT | ||
|
|
||
| git config core.sparsecheckout true | ||
| git checkout $1 | ||
| git read-tree -mu HEAD |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,11 @@ | ||
| #!/bin/sh | ||
|
|
||
| set -e | ||
|
|
||
| WORKDIR=$(dirname "$0") | ||
| WORKDIR=$(readlink -f "${WORKDIR}") | ||
|
|
||
| "$WORKDIR/sparse-checkout/setup-sparse-checkout.sh" | ||
| git submodule init | ||
| git submodule sync | ||
| git submodule update --depth=1 |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I suggest reverting this part in the CI.
Besides the fact, that it only does the additional work after the modules are already checked out, this part has broken the debug build https://github.com/ClickHouse/ClickHouse/actions/runs/5033995222/jobs/9028616459#step:7:3663
cc @nickitat
upd: it's only a suggestion though. Fixing in the same PR works too. But it's better to backport it too
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@Felixoid, that's exactly why I added this to the CI. Changes in #50037 break sparse checkout, so it must be fixed in #50037 before merging.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
it makes sense to check this script in CI, but maybe as a separate step with self-explanatory name. otherwise people will just waste time over and over again because obviously most of them have no clue about this logic. and they shouldn't.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Agree with Nikina. It's unclear now what is broken there, and why it differs from other steps
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Besides the fact that it's good to have a good explanation of failure, we should use one of the fastest builds instead of a long one
What do you think of BuilderBinRelease as the job to test the sparse checkout? We can rename it explicitly to
BuilderBinReleaseSparseSubmodulesThere was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, it was unobvious that we use sparse checkout for debug build only, sorry for that. But I don't like introducing another build just to check that sparse checkout works, because it will actually duplicate an ordinary build for no good reason. I would prefer to add some note about this to the report.
It does not matter, sparse checkout does not make a build slower, it just takes a few seconds to re-checkout submodules, and after that build continues as usual.
Release build is slower than debug build.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Release deb yes, but I am talking regarding the release binary build. See the report, 0:16:20 for the BuilderBinRelease vs 0:20:05 for BuilderDebDebug
I am speaking not about adding a new check, but renaming the existing one to reflect what we actually check there.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
#50200