Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ARROW-9224: [Dev][Archery] clone local source with --shared #7541

Closed
wants to merge 1 commit into from

Conversation

cyb70289
Copy link
Contributor

@cyb70289 cyb70289 commented Jun 25, 2020

archery benchmark diff runs "git clone --local" to clone source code
to /tmp. If /tmp and source directory are at different disk volumes, it
fails because "git clone --local" uses hard links which only works on
same volume. This patch replaces "--local" with "--shared".

Error log:
fatal: failed to create link '/tmp/arrow-archery-lwm7k6wm/origin_master/arrow/.git/objects/2d/80a4d2da431dbfe3c736b6349eb4a0bcb6ece5': Invalid cross-device link
subprocess.CalledProcessError: Command '['git', 'clone', '--local', PosixPath('/home/cyb/arrow'), '/tmp/arrow-archery-lwm7k6wm/origin_master/arrow']' returned non-zero exit status 128.

@github-actions
Copy link

@pitrou
Copy link
Member

pitrou commented Jun 25, 2020

Can you try using git clone --shared instead? It should avoid the copy.

@cyb70289
Copy link
Contributor Author

Can you try using git clone --shared instead? It should avoid the copy.

--shard works okay per my test. What about simply replacing git clone --local with git clone --shared?
From man page, the only danger of --shared is if I delete and prune the test branch when related test is running, looks not real.

       --shared, -s
           When the repository to clone is on the local machine, instead of using hard links, automatically setup .git/objects/info/alternates to share the
           objects with the source repository. The resulting repository starts out without any object of its own.

           NOTE: this is a possibly dangerous operation; do not use it unless you understand what it does. If you clone your repository using this option
           and then delete branches (or use any other Git command that makes any existing commit unreferenced) in the source repository, some objects may
           become unreferenced (or dangling). These objects may be removed by normal Git operations (such as git commit) which automatically call git gc
           --auto. (See git-gc(1).) If these objects are removed and were referenced by the cloned repository, then the cloned repository will become
           corrupt.

@pitrou
Copy link
Member

pitrou commented Jun 25, 2020

What about simply replacing git clone --local with git clone --shared?

+1 for me.

`archery benchmark diff` runs "git clone --local" to clone source code
to /tmp. If /tmp and source directory are at different disk volumes, it
fails because "git clone --local" uses hard links which only works on
same volume. This patch replaces "--local" with "--shared".

Error log:
fatal: failed to create link '/tmp/arrow-archery-lwm7k6wm/origin_master/arrow/.git/objects/2d/80a4d2da431dbfe3c736b6349eb4a0bcb6ece5': Invalid cross-device link
subprocess.CalledProcessError: Command '['git', 'clone', '--local', PosixPath('/home/cyb/arrow'), '/tmp/arrow-archery-lwm7k6wm/origin_master/arrow']' returned non-zero exit status 128.
@cyb70289 cyb70289 changed the title ARROW-9224: [Dev][Archery] Copy local repo on clone failure ARROW-9224: [Dev][Archery] clone local source with --shared Jun 26, 2020
@wesm
Copy link
Member

wesm commented Jun 26, 2020

thanks!

@wesm wesm closed this in a35c956 Jun 26, 2020
@cyb70289 cyb70289 deleted the archery branch June 26, 2020 12:35
sgnkc pushed a commit to sgnkc/arrow that referenced this pull request Jul 2, 2020
`archery benchmark diff` runs "git clone --local" to clone source code
to /tmp. If /tmp and source directory are at different disk volumes, it
fails because "git clone --local" uses hard links which only works on
same volume. This patch replaces "--local" with "--shared".

Error log:
fatal: failed to create link '/tmp/arrow-archery-lwm7k6wm/origin_master/arrow/.git/objects/2d/80a4d2da431dbfe3c736b6349eb4a0bcb6ece5': Invalid cross-device link
subprocess.CalledProcessError: Command '['git', 'clone', '--local', PosixPath('/home/cyb/arrow'), '/tmp/arrow-archery-lwm7k6wm/origin_master/arrow']' returned non-zero exit status 128.

Closes apache#7541 from cyb70289/archery

Authored-by: Yibo Cai <yibo.cai@arm.com>
Signed-off-by: Wes McKinney <wesm@apache.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants