Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SCM-928] added scm type 'mount-repos' to support google repo tool for checkout manifest.xml #92

Closed
wants to merge 3 commits into from

Conversation

Arnaud-Nauwynck
Copy link

@Arnaud-Nauwynck Arnaud-Nauwynck commented Jun 20, 2019

see jira https://issues.apache.org/jira/browse/SCM-928

as stated in https://maven.apache.org/scm.html
Checkouting maven sources can be done using google repo tool: download the file default.xml (called manifest in google repo tool), then doing "repo init"

repo init -u https://gitbox.apache.org/repos/asf/maven-sources.git
repo sync
repo start master --all

Unfortunatly, it does not work on some windows ... It requires to install python 2.7 but not 3, and mingw but not cygwin, it calls system process for ssh.exe but fails, and does not detect the correct pathes in cygwin, etc...

This PR contains a pure java implementation of the checkout command, that is compatible with maven-scm api. It implements the parsing of the repo manifest xml file, and call jgit sub-commands

You can test it by recompiling maven-scm, then typing either

mvn org.apache.maven.plugins:maven-scm-plugin:1.11.3-SNAPSHOT:checkout -DconnectionUrl=scm:mount-repos:default.xml -DcheckoutDirectory=checkout

or better using a new mojo that supports checkout OR update (does not clean the checkout dir if it already exists... so you can execute several times an update, or retry a partially failing checkout )

mvn org.apache.maven.plugins:maven-scm-plugin:1.11.3-SNAPSHOT:checkout-or-update -DconnectionUrl=scm:mount-repos:default.xml -DcheckoutDirectory=checkout

Notice that their are plenty of http 429 "Too Many Requests" will cloning some apache gitbox repositories, so this PR also contains in a separate commit a workaround for try-catch-retry on theses errors.

Remark 1:
It is implemented s a standard "composite design-pattern" on scm repository api, eventhough the current implementation that parse manifest.xml implicitely recognise jgit sub-repositories only.
By detecting other file format, it would be possible to mix git/hg/svn.. sub repositories

Remark 2:
I named this scm type 'mount-repos' instead of 'repo' (the original name of the google repo tool) because it looks like a mount-tab file in linux: it mounts some sub-dirs to delegate to other (scm) filesystems api.

Remark 3:
For reason 1 & 2, it makes sense to be properly integrated as a maven-scm provider type. It is much more general than simply getting maven own sources code, and could be used in some complex multi-repositories builds.

Remark 4:
Currently, only the checkout and update commands are implemented.
To fully supports maven scm api, we could also implements all remaining methods: diff/status/tag/update/edit/unedit/blame/mkdir/...

@hboutemy ... can you have a look? This work is inspired of problems discussed at hacker garten

@michael-o
Copy link
Member

I will probably need some time to process this..thank you for the contribution. Can you create a JIRA issue too?

@elharo elharo changed the title added scm type 'mount-repos' to support google repo tool for checkout manifest.xml [SCM-928] added scm type 'mount-repos' to support google repo tool for checkout manifest.xml Dec 23, 2020
@michael-o
Copy link
Member

I'll need time to process this.

@michael-o michael-o removed their request for review January 7, 2022 20:51
@michael-o
Copy link
Member

As per JIRA issue, closing this one.

@michael-o michael-o closed this Jan 7, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants