New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
jenkins often fails just due to cloning #155
Comments
Sounds as good solution for me - if i am understand well, you propose to do a shallow clone with submodules, and then look into "cache" (whatever it is) to find for example xgboost-.zip instead of Furthermore, we should modify Jenkins file to do (CC:@anmol):
|
Yes it is very annoying - @abal5 is it because of some jenkins setup or just the network/node is being slow? |
It's because scikit-learn repo is very large (like 100MB) and github throttles us. Happens with cub sometimes too. A shallow clone will help, and works as long as we use our own repo that we control (instead of losing track of remote repo and shallow will break as head moves to new commits). |
If I knew where the jenkins stuff was done (where all the clone commands are) I could fix things myself. I don't see things in the docker or jenkins files in the repo. Are they stored somewhere else? |
@pseudotensor @abal5 will know for sure but I think we're using the git checkout plugin for it, it's in the Jenkins file:
I'm not very knowledgeable about it, though. Probably need to dabble with the |
But where is the actual repo shown, etc.? I can try setting recursiveSubmodules: false and using the scripts/gitshallow_submodules.sh , but I'll ask @abal5 before breaking things and wasting time. |
@pseudotensor repo is defined in jenkins job itself. btw: what about disabling checkout from jenkins totally and running the @pseudotensor script? |
Resolved by #167 |
Solution could be to keep s3 private versions of each repo and sync from them instead of git cloning and keeping the s3 up to date with a jenkins job. Point is that we very infrequently change the commit we point to, and silly to grab from github every time. The other solution is to use my shallow clone stuff so that there is much less to download from github.
Commit message: "minor doc2"
The text was updated successfully, but these errors were encountered: