Add Ability to Recursively Clone Git Repo

[indiv0]

Currently, clippy-service only loads the contents of the specified repo as a zip. This causes issues for projects which have git submodules that they depend on (e.g. rusoto).

Would it be possible to either add a feature to use the git clone --recursive command to obtain the code along with the submodules, or replace the current functionality altogether?

[gnunicorn]

hello @indiv0,

thanks for your request. I opted for using the ZIP-Archive download, because it has the lower footprint. git clone can easily – as it syncs the entire repo – take a long time and consume lots of bandwidth. And as we are currently not keeping any cached version, it would download everything every time – that's an awefull lot of traffic.

I am wondering: are the submodules also exposed in the ZIP-Archive? Could we instead of fetching with git clone read that file and fetch those ZIPs, too (potentially recursively), instead?

In any case I'd only opt for that feature if we make all of that part of the 10min sandboxed limit, thus, if anyone wants to download too much stuff, it goes of their compile-time...

[mcarton]

What about git clone --depth 1? Never really used it but shouldn’t that have a lower footprint?

[indiv0]

Yeah, shallow clone should have a much smaller footprint. I checked the zip, and it does include the .gitmodules file so if you want to parse that and download them as well it should work.

[gnunicorn]

Yeah, shallow clone should have a much smaller footprint. I checked the zip, and it does include the .gitmodules file so if you want to parse that and download them as well it should work.

This sounds like a reasonable approach. We could download one level deep and unpack those, too. PRs about this would be welcome.