Alternatives to always-up-to-date submodules

[alexbw]

There are obvious downsides to having submodules always update to master (no master branch is 100% perfect all the time, and performance and memory leak regressions can be subtle and take awhile to catch and fix).

The downside to having submodules at fixed commits is stewardship overhead to make sure the submodules are as up-to-date as possible, but no more.

I lean more on the side of stability than ease-of-stewardship.

[soumith]

@alexbw as I said, I am okay with either. I think we should give the one a try, i.e. keeping submodules at fixed commits. If we have any issues, we would uncover them as we go.
Historically, I built torch-distro for @jonathantompson, as he was complaining that he missed the old torch7-distro, and with torch7-distro, we just had one giant repo of everything (which was automatically on master by definition). But torch-distro has gotten bigger than that one single use-case.

Also, it's great that you are taking the lead on this, dont tone it down at all :)

[clementfarabet]

This is a great initiative. I'd vote for versioning commits, and upgrade them only when certain strict conditions are met (such as travis giving you the green light). It's especially important for /torch7, /nn, /cutorch and /cunn, where any subtle regression becomes a huge pain to trace.

[soumith]

on board with that idea as well.

on the issue of travis, we have travis for torch7 and nn, but it almost always fails when running the unit tests, by just not responding for a long time. It seems to be a bug of VM + OpenBlas.

We also do not have travis testing for cutorch/cunn

If you guys have any ideas, let me know

[clementfarabet]

So about openblas, I think we should kill it for travis (if not everywhere...). This would speedup the build tremendously. What do you think?

For CUDA stuff, it's tricky to test for sure (no gpus on travis right?). Maybe we could host some of this testing somewhere else.

[alexbw]

Ok. I will remove the line in install.sh that automatically pulls all new repos and update all submodules to their current master branches.

After that, all new cool updates to included submodules have to be rolled in via a git submodule commit update. I think this is feasible, because if this distribution becomes a community standard, folks will pester us with github issues when it's time to update. I like the push better than pull model here :)

[clementfarabet]

Totally agreed!

[soumith]

👍

[alexbw]

Done in dbf367a. Tested on laptop.