Optimize Git clone/checkout

[ghost]

I'm in the process of testing out mina with some existing projects and enjoying it so far.

One downside I have noticed is that it it quite a bit slower than Vlad in some cases as Vlad keeps an "scm" folder on the server so that the whole repository doesn't need to be checked out each deploy, is this something that is planned?

[mikong]

We'll have this very soon. Thanks for pointing it out :)

[ghost]

On second thoughts, Vlad was very buggy in my experience when using the scm directory and I had to manually empty it in certain situations for the deploy to work. Perhaps a clean checkout as it happens with mina is the best option.

[stas]

Hmm, why this bug was closed?
Please leave it open as a feature request.
Thanks.

[rstacruz]

Renaming this issue to "Optimize Git clone/checkout".

I've been thinking about this. Perhaps #{deploy_to}/scm should be a bare repo, and the git:clone task will simply fetch new objects with it. New releases will clone from this bare repo. This essentially makes the bare repo a "cache" of the old Git objects. This should also be immune to crazy Git hacks like rebasing and push-forcing.

Not sure on the pitfalls of this yet, though. Definitely up for discussion and experimentation.

[ghost]

Like the idea of just having a cached repository and cloning from that... would get around some weird issues that occur with Vlad.

[rstacruz]

@thelucid Seems good in theory, but I'd like to do some benchmarks. The checking out "process" will still have to be done every time, it's just that you'll be skipping some network I/O, hypothetically.

[sarguru]

@rstacruz But for a big project and on a production server network I/O is costlier than running the "process". Your idea is cool. I just have a small suggestion. I wish if we could create a directory for the first time and check in the code via git clone and if that exists then pull in the directory on subsequent deploys.

[sosedoff]

How about just running git clone with --depth=1 ?

[sarguru]

http://blogs.gnome.org/simos/2009/04/18/git-clones-vs-shallow-git-clones/. I dont think it will be as useful as pull/fetch.

[rstacruz]

We'll most likely be implementing something like: (pseudo bash code ahead)

if [ ./scm exists ]; then
  cd scm && git fetch origin $commit
else
  git clone --bare $repository ./scm
fi

if [ $commit is a branch name or a tag name ]; then
  git clone ./scm ./tmp/the-current-build --depth 1 --recursive --branch $commit
else
  git clone ./scm ./tmp/the-current-build --recursive
  cd tmp/the-current-build && git checkout -b new-temp-branch $commit
fi

This 'caches' the objects into ./scm.

[sarguru]

@rstacruz I assume $commit is the last commit during deploy?

[rstacruz]

@sarguru It'll be whatever commit you've ran mina deploy with, or a revision that you manually specify using set :revision, 'production'. http://nadarei.co/mina/settings/revision.html

[rstacruz]

Actually, in the interest of optimization, we should probably change the default revision to be deployed as 'HEAD'.

[sarguru]

@rstacruz Agreed. HEAD would be handy on much more use-cases too, in repos where branching is more and revision to be deployed need not be the master. And thanks for the quick replies. Wonderful work.

[rstacruz]

Okay, this change makes the default deployable branch master, and introduces two new settings: :commit and :branch.

By default, :branch is master. If you would like to deploy another branch, just change that setting (set :branch, 'deploy'), or use set :commit, '04da8211f' to use a certain sha1.

Also, it deprecates the old :revision setting.

[ghost]

Beautiful, nice solution.