Use released Rust tar-balls instead of git repo

[frol]

It would be also great to have images for stable (1.6.0) and beta releases on Docker Hub. (You can create several branches with just RUST_VERSION=1.6.0 and RUST_VERSION=beta changes, and link them on Docker Hub)

P.S. Please, add a link to Debian image on Docker Hub (in Build Settings) so your images get rebuilt on every Debian image update.

[andrew-d]

Hi there, and thanks for the PR. It looks like the build on the Docker Hub is actually timing out - I'm going to try building with these changes and see if that drops things below the timeout. I've also emailed Docker support to see if it's possible to bump the limit on this particular repo.

I've also added a link to Debian, but that won't take effect until the build is working again.

[frol]

Unfortunatelly, it timed out again. If Docker Hub will not extend the time limit, you can try Travis or CircleCI (though they have only 25 hours per month free plan) to automate the build process.

[andrew-d]

Yeah, I tried using Travis, but it looks like it's over the limit there too :-(

[frol]

That is weird. Travis claims the following:

• Fair use
• Unlimited build minutes
• Unlimited repositories
• Unlimited collaborators

https://travis-ci.com/plans

Try to use travis_wait

[andrew-d]

I think the issue is actually this (from here):

If a script or test suite takes longer than 50 minutes (or 120 minutes on travis-ci.com), or if there is not log output for 10 minutes, it is terminated

Since the docker build command takes more than 50 minutes, Travis kills it 😞

[andrew-d]

Okay, so, I got this response from Docker:

Hi Andrew,

Thank you for contacting Docker Hub support.

Currently, the build time limit for automated builds is 2 hours.

I take it that you are referring to the andrewd/rust-musl repo.

My suggestion would be use an intermediate repo as the base image for this build. In other words, create another repo that is based on jessie and the image would be base "jessie" with apt-get update and all the dependencies, then on your actual rust-musl repo, you just have to compile rust and musl. If that still takes too long, you might have to split the compilation of rust and musl into separate repos. Then, you can just link the repositories in the build settings.

I understand that this seems a bit redundant, but we are not planning to change the build limit anytime soon, especially for the free tier users.

Thanks,

Support Engineer Docker

So, I have a couple options:

1. I can stop using an automated build and just build + push it myself. This is less than ideal, since I need to remember to do it, and you also lose the security guarantees from an automated build (i.e. is a random image that I push backdoored? No, but you have to trust me.).
2. I can try splitting the build up. Rust allows building each stage independently, so I could create andrewd/rust-musl-stage1, -stage2, and so on, and then have the final andrewd/rust-musl repository install the built compiler and then clean up. This does have the problem that the final image has all the intermediate layers, which will be very large.

Any thoughts?

[frol]

I have an idea based on your second option. After all stages are done, we can create a package or just a bundle, and push it to an external storage. The next step would be creating a simple image with installation of the package/bundle. We can probably even try to use Alpine for the final image.

[andrew-d]

Hm, that's actually an interesting idea. There would need to be some way of securely uploading to external storage, though - you can't just put a password into the Dockerfile or anyone would be able to upload anything 😦

I thought about doing some sort of Docker-in-Docker stuff too, but docker build doesn't support the --privileged flag, which is required for that to work.

I'd be interested in any ideas here!

[frol]

It wouldn't be an easy way, but we can try to pull last-stage-rust-image using https://github.com/samalba/docker-registry-debug and then unpack the necessary files from the tar-ball on the fly in the final image.

[frol]

FYI, I have asked a question on StackOverflow: http://stackoverflow.com/questions/34993610/is-there-a-way-to-download-a-docker-image-without-docker-daemon

[frol]

@andrew-d maybeg from StackOverflow wrote the following:

Do you really have to use Dockerhub? Because quay.io offers Squashed-Image-Download-Links!

I guess, using webhooks it should be possible to build all stages on quay.io, and then build the final image on Docker Hub. (This thing doesn't feel like an easy and reliable solution, but I don't have other ideas anyway.)