switch to herokuish + docker hub

[progrium]

The original reason we started using "prebuilt" stack image of buildstep is because Docker Hub was slow and did not compress images. Now, however, it does. In some cases it can be even faster than downloading a tarball. So firstly, we should switch back to the model of a simple docker pull from Docker Hub.

Second, Buildstep has become a compatibility wrapper for Herokuish. But just the other day, I merged most of those compatibility bits into Herokuish. I also added a Dockerfile that is basically the same as Buildstep. As such, Herokuish can now be used instead of Buildstep.

I would recommend using Herokuish from Docker Hub now instead of Buildstep. This will do several things:

1. Get people submitting issues to Herokuish instead of Buildstep (super frustrating)
2. Eliminate the mostly useless and no doubt confusing middle project of Buildstep
3. Allow Buildstep to freely be refactored into Buildkit

The switch also means Dokku will more immediately and more easily take advantage of security patches to upstream base layers. Something I've been hearing more and more about recently.

Other benefits of Herokuish:

• Herokuish is set up for better releases. Anybody can just submit a PR to release branch.
• Herokuish is close to having fully automatic testing of newly released buildpacks.
• Herokuish, among other things, is great for debugging Dokku pushes.

Later on, Dokku can come back to using what would then be Buildkit, which will orchestrate Herokuish builds (less code for Dokku), and support other builds via pluggable build strategies. For example, this would move Dockerfile building out of Dokku (even less code). But it would also open up more possibilities, such as what we're calling runtime images. Or other custom build strategies.

By the way "runtime images" are an exciting development providing new middleground between Dockerfile and buildpacks. They're lightweight prebuilt Docker images with ONBUILD hooks for a similar experience to Buildpacks except produces MUCH MUCH smaller images, while not being a fully custom Dockerfile. So they're as easy as Buildpacks (no special code, auto-detected, etc) with most benefits of Dockefile builds (Docker native, much more efficient).

[michaelshobbs]

Couple questions.

What is the official docker hub image path for herokuish? (gliderlabs/herokuish, ya?)
What would be the prescribed method to customizing what was the buildstep image?

For example, I build my own buildstep image with our custom list of apt packages installed and host it on s3. Then when we provision instances, we import using the s3 url. I suppose then we would have a custom herokuish image and import that into the correct namespace in the local docker daemon?

[progrium]

Yes, gliderlabs/herokuish:latest.

For now, I recommend keeping a variable in the Makefile but instead of pointing to an S3 URL, point to a Docker Hub image. I would then add logic that if a different variable is set, it would download a tarball, import as gliderlabs/herokuish:latest and go from there.

Ultimately, buildkit will provide a better way to do configure custom builds and base images, etc.

[patrickjahns]

Ping in here in light of https://github.com/gliderlabs/herokuish

I will create a PR later on. Also wondering if it is possible to simply create a config file/variable for Dokku where we specify the image name? (It would make it more easy to also allow for custom herokuish versions more easily).

Renaming/Retagging images in docker has been cumbersome and for some odd reason sometimes docker mixes tags up, when the original image was pulled and then removed...

[progrium]

There is a variable in the Makefile for how to get the image (which would be abandoned), but a variable for where it's used later could be reasonable. I still think if you want to use your own, you should just do it via image retagging.

[michaelshobbs]

It would be great to support either a docker hub image and/or the docker import style we do now. Less disruptive perhaps? I know I use the import feature in my productions stacks.

[progrium]

Yeah I think that is what I was originally thinking

[patrickjahns]

I am a bit confused here - should dokku check if the correct image is already installed and if not just issue a pull command (I think docker does this automatically anyway?)

So what needs to be changed are the deb files and the Makefile itself.
@michaelshobbs how do you propose to change the makefile to support both?

[michaelshobbs]

@patrickjahns see #1128

wasn't sure if this was going to work so I just did it. 😄

[josegonzalez]

@progrium whats the release process for herokuish?

[patrickjahns]

@michaelshobbs
that was really fast

Dropping the build for deb package is useful - just wonder now if we should issue a pull request for download herokuish automatically after install, or let the user do it. I can imagine that people will complain when they first deploy, that it takes for ever since docker will fetch the image first

[michaelshobbs]

yeah i think we should probably continue to package it up to save that install time. just need @progrium to tag builds in docker hub 😉

[michaelshobbs]

i'll let @josegonzalez chime in on this as he did the deb work.

[patrickjahns]

Can we create a deb file that basically just issues the command docker pull gliderlabs/herokuish ?

For the Makefile it shouldn`t be a problem to include right?

Also I am wondering if we force the user to pull the container on install. (What about users wanting to use their custom build? Then the deb/Makefile would download the image and the user needs to remove it again - wasted bandwith and time I believe)

[josegonzalez]

We can, it would just be nice to pin to a version for debugging purposes.

If users want a custom build, then they can wade through extra work. I'd rather the defaults work great and advanced installs be more effort than the other way around.

[michaelshobbs]

@josegonzalez 👍

[michaelshobbs]

was just revisiting this and realized that the original thought was to refactor buildstep (buildkit?) to be a lightweight wrapper around herokuish. is that right @progrium? #1128 cuts out buildstep completely. not sure that's what we really want. can someone confirm?

[progrium]

In the short term we can cut out buildstep entirely and that will allow it
to make its metamorphosis. It won't wrap herokuish, it will coordinate the
build. It's like a step before herokuish where it might decide to do a
docker build (which dokku does now itself), or do a runtime image build, or
something else.

On Fri, Jun 12, 2015 at 5:59 PM, Michael Hobbs notifications@github.com
wrote:

was just revisiting this and realized that the original thought was to
refactor buildstep (buildkit?) to be a lightweight wrapper around
herokuish. is that right @progrium https://github.com/progrium? #1128
#1128 cuts out buildstep
completely. not sure that's what we really want. can someone confirm?

—
Reply to this email directly or view it on GitHub
#1091 (comment).

Jeff Lindsay
http://progrium.com

[michaelshobbs]

cool. thanks for clarifying.

[josegonzalez]

Closing as there is a pr open for this #1128.