#44 Nightly builds

[ostrolucky]

So I have tested this one out on my fork, seems it works like a charm.

Here's what you got to do in Travis:

• Define DOCKERHUB_LOGIN and DOCKERHUB_PASSWORD environment variables
• Add a cron job

Notes:

• Build time: This doesn't add much build time and travis is limited to 50 minutes. Plenty enough
• There is no limit on number of tags or sizes
• Security: Output is silenced, so nobody will see credentials

[jakzal]

Thanks for working on this!

[jakzal]

Shall we just use latest and alpine? These are meant for the latest versions of both image flavours.

[ostrolucky]

No, I did it this way because I want a way to lock to specific version. Otherwise, same image pulled down later will get you tools with higher version, incompatible with your project.

[jakzal]

Hm.. this is not how daily builds usually work, so I got confused :) There's usually one (latest) daily build available.

I usually create tags right after new features or fixes are merged, and then a merge triggers a build.

How would creating a tag each day improve the situation for you? I'm worried we'd be generating lots of tags on docker hub.

[ostrolucky]

Generating lot of tags on docker hub is not a problem, I did mention this in OG post in this PR. But yes, it would be nice to optimize this later to NOT create new tag if nothing changed (no new versions of packages). But I suggest to do it later and not complicate this at this stage.

Here's how it improves situation for me:

1. New patch release of Easy Coding Standard has been released which solves some issues I have
2. There is no new release of phpqa (no, new releases are definitely not done often enough), but there is new nightly build, so I just require jakzal/phpqa:nightly-currentday
3. New major release of Easy Coding Standard has been released, which changes configuration format from .neon to .yaml
4. Build of my project is triggered again
5. Nothing is broken because I've locked to nightly-currentday instead of :latest

[jakzal]

Is the login/pass the only way? No tokens?

[ostrolucky]

I don't think so

[jakzal]

There's a way to trigger builds on docker hub afterall: #48

Haven't tested it yet.

[ostrolucky]

That approach couldn't be used because I want new tag based on %currentday.

[jakzal]

Since the image was already built in the current job we could probably push that one instead of building it again.

[ostrolucky]

Don't worry this doesn't build it from scratch. It reuses all layers and just slaps new tag on it, it's quick. Didn't find a way how to do it differently

[jakzal]

Perhaps creating a new tag and pushing it would work. The Current way does take ~3min for some reason.

[ostrolucky]

That's weird, it was quick for me. Of course provided other build steps were run before.

[ostrolucky]

Did some more digging, it really looks like there is no way to create custom tags when building via their API.

Regarding login credentials, quay.io allows to login via token. But I don't think this is worth migration. They also don't say anything about usage limits.

[ostrolucky]

Anything new? Or are you uncomfortable going forward? If that is so, let me know and I will create fork.

[jakzal]

I'll review this soon. Thanks again! 🍺

[jakzal]

FYI: https://forums.docker.com/t/is-there-a-max-number-of-tags-that-i-can-push-to-docker-hub-registry/1760/2

is there a limit on how many tags we can keep or is it unlimited?

The answer is going to change over time. Right now, there is no programmatic limit on the number of tags you can have, but there is a practical limit.

Some docker operations and Hub UI operations try to query all the tags at once, and that query can time out if there are too many tags. I’ve seen this happen some times at around the 100 mark when there really are 100+ different images. If you have multiple tags on the same image then you can probably go higher.

So in the near term I would suggest removing your older tags to keep below 100 unique images.

Over time we’ll improve the behavior of the APIs so they don’t time out or they page their results better, or we may put programmatic limits in place (if we did something like this, we’d let people know ahead of time and help those with lots of tags migrate).

[jakzal]

Ok, let's try it on jakzal/phpqa-nightly. I'm not comfortable with generating this many tags on the main repo. Especially for such an edge case.

In the same time we should work on:

• making sure latest and alpine tags are rebuilt daily (Trigger a build on docker hub during the daily travis build #48)
• working out a way to notify us when a new version of a tool is released (i.e. create an issue/PR) - automate later

[jakzal]

Seems to work :) https://hub.docker.com/r/jakzal/phpqa-nightly/tags/