Become part of Docker stdlib / the official Docker image: _/tusd

[kvz]

I think it would increase confidence and hence re-usability if our docker image were to become part of the Docker standard library. For reference, here's the guide with the requirements.

At first glance, it seems we'd be a good candidate already, and that it would not take much more than a few tweaks, a PR, and (this weighs heavier) commitment to provide security updates and respond to community feedback in a timely manner.

I believe the way the Docker image is now built is fully automatic tho, and so it would be as easy as making changes to the Dockerfile in master, correct?

If so, I for one would not mind taking that responsibility on me. I wanted to invite some discussion here first tho and see if there are things I'm overlooking and if there are maybe other volunteers, before taking this plunge.

/cc @Acconut @tersmitten @trickkiste

Refs: #127 #129

[Acconut]

Sound like a good idea to me, thank you for looking into this 👍 Could you explain how exact it would work, if we would join the stdlib (or whatever it's called)? It seems as if a new docker-library/tusd repository is needed which hosts the Dockerfiles. How do we ensure that this repository is updated? Could this be automated?

[kvz]

good point, I guess there's three ways:

1. we move the Dockerfile there, so there's one source of truth (and perhaps keep a local Dockerfile in a .infra dir just for experimentation around)
2. we maintain it here and automate a sync (but i don't suppose Docker will like having bots push to their repo)
3. we maintain it here and sync changes manually

I think 1. is probably best/easiest?

[trickkiste]

Count me in on supporting the effort. I have created the initial Dockerfile and had this in mind also. Business got my attention drained from the effort till now.

@Acconut did you guys create a tusio organisation on your own in the meantime? I have registered the name "tusio" there, but have not gotten further due to lack of time. However, since I am working on the tusd part of our project again, I will finish the process now.

Could you confirm, that my build process is correct. Because I am unable to get tusd to resume my uploads (through an NGINX reverse-proxy) but am having no problem to do so with master.tus.io with the exact same client setup.

Maybe I have not compiled some plugins ...?

[kvz]

@trickkiste we're using https://hub.docker.com/r/tusproject/tusd/ now.

I doubt that has to do with your build, more likely an Nginx config that's slightly off? Seeing any errors? Preferably paste them into a new issue to keep this thread clean-ish.

[trickkiste]

@kvz I added you to the owners team of the "tusio" Docker organisation, in case you prefer this name. I do not have any usernames of other core team members like @Acconut.

Would you care to add me to the Docker team on "tusproject"? I would like to collaborate on any efforts surrounding Docker.

[trickkiste]

Please add me to the "tusproject/tusd" Docker team. I would like to take care of having all tagged tusd versions available as Docker images.

[Acconut]

I would like to take care of having all tagged tusd versions available as Docker images.

Very much appreciated, what's your username or email?

[trickkiste]

Docker Hub
user: zitabel
email: mark@flaneur.tv

Github
user: trickkiste
email: mark@trickkiste.at

Please add me to the tusd organisation on Github as well, so I can hook up my own Docker builds to that repo as well! Read access is sufficient, as long as Third-party application access policy is set to No restrictions. Otherwise my Docker account will not see the repo.

[trickkiste]

I don't seem to be able to do anything on that Docker repo. Probably would need to be in the owners team to work on the build settings, trigger builds, etc.

[kvz]

Once things are settled here @trickkiste, i think you should be the lead in this, since you are obviously more experienced than I am. If at any point you feel you don't have the time/interest anymore to maintain that position, just let me know and I'll try to step up. How does that sound?

[trickkiste]

@kvz, I accept. I will see, that I get tusd accepted. For ongoing maintenance, I would certainly appreciate assistance though!

[kvz]

Very well! We have a big stake in that so happy to assist where needed, just ping!

[trickkiste]

@kvz can you please fork this to tus github organisation and give me admin/owner access to the repo: https://github.com/docker-library/official-images

My github user is trickkiste.

[kvz]

It's done! 👌👍 Sent from mobile, pardon the brevity.

…

On 16 Jul 2017, at 20:35, DI FH Markus Kienast ***@***.***> wrote: trickkiste

[kvz]

Hi @trickkiste I was wondering what the next steps are for getting _/tusd off the ground? I know you're busy so perhaps our crew can take a few of your plate?

[kvz]

@trickkiste don't want to be rude but if you could just let us know where you left off and what's still to do, i'd like to continue this one! :)

[thirsch]

Hi @trickkiste and @kvz, any update on this? I'd like to help as well!

@kvz If ok, I'll send you a pr for adding the current Dockerfile to the official-images clone. But could you please update the fork before I'm starting to work on it? Is it possible to join directly in the group for tus or should I fork on my organization and send a pr from there?

Best regards,
Thomas

[Acconut]

@thirsch Unfortunately, there have been no updates since the latest message from @kvz. However, we are still very much interested in continuing with this journey, so I recreated the fork at https://github.com/tus/official-images to be up-to-date and also provided you with write access. Is there anything else we can help you with?

[thirsch]

@Acconut thanks! Yes, there is something else. Could you please fork https://github.com/docker-library/docs as well and grant write access to me? We have to supply a doc along with the PR for the image refs.

As we have to keep the Dockerfile somewhere on our own repo, shall we go for a separate repo with only the Dockerfiles or a directory structure like docker/{platform}/Dockerfile for example, in this repo, where platform is alpine at the moment?

[Acconut]

Could you please fork https://github.com/docker-library/docs as well and grant write access to me?

Sure, consider it done.

As we have to keep the Dockerfile somewhere on our own repo, shall we go for a separate repo with only the Dockerfiles or a directory structure like docker/{platform}/Dockerfile for example, in this repo, where platform is alpine at the moment?

I would prefer to keep them inside the tusd repository, so we when make changes to the tusd setup, we can also directly change the corresponding Dockerfile. One question though since I am not as experienced in Docker: Do we need multiple Dockerfiles? What would other platforms be and why would people choose another platform?

[kvz]

@thirsch very happy to see you taking charge of this 😍

[thirsch]

Could you please fork https://github.com/docker-library/docs as well and grant write access to me?

Sure, consider it done.

Thanks!

As we have to keep the Dockerfile somewhere on our own repo, shall we go for a separate repo with only the Dockerfiles or a directory structure like docker/{platform}/Dockerfile for example, in this repo, where platform is alpine at the moment?

I would prefer to keep them inside the tusd repository, so we when make changes to the tusd setup, we can also directly change the corresponding Dockerfile. One question though since I am not as experienced in Docker: Do we need multiple Dockerfiles? What would other platforms be and why would people choose another platform?

A common thing in docker is to choose whether to use alpine or debian as your base image. Most official images provide for example debian-stretch and alpine based images. But there could also be a ubuntu or another debian based version.

Let's start with the existing Dockerfile based on Alpine and get the image online at the official images repo. We can change the structure later as well.

[Acconut]

Sounds like a good plan 👍

[thirsch]

@Acconut @kvz I've prepared the required files. Could one of you please review my changes before we create the pr? Unfortunately, the provided toolchain for generating the final README.md for Docker Hub does only work after the first version has been merged back to the official repo.

https://github.com/tus/official-images/blob/master/library/tusd
This file contains the references for building the images and listing the tags. I started with version 0.13.1 (aliased with tags 0.13 and latest). In the next couple of days I'll send you a pr for the tusd main repo to incorporate a bash script to generate the library-file automatically based on the git tags.

https://github.com/tus/docs/tree/add-tusd-to-hub/tusd
Within this folder all relevant parts for assembling the overview (docs) page for Docker Hub are provided. The above mentioned toolchain will produce a README.md based on this files.

It's not much to read, as I'd suggest to start small and add new things and features from time to time.

Well, maybe we have to change our Dockerfile according to https://github.com/tus/official-images#user-content-consistency, but as we're basically not providing a shell, it might be ok.

We have to send both pr's close together to the official-images repos.

[kvz]

Hi, I think this all looks really good, thank you! One thing I'd ask is for the docs repo to be called official-images-docs or similar, to group it with the other docker-required repo. Is that possible?

[thirsch]

Hi, sure, I did the same with my local clone. As I don't have enough permissions, you have to do it. Should I create the pr's back to the official repo or do you want to do it?

[kvz]

Okay, renamed here: https://github.com/tus/official-images-docs

But i apologize I'm not sure what you mean by:

Should I create the pr's back to the official repo or do you want to do it?

[thirsch]

But i apologize I'm not sure what you mean by:

Should I create the pr's back to the official repo or do you want to do it?

Sorry for my unclear statement. I was talking about who should open the PRs to bring back the changes from our forks to the upstream for both repos, official-images and docs.

[Acconut]

First of all, I would like to apologize for the delay on my side. The last week was quite busy.

Could one of you please review my changes before we create the pr?

I had a look at the linked files and they all look fine, although I have to admit that my experience here is a bit limited :)

I was talking about who should open the PRs to bring back the changes from our forks to the upstream for both repos, official-images and docs.

It would likely be the easiest solution if you could open the PRs on your own. Or do you need additional permissions for this?

[thirsch]

Thanks for your review, @Acconut! I just created the PRs and it looks like I've got sufficient permissions to do so. Let's see what the guys say to our request to add tusd to the official images list :-)

[Acconut]

Perfect! Just for reference, those are the two PRs, aren't they?

docker-library/docs#1523
docker-library/official-images#6240

[thirsch]

Hi @Acconut, we got the first feedback on our PR from the official-images maintainers. Please head over and have a look at the comments from @tianon.

Basically they say, we shouldn't build tusd during the Docker build as this will bloat up the build context and they don't need/want any diffs from the sources. He did a lot of work to show an example on to illustrate a new Dockerfile in his comment.

What do you think about? Should I change the Dockerfile to the new process?

[Acconut]

I think @tianon makes a good point there by saying

What I'd recommend doing is creating a separate "release" Dockerfile which consumes the official released binaries directly

So that would mean keeping the current Dockerfile which builds tusd from source and adding an additional Dockerfile which pulls the binary release files in. For me, that sounds like a good deal since there might be people how like to run tusd from source using Docker.

Do you think that this is a good approach as well?

[thirsch]

Hey @Acconut, @kvz - We've got a ping from Tianon. I've been busy with other projects, sorry.

I've created another Dockerfile that would comply to the requirements. A few points are not handled within the file:

• It's based on buster-slim and not alpine as our binaries here can't be used without modifying the build command.
• The version is fixed to 1.2.0 and we would need some script or workflow to have new Dockerfiles for new version.
• Same for the hash. I've manually created the hash for the current release 1.2.0 but this should be someone automated in the release workflow.

I'm asking Tianon over there, if it would be sufficient to start with this and come up with further optimisation later.

We should think about putting the official docker related stuff in a new repository as we first need to prepare a release here to have binaries and the download-page updated with the latest version to be able to get the docker image built. But then the Dockerfile is not in the same commit/tag and it might be confusing if the docker version (e. g.) 1.2.0 is not in the tag of the source 1.2.0.

Whats do you think about the open points?

[Acconut]

Hi @thirsch, no worries, I am glad that you still find the time :)

I'm asking Tianon over there, if it would be sufficient to start with this and come up with further optimisation later.

That would be a good approach, I believe.

We should think about putting the official docker related stuff in a new repository as we first need to prepare a release here to have binaries and the download-page updated with the latest version to be able to get the docker image built.

I am not sure if a separate repository is needed. The releases of tusd are done automated in GitHub Actions (https://github.com/tus/tusd/blob/master/.github/workflows/main.yml#L40-L46). So after these lines are executed, the packed archives should be available on the download pages and the docker image can be built. Would that help with keeping the Dockerfile inside this repository?

[thirsch]

Hi @Acconut, sorry for the big delay. I've been assigned to another project and therefore did not have the time to further work on this.

The issue is, the official image should contain a hash verification step as shown here: docker-library/official-images#6240 (comment)

As I understand it, we must produce the binary as a release version and afterwards update the Dockerfile to contain the hash for the verification step?

Is it possible to not only publish a release, but also commit a new Dockerfile within GitHub Actions? I've not done much with GitHub Actions so far, sorry for the question.

[Acconut]

Hi @thePanz, hope you are all well! I have also been quite busy in the meantime!

As I understand it, we must produce the binary as a release version and afterwards update the Dockerfile to contain the hash for the verification step?

Seems so, yes. This makes this matter much more complex though.

Is it possible to not only publish a release, but also commit a new Dockerfile within GitHub Actions? I've not done much with GitHub Actions so far, sorry for the question.

Yes, that should be possible. We can modify files in the workspace during GitHub Actions is running and then use an action like https://github.com/marketplace/actions/add-commit to commit the changes. Of course, we than have to make sure that this commit is then only used for producing the Docker image, if I understand this correctly.

[omBratteng]

@Acconut let me know if you want me to take a look at this.

[Acconut]

@omBratteng Yes, that would be amazing! IIRC, the requirements to become an official image are not easy to meet but should be doable! Let me know if you need help in that journey.