hub.docker.com repositories all need documentation

[rfay]

The Problem

Our Docker-Sponsored-Open-Source application is at risk because we don't have appropriate docs on all the DDEV repositories.

Email from them:

Thank you for your interest in renewing your membership with the Docker-Sponsored Open Source Program. Upon reviewing your application for DDEV, we determined that while your project meets most of the program requirements, there is a lack of documentation in one or more of your repositories on Docker Hub.

As stated in the Qualification Criteria section of our webpage, we encourage the authors of open-source projects to include documentation that meets the recommended community standards. This means a detailed project description on your Docker Hub repository pages that includes a link to your project source code, licensing information, and a general overview. Projects lacking this information might not receive the Docker Sponsored Open Source badge for their images on Docker Hub.

If you still wish to proceed with the renewal and continue benefiting from the Docker-Sponsored Open Source Program, we kindly ask you to reply to this email once you have updated the documentation for your project. Please keep in mind that failure to provide the necessary updates or respond to this email within 2 working weeks may result in the de-provisioning of your benefits.

Their web page has added this new requirement:

Your project repositories on Docker Hub must have documentation that meets the recommended community standards. We recommend a detailed project description on your Docker Hub pages that includes a link to your project in its respective source code repository and contributing guidelines. Projects lacking this information will not receive a Docker-Sponsored Open Source membership.

Solution:

What we need to do:

• Each DDEV repo should have link to the repository it comes from
• And license info (Apache2).
• Meet recommended community standards

[rfay]

Response from them, (thanks!)

Thank you for the reply. To avoid disruption to your membership, I will approve the renewal so that your project can continue using the benefits of the Docker-Sponsored Open Source program. We will check back in on the documentation requirement in 8 weeks.

[rfay]

If you'd like to work on this, I'll help, and would love to have you work on it. You need to be a community member with some history to be trusted with the hub.docker.com privileges of course.

[bmartinez287]

I imagine this needs a template to get started.
https://hub.docker.com/_/nginx
https://hub.docker.com/_/redis
https://hub.docker.com/_/alpine
I used the images above to come up with the template below.
Some DDEV images could probably use more info as they play unique roles but we could start with a generic template and add to those that need more.

Template file

Quick reference

Maintained by:
the DDEV Docker Maintainers

Where to get help:
DDEV Community Slack or Stack Overflow

Quick reference (cont.)

Where to file issues:
https://github.com/ddev/ddev/issues

Documentation:
https://ddev.readthedocs.io/en/stable/
https://ddev.com/

What is DDEV?

DDEV is an open source tool for launching local web development environments in minutes. It supports PHP, Node.js, and Python (experimental).

These environments can be extended, version controlled, and shared, so you can take advantage of a Docker workflow without Docker experience or bespoke configuration. Projects can be changed, powered down, or removed as easily as they’re started.

License

View license information for the software contained in this image.

As with all Docker images, these likely also contain other software which may be under other licenses (such as Bash, etc from the base distribution, along with any direct or indirect dependencies of the primary software being contained).

Some additional license information which was able to be auto-detected might be found in repository's ddev/ directory.

As for any pre-built image usage, it is the image user's responsibility to ensure that any use of this image complies with any relevant licenses for all software contained within.

[rfay]

Awesome!

[jpoesen]

🤚

I'll give one a go if you'll have me. Based on that workload I'll be able to let you know what more I can actually commit to (I'm in the midst of doing a bunch of other technical writing).

[rfay]

Would love to have you do this @jpoesen ! I can give you privs over there. Please hit me up in Discord and I'll set you up, we can talk through and do a screenshare of what's going on.

[rfay]

The list of repos to edit is in https://hub.docker.com/?namespace=ddev

Edits/comments on #5847 (comment) are welcome.

We'll start with seeing how it goes with ddev-webserver

In each of these we want to give a little context about the reason the repo exists.

[rfay]

The primary goal here should be to make these listings useful to people who land on them.

Secondary goal is to satisfy the kind folks at Docker who do the DSOS.

However, any actual documentation improvements belong in the relevant GitHub repository, normally ddev/ddev. It can be properly maintained there, and won't be properly maintained if kept separate.

@jpoesen please take a look and see if Dockerhub provides a way to bring the description in from github.

Thanks!

[rfay]

We've expanded the scope here to move the docs into DDEV github repos, which is how it should be.

@jpoesen is studying this, and found these references already:

• Docker official docs
• https://github.com/marketplace/actions/synchronize-github-description-and-readme-md-with-docker-hub
• https://github.com/marketplace/actions/update-container-description-action

[rfay]

Dockerhub autobuilding was another possible solution to updating the description, but AFAICT it doesn't allow the flexibility that we currently have in pushing images. For example, our many db images are all pushed at once to different repositories and with slightly different builds, even though they share a common Dockerfile. https://docs.docker.com/docker-hub/builds/how-builds-work/

[tyler36]

https://hub.docker.com/?namespace=ddev

Link does not show expected search results:

[bmartinez287]

Ooo I thought I clicked that at one point but maybe I just went to https://hub.docker.com/u/ddev

[rfay]

Maybe you have to be logged in or member of the org?

[tyler36]

I am logged in so I guess you need to be member of org?

https://hub.docker.com/u/ddev says "Displaying 1 to 25 of 28 repositories" not sure if that is "all" of them.

[rfay]

Thanks, yes, that's a fine link, and you can get to the next page at the bottom, and that is all of them.

[jpoesen]

@rfay

I expanded @bmartinez287's template as a gist here: https://gist.github.com/jpoesen/caf76ff57ef3b2dd903d1e821f86b35c. Feel free to suggest further tweaks.

I've got a basic action working that gets triggered on updates to readme.md on main.

• Example github repo: https://github.com/jpoesen/demo
• Linked docker repo: https://hub.docker.com/r/jpoesen/demo-repo

Next step: discuss the logic.

Some dockerfiles are contained within ddev/ddev, while others are in their own ddev/xyz repo, correct?

For those that have their own repo, my current action will work (triggers sync on readme.md update).

For those within ddev/ddev I'm not exactly sure. We could explicitly add each readme file to watch (such as https://github.com/ddev/ddev/blob/master/containers/ddev-webserver/README.md). There's only a handful of them or so, but the action would need to be updated when containers are added or removed from the repo.

[rfay]

Some dockerfiles are contained within ddev/ddev, while others are in their own ddev/xyz repo, correct?

The only ones that matter much are in ddev/ddev "containers" directory. https://github.com/ddev/ddev/tree/master/containers

There is one in ddev/ddev ".gitpod/images", so I guess we should include that. https://github.com/ddev/ddev/tree/master/.gitpod/images

[rfay]

The template looks great, except support should be improved as mentioned on the gist,

We use Discord, not slack, but the best link is https://ddev.readthedocs.io/en/stable/users/support/

[rfay]

This looks just fantastic to me.

Does anything have to be done on the dockerhub side before it gets deployed?

[rfay]

Maybe we can have a minor chat here or in Discord about how to go forward, what's required in terms of long-term maintenance, what's required to get this initially deployed.

[rfay]

I know you will, but please take a look @stasadev @bmartinez287

[jpoesen]

what's required in terms of long-term maintenance, what's required to get this initially deployed.

Setup

• Add 2 items to the GH DDEV org secrets (DOCKERHUB_USERNAME and DOCKERHUB_PASSWORD).
• For the ddev/ddev GH repo and the other container repos:
  • Replace each container's README.md with the template + copy over / fill in missing info. We / I can already do this even with no automation in place yet.
  • Add the sync workflow/action to the GH repo (https://github.com/jpoesen/demo/blob/main/.github/workflows/main.yml).

Note:
Only updating any of the specified README files on main will trigger the sync, nothing else.

Maintenance:

• Find a better place for the template than a gist under my account.
• Don't manually update DH READMEs. Update GH READMEs on main, which will trigger the sync.

Not sure what other maintenance tasks there are.

[rfay]

Here's how to Dockerhub stuff is done. You can't do passwords with dockerhub, have to do token:

ddev/.github/workflows/push-tagged-image.yml

Lines 50 to 54 in 4f65cfc

	- name: Login to DockerHub
	uses: docker/login-action@v3
	with:
	username: ${{ secrets.DOCKERHUB_USERNAME }}
	password: ${{ secrets.DOCKERHUB_TOKEN }}

A good next step would be for you to use https://github.com/ddev/ddev/actions/workflows/push-tagged-image.yml in your own fork. Developer docs show how to do this. You can add your code in there to do the updates, and of course push to your own Docker repo.

I'm pretty sure that we should do the update on push, rather than on change to README.

The template can go into the containers directory, true?

[rfay]

No pressure, but a little pressure. I don't want to wake up one day with an email from them :) thank you!

[rfay]

Would you like somebody else to push forward with this @jpoesen ? Thanks so much for pioneering this.

[jpoesen]

Sorry, the last 2 weeks were busier than planned.

The template can go into the containers directory, true?

Yes, I'll create a PR for that.

I'm pretty sure that we should do the update on push, rather than on change to README.

I'm not sure why. It doesn't seem like the action of pushing a tagged image changes the container's README file, so we'd be overwriting the dockerhub READMEs with the same content each time.

Unless I'm mistaken, the container READMEs are only updated manually, so it's more logical we update the linked dockerhub repo at that time, no?

Quick recap, just to be sure:

• Each container in ddev/ddev/containers has a README that needs to be manually updated using the new template we're discussing
• Each container has a corresponding dockerhub repo whose README should be updated (synced really) based on the container's README. The exact triggering event for this is TBD.

Correct?
Happy to discuss on a call if that makes things easier.

[rfay]

I'm not sure why. It doesn't seem like the action of pushing a tagged image changes the container's README file, so we'd be overwriting the dockerhub READMEs with the same content each time.

It works for me either way! As long as it's understood that we're changing the README only, and that's separate from the image push.

It's not too hard to trigger on git diff of the README. The github action could check that and do what's needed.

I think we should be fine, and I know you're busy, so feel free to push back if it's too much and you can just manage things. Thanks so much for pioneering this!

[jpoesen]

Not having been involved in the development and release process at all:

Do new releases {always | often | sometimes | never } involve a related change in a container's repo? Like a build summary or a custom build-specific message, or something else?

At the moment the GH action I'm using is triggered on any change to a container's README file, so no diffing needed, unless I'm misunderstanding something.

[rfay]

It will be very unusual to change the README in the containers directory. We just need to get a good starting one in there.

Our images are unlikely to be used separately, and it's not encouraged. So mostly we just want to do what they ask us, do a decent job of it, and things should be OK.

[rfay]

I'll start working on this using your excellent pioneering work if I don't hear from you soon. Thanks for all the effort!

[rfay]

• feat: Keep hub.docker.com repositories updated, fixes #5847 [skip ci] #6032

• I added the suggested template as a starter for ddev-webserver

• experimenting with using goreleaser, since we use it for everything else and it claims to have dockerhub for this. Asking about it in their discord, because it seems incomplete

• In addition to using the suggested uses: peter-evans/dockerhub-description@v4 github action, we also could use the peter-evans dockerhub repo to do this, see https://hub.docker.com/r/peterevans/dockerhub-description, "Using the Docker image independently of GitHub Actions". The basic github action is geared toward a single README in a single repo, so is not an exact match, or at least will be somewhat repetitive to implement.

[rfay]

This is adequately completed in #6032

@jpoesen would love it if you'd review that. I think it will be OK, built it on the template.