Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

first shot at testing build matrices #6

Merged
merged 24 commits into from
Sep 9, 2021
Merged

Conversation

vsoch
Copy link
Contributor

@vsoch vsoch commented Sep 4, 2021

this is a first try to test a real use case of docker build matrices, as discussed:

https://vsoch.github.io/uptodate/docs/#/user-guide/user-guide?id=docker-build

Likely I'll need a bit of iteration before this is ready to merge, as it's the first time I'm really using my own action in this context :p

Signed-off-by: vsoch vsoch@users.noreply.github.com

Signed-off-by: vsoch <vsoch@users.noreply.github.com>
Signed-off-by: vsoch <vsoch@users.noreply.github.com>
@vsoch
Copy link
Contributor Author

vsoch commented Sep 4, 2021

I anticipated this bug (and it happened) so I need to fix that : vsoch/uptodate#19

likely we will need something more detailed in the future, e.g., to check that
a buildarg has a default or not (and allow those that have defaults) but for now since
lets just skip all Dockerfile with build args for docker list

Signed-off-by: vsoch <vsoch@users.noreply.github.com>
Signed-off-by: vsoch <vsoch@users.noreply.github.com>
Signed-off-by: vsoch <vsoch@users.noreply.github.com>
@vsoch
Copy link
Contributor Author

vsoch commented Sep 4, 2021

okay that worked! Next I'll need to actually parse those build args :P

Signed-off-by: vsoch <vsoch@users.noreply.github.com>
Signed-off-by: vsoch <vsoch@users.noreply.github.com>
@vsoch
Copy link
Contributor Author

vsoch commented Sep 4, 2021

That worked!

docker build -f Dockerfile --build-arg ubuntu_version=16.04 --build-arg llvm_version=5.0.1 -t ghcr.io/TODOPATHANDTAG

So next I think the action should also output a suggested container name, e.g. ubuntu-clang (some tag)? @davidbeckingsale what do you think should be our naming convention? I was thinking it would be easy enough to derive the container name from the path, e.g.,

ubuntu/clang/Dockerfile == ubuntu-clang

And then in the GitHub action the user can combine that with a registry, e.g., here would be:

ghcr.io/rse-radiuss/ubuntu-clang:<sometag>

But what we need to figure out is how to specify what the tag is. In the case of the build matrix, since we have one or more build args and there isn't a primary one, we would need a strategy to ensure unique names. if we include every build arg, the container names are going to get really long. I'm going outside now for a bit but if you read this before I get back let me know your thoughts! Either we come up with an automated strategy, or ask the person in the config file to specify one of the build args to be the version of interest (and perhaps this would reduce the size of the matrix if we build fewer combinations that would be redundant). Or we nix the build matrix and require explicit pairs of things (e.g., instead of all combinations of List A and B, we just pair up A and B. You can take a look at the current builds above (e.g., the files too) to see a simple matrix (which is going to get much more complicated!)

@vsoch
Copy link
Contributor Author

vsoch commented Sep 5, 2021

okay some ideas after thinking on my walk/run:

  • include all build args/versions with tags this assumes a small number of build args, and just output versions. E.g., each build arg would have an optional key (and default to the name) so --build-arg ubuntu_version=16.04 --build-arg llvm_version=5.0.1 would just be ghcr.io/rse-radiuss/ubuntu-clang:ubuntu-16.04-clang-5.0.1. That's very redundant because the path of the folder is already represented (ubuntu/clang) and then having the build args makes it appear twice (but as a tag). This is simple to implement but kind of ugly.
  • Build a paired list instead of a matrix: instead of having ALL versions of llvm with ALL versions of ubuntu, we'd pair them up. This would make it harder to automatically update because we'd have to be able to have a way to not just add new entries, but to match them to other build args. But that would guarantee that, for example, for ubuntu 16.04 there was only ONE version of llvm, so we would just call it ghcr.io/rse-radiuss/ubuntu-clang:16.04.
  • Pin a specific version across all builds: If we wanted to, say, always use the most up to date version of llvm, e.g., maybe jumping from major releases (4.0.0 to 5.0.0) across ubuntu versions, then it would be easy to say "ubuntu 16.04 is always going to have the same version of llvm we don't need it in the tag" and we'd again do ghcr.io/rse-radiuss/ubuntu-clang:16.04. We could also flip that and decide to always use the latest ubuntu in which case we'd have ghcr.io/rse-radiuss/ubuntu-clang:5.0.0 (where the version corresponds to llvm).

I'll chew on these ideas a bit - I guess based on the current repos it seems like the middle one (having pairs of variables) is what matches best now, but having a conversation about what are the needs of the teams that use these bases would be useful here.

Signed-off-by: vsoch <vsoch@users.noreply.github.com>
Signed-off-by: vsoch <vsoch@users.noreply.github.com>
Signed-off-by: vsoch <vsoch@users.noreply.github.com>
@vsoch
Copy link
Contributor Author

vsoch commented Sep 5, 2021

okay updated the titles in the steps above to show the start of the directory and build args (so it's easier to see what is being done!)

Signed-off-by: vsoch <vsoch@users.noreply.github.com>
Signed-off-by: vsoch <vsoch@users.noreply.github.com>
Signed-off-by: vsoch <vsoch@users.noreply.github.com>
Signed-off-by: vsoch <vsoch@users.noreply.github.com>
Signed-off-by: vsoch <vsoch@users.noreply.github.com>
Signed-off-by: vsoch <vsoch@users.noreply.github.com>
Signed-off-by: vsoch <vsoch@users.noreply.github.com>
Signed-off-by: vsoch <vsoch@users.noreply.github.com>
Signed-off-by: vsoch <vsoch@users.noreply.github.com>
Signed-off-by: vsoch <vsoch@users.noreply.github.com>
Signed-off-by: vsoch <vsoch@users.noreply.github.com>
Signed-off-by: vsoch <vsoch@users.noreply.github.com>
Signed-off-by: vsoch <vsoch@users.noreply.github.com>
Signed-off-by: vsoch <vsoch@users.noreply.github.com>
@vsoch
Copy link
Contributor Author

vsoch commented Sep 9, 2021

I'm going to merge this for now since we at least have working containers - it would be nice to have a working build cache but we can address that later.

@vsoch vsoch merged commit f59dec3 into main Sep 9, 2021
@vsoch vsoch deleted the add/docker-build-testing branch September 9, 2021 18:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant