refactor: Add build GHA image #209
Conversation
|
Sorry for the noise, I created this PR a little bit early it looks like |
|
Want to squash for us? |
|
Happy to. Bare with me, git and I dont get alone sometimes :) (I usually have squash turns on automatically in my repos) |
jesse-peters
force-pushed
the
add-gha-build-container
branch
from
578cbf0
to
87d46f5
Compare
Move dockerfile to dir. Add to makefile. Add other deps build needs addl permissions Change image name need protobuf dockerfile hash was in the wrong spot in GHA still not seeing the cache marker Checking out the repo first would help Add pkg-config and use latest digest in builder needs not jobs in GHA imageid not digest from the docker build id missing on build step retrigger with new cache to get output it must have been digest, but the other issue bocked it try defining the tag earlier need a new cache to trigger the image push gha syntax error can't have uppercase hash was too long, use built in GHA to create a single Add vale, protoc. Cleanup dockerfile timeout split apart for speed That wasnt faster. Lets keep the old ones for now too. need to use a different cache key
jesse-peters
force-pushed
the
add-gha-build-container
branch
from
87d46f5
to
2d765ec
Compare
|
@cla-bot check |
|
In order to contribute to an Aurae project you must sign and agree to the CLA. Comment |
|
The cla-bot has been summoned, and re-checked this pull request! |
|
I think I'll need a project maintainer to add write permissions containers to this project. I'm using I think its here for an org: |
|
@cla-bot check |
|
The cla-bot has been summoned, and re-checked this pull request! |
|
Hey @jesse-peters I hate to sound like the idiot who doesn't understand how to use GitHub, but I have spent the last 30 minutes trying to understand what it is I am supposed to do to help you out here and I honestly am stumped. What exactly do you need? Can you be as descriptive as possible? Is there any way you can share a "this specific thing is failing and if you make this specific thing go away we will be happy" style description? |
|
I think Daegalus provided some instructions in discord. The docker image is failing when it pushes to the docker registry with a 403. I hit this issue when developing on my branch, and thought I had to solve it with the steps above. This other solution may be what we need though. Sorry to send you on a wild goose chase if this solves it!! |
I took another whack at this using a few different methods. The time to run is still a bit longer than native GHA jobs, but with a self-hosted runner I think it would be pretty similar.
I'm building the image as the first step in the pipeline, then using that image to run the jobs on downstream. I have introduced cargo-chef to package the dependencies, without needing to compile the code itself.
Using minideb we launch a container that has the rust requirements (but without extras like cargo-chef). This runs the actual make steps, and uses native GHA caching for the rust modules.
I would be happy to setup and own a self-hosted runner and docker registry. I'm happy to chat about that in discord too (kuxaku) :)
Most of the build layers should cache across branches too (using buildkits native GHA cache), so the first time it runs should be the longest.
Biggest Risks:
Out of scope:
deploystep yet.I kept the original two GHA jobs in the pipeline for reference. If/when this gets merged they can be ran alongside for a bit to ensure its working, or deleted in a second PR.