Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
Implement support for gitlab CI #1539
You can find gitlab CI results here:
Oh wow I didn't realize CI had to be working by tomorrow.. At the moment, I don't think so, but I will have some more time to work on it this evening.
Here's the current status: everything works except the
I am in the process of recreating the docker environment as much as I can locally so I can get away from using the gitlab CI to debug this, since the CI is quite slow for debugging these issues.
All other tests are running/passing, so that's a good sign :)
Note: this PR does NOT contain my latest changes, they're in my gitlab.com/craftyguy/pmbootstrap repo (but even those are not 100% working as indicated earlier in this comment).
Ok I've managed to get a bit closer, by creating the /dev/loopX devices with mknod (luckily docker is run with --privileged in gitlab.com's public CI runner!).
Now I'm hung up on this:
Interestingly enough, the partitions are there:
But the nodes aren't under /dev:
Naturally I run kpartx to update the kernel, but that doesn't seem to be working as expected:
Even forcing it, with verbose set, shows they should have been created, but they aren't:
This is likely some weird docker/container magic causing this, I'll research more tomorrow. I think I'm very close to getting this to work, assuming the image creation is one of the last steps in the
I've been able to get around the loop issues (apparently partitions are showing up in /dev/mapper/loopXpY in the docker container), and have patched partitions.py in my latest implementation.
There now seems to be some issue with not being able to ssh to the qemu instance that is created by the test. Not sure if it's a problem with my local docker config, but I fired it off to the gitlab.com CI just to see if it is also experiencing this issue. Getting closer!
I do not think we can use the gitlab.com shared runners for the qemu tests, since it requires KVM and it seems there is some variability in runners that are in the 'shared runner' pool such that not all of them have KVM, enabled properly.
I am currently investigating whether I can run a runner to meet our requirements, that we should then be able to use from gitlab.com's CI.
I've successfully created a custom gitlab runner for running pmbootstrap CI stuff (using virtualbox executor), and will host this system until we can come up with a more permanent solution. With help fom @ollieparanoid, we were able to resolve the loop device issues once and for all (hopefully).
So now all qemu-related tests are passing (xfce4 and plasma). I've enabled all pytest tests now and have about 5 failures that I will continue to debug.
I've gotten all tests to pass, and the latest changes are force-pushed to this PR.
Here's the output/results: https://gitlab.com/craftyguy/pmbootstrap/pipelines/24474062
It's important to note that the qemu tests use a specific runner that I am hosting. I'll need access to the gitlab pmbootstrap project to configure it, or I can provide instructions on how to do so if necessary.
Total run time on the gitlab CI stuff is about 8.5 minutes, compared to ~20 minutes for Travis :P
Added a few nitpicks. You did an amazing job with this, can't wait until we move to gitlab \o/ The artifacts and the increased speed are nice improvements!
referenced this pull request
Jun 25, 2018
Latest feedback incorporated, though I still have some questions about some of it (see above comments to in-line comments)
Results are here: https://gitlab.com/craftyguy/pmbootstrap/pipelines/24555986
Thanks for the update, I think it's pretty close now!
Some more comments/discussion points:
- I've removed my "(note to myself: does this work?)" comment, it's outdated and I didn't mean to make it part of the review (also removed your follow up question to the comment)
.gitlab/config.tomladded on purpose? If not, please add it to the gitignore. I'm not sure if you have put
<REDACTED>there, or if GitHub did it. In the latter case I would recommend to generate new passwords.
- How about we run shellcheck on the gitlab CI scripts as well (by adding them to
- With the modifications in
test/check_checksums.py, the Travis CI code won't work as expected anymore. I think this is good, because then we don't carry around legacy code. But I would recommend that we remove the other Travis CI code as well (
I added that on purpose, to serve as a reference configuration file for folks who want to also run qemu runners. I redacted the passwords, tokens myself. This file is manually updated to reflect my configuration (which is in /etc/gitlab-runner/config.toml on the server and not in any repo).
I'll add this in!
Ok, I'll add the removal of the travis-related files to this PR.
Changes implemented, and passing gitlab CI. I suspect this will now fail Travis CI though since there's a commit in this PR to remove the travis stuff :)