Improve CI pipeline and integration tests

[dklimpel]

I have opened a few PR to move from Travis CI to Github and update integration tests.

The following proposal for processing the PR.

Independent of this, new feature:

• Add new runtime environment variable to dgoss to copy additional files #898
• Add pipeline for build goss docker image #909

Independent of this, fix docs:

1. Fix building mkdocs #900
2. Update docs to fix linting issues #904
3. fix path pattern in documentation pipeline #919

bring back linter:

1. Add github golang lint CI pipeline #899
2. later some PR fix linting issues
3. fix some linting failures and configure golangci-lint #922
4. bring back linter for windows

update integration test docker images:

1. bump alpine linux to 3.19 #907
2. Add Debian Bookworm to integration-tests #902
3. Replace centos7 with rockylinux9 in integration-tests #906
4. Add Ubuntu Noble to integration-tests #908
5. new PR to remove old images
6. Build own docker images for integration tests #901
7. new PR to uses new docker images from ghcr (incl. new MD5 checksums to bypass build in CI)
8. decide when to build new docker test images

After that it is possible to move from Travis CI to Github Actions.

• add unit tests to ci #928
• ci: add integation tests to GHA CI #946
• Add Github action to build release artifacts #916
• update call of linter in makefile (example) (fix some linting failures and configure golangci-lint #922)
• rebuilding / remove the scripts in the ci folder
• review if codeclimate is needed anymore
• remove Code Climate from project #933

It may be useful to provide all CIs with a uniform wording and an optimized sequence at the end.

[aelsabbahy]

You've been busy, love it!

I'll go through these PRs and start reviewing/merging the ones with least dependencies.

Two questions regarding integration-tests:

1. Did you ever find a fix for the systemd in docker thing?
2. I haven't looked at the PRs yet, do they leverage the make targets?
   • If not, I want to make sure it's reproducible locally outside of CI if possible.
   • There's also act - written by a former colleague of mine, one of the best engineers I had the pleasure of working with. I've never personally used it.

[dklimpel]

Did you ever find a fix for the systemd in docker thing?

The topic is very annoying. Docker is simply not made for running multiple processes and monitoring them from a separate process (systemd)- there are solutions to start multiple processes in a container. But that won't help, because the container is used to simulate an operating system. The whole thing can only be solved by calling docker run with the correct parameters.

I haven't looked at the PRs yet, do they leverage the make targets?

All these changes run with Travis CI, Github workflow and locally on my ubuntu machine (make test-all)

One issue remains open from my point of view. The test in which runlevel a service runs is not compatible with current linux versions. My knowledge of go is not great enough to investigate this.

[aelsabbahy]

@dklimpel random question, are you on gophers slack by any chance?

[aelsabbahy]

Between #900 and #904 is there a dependency or order to merge them. Both have failing builds currently.

After those are merged, then the long awaited journey to move off of Travis begins.

[dklimpel]

Both have different failing jobs. And both fix different jobs. I would start with #900 and then merge main to #904. Then should #904 not failing anymore.

[aelsabbahy]

#900 merged, #904 updated

[aelsabbahy]

One issue remains open from my point of view. The test in which runlevel a service runs is not compatible with current linux versions. My knowledge of go is not great enough to investigate this.

Which Linux (branch) is failing, I can checkout that brance and investigate next week. Wonder if it's a Goss bug, or working as intended.

[dklimpel]

Which Linux (branch) is failing, I can checkout that brance and investigate next week. Wonder if it's a Goss bug, or working as intended.

It is a problem with Debian and Ubuntu:

• Add Debian Bookworm to integration-tests #902
• Add Ubuntu Noble to integration-tests #908

Also on my local Ubuntu machine.

[aelsabbahy]

Okay, thanks. I'll check them both out.

Which PRs are next for documentation?

#919?

[dklimpel]

I have tried to put it into a sorted list above.

#904 fix the linting issues. After that there should be a valid documentation. #919 fixing for upcoming changes that the documentation pipeline becomes triggered.

[dklimpel]

GitHub creates a workflow when push to master automatically.
https://github.com/goss-org/goss/actions/workflows/pages/pages-build-deployment

IMHO this is failing because this is a default job with Jekyll theme. Probably this can be disabled in project settings. https://github.com/goos-org/goss/settings/pages - Change to source "GitHub Action". Readthedocs should not need this, because it is working by triggers, I think. GitHub pages is not used here.

[aelsabbahy]

Changed, I guess next PR to be merged will validate this?

Also, please continue to use this issue to let me know the next PR in the chain. I find it a lot easier to track on here.

This is an amazing level of work by the way, much appreciated. It's something I've wanted for a long time. Unfortunately, due to limited time I never got around to it, my focus tends to be:

1. Bugs/security findings
2. Features
3. Everything else (refactor, CI, etc..)

[dklimpel]

Changed, I guess next PR to be merged will validate this?

Yes, it is.

My suggestion for the next steps.

• fix path pattern in documentation pipeline #919 finish the topic docs and get a reliable pipeline. It is no change of program code.
• Add github golang lint CI pipeline #899 to bring back linting to prevent bugs

[dklimpel]

@dklimpel random question, are you on gophers slack by any chance?

Unfortunately not.

[aelsabbahy]

Hey @dklimpel , if you don't mind. Let me know on here the next PR that's ready and I'll review.

This is an awesome amount of work you put it, it's greatly appreciated!

[dklimpel]

To improve the code:

• fix some linting failures and configure golangci-lint #922
  or bring tests up to date
• bump alpine linux to 3.19 #907

[aelsabbahy]

Holding off on merging more PRs until Travis oss credits are replenished.

Don't want CI to get in a broken state with unclear traceability on what caused it.

This is an awesome amount of work. Can't thank you enough for taking the time to do this.

[dklimpel]

Holding off on merging more PRs until Travis oss credits are replenished.

Ok.

[dklimpel]

I think #928 can help. This enables unit tests with GitHub Pipeline.

[aelsabbahy]

Using this as a central coordination point. What PRs are ready for merge, so I can start going through them.

[dklimpel]

I would recommend finalizing a few topics before we take a look at the docker images.

• bring unit tests to GH action
  add unit tests to ci #928
• remove not needed codeclimate
  remove Code Climate from project #933
• update docs and close Docs: "docker" container in kgoss docs #931
  docs: Update and more general wording for "docker container" #934
• Final steps to new documentation #925
• new release to fix
  Critical CVE on goss #941

[aelsabbahy]

We are unable to start your build at this time. You exceeded the number of users allowed for your plan. Please review your plan details and follow the steps to resolution.

Dealing with more travis-ci issues, waiting on support. Figured it's relevant to this issue given the work-effort here =)

[dklimpel]

If you would like to run the integration tests with GHA:

• ci: add integation tests to GHA CI #946

[dklimpel]

I have added a replacement of CentOS in:

• Replace centos7 with rockylinux9 in integration-tests #906

The creation of binarries for tags / releases is included there:

• Add Github action to build release artifacts #916

IMHO this would be the last step to replace the functionalities of travis CI.

After that, there will certainly be some clean-up work to do, such as renaming variables like "$TRAVIS_TAG".