Add requirements files for python dependencies

[rhmdnd]

This commit adds two separate requirements files that you can feed into
pip for installing python dependencies.

This makes it easier for contributors to install dependencies by having
them isolated to a couple specific files, instead of having to reference
build documentation.

[rhmdnd]

This overlaps with #10474 - but ideally at the end of the process all dependencies will either be in a bindep or requirements file.

[github-actions]

Start a new ephemeral environment with changes proposed in this pull request:

Fedora Environment

Oracle Linux 8 Environment

[matejak]

Would it be possible to use this in https://github.com/ComplianceAsCode/content/blob/master/.github/workflows/gate.yaml? i.e. to have the dnf-powered dep install stage followed by pip where it is applicable?

[evgenyz]

Would it be possible to use this in https://github.com/ComplianceAsCode/content/blob/master/.github/workflows/gate.yaml? i.e. to have the dnf-powered dep install stage followed by pip where it is applicable?

I'd just add that this stage could be optimized into dnf builddep foobar.spec where applicable. The apt-get manager on the other hand can only install specs from the version of the package in the repo AFAIK (so any new deps added during development cycle would cause troubles).

[rhmdnd]

Would it be possible to use this in https://github.com/ComplianceAsCode/content/blob/master/.github/workflows/gate.yaml? i.e. to have the dnf-powered dep install stage followed by pip where it is applicable?

You mean using the requirements.txt file during the python package install process?

diff --git a/.github/workflows/gate.yaml b/.github/workflows/gate.yaml
index ee43e6e2e4..0696d1bd46 100644
--- a/.github/workflows/gate.yaml
+++ b/.github/workflows/gate.yaml
@@ -60,7 +60,7 @@ jobs:
       - name: Install Deps
         run: apt-get install -y ansible-lint bats check cmake expat libopenscap8 libxml2-utils ninja-build python3-github python3-jinja2 python3-setuptools python3-pip python3-pytest python3-pytest-cov xsltproc
       - name: Install deps python
-        run: pip3 install ruamel.yaml yamlpath pyyaml==5.4.1
+        run: pip3 install -r requirements.txt
       - name: Checkout
         uses: actions/checkout@v2
       - name: Build

[rhmdnd]

Looks like we have some issues where the workflows can't find the requirements files.

[matejak]

Looks like we have some issues where the workflows can't find the requirements files.

It may be that checking out the repo first and then to download deps (i.e. right before building the content) could help ;-)

[rhmdnd]

Looks like we have some issues where the workflows can't find the requirements files.

It may be that checking out the repo first and then to download deps (i.e. right before building the content) could help ;-)

After I looked a little closer I saw that checkout was happening right after the pip install step. Thanks!

[rhmdnd]

Rebasing to pull in #10540

[rhmdnd]

Clean CI run now that we're past the testing farm issue (for now).

Should be good for another round of reviews.

[jan-cerny]

@rhmdnd @Mab879 What are your plans?

[Mab879]

@rhmdnd This looks good to me. Can you please rebase to fix the conflicts.

[codeclimate]

Code Climate has analyzed commit e014370 and detected 0 issues on this pull request.

The test coverage on the diff in this pull request is 100.0% (50% is the threshold).

This pull request will bring the total coverage in the repository to 52.8% (0.0% change).

View more on Code Climate.