Skip to content

Conversation

@zvigrinberg
Copy link
Contributor

@zvigrinberg zvigrinberg commented Jan 8, 2024

Description

For CI/CD pipelines, it's a lot of times better to install requirements.txt manifest in a dedicated agent/slave/computer/container that contains the desired python & pip versions, and invoke the rhda analysis in different agent/slave/computer/container ( let's denote it as "node") - that contains the whole infrastructure required to invoke the analysis alone ( including python and pip binaries, but the problem is that it "stuck" with a given versions of python and pip in this node, hence very limited to the versions of packages in requirements.txt).

For that purpose, and in such cases that we need to separate the analysis and the pip install to run in different steps/stages and even on different nodes, this PR introduces two environment variables, these two env variables will need to be populated with the output of the commands (pip show and pip freeze commands) encoded in base64 ( in order to preserve new lines in environment variables, the library will decode it to ASCII plain-text before passing it to the logic of the API).

  1. EXHORT_PIP_FREEZE - need to be populated with the pip freeze --all output after pip install on requirements.txt finished on the python node.
  2. EXHORT_PIP_SHOW - need to be populated with the output of pip freeze show listOfAllPackagesInPipFreeze.
    Example how to retrieve in shell script:
# get all packages names in a list.
export SHOW_LIST=$(pip freeze --all | awk -F '==' '{print $1}')
# run pip show with all the packages from the list
pip show $SHOW_LIST > pip_show.txt
export EXHORT_PIP_SHOW=$(cat pip_show.txt | base64 -w0)

needs to work together with setting EXHORT_PYTHON_VIRTUAL_ENV=false ( this is the default so not specifying this setting is just fine).

Off course, you need to use pipelines workspaces/shared volumes/pipes/fifos/ ipc shared memory/networking in order to pass the data from one node to another.

in case the two new environment variables are not populated, then the logic will be the same ( taking the pip freeze and pip show of the environment formed by pip and python binaries passed to the library.

Checklist

  • I have followed this repository's contributing guidelines.
  • I will adhere to the project's code of conduct.

Signed-off-by: Zvi Grinberg <zgrinber@redhat.com>
@zvigrinberg zvigrinberg force-pushed the feature/pip-externalize-freeze-and-show branch from b72d4ae to 311d9aa Compare January 9, 2024 12:38
Signed-off-by: Zvi Grinberg <zgrinber@redhat.com>
@zvigrinberg zvigrinberg merged commit 87af6eb into guacsec:main Jan 9, 2024
zvigrinberg added a commit to guacsec/trustify-da-java-client that referenced this pull request Feb 5, 2024
Implement guacsec/trustify-da-javascript-client#92
for Java-API

Developed to Ease and make life easier for running rhda analysis for
python pip requirements.txt manifest, as every requirements.txt with
certain versions for its package is tailored for a certain python
version.
This enables separating the installation of the requirements.txt from
the machine that runs the RHDA analysis ( will be used mainly from
jenkins pipelines together with RHDA Jenkins pipeline).

Signed-off-by: Zvi Grinberg <zgrinber@redhat.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant