From 225e4edd4ebb580371ac4c0e12c689a3a5ebe69f Mon Sep 17 00:00:00 2001 From: Gang Li Date: Sat, 11 May 2024 20:12:12 +0800 Subject: [PATCH 01/12] Update the README --- README.md | 27 +++++++++++++++++++++++---- 1 file changed, 23 insertions(+), 4 deletions(-) diff --git a/README.md b/README.md index af0f6c89..d52a4506 100644 --- a/README.md +++ b/README.md @@ -60,6 +60,7 @@ This command will upload the repo in tarball to S3. It will auto-detect if the tarball is for maven or npm * For maven type, it will: + * Scan the tarball for all paths and collect them all. * Check the existence in S3 for all those paths. * Filter out the paths in tarball based on: @@ -70,10 +71,8 @@ It will auto-detect if the tarball is for maven or npm * Upload these artifacts to S3 with metadata of the product. * If the artifacts already exists in S3, update the metadata of the product by appending the new product. - * NPM type (TBH): We need to know the exact tarball structure of npm repo - * For both types, after uploading the files, regenerate/refresh the index files for these paths. @@ -107,10 +106,30 @@ This command will refresh the index.html for the specified path. * Note that if the path is a NPM metadata path which contains package.json, this refreshment will not work because this type of folder will display the package.json instead of the index.html in http request. -### charon-validate: validate the checksum of files in specified path in a maven repository +### charon-cf-check: check the invalidation status of the specified invalidation id for AWS CloudFront + +```bash +usage: charon cf check $invalidation_id [-t, --target] [-D, --debug] [-q, --quiet] +``` + +### charon-cf-invalidate: do invalidating on AWS CloudFront for the specified paths + +```bash +usage: charon cf invalidate [-t, --target] [-p, --path] [-f, --path-file] [-D, --debug] [-q, --quiet] +``` + +### charon-checksum-validate: validate the checksum of files in specified path in a maven repository ```bash -usage: charon validate $path [-t, --target] [-f, --report_file_path] [-i, --includes] [-r, --recursive] [-D, --debug] [-q, --quiet] +usage: charon checksum validate $path [-t, --target] [-f, --report_file_path] [-i, --includes] [-r, --recursive] [-D, --debug] [-q, --quiet] ``` This command will validate the checksum of the specified path for the maven repository. It will calculate the sha1 checksum of all artifact files in the specified path and compare with the companied .sha1 files of the artifacts, then record all mismatched artifacts in the report file. If some artifact files misses the companied .sha1 files, they will also be recorded. + +### charon-checksum-refresh: refresh the checksum files for the artifacts in the specified maven repository + +```bash +usage: charon checksum refresh [-t, --target] [-p, --path] [-f, --path-file] [-D, --debug] [-q, --quiet] +``` + +This command will refresh the checksum files for the specified artifact files in the maven repository. Sometimes the checksum files are not matched with the artifacts by some reason, so this command will do the refresh to make it match again. It will calculate the checksums of all artifact files in the specified path and compare with the companied checksum files of the artifacts, if the checksum are not matched, they will be refreshed. From 57da84ac9fb2929ce71845285796e4fe83f40e4f Mon Sep 17 00:00:00 2001 From: Gang Li Date: Sun, 2 Jun 2024 11:20:34 +0800 Subject: [PATCH 02/12] Mv Dockerfile to image folder and fix some issues * Use ubi8-minimal as base image * Removed some useless files in image --- Dockerfile | 58 ------------------------------------- config/charon.yaml.sample | 5 ++++ image/Containerfile | 60 +++++++++++++++++++++++++++++++++++++++ 3 files changed, 65 insertions(+), 58 deletions(-) delete mode 100644 Dockerfile create mode 100644 image/Containerfile diff --git a/Dockerfile b/Dockerfile deleted file mode 100644 index 8111c485..00000000 --- a/Dockerfile +++ /dev/null @@ -1,58 +0,0 @@ -#### -# This Dockerfile is used to build the container for charon -# -# charon requires python 3 -# -# 0. Step into the project dir -# -# 1. Build the image -# docker/podman build -t charon:1.0.0 . -# -# 2. Run the container as daemon, mount the host ~/upload/ path to container /root/upload/ path, -# the uploading path is the dir location where you will upload the tarballs from -# add -e to set specific environment variables, such as: AWS_PROFILE, aws_endpoint_url, bucket -# docker/podman run -dit -v ~/upload/:/root/upload/ --name charon charon:1.0.0 -# -# 3. Execute the container -# docker/podman exec -it charon bash -# -# 4. Start using uploader -# charon upload/delete from /root/upload/... -### - -# parser directive, always points to the latest release of the version 1 syntax, -# automatically checks for updates before building, making sure using the most current version -# syntax=docker/dockerfile:1 -FROM python:3.8 - -# ensure the latest version of pip -RUN pip3 install --no-cache-dir --upgrade pip - -RUN adduser charon -USER charon -WORKDIR /home/charon - -# pip respects TMPDIR to set another enough disk space for pip packages installation -ENV TMPDIR="/home/charon/tmp" - -# install all required packages -COPY --chown=charon:charon requirements.txt ./ -RUN pip3 install --user --no-cache-dir -r requirements.txt - -# prepare configs for charon -ADD ./config/charon.conf /home/charon/.charon/charon.conf -ADD ./config/aws-credentials /home/charon/.aws/credentials - -# prepare templates for charon -ADD ./template/index.html.j2 /home/charon/.charon/template/index.html.j2 -ADD ./template/maven-metadata.xml.j2 /home/charon/.charon/template/maven-metadata.xml.j2 - -ENV PATH="/home/charon/.local/bin:${PATH}" -COPY --chown=charon:charon . . - -# install charon -RUN pip3 install --user --no-cache-dir . - -# this will be invoked when container runs, charon will directly setup -# from the container and keep running as long as the bash is active -CMD ["bash"] diff --git a/config/charon.yaml.sample b/config/charon.yaml.sample index 97c37d4a..b3e0fb71 100644 --- a/config/charon.yaml.sample +++ b/config/charon.yaml.sample @@ -1,3 +1,6 @@ +#aws_profile: ${profile} +#aws_cf_enable: True + ignore_patterns: - ".*^(redhat).*" - ".*snapshot.*" @@ -30,3 +33,5 @@ targets: - bucket: "stage-npm-npmjs" prefix: / registry: "npm.stage.registry.redhat.com" + +#manifest_bucket: manifest \ No newline at end of file diff --git a/image/Containerfile b/image/Containerfile new file mode 100644 index 00000000..34074ff2 --- /dev/null +++ b/image/Containerfile @@ -0,0 +1,60 @@ +#### +# This Containerfile is used to build the container for charon +# +# charon requires python 3 +# +# 0. Step into the project dir +# +# 1. Build the image +# docker/podman build -t charon:1.0.0 -f image/Containerfile . +# +# 2. Run the container as daemon, mount the host ~/upload/ path to container /root/upload/ path, +# the uploading path is the dir location where you will upload the tarballs from +# add -e to set specific environment variables, such as: AWS_PROFILE, aws_endpoint_url, bucket +# docker/podman run -dit -v ~/upload/:/home/charon/upload/ --name charon charon:1.0.0 +# +# 3. Execute the container +# docker/podman exec -it charon bash +# +# 4. Start using uploader +# charon upload/delete from /home/charon/upload/... +### +FROM registry.access.redhat.com/ubi8-minimal:latest + +LABEL description="Charon upload image" \ + summary="Charon upload image" \ + maintainer="RedHat SPMM Team " \ + vendor="Red Hat, Inc." \ + distribution-scope="public" \ + vcs-type="git" + +ARG USER=charon +ARG UID=10000 +ARG HOME_DIR=/home/${USER} + +WORKDIR ${HOME_DIR} + +USER root + +RUN microdnf install -y python3.9 jq shadow-utils \ + && microdnf clean all +RUN pip3 install --no-cache-dir --upgrade pip +RUN useradd -d ${HOME_DIR} -u ${UID} -g 0 -m -s /bin/bash ${USER} \ + && chown ${USER}:0 ${HOME_DIR} \ + && chmod -R g+rwx ${HOME_DIR} \ + && chmod g+rw /etc/passwd + +COPY ./charon ./charon +COPY ./requirements.txt ./setup.py ./ + +RUN pip3 install --no-cache-dir -r ./requirements.txt +RUN pip3 install --no-cache-dir . + +USER ${USER} + +ENV HOME=${HOME_DIR} \ + LANG=en_US.UTF-8 + +# this will be invoked when container runs, charon will directly setup +# from the container and keep running as long as the bash is active +CMD ["bash"] From 61019173d4907c6098fd50a02f18c92518cc948f Mon Sep 17 00:00:00 2001 From: Gang Li Date: Tue, 4 Jun 2024 15:37:24 +0800 Subject: [PATCH 03/12] Add RH root CA to the image --- image/2022-IT-Root-CA.pem | 37 +++++++++++++++++++++++++++++++++++++ image/Containerfile | 5 ++++- 2 files changed, 41 insertions(+), 1 deletion(-) create mode 100644 image/2022-IT-Root-CA.pem diff --git a/image/2022-IT-Root-CA.pem b/image/2022-IT-Root-CA.pem new file mode 100644 index 00000000..dbe10450 --- /dev/null +++ b/image/2022-IT-Root-CA.pem @@ -0,0 +1,37 @@ +-----BEGIN CERTIFICATE----- +MIIGcjCCBFqgAwIBAgIFICIEEFwwDQYJKoZIhvcNAQEMBQAwgaMxCzAJBgNVBAYT +AlVTMRcwFQYDVQQIDA5Ob3J0aCBDYXJvbGluYTEQMA4GA1UEBwwHUmFsZWlnaDEW +MBQGA1UECgwNUmVkIEhhdCwgSW5jLjETMBEGA1UECwwKUmVkIEhhdCBJVDEZMBcG +A1UEAwwQSW50ZXJuYWwgUm9vdCBDQTEhMB8GCSqGSIb3DQEJARYSaW5mb3NlY0By +ZWRoYXQuY29tMCAXDTIzMDQwNTE4MzM0NFoYDzIwNTIwNDAyMTgzMzQ0WjCBozEL +MAkGA1UEBhMCVVMxFzAVBgNVBAgMDk5vcnRoIENhcm9saW5hMRAwDgYDVQQHDAdS +YWxlaWdoMRYwFAYDVQQKDA1SZWQgSGF0LCBJbmMuMRMwEQYDVQQLDApSZWQgSGF0 +IElUMRkwFwYDVQQDDBBJbnRlcm5hbCBSb290IENBMSEwHwYJKoZIhvcNAQkBFhJp +bmZvc2VjQHJlZGhhdC5jb20wggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIKAoIC +AQCxuloEVglzWXZ9FFFUOSVdpRIB2jW5YBpwgMem2fPZeWIIvrVQ6PL9XNenDOXu +BHbShD/PApxi/ujSZyOIjLsNh7WDO+0NqpkfTyB9wUYAhx3GTIGY75RSoyZy1yKb +ZDTKv+rSfui9IlstAMz6L3OQLZES9zAYK8ICiDUwTeNZ7quA6qf0Kam2LyuBc/bl +BI7WFLOGGWY135P1OUXJgnJUsMhnYMTgvZQyJ2P7eLQpiR8TOr5ZI6CYapiyG64L +nkr/rsALjSxoUo09Yai1CVO66VFJ/XgMNt3mzQtLDMPXiKUuwsBsgvo4QvLjkXYI +ii+/YQyQaypsKctG8mefKkTT1kRDKj4LNdTRRgd5tco+b4+O/4upt8mIsx1+tbdM +LNGEz3Jqd0sj8Fl4Rzus+W+enzXmMfZH86X6bU5tMvueuFd5LV+M9XzliscaEQMK +EQ7CC72ldrOK2K12Gjb7bu8dKq+aSlNuWK+Gz1NvbwYpaCBYp0JoryvHEq5jrCLP +lTkuJQ3HaaAf+4LaBm8no9xK2VbDf6l/7Htb5I5LnAAZi0/5TzH07NhHoIeMSmTE +Ea07i/i5lbhM2qbx6pfLukg24HLCKTdi4Fo6/JqPWH6/3eI55NsoWSmoDdTiLg4v +1G/rgUVr2N6F36GTYMGqiITvvd4Qm3i9XOTQvsx8RJx4JQIDAQABo4GoMIGlMB0G +A1UdDgQWBBS1+o3lCnihCZXbTSGGlWpZT0nIizAfBgNVHSMEGDAWgBS1+o3lCnih +CZXbTSGGlWpZT0nIizAPBgNVHRMBAf8EBTADAQH/MA4GA1UdDwEB/wQEAwIBhjAR +BglghkgBhvhCAQEEBAMCAQYwLwYDVR0fBCgwJjAkoCKgIIYeaHR0cDovL29jc3Au +cmVkaGF0LmNvbS9jcmwucGVtMA0GCSqGSIb3DQEBDAUAA4ICAQCDLaGTS0g2HmMS +g0i6Z0RVDC7sSnWFgEk2ZO1WUQj5WkFVS7gWxed/mXCzeL2EV1Pd22YKHM1eU1vo +6b03cbNRXlRGGFksmQeM9h2sVjbP0hRZxqqfI+UW223N8E+qK3wSa8m6nhOfIJie +DD9s8CdL1VT6l4qq2gR8mVBW7EZ+Ux5u+AMXpN4WPEkcLer2djbfhXoPsJ4r5CcX +vh7W5rCZbo+0oBI5hrTlG4Tjhv1atqLhMmssjn8NbRrnhrbGF7w8NxFts69GkKDB +UIXr1pWZSAuRELlIxmvh5ZSX5YTbFmDuTvmNx8RPPy6OY4W1v1BUKp0HyJTi07s2 +8SN+n9htHPHX9XBZctQmOSFLiqhi15LIqI54tR2tSgwH3Z5moh4sy6MuApXstsu4 +qtkII2KZk3SottI8MOS6zqKrU7jPou6ZE0fznNiu23Q3Ksuuj6mBkLVw3bQe68Vm +NUTDac1oVzc8d5NMbx5kVb4Lahq+SATVFC8NK9G/Pk1AiwO8WhKffySsLeO5nMib +4BOVq0qFoAi8YCFuJOl9FlH1dPW/TnqlTQMQNhXpzGjU3HV3lr/Mk+ghNgIYcLcz +pEBsiGwKOVW4nYKIqPLn/36Ao/kfXeAdJhaAZq1SkTbeqNiwHQm3KNHzNObmjD0f +56vmq8fwQYIcazjrygWiaOnoep/SMw== +-----END CERTIFICATE----- diff --git a/image/Containerfile b/image/Containerfile index 34074ff2..2612a44f 100644 --- a/image/Containerfile +++ b/image/Containerfile @@ -23,7 +23,7 @@ FROM registry.access.redhat.com/ubi8-minimal:latest LABEL description="Charon upload image" \ summary="Charon upload image" \ - maintainer="RedHat SPMM Team " \ + maintainer="RedHat SPMM Dev Team" \ vendor="Red Hat, Inc." \ distribution-scope="public" \ vcs-type="git" @@ -44,6 +44,9 @@ RUN useradd -d ${HOME_DIR} -u ${UID} -g 0 -m -s /bin/bash ${USER} \ && chmod -R g+rwx ${HOME_DIR} \ && chmod g+rw /etc/passwd +ADD ./image/2022-IT-Root-CA.pem /etc/pki/ca-trust/source/anchors/2022-IT-Root-CA.pem +RUN update-ca-trust extract + COPY ./charon ./charon COPY ./requirements.txt ./setup.py ./ From 3cfde6c2c7350147dbc8daa0d2fa882a2055003b Mon Sep 17 00:00:00 2001 From: Gang Li Date: Thu, 4 Jul 2024 14:31:06 +0800 Subject: [PATCH 04/12] Update Containerfile * Remove context dependency when build * Change install way for charon to wheel --- image/Containerfile | 19 ++++++++++--------- 1 file changed, 10 insertions(+), 9 deletions(-) diff --git a/image/Containerfile b/image/Containerfile index 2612a44f..6989176a 100644 --- a/image/Containerfile +++ b/image/Containerfile @@ -31,12 +31,13 @@ LABEL description="Charon upload image" \ ARG USER=charon ARG UID=10000 ARG HOME_DIR=/home/${USER} +ARG GIT_BRANCH=main WORKDIR ${HOME_DIR} USER root -RUN microdnf install -y python3.9 jq shadow-utils \ +RUN microdnf install -y git-core python3.9 shadow-utils \ && microdnf clean all RUN pip3 install --no-cache-dir --upgrade pip RUN useradd -d ${HOME_DIR} -u ${UID} -g 0 -m -s /bin/bash ${USER} \ @@ -44,20 +45,20 @@ RUN useradd -d ${HOME_DIR} -u ${UID} -g 0 -m -s /bin/bash ${USER} \ && chmod -R g+rwx ${HOME_DIR} \ && chmod g+rw /etc/passwd -ADD ./image/2022-IT-Root-CA.pem /etc/pki/ca-trust/source/anchors/2022-IT-Root-CA.pem +RUN git clone -b ${GIT_BRANCH} --depth 1 https://github.com/Commonjava/charon.git +RUN cp ./charon/image/2022-IT-Root-CA.pem /etc/pki/ca-trust/source/anchors/2022-IT-Root-CA.pem RUN update-ca-trust extract -COPY ./charon ./charon -COPY ./requirements.txt ./setup.py ./ +RUN pip3 install --no-cache-dir -r ./charon/requirements.txt +RUN pip3 wheel ./charon +RUN pip3 install --no-cache-dir ./*.whl +RUN rm -rf ./charon ./*.whl -RUN pip3 install --no-cache-dir -r ./requirements.txt -RUN pip3 install --no-cache-dir . +RUN microdnf remove git-core shadow-utils && microdnf clean all USER ${USER} ENV HOME=${HOME_DIR} \ LANG=en_US.UTF-8 -# this will be invoked when container runs, charon will directly setup -# from the container and keep running as long as the bash is active -CMD ["bash"] +CMD ["/usr/local/bin/charon"] From 25e43637190d1e9b2f0519ea111737346ee3585d Mon Sep 17 00:00:00 2001 From: Gang Li Date: Thu, 4 Jul 2024 15:09:45 +0800 Subject: [PATCH 05/12] Update Containerfile to use multi-stage build to reduce size --- image/Containerfile | 34 +++++++++++++++++----------------- 1 file changed, 17 insertions(+), 17 deletions(-) diff --git a/image/Containerfile b/image/Containerfile index 6989176a..ecdf29c1 100644 --- a/image/Containerfile +++ b/image/Containerfile @@ -19,42 +19,42 @@ # 4. Start using uploader # charon upload/delete from /home/charon/upload/... ### -FROM registry.access.redhat.com/ubi8-minimal:latest +FROM registry.access.redhat.com/ubi8-minimal:latest as builder -LABEL description="Charon upload image" \ - summary="Charon upload image" \ - maintainer="RedHat SPMM Dev Team" \ - vendor="Red Hat, Inc." \ - distribution-scope="public" \ - vcs-type="git" +ARG GIT_BRANCH=main + +RUN microdnf install -y git-core python3.9 && microdnf clean all +RUN git clone -b ${GIT_BRANCH} --depth 1 https://github.com/Commonjava/charon.git +RUN pip3 install --no-cache-dir --upgrade pip +RUN pip3 wheel ./charon + +FROM registry.access.redhat.com/ubi8-minimal:latest ARG USER=charon ARG UID=10000 ARG HOME_DIR=/home/${USER} -ARG GIT_BRANCH=main WORKDIR ${HOME_DIR} USER root -RUN microdnf install -y git-core python3.9 shadow-utils \ - && microdnf clean all -RUN pip3 install --no-cache-dir --upgrade pip +RUN microdnf install -y python3.9 shadow-utils && microdnf clean all RUN useradd -d ${HOME_DIR} -u ${UID} -g 0 -m -s /bin/bash ${USER} \ && chown ${USER}:0 ${HOME_DIR} \ && chmod -R g+rwx ${HOME_DIR} \ && chmod g+rw /etc/passwd -RUN git clone -b ${GIT_BRANCH} --depth 1 https://github.com/Commonjava/charon.git -RUN cp ./charon/image/2022-IT-Root-CA.pem /etc/pki/ca-trust/source/anchors/2022-IT-Root-CA.pem +COPY --from=builder ./charon/image/2022-IT-Root-CA.pem /etc/pki/ca-trust/source/anchors/2022-IT-Root-CA.pem RUN update-ca-trust extract -RUN pip3 install --no-cache-dir -r ./charon/requirements.txt -RUN pip3 wheel ./charon +COPY --from=builder ./charon/requirements.txt ./ +COPY --from=builder ./*.whl ./ +RUN pip3 install --no-cache-dir --upgrade pip +RUN pip3 install --no-cache-dir -r ./requirements.txt RUN pip3 install --no-cache-dir ./*.whl -RUN rm -rf ./charon ./*.whl +RUN rm -rf ./requirements.txt ./*.whl -RUN microdnf remove git-core shadow-utils && microdnf clean all +RUN microdnf remove shadow-utils && microdnf clean all USER ${USER} From 074dcd42be49ea84c427c40be013aa60ecdd4100 Mon Sep 17 00:00:00 2001 From: Gang Li Date: Sun, 7 Jul 2024 09:46:34 +0800 Subject: [PATCH 06/12] Adjust requirements.txt Just need semantic-version instead of setuptools-rust --- .gitignore | 2 ++ requirements.txt | 4 ++-- 2 files changed, 4 insertions(+), 2 deletions(-) diff --git a/.gitignore b/.gitignore index f4df6301..6db38818 100644 --- a/.gitignore +++ b/.gitignore @@ -13,6 +13,8 @@ package/ .local local .DS_Store +*.whl + # Unit test __pytest_reports diff --git a/requirements.txt b/requirements.txt index 4f1b20ad..428872a0 100644 --- a/requirements.txt +++ b/requirements.txt @@ -1,4 +1,3 @@ -setuptools-rust==1.7.0 Jinja2==3.1.4 boto3==1.28.46 botocore==1.31.46 @@ -8,4 +7,5 @@ PyYAML==6.0.1 defusedxml==0.7.1 subresource-integrity==0.2 jsonschema==4.19.0 -urllib3==1.26.18 \ No newline at end of file +urllib3==1.26.18 +semantic-version==2.10.0 \ No newline at end of file From e5596eb18acea33870d5d8dc5e8b3e1faba75e47 Mon Sep 17 00:00:00 2001 From: Gang Li Date: Sun, 7 Jul 2024 10:33:21 +0800 Subject: [PATCH 07/12] Adjust requirements.txt - add lower version bounds --- requirements.txt | 22 +++++++++++----------- tests/requirements.txt | 4 ++-- 2 files changed, 13 insertions(+), 13 deletions(-) diff --git a/requirements.txt b/requirements.txt index 428872a0..8ec8ce9d 100644 --- a/requirements.txt +++ b/requirements.txt @@ -1,11 +1,11 @@ -Jinja2==3.1.4 -boto3==1.28.46 -botocore==1.31.46 -click==8.1.7 -requests==2.31.0 -PyYAML==6.0.1 -defusedxml==0.7.1 -subresource-integrity==0.2 -jsonschema==4.19.0 -urllib3==1.26.18 -semantic-version==2.10.0 \ No newline at end of file +Jinja2>=3.1.4 +boto3>=1.28.46 +botocore>=1.31.46 +click>=8.1.7 +requests>=2.31.0 +PyYAML>=6.0.1 +defusedxml>=0.7.1 +subresource-integrity>=0.2 +jsonschema>=4.19.0 +urllib3>=1.26.18 +semantic-version>=2.10.0 \ No newline at end of file diff --git a/tests/requirements.txt b/tests/requirements.txt index 4acad34d..43ac571a 100644 --- a/tests/requirements.txt +++ b/tests/requirements.txt @@ -5,5 +5,5 @@ pytest-cov pytest-html flake8 requests-mock -moto==5.0.3 -python-gnupg==0.5.0 +moto>=5.0.3,<6 +python-gnupg>=0.5.0,<1 From 1f534391f04e8e0e9d0f9f39e46121de6980786d Mon Sep 17 00:00:00 2001 From: Gang Li Date: Mon, 8 Jul 2024 10:00:35 +0800 Subject: [PATCH 08/12] Add setuptools for pkg_resources module missing --- requirements.txt | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/requirements.txt b/requirements.txt index 8ec8ce9d..a2f622f9 100644 --- a/requirements.txt +++ b/requirements.txt @@ -8,4 +8,5 @@ defusedxml>=0.7.1 subresource-integrity>=0.2 jsonschema>=4.19.0 urllib3>=1.26.18 -semantic-version>=2.10.0 \ No newline at end of file +semantic-version>=2.10.0 +setuptools>=70.0.0 From 5733f9321212fd92d4cfdce9704934014c6806d9 Mon Sep 17 00:00:00 2001 From: Gang Li Date: Mon, 8 Jul 2024 10:20:44 +0800 Subject: [PATCH 09/12] Add dependencies in setup.py --- setup.py | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/setup.py b/setup.py index 0c11c688..dbfdd940 100755 --- a/setup.py +++ b/setup.py @@ -50,4 +50,18 @@ entry_points={ "console_scripts": ["charon = charon.cmd:cli"], }, + install_requires=[ + "Jinja2>=3.1.4", + "boto3>=1.28.46", + "botocore>=1.31.46", + "click>=8.1.7", + "requests>=2.31.0", + "PyYAML>=6.0.1", + "defusedxml>=0.7.1", + "subresource-integrity>=0.2", + "jsonschema>=4.19.0", + "urllib3>=1.26.18", + "semantic-version>=2.10.0", + "setuptools>=70.0.0", + ], ) From c56dab2ad809acde06d8b2643179b2f43e7210cd Mon Sep 17 00:00:00 2001 From: Gang Li Date: Mon, 8 Jul 2024 18:05:18 +0800 Subject: [PATCH 10/12] Update Containerfile to use pip wheel way to install --- image/Containerfile | 10 ++++------ 1 file changed, 4 insertions(+), 6 deletions(-) diff --git a/image/Containerfile b/image/Containerfile index ecdf29c1..8e38acbc 100644 --- a/image/Containerfile +++ b/image/Containerfile @@ -23,7 +23,7 @@ FROM registry.access.redhat.com/ubi8-minimal:latest as builder ARG GIT_BRANCH=main -RUN microdnf install -y git-core python3.9 && microdnf clean all +RUN microdnf install -y git-core python3.12 python3.12-pip && microdnf clean all RUN git clone -b ${GIT_BRANCH} --depth 1 https://github.com/Commonjava/charon.git RUN pip3 install --no-cache-dir --upgrade pip RUN pip3 wheel ./charon @@ -38,7 +38,7 @@ WORKDIR ${HOME_DIR} USER root -RUN microdnf install -y python3.9 shadow-utils && microdnf clean all +RUN microdnf install -y python3.12 python3.12-pip shadow-utils && microdnf clean all RUN useradd -d ${HOME_DIR} -u ${UID} -g 0 -m -s /bin/bash ${USER} \ && chown ${USER}:0 ${HOME_DIR} \ && chmod -R g+rwx ${HOME_DIR} \ @@ -47,14 +47,12 @@ RUN useradd -d ${HOME_DIR} -u ${UID} -g 0 -m -s /bin/bash ${USER} \ COPY --from=builder ./charon/image/2022-IT-Root-CA.pem /etc/pki/ca-trust/source/anchors/2022-IT-Root-CA.pem RUN update-ca-trust extract -COPY --from=builder ./charon/requirements.txt ./ COPY --from=builder ./*.whl ./ RUN pip3 install --no-cache-dir --upgrade pip -RUN pip3 install --no-cache-dir -r ./requirements.txt RUN pip3 install --no-cache-dir ./*.whl -RUN rm -rf ./requirements.txt ./*.whl +RUN rm ./*.whl -RUN microdnf remove shadow-utils && microdnf clean all +RUN microdnf remove python3.12-pip shadow-utils && microdnf clean all USER ${USER} From 32dcf26ea71f66c48d2a0a9114fdae9b835c9e9e Mon Sep 17 00:00:00 2001 From: Gang Li Date: Wed, 10 Jul 2024 16:55:39 +0800 Subject: [PATCH 11/12] Remove ca update for RH-IT-Root as it is used internally --- image/2022-IT-Root-CA.pem | 37 ------------------------------------- image/Containerfile | 3 --- 2 files changed, 40 deletions(-) delete mode 100644 image/2022-IT-Root-CA.pem diff --git a/image/2022-IT-Root-CA.pem b/image/2022-IT-Root-CA.pem deleted file mode 100644 index dbe10450..00000000 --- a/image/2022-IT-Root-CA.pem +++ /dev/null @@ -1,37 +0,0 @@ ------BEGIN CERTIFICATE----- -MIIGcjCCBFqgAwIBAgIFICIEEFwwDQYJKoZIhvcNAQEMBQAwgaMxCzAJBgNVBAYT -AlVTMRcwFQYDVQQIDA5Ob3J0aCBDYXJvbGluYTEQMA4GA1UEBwwHUmFsZWlnaDEW -MBQGA1UECgwNUmVkIEhhdCwgSW5jLjETMBEGA1UECwwKUmVkIEhhdCBJVDEZMBcG -A1UEAwwQSW50ZXJuYWwgUm9vdCBDQTEhMB8GCSqGSIb3DQEJARYSaW5mb3NlY0By -ZWRoYXQuY29tMCAXDTIzMDQwNTE4MzM0NFoYDzIwNTIwNDAyMTgzMzQ0WjCBozEL -MAkGA1UEBhMCVVMxFzAVBgNVBAgMDk5vcnRoIENhcm9saW5hMRAwDgYDVQQHDAdS -YWxlaWdoMRYwFAYDVQQKDA1SZWQgSGF0LCBJbmMuMRMwEQYDVQQLDApSZWQgSGF0 -IElUMRkwFwYDVQQDDBBJbnRlcm5hbCBSb290IENBMSEwHwYJKoZIhvcNAQkBFhJp -bmZvc2VjQHJlZGhhdC5jb20wggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIKAoIC -AQCxuloEVglzWXZ9FFFUOSVdpRIB2jW5YBpwgMem2fPZeWIIvrVQ6PL9XNenDOXu -BHbShD/PApxi/ujSZyOIjLsNh7WDO+0NqpkfTyB9wUYAhx3GTIGY75RSoyZy1yKb -ZDTKv+rSfui9IlstAMz6L3OQLZES9zAYK8ICiDUwTeNZ7quA6qf0Kam2LyuBc/bl -BI7WFLOGGWY135P1OUXJgnJUsMhnYMTgvZQyJ2P7eLQpiR8TOr5ZI6CYapiyG64L -nkr/rsALjSxoUo09Yai1CVO66VFJ/XgMNt3mzQtLDMPXiKUuwsBsgvo4QvLjkXYI -ii+/YQyQaypsKctG8mefKkTT1kRDKj4LNdTRRgd5tco+b4+O/4upt8mIsx1+tbdM -LNGEz3Jqd0sj8Fl4Rzus+W+enzXmMfZH86X6bU5tMvueuFd5LV+M9XzliscaEQMK -EQ7CC72ldrOK2K12Gjb7bu8dKq+aSlNuWK+Gz1NvbwYpaCBYp0JoryvHEq5jrCLP -lTkuJQ3HaaAf+4LaBm8no9xK2VbDf6l/7Htb5I5LnAAZi0/5TzH07NhHoIeMSmTE -Ea07i/i5lbhM2qbx6pfLukg24HLCKTdi4Fo6/JqPWH6/3eI55NsoWSmoDdTiLg4v -1G/rgUVr2N6F36GTYMGqiITvvd4Qm3i9XOTQvsx8RJx4JQIDAQABo4GoMIGlMB0G -A1UdDgQWBBS1+o3lCnihCZXbTSGGlWpZT0nIizAfBgNVHSMEGDAWgBS1+o3lCnih -CZXbTSGGlWpZT0nIizAPBgNVHRMBAf8EBTADAQH/MA4GA1UdDwEB/wQEAwIBhjAR -BglghkgBhvhCAQEEBAMCAQYwLwYDVR0fBCgwJjAkoCKgIIYeaHR0cDovL29jc3Au -cmVkaGF0LmNvbS9jcmwucGVtMA0GCSqGSIb3DQEBDAUAA4ICAQCDLaGTS0g2HmMS -g0i6Z0RVDC7sSnWFgEk2ZO1WUQj5WkFVS7gWxed/mXCzeL2EV1Pd22YKHM1eU1vo -6b03cbNRXlRGGFksmQeM9h2sVjbP0hRZxqqfI+UW223N8E+qK3wSa8m6nhOfIJie -DD9s8CdL1VT6l4qq2gR8mVBW7EZ+Ux5u+AMXpN4WPEkcLer2djbfhXoPsJ4r5CcX -vh7W5rCZbo+0oBI5hrTlG4Tjhv1atqLhMmssjn8NbRrnhrbGF7w8NxFts69GkKDB -UIXr1pWZSAuRELlIxmvh5ZSX5YTbFmDuTvmNx8RPPy6OY4W1v1BUKp0HyJTi07s2 -8SN+n9htHPHX9XBZctQmOSFLiqhi15LIqI54tR2tSgwH3Z5moh4sy6MuApXstsu4 -qtkII2KZk3SottI8MOS6zqKrU7jPou6ZE0fznNiu23Q3Ksuuj6mBkLVw3bQe68Vm -NUTDac1oVzc8d5NMbx5kVb4Lahq+SATVFC8NK9G/Pk1AiwO8WhKffySsLeO5nMib -4BOVq0qFoAi8YCFuJOl9FlH1dPW/TnqlTQMQNhXpzGjU3HV3lr/Mk+ghNgIYcLcz -pEBsiGwKOVW4nYKIqPLn/36Ao/kfXeAdJhaAZq1SkTbeqNiwHQm3KNHzNObmjD0f -56vmq8fwQYIcazjrygWiaOnoep/SMw== ------END CERTIFICATE----- diff --git a/image/Containerfile b/image/Containerfile index 8e38acbc..6ee0c02c 100644 --- a/image/Containerfile +++ b/image/Containerfile @@ -44,9 +44,6 @@ RUN useradd -d ${HOME_DIR} -u ${UID} -g 0 -m -s /bin/bash ${USER} \ && chmod -R g+rwx ${HOME_DIR} \ && chmod g+rw /etc/passwd -COPY --from=builder ./charon/image/2022-IT-Root-CA.pem /etc/pki/ca-trust/source/anchors/2022-IT-Root-CA.pem -RUN update-ca-trust extract - COPY --from=builder ./*.whl ./ RUN pip3 install --no-cache-dir --upgrade pip RUN pip3 install --no-cache-dir ./*.whl From dae5372845cde5b40327a9c658df98085cfe4796 Mon Sep 17 00:00:00 2001 From: Gang Li Date: Wed, 10 Jul 2024 17:31:40 +0800 Subject: [PATCH 12/12] Update version to 1.3.2 in setup.py --- setup.py | 6 +----- 1 file changed, 1 insertion(+), 5 deletions(-) diff --git a/setup.py b/setup.py index dbfdd940..c09e3784 100755 --- a/setup.py +++ b/setup.py @@ -15,12 +15,8 @@ """ from setuptools import setup, find_packages -version = "1.3.1" +version = "1.3.2" -# f = open('README.md') -# long_description = f.read().strip() -# long_description = long_description.split('split here', 1)[1] -# f.close() long_description = """ This charon is a tool to synchronize several types of artifacts repository data to RedHat Ronda service (maven.repository.redhat.com).