NEXUS-38651 - Missing sha256 and sha512 checksums when uploading to raw repository with curl

[matejsp]

• What problem are you trying to solve?
  There is difference if you upload a file via nexus UI or via curl. The difference is in missing sha256 and sha512 hashes.
  I would like to have sha256 and sha512 automatically generated.

for f in * **/* ; do
    if [[ -f "$f" ]]; then
        echo "$f uploading to ${NEXUS_REPOSITORY_URL}/${HOSTED_REPOSITORY}/${PACKAGE_NAME}/${PACKAGE_VERSION}/${f}"
        curl -u ${NEXUS_USERNAME}:${NEXUS_PASSWORD} --upload-file ./${f} ${NEXUS_REPOSITORY_URL}/${HOSTED_REPOSITORY}/${PACKAGE_NAME}/${PACKAGE_VERSION}/${f}
    fi
done

Via CURL:

Via UI:

• Do you have a workaround you are using at present?
  No.

• What feature or behavior is this required for?
  In our CI we pin sha256 checksum so we know that the file was not altered on nexus.

• How could we solve this issue? (Not knowing is okay!)

One way is to always automatically calculate Sha256 and Sha512.

Alternative is how Artifactory does this (https://jfrog.com/help/r/jfrog-rest-apis/deploy-artifact-by-checksum):

for file in $(find a_folder -type f)
do
    ARTIFACT_MD5_CHECKSUM=$(md5sum $file | awk '{print $1}')
    ARTIFACT_SHA1_CHECKSUM=$(shasum -a 1 $file | awk '{ print $1 }')
    ARTIFACT_SHA256_CHECKSUM=$(shasum -a 256 $file | awk '{ print $1 }')

    echo curl --upload-file $file \
            --header "X-Checksum-MD5:${ARTIFACT_MD5_CHECKSUM}" \
            --header "X-Checksum-Sha1:${ARTIFACT_SHA1_CHECKSUM}" \
            --header "X-Checksum-Sha256:${ARTIFACT_SHA256_CHECKSUM}" \
            -u "admin:${APIKEY}" \
            -v http://URL/$file
done

• Tell us about your Nexus Repository deployment: what version, operating system, and database are you using?
  Sonatype Nexus Repository OSS 3.52.0-01

[mrprescott]

Thanks for filing this. We have this tracked internally as NEXUS-38651.

[igreenfield]

@mrprescott What is the status of this? is there any workaround for this?

[miguelgmedina]

We also experience this issue, we are using OSS 3.65.0-02 on OpenShift. Using Python requests library(not curl directly). One question, does the PRO version have this same issue?

[miguelgmedina]

We also experience this issue, we are using OSS 3.65.0-02 on OpenShift. Using Python requests library(not curl directly). One question, does the PRO version have this same issue?

I have tested this, the issue is the same for both OSS and PRO versions (3.65.0-02)

[miguelgmedina]

Is there any workaround to add these sha256sum or sha512sum AFTER uploading the raw packages?

[miguelgmedina]

As a workaround, instead of using the repository URL, use the Rest API for component upload. With Python:

def upload_artifact(nexus_url, repository_name, file_path, artifact_path, skip_ssl_verification, nexus_username, nexus_password):
    upload_url = f"https://{nexus_url}/service/rest/v1/components?repository={repository_name}"

    auth = HTTPBasicAuth(nexus_username, nexus_password)

    with open(file_path, 'rb') as file:
        payload = {
        'raw.directory': (None, os.path.dirname(artifact_path)),
        'raw.asset1.filename': (None, os.path.basename(file_path)),
        'raw.asset1': ( os.path.basename(file_path), file)
        }
    
        response = requests.post(upload_url, files=payload, verify=not skip_ssl_verification, auth=auth)

    return response

[m4rii0]

There is already a pull request adding this feature (#100), what else is needed to merge it?

[nblair]

Good news - we have completed a set of code changes that will result in the sha256 and sha512 checksums being correctly stored with the raw component when uploaded via PUT /repository/{repositoryName}/. These changes are expected to be delivered in the May 3.68.0 release. Thanks for your patience!