jf audit fail with Gradle version 8

[janardhanajl]

Describe the bug

Issue description:

jf audit does not work with Gradle 8. You need to change how the dependencies are collected if no specific publishing is defined. Resolving the archives configuration is no longer allowed with Gradle 8:

Current behavior

Observing the below error:

gradle-xray-audit-test# jf audit --gradle  

08:40:46 [🔵Info] Log path: /root/.jfrog/logs/jfrog-cli.2023-02-06.08-40-46.7516.log

Downloading https://services.gradle.org/distributions/gradle-8.0-rc-2-bin.zip

CalculatCalculating GrCalcCCCalculating Gradle dependencies ∙∙∙∙∙∙        

Calculating Gradle dependencies ∙∙∙∙∙∙        

Welcome to Gradle 8.0-rc-2!


For more details see https://docs.gradle.org/8.0-rc-2/release-notes.html

Calculating Gradle dependencies ●∙∙∙∙∙        

Starting a Gradle Daemon (subsequent builds will be faster)

FAILURE: Build failed with an exception.

 
* What went wrong:

Could not determine the dependencies of task ':artifactoryPublish'.

> Resolving dependency configuration 'archives' is not allowed as it is defined as 'canBeResolved=false'.

 Instead, a resolvable ('canBeResolved=true') dependency configuration that extends 'archives' should be resolved.


* Try:ating Gradle dependencies ∙∙∙●∙∙        

> Run with --stacktrace option to get the stack trace.

> Run with --info or --debug option to get more log output.

> Run with --scan to get full insights.

Reproduction steps

1. Have a gradle project.
2. Install the latest JFrog CLI version.
3. Configure JFrog CLI with JFrog Platform.
4. Install Gradle version Gradle 8.0-rc-2
5. Run the jf audit --gradle

Expected behavior

jf audit --gradle, should work as expected and display the vulnerability result

JFrog CLI version

2.34.1

Operating system type and version

ubuntu 22.04, 18.04, mac os

JFrog Artifactory version

No response

JFrog Xray version

No response

[omerzi]

Hey @janardhanajl, we recently made significant improvements to the audit command using Gradle. Please upgrade your JFrog CLI version to v2.36.1 and let me know if the issue has been resolved. Thanks!

[schmist]

@omerzi we cannot use version 2.36.1 since it adds an additional dependency which cannot be loaded in our environment (no Internet connection) and loading through Artifactory seems not to work. But it seems that it also works with 2.36.0. Is that correct?

[yahavi]

@schmist
Could you please provide more information about the dependency that cannot be loaded into your environment? What is the error you see?

[schmist]

@yahavi
When running the audit command, Gradle cannot pull the additional dependency (com.jfrog:gradle-dep-tree:2.2.0), since we build in an environment disconnected from the Internet. According to the 2.36.1 source, there should be the possibility to load that dependency from a configured Artifactory, but the generated Gradle init script (generated by the audit command) always looks like this:

initscript {
    repositories { 
		mavenCentral()
    }
    dependencies {
        classpath 'com.jfrog:gradle-dep-tree:2.2.0'
    }
}

allprojects {
	repositories { 
	}
    apply plugin: com.jfrog.GradleDepTree
}

The local Artifactory is configured in the Jfrog CLI config file.

[omerzi]

Hey @schmist, thank you for informing us of this issue.
When you're saying that your local Artifactory is configured in the JFrog CLI config file, do you mean that you have a gradle.yaml file under the .jfrog/projects folder? Did you configure it through the jf gradlec command?

[schmist]

@omerzi thanks for the hint with the gradle.yaml. Unfortunately the init script is still not completely correct:

initscript {
    repositories { 
		maven {
			url "https://artifactory.cl.ergon.ch/artifactory//artifactory/oss-releases"
			credentials {
				username = '***'
				password = '***'
			}
		}
		mavenCentral()
    }
    dependencies {
        classpath 'com.jfrog:gradle-dep-tree:2.2.0'
    }
}

allprojects {
	repositories { 
		maven {
			url "https://artifactory.cl.ergon.ch/artifactory/proxy-maven-central"
			credentials {
				username = '***'
				password = '***'
			}
		}
	}
    apply plugin: com.jfrog.GradleDepTree
}

There is a problem when constructing the url for the oss-releases repository. According to the code there is a parameter called releases-repo which when set is used to construct the url instead of the constant artifactory/oss-releases. Where can it be configured? Alternatively the artifactory prefix could be removed from the constant.

[omerzi]

Hey @schmist, @janardhanajl
We just released JFrog CLI v2.37.0 and added a new environment variable - "JFROG_CLI_RELEASES_REPO":

Configured Artifactory repository name from which to download the jar needed by the mvn/gradle command.
This environment variable's value format should be <server ID configured by the 'jf c add' command>/<repo name>.
The repository should proxy https://releases.jfrog.io.
This environment variable is used by the 'jf mvn' and 'jf gradle' commands and the 'jf audit' when used for maven or gradle projects.

To use this env, simply provide a server-id/remote-repository-name to the 'JFROG_CLI_RELEASES_REPO' environment variable. Please note that the remote repository must be set to proxy 'https://releases.jfrog.io/'. If you have any questions or encounter any issues, please let us know. We appreciate your feedback and hope this feature will enhance your experience with JFrog CLI.

[Persi]

@omerzi I've set the new Variable JFROG_CLI_RELEASES_REPO but it seems to be ignored by current jf audit command (version 2.37.1). This could probably be the case, because we do not have an internal Artifactory, so instead the variable points to a Nexus repository with anonymous access proxying the official JFrog OSS repo.

The scan is failing with timeouts:

Could not resolve all files for configuration 'classpath'.
> Could not resolve com.jfrog:gradle-dep-tree:2.2.0.
  Required by:
      unspecified:unspecified:unspecified
   > Could not resolve com.jfrog:gradle-dep-tree:2.2.0.
      > Could not get resource 'https://repo.maven.apache.org/maven2/com/jfrog/gradle-dep-tree/2.2.0/gradle-dep-tree-2.2.0.pom'.
         > Could not GET 'https://repo.maven.apache.org/maven2/com/jfrog/gradle-dep-tree/2.2.0/gradle-dep-tree-2.2.0.pom'.
            > Connect to repo.maven.apache.org:443 [repo.maven.apache.org/146.75.116.215] failed: Connect timed out

The only option I see for me would be, to add a configuration option to configure a self written init.gradle to be able to configure the correct Gradle plugin repository. If i do this locally and call the generateDepTrees task it works as expected.

init.gradle:

initscript {
    repositories {
        maven { url "https://internalnexusdomain/raw-proxy-jfrog-artifactory-oss"}
    }
    dependencies {
        classpath 'com.jfrog:gradle-dep-tree:+'
    }
}

allprojects {
    apply plugin: com.jfrog.GradleDepTree
}

Result:

./gradlew -I init.gradle generateDepTrees -Dcom.jfrog.depsTreeOutputFile=gradledeptree.out -Dcom.jfrog.includeAllBuildFiles=true

> Configure project :
Building for Keycloak Release 21.0.2

BUILD SUCCESSFUL in 396ms
1 actionable task: 1 up-to-date

Is there a chance you could add an init script path config parameter to jf gradlec?

[omerzi]

Hi @Persi, thank you for letting us know about your issue. Could you please provide us with the debug logs for the jf audit command? To enable DEBUG logs, please set the JFROG_CLI_LOG_LEVEL environment variable to DEBUG.

Could you also provide us with the value you have set for the JFROG_CLI_RELEASES_REPO environment variable? It would be helpful if you could also provide some more details about your configuration.

[Persi]

Hi @omerzi, thanks for your fast feedback!

I've played a bit to get around our download timeout mentioned above. This is my current setup:

jf cli config:

jf config add nexusnb --artifactory-url https://internalnexusdomain/repository --interactive=false
JFROG_CLI_RELEASES_REPO=nexusnb/raw-proxy-jfrog-artifactory-oss

gradle.yaml for jf cli in the relevant project:

version: 1
type: gradle
resolver:
  repo: nexusnb/raw-proxy-jfrog-artifactory-oss
  serverId: nexusnb
deployer:
  deployMavenDescriptors: true
  deployIvyDescriptors: true
  ivyPattern: '[organization]/[module]/ivy-[revision].xml'
  artifactPattern: '[organization]/[module]/[revision]/[artifact]-[revision](-[classifier]).[ext]'
useWrapper: true

Debug log of the audit call:

jf audit --url $CLOUD_SERVICE_BASE_URL/xray --user $SECURITY_SCAN_USER --password $SECURITY_SCAN_PASSWORD --watches "${XRAY_WATCHES}" --gradle --use-wrapper --exclude-test-deps
08:14:35 [Debug] JFrog CLI version: 2.37.1
08:14:35 [Debug] OS/Arch: linux/amd64
08:14:35 [Debug] Sending HTTP GET request to: https://company.jfrog.io/xray/api/v1/system/version
08:14:36 [Info] JFrog Xray version is: 3.73.8
08:14:36 [Info] Auditing project...
08:14:36 [Debug] Using resolver config from /projectpath/.jfrog/projects/gradle.yaml
08:14:36 [Error] audit failed while building gradle dependency tree:
either username/password or access token must be set for https://internalnexusdomain/repository/

Our Nexus proxy repository is accessible via anonymous but jf cli seems to enforce credentials. If I do not provide a gradle.yaml in the project I ran into the download timeout because of our internet proxy. Which I cannot configure either.

With basic gradle init scripts it works, so for me the easiest way would be to provide my own init.gradle and tell jf cli via gradlec to use it instead of generate a new one on each run.

[omerzi]

Hi @Persi, thank you for providing the detailed information. I have a question regarding the functionality of your init script. Will it work correctly if you configure your repository in the following manner:

initscript {
    repositories {
        maven { 
            url "https://internalnexusdomain/raw-proxy-jfrog-artifactory-oss"
            credentials {
                username = ""
                password = ""
            }
        }
    }
    dependencies {
        classpath 'com.jfrog:gradle-dep-tree:+'
    }
}

allprojects {
    apply plugin: com.jfrog.GradleDepTree
}

Thanks.

[Persi]

Hi @omerzi, your init script looks more or less exactly the same as mine:

initscript {
    repositories {
        maven { 
            url "https://internalnexusdomain/repository/raw-proxy-jfrog-artifactory-oss"
        }
    }
    dependencies {
        classpath 'com.jfrog:gradle-dep-tree:+'
    }
}

allprojects {
    apply plugin: com.jfrog.GradleDepTree
}

and both scripts works as expected.

But if I run jf scan locally or in our ci pipelines, I get this error with the above mentioned gradle.yaml:

jf audit --url https://companydomain/xray --user USERNAME --password PASSWORD --watches "WATCHNAME" --gradle --use-wrapper --exclude-test-deps
09:14:00 [🔵Info] Log path: /PATH/.jfrog/logs/jfrog-cli.2023-05-22.09-14-00.24628.log
09:14:00 [🚨Error] audit failed while building gradle dependency tree:
either username/password or access token must be set for https://internalnexusdomain/repository/