Java: Add tests for autobuilder-detected diagnostics #11943

smowton · 2023-01-20T17:47:59Z

No description provided.

tamasvajk · 2023-01-25T07:57:59Z

java/ql/integration-tests/all-platforms/java/diagnostics/no-build-system/test.py

@@ -0,0 +1,8 @@
+import os
+from create_database_utils import *
+from diagnostics_test_utils import *


The CI jobs are failing with ModuleNotFoundError: No module named 'diagnostics_test_utils'. Did you forget to push the util file?

This depends on an internal PR

atorralba

Test cases mostly LGTM. Added some recommendations regarding markdown links — see internal PR as well.

atorralba · 2023-02-22T10:45:35Z

...ion-tests/all-platforms/java/diagnostics/android-gradle-incompatibility/diagnostics.expected

@@ -0,0 +1,28 @@
+{
+  "markdownMessage": "An Android build may have failed. Ensure the Code Scanning workflow installs required dependencies, and that the [https://developer.android.com/studio/releases/gradle-plugin#updating-gradle](Gradle and Android SDK versions are compatible). Suspicious output line: `   > Minimum supported Gradle version is 7.4. Current version is 7.3. If using the gradle wrapper, try editing the distributionUrl in <test-root-directory>/gradle/wrapper/gradle-wrapper.properties to gradle-7.4-all.zip`",


Suggested change

"markdownMessage": "An Android build may have failed. Ensure the Code Scanning workflow installs required dependencies, and that the [https://developer.android.com/studio/releases/gradle-plugin#updating-gradle](Gradle and Android SDK versions are compatible). Suspicious output line: ` > Minimum supported Gradle version is 7.4. Current version is 7.3. If using the gradle wrapper, try editing the distributionUrl in <test-root-directory>/gradle/wrapper/gradle-wrapper.properties to gradle-7.4-all.zip`",

"markdownMessage": "An Android build may have failed. Ensure the Code Scanning workflow installs required dependencies, and that the [Gradle and Android SDK versions are compatible](https://developer.android.com/studio/releases/gradle-plugin#updating-gradle). Suspicious output line: ` > Minimum supported Gradle version is 7.4. Current version is 7.3. If using the gradle wrapper, try editing the distributionUrl in <test-root-directory>/gradle/wrapper/gradle-wrapper.properties to gradle-7.4-all.zip`",

atorralba · 2023-02-22T10:45:51Z

...ion-tests/all-platforms/java/diagnostics/android-gradle-incompatibility/diagnostics.expected

+  }
+}
+{
+  "markdownMessage": "An Android build may have failed. Ensure the Code Scanning workflow installs required dependencies, and that the [https://developer.android.com/studio/releases/gradle-plugin#updating-gradle](Gradle and Android SDK versions are compatible). Suspicious output line: `Caused by: java.lang.RuntimeException: Minimum supported Gradle version is 7.4. Current version is 7.3. If using the gradle wrapper, try editing the distributionUrl in <test-root-directory>/gradle/wrapper/gradle-wrapper.properties to gradle-7.4-all.zip`",


Suggested change

"markdownMessage": "An Android build may have failed. Ensure the Code Scanning workflow installs required dependencies, and that the [https://developer.android.com/studio/releases/gradle-plugin#updating-gradle](Gradle and Android SDK versions are compatible). Suspicious output line: `Caused by: java.lang.RuntimeException: Minimum supported Gradle version is 7.4. Current version is 7.3. If using the gradle wrapper, try editing the distributionUrl in <test-root-directory>/gradle/wrapper/gradle-wrapper.properties to gradle-7.4-all.zip`",

"markdownMessage": "An Android build may have failed. Ensure the Code Scanning workflow installs required dependencies, and that the [Gradle and Android SDK versions are compatible](https://developer.android.com/studio/releases/gradle-plugin#updating-gradle). Suspicious output line: `Caused by: java.lang.RuntimeException: Minimum supported Gradle version is 7.4. Current version is 7.3. If using the gradle wrapper, try editing the distributionUrl in <test-root-directory>/gradle/wrapper/gradle-wrapper.properties to gradle-7.4-all.zip`",

atorralba · 2023-02-22T10:48:13Z

java/ql/integration-tests/all-platforms/java/diagnostics/dependency-error/diagnostics.expected

@@ -0,0 +1,14 @@
+{
+  "markdownMessage": "A dependency failed to download. Check that all dependencies are available, and [https://github.com/Azure/actions-workflow-samples/blob/master/assets/create-secrets-for-GitHub-workflows.md#set-up-secrets-in-github-action-workflows](supply credentials for any private dependencies). Suspicious output line: `Caused by: org.eclipse.aether.transfer.ArtifactNotFoundException: Could not find artifact junit:junit-nonesuch:jar:4.11 in central (https://repo.maven.apache.org/maven2)`",


Suggested change

"markdownMessage": "A dependency failed to download. Check that all dependencies are available, and [https://github.com/Azure/actions-workflow-samples/blob/master/assets/create-secrets-for-GitHub-workflows.md#set-up-secrets-in-github-action-workflows](supply credentials for any private dependencies). Suspicious output line: `Caused by: org.eclipse.aether.transfer.ArtifactNotFoundException: Could not find artifact junit:junit-nonesuch:jar:4.11 in central (https://repo.maven.apache.org/maven2)`",

"markdownMessage": "A dependency failed to download. Check that all dependencies are available, and [supply credentials for any private dependencies](https://github.com/Azure/actions-workflow-samples/blob/master/assets/create-secrets-for-GitHub-workflows.md#set-up-secrets-in-github-action-workflows). Suspicious output line: `Caused by: org.eclipse.aether.transfer.ArtifactNotFoundException: Could not find artifact junit:junit-nonesuch:jar:4.11 in central (https://repo.maven.apache.org/maven2)`",

atorralba · 2023-02-22T10:48:38Z

...l/integration-tests/all-platforms/java/diagnostics/java-version-too-old/diagnostics.expected

@@ -0,0 +1,28 @@
+{
+  "markdownMessage": "Your project may need a different JDK version. Ensure your Code Scanning workflow file has [https://github.com/actions/setup-java#eclipse-temurin](an appropriate `setup-java` step). Suspicious output line: `> Could not target platform: 'Java SE 11' using tool chain: 'JDK 8 (1.8)'.`",


Suggested change

"markdownMessage": "Your project may need a different JDK version. Ensure your Code Scanning workflow file has [https://github.com/actions/setup-java#eclipse-temurin](an appropriate `setup-java` step). Suspicious output line: `> Could not target platform: 'Java SE 11' using tool chain: 'JDK 8 (1.8)'.`",

"markdownMessage": "Your project may need a different JDK version. Ensure your Code Scanning workflow file has [an appropriate `setup-java` step](https://github.com/actions/setup-java#eclipse-temurin). Suspicious output line: `> Could not target platform: 'Java SE 11' using tool chain: 'JDK 8 (1.8)'.`",

atorralba · 2023-02-22T10:48:59Z

...l/integration-tests/all-platforms/java/diagnostics/java-version-too-old/diagnostics.expected

+  }
+}
+{
+  "markdownMessage": "Your project may need a different JDK version. Ensure your Code Scanning workflow file has [https://github.com/actions/setup-java#eclipse-temurin](an appropriate `setup-java` step). Suspicious output line: `Caused by: java.lang.IllegalArgumentException: Could not target platform: 'Java SE 11' using tool chain: 'JDK 8 (1.8)'.`",


Suggested change

"markdownMessage": "Your project may need a different JDK version. Ensure your Code Scanning workflow file has [https://github.com/actions/setup-java#eclipse-temurin](an appropriate `setup-java` step). Suspicious output line: `Caused by: java.lang.IllegalArgumentException: Could not target platform: 'Java SE 11' using tool chain: 'JDK 8 (1.8)'.`",

"markdownMessage": "Your project may need a different JDK version. Ensure your Code Scanning workflow file has [an appropriate `setup-java` step](https://github.com/actions/setup-java#eclipse-temurin). Suspicious output line: `Caused by: java.lang.IllegalArgumentException: Could not target platform: 'Java SE 11' using tool chain: 'JDK 8 (1.8)'.`",

atorralba · 2023-02-22T10:50:46Z

...egration-tests/all-platforms/java/diagnostics/multiple-candidate-builds/diagnostics.expected

+  }
+}
+{
+  "markdownMessage": "If you want to use one of the candidate build systems and directories (see previous warnings), please [https://docs.github.com/en/github-ae@latest/code-security/code-scanning/automatically-scanning-your-code-for-vulnerabilities-and-errors/configuring-the-codeql-workflow-for-compiled-languages#adding-build-steps-for-a-compiled-language](supply a manual a build command)",


Suggested change

"markdownMessage": "If you want to use one of the candidate build systems and directories (see previous warnings), please [https://docs.github.com/en/github-ae@latest/code-security/code-scanning/automatically-scanning-your-code-for-vulnerabilities-and-errors/configuring-the-codeql-workflow-for-compiled-languages#adding-build-steps-for-a-compiled-language](supply a manual a build command)",

"markdownMessage": "If you want to use one of the candidate build systems and directories (see previous warnings), please [supply a manual a build command](https://docs.github.com/en/github-ae@latest/code-security/code-scanning/automatically-scanning-your-code-for-vulnerabilities-and-errors/configuring-the-codeql-workflow-for-compiled-languages#adding-build-steps-for-a-compiled-language)",

atorralba · 2023-02-22T10:51:01Z

java/ql/integration-tests/all-platforms/java/diagnostics/no-build-system/diagnostics.expected

@@ -0,0 +1,14 @@
+{
+  "markdownMessage": "Could not find a Gradle, Maven or Ant top-level project to build. Please [https://docs.github.com/en/github-ae@latest/code-security/code-scanning/automatically-scanning-your-code-for-vulnerabilities-and-errors/configuring-the-codeql-workflow-for-compiled-languages#adding-build-steps-for-a-compiled-language](supply a manual build command)",


Suggested change

"markdownMessage": "Could not find a Gradle, Maven or Ant top-level project to build. Please [https://docs.github.com/en/github-ae@latest/code-security/code-scanning/automatically-scanning-your-code-for-vulnerabilities-and-errors/configuring-the-codeql-workflow-for-compiled-languages#adding-build-steps-for-a-compiled-language](supply a manual build command)",

"markdownMessage": "Could not find a Gradle, Maven or Ant top-level project to build. Please [supply a manual build command](https://docs.github.com/en/github-ae@latest/code-security/code-scanning/automatically-scanning-your-code-for-vulnerabilities-and-errors/configuring-the-codeql-workflow-for-compiled-languages#adding-build-steps-for-a-compiled-language)",

atorralba · 2023-02-22T10:51:20Z

...integration-tests/all-platforms/java/diagnostics/no-gradle-test-classes/diagnostics.expected

@@ -0,0 +1,28 @@
+{
+  "markdownMessage": "Built a Gradle project without the [https://docs.gradle.org/current/userguide/gradle_wrapper.html](Gradle wrapper). This may use an incompatible version of Gradle",


Suggested change

"markdownMessage": "Built a Gradle project without the [https://docs.gradle.org/current/userguide/gradle_wrapper.html](Gradle wrapper). This may use an incompatible version of Gradle",

"markdownMessage": "Built a Gradle project without the [Gradle wrapper](https://docs.gradle.org/current/userguide/gradle_wrapper.html). This may use an incompatible version of Gradle",

atorralba · 2023-02-22T10:51:34Z

...integration-tests/all-platforms/java/diagnostics/no-gradle-test-classes/diagnostics.expected

+  }
+}
+{
+  "markdownMessage": "Gradle project does not define a `testClasses` goal. [https://docs.github.com/en/github-ae@latest/code-security/code-scanning/automatically-scanning-your-code-for-vulnerabilities-and-errors/configuring-the-codeql-workflow-for-compiled-languages#adding-build-steps-for-a-compiled-language](Supply a manual build command) that builds the code that should be analyzed. Suspicious output line: `org.gradle.execution.TaskSelectionException: Task 'testClasses' not found in root project 'no-gradle-test-classes'.`",


Suggested change

"markdownMessage": "Gradle project does not define a `testClasses` goal. [https://docs.github.com/en/github-ae@latest/code-security/code-scanning/automatically-scanning-your-code-for-vulnerabilities-and-errors/configuring-the-codeql-workflow-for-compiled-languages#adding-build-steps-for-a-compiled-language](Supply a manual build command) that builds the code that should be analyzed. Suspicious output line: `org.gradle.execution.TaskSelectionException: Task 'testClasses' not found in root project 'no-gradle-test-classes'.`",

"markdownMessage": "Gradle project does not define a `testClasses` goal. [Supply a manual build command](https://docs.github.com/en/github-ae@latest/code-security/code-scanning/automatically-scanning-your-code-for-vulnerabilities-and-errors/configuring-the-codeql-workflow-for-compiled-languages#adding-build-steps-for-a-compiled-language) that builds the code that should be analyzed. Suspicious output line: `org.gradle.execution.TaskSelectionException: Task 'testClasses' not found in root project 'no-gradle-test-classes'.`",

atorralba · 2023-02-22T10:52:17Z

java/ql/integration-tests/all-platforms/java/diagnostics/no-gradle-wrapper/diagnostics.expected

@@ -0,0 +1,14 @@
+{
+  "markdownMessage": "Built a Gradle project without the [https://docs.gradle.org/current/userguide/gradle_wrapper.html](Gradle wrapper). This may use an incompatible version of Gradle",


Suggested change

"markdownMessage": "Built a Gradle project without the [https://docs.gradle.org/current/userguide/gradle_wrapper.html](Gradle wrapper). This may use an incompatible version of Gradle",

"markdownMessage": "Built a Gradle project without the [Gradle wrapper](https://docs.gradle.org/current/userguide/gradle_wrapper.html). This may use an incompatible version of Gradle",

… format

The CLI does this now. Also omit default arguments to the integration test script.

smowton requested a review from a team as a code owner January 20, 2023 17:47

github-actions bot added the Java label Jan 20, 2023

smowton force-pushed the smowton/admin/java-autobuilder-diagnostic-tests branch 2 times, most recently from 1dc7c87 to 739e588 Compare January 24, 2023 17:07

tamasvajk reviewed Jan 25, 2023

View reviewed changes

smowton changed the title ~~Java: Add test for missing build system~~ Java: Add tests for autobuilder-detected diagnostics Jan 30, 2023

smowton force-pushed the smowton/admin/java-autobuilder-diagnostic-tests branch from 88fafe7 to 203adb1 Compare January 30, 2023 12:24

smowton force-pushed the smowton/admin/java-autobuilder-diagnostic-tests branch 2 times, most recently from 7263139 to 62b353a Compare February 7, 2023 18:21

smowton force-pushed the smowton/admin/java-autobuilder-diagnostic-tests branch 5 times, most recently from 50ba154 to 2c55231 Compare February 21, 2023 15:46

atorralba reviewed Feb 22, 2023

View reviewed changes

atorralba previously approved these changes Feb 22, 2023

View reviewed changes

smowton added 9 commits February 22, 2023 17:43

Add test for missing Java build system

40e45d3

Java no-build-system test: switch to single-file diagnostics.expected…

7a2c6b5

… format

Switch to using language-specific diagnostic environment variable

fd1ce5d

Add tests for Java autobuilder errors

57ac951

Replace test absolute directory

53ac1e0

Diagnostics tests: don't create own diagnostics directory

2c21679

The CLI does this now. Also omit default arguments to the integration test script.

Update test expectations

73e4480

Fix diagnostic markdown links

c4640a6

Fix multiple-candidate-builds message

560a341

smowton dismissed atorralba’s stale review via 560a341 February 22, 2023 17:43

smowton force-pushed the smowton/admin/java-autobuilder-diagnostic-tests branch from a54d086 to 560a341 Compare February 22, 2023 17:43

atorralba approved these changes Feb 23, 2023

View reviewed changes

smowton merged commit 748387a into github:main Feb 23, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Java: Add tests for autobuilder-detected diagnostics #11943

Java: Add tests for autobuilder-detected diagnostics #11943

Uh oh!

smowton commented Jan 20, 2023

Uh oh!

tamasvajk Jan 25, 2023

Uh oh!

smowton Jan 25, 2023

Uh oh!

atorralba left a comment

Uh oh!

atorralba Feb 22, 2023

Uh oh!

atorralba Feb 22, 2023

Uh oh!

atorralba Feb 22, 2023

Uh oh!

atorralba Feb 22, 2023

Uh oh!

atorralba Feb 22, 2023

Uh oh!

atorralba Feb 22, 2023

Uh oh!

atorralba Feb 22, 2023

Uh oh!

atorralba Feb 22, 2023

Uh oh!

atorralba Feb 22, 2023

Uh oh!

atorralba Feb 22, 2023

Uh oh!

Uh oh!

		@@ -0,0 +1,28 @@
		{
		"markdownMessage": "An Android build may have failed. Ensure the Code Scanning workflow installs required dependencies, and that the [https://developer.android.com/studio/releases/gradle-plugin#updating-gradle](Gradle and Android SDK versions are compatible). Suspicious output line: ` > Minimum supported Gradle version is 7.4. Current version is 7.3. If using the gradle wrapper, try editing the distributionUrl in <test-root-directory>/gradle/wrapper/gradle-wrapper.properties to gradle-7.4-all.zip`",

		@@ -0,0 +1,14 @@
		{
		"markdownMessage": "A dependency failed to download. Check that all dependencies are available, and [https://github.com/Azure/actions-workflow-samples/blob/master/assets/create-secrets-for-GitHub-workflows.md#set-up-secrets-in-github-action-workflows](supply credentials for any private dependencies). Suspicious output line: `Caused by: org.eclipse.aether.transfer.ArtifactNotFoundException: Could not find artifact junit:junit-nonesuch:jar:4.11 in central (https://repo.maven.apache.org/maven2)`",

		@@ -0,0 +1,28 @@
		{
		"markdownMessage": "Your project may need a different JDK version. Ensure your Code Scanning workflow file has [https://github.com/actions/setup-java#eclipse-temurin](an appropriate `setup-java` step). Suspicious output line: `> Could not target platform: 'Java SE 11' using tool chain: 'JDK 8 (1.8)'.`",

	"markdownMessage": "If you want to use one of the candidate build systems and directories (see previous warnings), please [https://docs.github.com/en/github-ae@latest/code-security/code-scanning/automatically-scanning-your-code-for-vulnerabilities-and-errors/configuring-the-codeql-workflow-for-compiled-languages#adding-build-steps-for-a-compiled-language](supply a manual a build command)",
	"markdownMessage": "If you want to use one of the candidate build systems and directories (see previous warnings), please [supply a manual a build command](https://docs.github.com/en/github-ae@latest/code-security/code-scanning/automatically-scanning-your-code-for-vulnerabilities-and-errors/configuring-the-codeql-workflow-for-compiled-languages#adding-build-steps-for-a-compiled-language)",

		@@ -0,0 +1,14 @@
		{
		"markdownMessage": "Could not find a Gradle, Maven or Ant top-level project to build. Please [https://docs.github.com/en/github-ae@latest/code-security/code-scanning/automatically-scanning-your-code-for-vulnerabilities-and-errors/configuring-the-codeql-workflow-for-compiled-languages#adding-build-steps-for-a-compiled-language](supply a manual build command)",

		@@ -0,0 +1,28 @@
		{
		"markdownMessage": "Built a Gradle project without the [https://docs.gradle.org/current/userguide/gradle_wrapper.html](Gradle wrapper). This may use an incompatible version of Gradle",

Java: Add tests for autobuilder-detected diagnostics #11943

Java: Add tests for autobuilder-detected diagnostics #11943

Uh oh!

Conversation

smowton commented Jan 20, 2023

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

atorralba left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!