Allow lang-painless maven package to add compile dependencies #11356

prudhvigodithi · 2023-11-28T00:37:59Z

Description

Allow lang-painless maven package to add compile dependencies

Tested with `publishToMavenLocal`

../../gradlew :modules:lang-painless:publishToMavenLocal

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
  <!-- This module was also published with a richer model, Gradle metadata,  -->
  <!-- which should be used instead. Do not delete the following line which  -->
  <!-- is to indicate to Gradle or any Gradle module metadata file consumer  -->
  <!-- that they should prefer consuming it instead. -->
  <!-- do_not_remove: published-with-gradle-metadata -->
  <modelVersion>4.0.0</modelVersion>
  <groupId>org.opensearch.plugin</groupId>
  <artifactId>lang-painless</artifactId>
  <version>3.0.0-SNAPSHOT</version>
  <name>lang-painless</name>
  <description>OpenSearch subproject :modules:lang-painless</description>
  <dependencies>
    <dependency>
      <groupId>org.antlr</groupId>
      <artifactId>antlr4-runtime</artifactId>
      <version>4.11.1</version>
      <scope>compile</scope>
      <exclusions>
        <exclusion>
          <groupId>*</groupId>
          <artifactId>*</artifactId>
        </exclusion>
      </exclusions>
    </dependency>
    <dependency>
      <groupId>org.ow2.asm</groupId>
      <artifactId>asm-util</artifactId>
      <version>9.6</version>
      <scope>compile</scope>
      <exclusions>
        <exclusion>
          <groupId>*</groupId>
          <artifactId>*</artifactId>
        </exclusion>
      </exclusions>
    </dependency>
    <dependency>
      <groupId>org.ow2.asm</groupId>
      <artifactId>asm-tree</artifactId>
      <version>9.6</version>
      <scope>compile</scope>
      <exclusions>
        <exclusion>
          <groupId>*</groupId>
          <artifactId>*</artifactId>
        </exclusion>
      </exclusions>
    </dependency>
    <dependency>
      <groupId>org.ow2.asm</groupId>
      <artifactId>asm-commons</artifactId>
      <version>9.6</version>
      <scope>compile</scope>
      <exclusions>
        <exclusion>
          <groupId>*</groupId>
          <artifactId>*</artifactId>
        </exclusion>
      </exclusions>
    </dependency>
    <dependency>
      <groupId>org.ow2.asm</groupId>
      <artifactId>asm-analysis</artifactId>
      <version>9.6</version>
      <scope>compile</scope>
      <exclusions>
        <exclusion>
          <groupId>*</groupId>
          <artifactId>*</artifactId>
        </exclusion>
      </exclusions>
    </dependency>
    <dependency>
      <groupId>org.ow2.asm</groupId>
      <artifactId>asm</artifactId>
      <version>9.6</version>
      <scope>compile</scope>
      <exclusions>
        <exclusion>
          <groupId>*</groupId>
          <artifactId>*</artifactId>
        </exclusion>
      </exclusions>
    </dependency>
    <dependency>
      <groupId>org.opensearch.plugin</groupId>
      <artifactId>opensearch-scripting-painless-spi</artifactId>
      <version>3.0.0-SNAPSHOT</version>
      <scope>compile</scope>
    </dependency>
  </dependencies>
  <inceptionYear>2021</inceptionYear>
  <licenses>
    <license>
      <name>The Apache Software License, Version 2.0</name>
      <url>http://www.apache.org/licenses/LICENSE-2.0.txt</url>
      <distribution>repo</distribution>
    </license>
  </licenses>
  <developers>
    <developer>
      <name>OpenSearch</name>
      <url>https://github.com/opensearch-project/OpenSearch</url>
    </developer>
  </developers>
  <url>https://github.com/prudhvigodithi/OpenSearch</url>
  <scm>
    <url>git@github.com:prudhvigodithi/OpenSearch.git</url>
  </scm>
</project>

Related Issues

Resolves #11353
(Part of ~opensearch-project/opensearch-build#4161)

Check List

~~New functionality includes testing.~~
~~All tests pass~~
~~New functionality has been documented.~~
~~New functionality has javadoc added~~
~~Commit changes are listed out in CHANGELOG.md file (See: Changelog)~~

Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Prudhvi Godithi <pgodithi@amazon.com>

github-actions · 2023-11-28T00:43:37Z

❌ Gradle check result for d82b614: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Signed-off-by: Prudhvi Godithi <pgodithi@amazon.com>

github-actions · 2023-11-28T01:01:34Z

Compatibility status:

Checks if related components are compatible with change b4fa136

Incompatible components

Skipped components

Compatible components

Compatible components: [https://github.com/opensearch-project/security-analytics.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/reporting.git, https://github.com/opensearch-project/opensearch-oci-object-storage.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/custom-codecs.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/cross-cluster-replication.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/performance-analyzer.git, https://github.com/opensearch-project/performance-analyzer-rca.git, https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/sql.git]

github-actions · 2023-11-28T01:33:10Z

❌ Gradle check result for e8d4d4a: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions · 2023-11-28T01:43:01Z

❌ Gradle check result for b4fa136: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

reta · 2023-11-28T15:04:10Z

modules/lang-painless/build.gradle

@@ -82,9 +82,16 @@ shadowJar {
  }
 }

-tasks.validateNebulaPom.dependsOn tasks.generatePomFileForShadowPublication
+// Allowing the task generatePomFileForNebulaPublication to generate the POM file with added compile dependencies.
+gradle.taskGraph.whenReady {


@prudhvigodithi I don't think we are addressing the root of the problem here, there are multiple publications that conflict (apparently, for no reason):

> Task :modules:lang-painless:publishShadowPublicationToMavenLocal Multiple publications with coordinates 'org.opensearch.plugin:lang-painless:3.0.0-SNAPSHOT' are published to repository 'mavenLocal'. The publications 'nebula' in project ':modules:lang-painless' and 'shadow' in project ':modules:lang-painless' will overwrite each other!

I am looking why that happens, mind if I push some changes to your pull request?

Sure @reta please do, I have just focused on the POM generation, which has the problem with the task generatePomFileForShadowPublication coming from the shadow plugin.

I assume there is a some issue with shadow plugin with gradle version 8.4, as notice the POM file generated (via generatePomFileForShadowPublication) also is not in right structure where it has the XML ending value <dependencies/>, but not the starting.

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd"> <modelVersion>4.0.0</modelVersion> <groupId>org.opensearch.plugin</groupId> <artifactId>lang-painless</artifactId> <version>2.10.0</version> <name>lang-painless</name> <description>An easy, safe and fast scripting language for OpenSearch</description> <inceptionYear>2021</inceptionYear> <licenses> <license> <name>The Apache Software License, Version 2.0</name> <url>http://www.apache.org/licenses/LICENSE-2.0.txt</url> <distribution>repo</distribution> </license> </licenses> <developers> <developer> <name>OpenSearch</name> <url>https://github.com/opensearch-project/OpenSearch</url> </developer> </developers> <url>https://github.com/opensearch-project/OpenSearch.git</url> <scm> <url>https://github.com/opensearch-project/OpenSearch.git</url> </scm> <dependencies/> </project>

Since generatePomFileForNebulaPublication already does the POM generation, I have disabled the generatePomFileForShadowPublication.

Also @reta I see task publishShadowPublicationToMavenLocal works on my local.

../../gradlew :modules:lang-painless:publishShadowPublicationToMavenLocal Picked up JAVA_TOOL_OPTIONS: -Dlog4j2.formatMsgNoLookups=true > Configure project : ========================= WARNING ========================= Backwards compatibility tests are disabled! See https://github.com/opensearch-project/OpenSearch/issues/4173 =========================================================== ======================================= OpenSearch Build Hamster says Hello! Gradle Version : 8.4 OS Info : Mac OS X 13.6 (aarch64) JDK Version : 11 (Homebrew JDK) JAVA_HOME : /opt/homebrew/Cellar/openjdk@11/11.0.16.1/libexec/openjdk.jdk/Contents/Home Random Testing Seed : AD775C78920E6AF4 In FIPS 140 mode : false ======================================= BUILD SUCCESSFUL in 5s 54 actionable tasks: 6 executed, 3 from cache, 45 up-to-date

@prudhvigodithi the issue is that Shadow publication does not generate the correct POM file anymore (the dependencies) and Nebula does not generate the POM we need either (dependencies are but not other elements), I think I found the place where is is happening.

prudhvigodithi · 2023-11-29T17:57:48Z

Closing this PR as @reta opened a new PR #11369 to completely remove the shadowjar for lang-painless. Thanks

github-actions bot added the bug Something isn't working label Nov 28, 2023

lang-painless to add maven compile dependencies

e8d4d4a

Signed-off-by: Prudhvi Godithi <pgodithi@amazon.com>

prudhvigodithi force-pushed the main branch from d82b614 to e8d4d4a Compare November 28, 2023 00:42

Empty Commmit

b4fa136

Signed-off-by: Prudhvi Godithi <pgodithi@amazon.com>

peterzhuamazon approved these changes Nov 28, 2023

View reviewed changes

reta reviewed Nov 28, 2023

View reviewed changes

reta mentioned this pull request Nov 28, 2023

Fix Shadow JAR dependency publication #11369

Merged

8 tasks

prudhvigodithi closed this Nov 29, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Allow lang-painless maven package to add compile dependencies #11356

Allow lang-painless maven package to add compile dependencies #11356

prudhvigodithi commented Nov 28, 2023 •

edited

Loading

github-actions bot commented Nov 28, 2023

github-actions bot commented Nov 28, 2023 •

edited

Loading

github-actions bot commented Nov 28, 2023

github-actions bot commented Nov 28, 2023

reta Nov 28, 2023

prudhvigodithi Nov 28, 2023

prudhvigodithi Nov 28, 2023 •

edited

Loading

prudhvigodithi Nov 28, 2023

reta Nov 28, 2023 •

edited

Loading

prudhvigodithi commented Nov 29, 2023

Allow lang-painless maven package to add compile dependencies #11356

Allow lang-painless maven package to add compile dependencies #11356

Conversation

prudhvigodithi commented Nov 28, 2023 • edited Loading

Description

Tested with publishToMavenLocal

Related Issues

Check List

github-actions bot commented Nov 28, 2023

github-actions bot commented Nov 28, 2023 • edited Loading

Compatibility status:

Incompatible components

Skipped components

Compatible components

github-actions bot commented Nov 28, 2023

github-actions bot commented Nov 28, 2023

reta Nov 28, 2023

Choose a reason for hiding this comment

prudhvigodithi Nov 28, 2023

Choose a reason for hiding this comment

prudhvigodithi Nov 28, 2023 • edited Loading

Choose a reason for hiding this comment

prudhvigodithi Nov 28, 2023

Choose a reason for hiding this comment

reta Nov 28, 2023 • edited Loading

Choose a reason for hiding this comment

prudhvigodithi commented Nov 29, 2023

prudhvigodithi commented Nov 28, 2023 •

edited

Loading

Tested with `publishToMavenLocal`

github-actions bot commented Nov 28, 2023 •

edited

Loading

prudhvigodithi Nov 28, 2023 •

edited

Loading

reta Nov 28, 2023 •

edited

Loading