[SPARK-42382][BUILD] Upgrade `cyclonedx-maven-plugin` to 2.7.5 #40065

LuciferYang · 2023-02-17T15:15:38Z

What changes were proposed in this pull request?

This pr aims upgrade cyclonedx-maven-plugin from 2.7.3 to 2.7.5

Why are the changes needed?

The release notes as follows:

On the other hand, we can upgrade to use maven 3.9.0 to build Spark after upgrading cyclonedx-maven-plugin to 2.7.5, otherwise, the build error described in SPARK-42380 will occur.

Does this PR introduce any user-facing change?

No

How was this patch tested?

Pass GitHub Actions.
Manual check the cyclonedx.xml file can be generated normally.

LuciferYang · 2023-02-17T15:45:57Z

cc @dongjoon-hyun

dongjoon-hyun

Is there a way to see the reported error in Spark GitHub Action jobs, @LuciferYang ?

LuciferYang · 2023-02-17T17:12:54Z

I make another one build with maven 3.8.7 + cyclonedx-maven-plugin 2.7.4 https://github.com/LuciferYang/spark/actions/runs/4205904014/jobs/7298678641

dongjoon-hyun · 2023-02-17T17:38:34Z

I mean in our GitHub Action repo. We are using CycloneDX 2.7.3, aren't we?

I make another one build with maven 3.8.7 + cyclonedx-maven-plugin 2.7.4 https://github.com/LuciferYang/spark/actions/runs/4205904014/jobs/7298678641

LuciferYang · 2023-02-17T18:02:57Z

Yes, we use CycloneDX 2.7.3. So I should not explain that 2.7.4 has such issue in the pr description, because it does not affect Spark now, am I right?

LuciferYang · 2023-02-17T18:06:22Z

Please let me explain my intention more:

First of all, I want to update maven to 3.9.0(keep use CycloneDX 2.7.3), then I found the following error:

[ERROR] An error occurred attempting to read POM
org.codehaus.plexus.util.xml.pull.XmlPullParserException: UTF-8 BOM plus xml decl of ISO-8859-1 is incompatible (position: START_DOCUMENT seen <?xml version="1.0" encoding="ISO-8859-1"... @1:42) 
    at org.codehaus.plexus.util.xml.pull.MXParser.parseXmlDeclWithVersion (MXParser.java:3423)
    at org.codehaus.plexus.util.xml.pull.MXParser.parseXmlDecl (MXParser.java:3345)
    at org.codehaus.plexus.util.xml.pull.MXParser.parsePI (MXParser.java:3197)
    at org.codehaus.plexus.util.xml.pull.MXParser.parseProlog (MXParser.java:1828)
    at org.codehaus.plexus.util.xml.pull.MXParser.nextImpl (MXParser.java:1757)
    at org.codehaus.plexus.util.xml.pull.MXParser.next (MXParser.java:1375)
    at org.apache.maven.model.io.xpp3.MavenXpp3Reader.read (MavenXpp3Reader.java:3940)
    at org.apache.maven.model.io.xpp3.MavenXpp3Reader.read (MavenXpp3Reader.java:612)
    at org.apache.maven.model.io.xpp3.MavenXpp3Reader.read (MavenXpp3Reader.java:627)
    at org.cyclonedx.maven.BaseCycloneDxMojo.readPom (BaseCycloneDxMojo.java:759)
    at org.cyclonedx.maven.BaseCycloneDxMojo.readPom (BaseCycloneDxMojo.java:746)
    at org.cyclonedx.maven.BaseCycloneDxMojo.retrieveParentProject (BaseCycloneDxMojo.java:694)
    at org.cyclonedx.maven.BaseCycloneDxMojo.getClosestMetadata (BaseCycloneDxMojo.java:524)
    at org.cyclonedx.maven.BaseCycloneDxMojo.convert (BaseCycloneDxMojo.java:481)
    at org.cyclonedx.maven.CycloneDxMojo.execute (CycloneDxMojo.java:70)
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:126)
    at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute2 (MojoExecutor.java:342)
    at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute (MojoExecutor.java:330)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:213)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:175)
    at org.apache.maven.lifecycle.internal.MojoExecutor.access$000 (MojoExecutor.java:76)
    at org.apache.maven.lifecycle.internal.MojoExecutor$1.run (MojoExecutor.java:163)
    at org.apache.maven.plugin.DefaultMojosExecutionStrategy.execute (DefaultMojosExecutionStrategy.java:39)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:160)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:105)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:73)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:53)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:118)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:260)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:172)
    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:100)
    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:821)
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:270)
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:192)
    at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:498)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:282)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:225)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:406)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:347)

I think We should see similar errors here: https://github.com/LuciferYang/spark/actions/runs/4206035140/jobs/7299042843 later

then I want to test maven 3.9.0 + CycloneDX 2.7.4 couple of days ago, but there an error same as maven 3.8.7 + cyclonedx-maven-plugin 2.7.4, I think we should see it here: https://github.com/LuciferYang/spark/runs/11424487074 later
then I test maven 3.9.0 + CycloneDX 2.7.5 today, there is no above issues(we can check https://github.com/LuciferYang/spark/runs/11424568023 later).

So If I want to upgrade Spark to use maven 3.9.0, I must upgrade cyclonedx-maven-plugin to 2.7.5, I should upgrade them in one or two pr?

dongjoon-hyun · 2023-02-17T19:10:15Z

I'm trying to assess the issue. So, those combination issue is not the AS-IS Apache Spark issue in both master/branch-3.4, right?

FYI, Cyclone plugin 2.7.4 issue is a known one. When I started SBOM works, 2.7.4 was the lastest but was unusable across multiple ASF projects. That was the main reason I chose 2.7.3 instead of the latest at that time. I'm not quite sure if 2.7.5 is stable enough.

Anyway, we can apply this PR on master branch for Apache Spark 3.5.0 only separately from the Maven issue. Maven is also another big issues always.

LuciferYang · 2023-02-17T19:22:15Z

Yeah, Spark 3.4.0 does not need this pr.

dongjoon-hyun · 2023-02-17T21:19:02Z

If you don't mind, please allow me one or two days. I'll check this during weekend~ Thank you for your patience always.

LuciferYang · 2023-02-18T04:01:54Z

@dongjoon-hyun found a new issue related to 2.7.5: CycloneDX/cyclonedx-maven-plugin#284

dongjoon-hyun · 2023-02-18T04:43:54Z

Got it. Thank you for informing.

LuciferYang · 2023-02-18T08:25:33Z

I think we should wait for 2.7.6 or higher to test usability, then we can reuse this jira. I will close this pr first, thanks @dongjoon-hyun

dongjoon-hyun · 2023-02-18T22:43:33Z

+1 for your decision, @LuciferYang . Thank you for letting me know before I started my work~ :)

steveloughran · 2023-02-23T11:08:35Z

I'm hitting this when trying to build hadoop having updated maven via homebrew so as to get spark to work. joy.

dongjoon-hyun · 2023-04-10T19:28:04Z

This PR is superseded by #40726 .

This PR aims to upgrade `cyclonedx-maven-plugin` from `2.7.3` to `2.7.6` to bring the latest bug fixes. - https://github.com/CycloneDX/cyclonedx-maven-plugin/releases/tag/cyclonedx-maven-plugin-2.7.6 - https://github.com/CycloneDX/cyclonedx-maven-plugin/releases/tag/cyclonedx-maven-plugin-2.7.5 - https://github.com/CycloneDX/cyclonedx-maven-plugin/releases/tag/cyclonedx-maven-plugin-2.7.4 Historically, there was some issue reports on the previous versions with the latest Maven. - apache/spark#40065 - apache/arrow#35086 Now, 2.7.6 is verified in Apache Spark, ORC, Arrow community as of today. - [ORC-1407: Upgrade cyclonedx-maven-plugin to 2.7.6](apache/orc#1463) - [SPARK-42382: Upgrade cyclonedx-maven-plugin to 2.7.6](apache/spark#40726) - [GH-35086: Upgrade CycloneDX Maven plugin version](apache/arrow#35092)

upgrade

daacba3

LuciferYang marked this pull request as draft February 17, 2023 15:15

github-actions bot added the BUILD label Feb 17, 2023

LuciferYang marked this pull request as ready for review February 17, 2023 15:45

dongjoon-hyun reviewed Feb 17, 2023

View reviewed changes

LuciferYang mentioned this pull request Feb 17, 2023

[SPARK-42380][BUILD] Upgrade maven to 3.9.0 #40068

Closed

LuciferYang closed this Feb 18, 2023

This was referenced Mar 2, 2023

[SPARK-42646][BUILD] Upgrade cyclonedx from 2.7.3 to 2.7.5 #40247

Closed

[SPARK-42673][BUILD] Make build/mvn build Spark only with the verified maven version #40283

Closed

This was referenced Apr 13, 2023

ZOOKEEPER-4688: Upgrade cyclonedx-maven-plugin to 2.7.6 apache/zookeeper#1995

Closed

PARQUET-2275: Upgrade cyclonedx-maven-plugin to 2.7.6 apache/parquet-java#1057

Merged

dongjoon-hyun mentioned this pull request Apr 13, 2023

HIVE-27263: Upgrade cyclonedx-maven-plugin to 2.7.6 apache/hive#4235

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[SPARK-42382][BUILD] Upgrade `cyclonedx-maven-plugin` to 2.7.5 #40065

[SPARK-42382][BUILD] Upgrade `cyclonedx-maven-plugin` to 2.7.5 #40065

LuciferYang commented Feb 17, 2023 •

edited

Loading

LuciferYang commented Feb 17, 2023

dongjoon-hyun left a comment

LuciferYang commented Feb 17, 2023 •

edited

Loading

dongjoon-hyun commented Feb 17, 2023

LuciferYang commented Feb 17, 2023

LuciferYang commented Feb 17, 2023 •

edited

Loading

dongjoon-hyun commented Feb 17, 2023

LuciferYang commented Feb 17, 2023

dongjoon-hyun commented Feb 17, 2023

LuciferYang commented Feb 18, 2023

dongjoon-hyun commented Feb 18, 2023

LuciferYang commented Feb 18, 2023

dongjoon-hyun commented Feb 18, 2023

steveloughran commented Feb 23, 2023

dongjoon-hyun commented Apr 10, 2023

[SPARK-42382][BUILD] Upgrade cyclonedx-maven-plugin to 2.7.5 #40065

[SPARK-42382][BUILD] Upgrade cyclonedx-maven-plugin to 2.7.5 #40065

Conversation

LuciferYang commented Feb 17, 2023 • edited Loading

What changes were proposed in this pull request?

Why are the changes needed?

Does this PR introduce any user-facing change?

How was this patch tested?

LuciferYang commented Feb 17, 2023

dongjoon-hyun left a comment

Choose a reason for hiding this comment

LuciferYang commented Feb 17, 2023 • edited Loading

dongjoon-hyun commented Feb 17, 2023

LuciferYang commented Feb 17, 2023

LuciferYang commented Feb 17, 2023 • edited Loading

dongjoon-hyun commented Feb 17, 2023

LuciferYang commented Feb 17, 2023

dongjoon-hyun commented Feb 17, 2023

LuciferYang commented Feb 18, 2023

dongjoon-hyun commented Feb 18, 2023

LuciferYang commented Feb 18, 2023

dongjoon-hyun commented Feb 18, 2023

steveloughran commented Feb 23, 2023

dongjoon-hyun commented Apr 10, 2023

[SPARK-42382][BUILD] Upgrade `cyclonedx-maven-plugin` to 2.7.5 #40065

[SPARK-42382][BUILD] Upgrade `cyclonedx-maven-plugin` to 2.7.5 #40065

LuciferYang commented Feb 17, 2023 •

edited

Loading

LuciferYang commented Feb 17, 2023 •

edited

Loading

LuciferYang commented Feb 17, 2023 •

edited

Loading