Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

plugins not fetched #23

Open
avodonosov opened this issue Nov 29, 2020 · 11 comments
Open

plugins not fetched #23

avodonosov opened this issue Nov 29, 2020 · 11 comments

Comments

@avodonosov
Copy link

avodonosov commented Nov 29, 2020

Plugins seem to be not fetched at all.

I've tried on two projects: https://github.com/redisson/redisson and https://github.com/eclipse/winery.
After successfully completing the resolve-dependencies, their build in offline mode fails.

For redisson:
[ERROR] Plugin org.apache.maven.plugins:maven-surefire-plugin:2.12.4 or one of its dependencies could not be resolved: Cannot access central (https://repo.maven.apache.org/maven2) in offline mode and the artifact org.apache.maven.plugins:maven-surefire-plugin:jar:2.12.4 has not been downloaded from it before. -> [Help 1]

For winery:
[ERROR] Plugin org.apache.maven.plugins:maven-jar-plugin:3.0.0 or one of its dependencies could not be resolved: Cannot access central (https://repo.maven.apache.org/maven2) in offline mode and the artifact org.apache.maven.plugins:maven-jar-plugin:jar:3.0.0 has not been downloaded from it before. -> [Help 1]

These plugins are not dynamic dependencies, they are directly mentioned in the build -> plugins sections of the corresponding projects.

@Theoderich
Copy link
Collaborator

Thanks for the report. I was able to reproduce the issue. I will look into at as soon as I have time.

@ScrambledRK
Copy link

ScrambledRK commented Dec 2, 2020

I have the same issue, with a different plugin. I reference it in the pom under build/plugins and I even tried adding it as a dynamic plugin as well. Neither worked.

Plugin org.apache.maven.plugins:maven-clean-plugin:3.1.0 or one of its dependencies could not be resolved: Cannot access central (https://repo.maven.apache.org/maven2) in offline mode and the artifact org.apache.maven.plugins:maven-clean-plugin:jar:3.1.0 has not been downloaded from it before.
<properties>
    <maven-clean-plugin.version>3.1.0</maven-clean-plugin.version>
    <go-offline-maven-plugin.version>1.2.5</go-offline-maven-plugin.version>
</properties>

<build>
    <plugins>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-clean-plugin</artifactId>
            <version>${maven-clean-plugin.version}</version>
        </plugin>

        <plugin>
            <groupId>de.qaware.maven</groupId>
            <artifactId>go-offline-maven-plugin</artifactId>
            <version>${go-offline-maven-plugin.version}</version>
            <configuration>
                <dynamicDependencies>
                    <dynamicDependency>
                        <groupId>org.apache.maven.plugins</groupId>
                        <artifactId>maven-clean-plugin</artifactId>
                        <version>${maven-clean-plugin.version}</version>
                        <repositoryType>PLUGIN</repositoryType>
                    </dynamicDependency>   
                </dynamicDependencies>
            </configuration>
        </plugin>
    </plugins>
</build>
mvn de.qaware.maven:go-offline-maven-plugin:resolve-dependencies -Dmaven.repo.local=./binaries

For some reason it downloaded the version 2.5 into org\apache\maven\plugins\maven-clean-plugin and I cannot figure out where this version could come from. There is no dependency with that version anywhere in my repository. Maybe because I am using properties to resolve the version?

Edit: 2.5 is the default version of the maven-clean-plugin and is coming from a "naked" module without any parent pom, without a go-offline definition. Once I define the version of the maven-clean-plugin other plugins start to fail (downloading the wrong version) - so I think the issue comes from pom's using the default configuration somehow.

My actual usecase is a quite complex multi-module setup including parent pom's defining the go-offline-maven-plugin and all other plugins. So each of the sub-modules is "going offline" one by one, all into the same repository. The child poms are very sparse compared to the shared parent pom.

@ScrambledRK
Copy link

ScrambledRK commented Dec 3, 2020

As a workaround I "went offline" by deleting my local repository, building my project and saving the resulting local repository.
Even then I encountered errors when trying to build offline using said repository via -Dmaven.repo.local=./binaries.

The issue was that empty aggregator poms, not defining anything beside child modules, still required some plugins that were not downloaded. Ensuring that even aggregator poms define all the same plugins resolved this issue for me.

I have yet to try the de.qaware.maven:go-offline-maven-plugin:resolve-dependencies once again, but I am short on time at the moment.

@Theoderich
Copy link
Collaborator

I have identified the issue: The plugin filters duplicate Plugins before fetching. Unfortunately, if a plugin was defined multiple times with different versions, all but the first version was filtered. Different sections of the same plugin where also ignored.
In have completely removed the filtering since the performance-impact seems more or less non-existant.
The fix is pushed to master. You can try it by building the master and explicitly running the 1.2.8-SNAPSHOT version of the plugin. I will release it to maven-central soon if no further bugs are reported

@avodonosov
Copy link
Author

avodonosov commented Dec 7, 2020

After building and running the current SNAPSHOT version for winery with mvn -Pjava de.qaware.maven:go-offline-maven-plugin:1.2.8-SNAPSHOT:resolve-dependencies I noticed the following in the output:

[INFO] --- go-offline-maven-plugin:1.2.8-SNAPSHOT:resolve-dependencies (default-cli) @ winery ---
[WARNING] The POM for com.github.ust-edmm.edmm:edmm-core:jar:v1.0.6 is missing, no dependency information available
[ERROR] Error downloading dependencies for project
[ERROR] Failure to find com.github.ust-edmm.edmm:edmm-core:jar:v1.0.6 in https://repo.maven.apache.org/maven2 was cached in the local repository, resolution will not be reattempted until the update interval of central has elapsed or updates are forced
[ERROR] Error downloading plugin dependencies for project
[ERROR] Failure to find org.ethereum:solcJ-all:jar:0.4.25 in https://repo.maven.apache.org/maven2 was cached in the local repository, resolution will not be reattempted until the update interval of central has elapsed or updates are forced
[WARNING] Failure to find com.github.ust-edmm.edmm:edmm-core:jar:v1.0.6 in https://repo.maven.apache.org/maven2 was cached in the local repository, resolution will not be reattempted until the update interval of central has elapsed or updates are forced
[WARNING] Failure to find org.ethereum:solcJ-all:jar:0.4.25 in https://repo.maven.apache.org/maven2 was cached in the local repository, resolution will not be reattempted until the update interval of central has elapsed or updates are forced
...
[INFO] BUILD SUCCESS

Don't know is that bad or not. Investigating in the dependency tree built with mvn org.apache.maven.plugins:maven-dependency-plugin:3.1.2:tree > target/dependency-tree I don't see ethereum:solcJ-al at all.

@avodonosov
Copy link
Author

avodonosov commented Dec 7, 2020

When executing winery build after that with mvn -X -Pjava --offline package I've got:

[ERROR] Failed to execute goal org.web3j:web3j-maven-plugin:4.2.0:generate-sources (generate-sources) on project org.eclipse.winery.accountability: Execution generate-sources of goal org.web3j:web3j-maven-plugin:4.2.0:generate-sources failed: Plugin org.web3j:web3j-maven-plugin:4.2.0 or one of its dependencies could not be resolved: Cannot access central (https://repo.maven.apache.org/maven2) in offline mode and the artifact org.ethereum:solcJ-all:jar:0.4.25 has not been downloaded from it before. -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.web3j:web3j-maven-plugin:4.2.0:generate-sources (generate-sources) on project org.eclipse.winery.accountability: Execution generate-sources of goal org.web3j:web3j-maven-plugin:4.2.0:generate-sources failed: Plugin org.web3j:web3j-maven-plugin:4.2.0 or one of its dependencies could not be resolved: Cannot access central (https://repo.maven.apache.org/maven2) in offline mode and the artifact org.ethereum:solcJ-all:jar:0.4.25 has not been downloaded from it before.

	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:212)
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
	at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)
	at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80)
	at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)
	at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
	at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:307)
	at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:193)
	at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:106)
	at org.apache.maven.cli.MavenCli.execute(MavenCli.java:863)
	at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:288)
	at org.apache.maven.cli.MavenCli.main(MavenCli.java:199)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
	at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
	at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
	at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
Caused by: org.apache.maven.plugin.PluginExecutionException: Execution generate-sources of goal org.web3j:web3j-maven-plugin:4.2.0:generate-sources failed: Plugin org.web3j:web3j-maven-plugin:4.2.0 or one of its dependencies could not be resolved: Cannot access central (https://repo.maven.apache.org/maven2) in offline mode and the artifact org.ethereum:solcJ-all:jar:0.4.25 has not been downloaded from it before.
	at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:106)
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:207)
	... 20 more
Caused by: org.apache.maven.plugin.PluginResolutionException: Plugin org.web3j:web3j-maven-plugin:4.2.0 or one of its dependencies could not be resolved: Cannot access central (https://repo.maven.apache.org/maven2) in offline mode and the artifact org.ethereum:solcJ-all:jar:0.4.25 has not been downloaded from it before.
	at org.apache.maven.plugin.internal.DefaultPluginDependenciesResolver.resolveInternal(DefaultPluginDependenciesResolver.java:218)
	at org.apache.maven.plugin.internal.DefaultPluginDependenciesResolver.resolve(DefaultPluginDependenciesResolver.java:149)
	at org.apache.maven.plugin.internal.DefaultMavenPluginManager.createPluginRealm(DefaultMavenPluginManager.java:400)
	at org.apache.maven.plugin.internal.DefaultMavenPluginManager.setupPluginRealm(DefaultMavenPluginManager.java:372)
	at org.apache.maven.plugin.DefaultBuildPluginManager.getPluginRealm(DefaultBuildPluginManager.java:231)
	at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:102)
	... 21 more
Caused by: org.eclipse.aether.resolution.ArtifactResolutionException: Cannot access central (https://repo.maven.apache.org/maven2) in offline mode and the artifact org.ethereum:solcJ-all:jar:0.4.25 has not been downloaded from it before.
	at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolve(DefaultArtifactResolver.java:444)
	at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolveArtifacts(DefaultArtifactResolver.java:246)
	at org.eclipse.aether.internal.impl.DefaultRepositorySystem.resolveDependencies(DefaultRepositorySystem.java:367)
	at org.apache.maven.plugin.internal.DefaultPluginDependenciesResolver.resolveInternal(DefaultPluginDependenciesResolver.java:210)
	... 26 more
Caused by: org.eclipse.aether.transfer.ArtifactNotFoundException: Cannot access central (https://repo.maven.apache.org/maven2) in offline mode and the artifact org.ethereum:solcJ-all:jar:0.4.25 has not been downloaded from it before.
	at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolve(DefaultArtifactResolver.java:391)
	... 29 more
Caused by: org.eclipse.aether.transfer.RepositoryOfflineException: Cannot access central (https://repo.maven.apache.org/maven2) in offline mode
	at org.eclipse.aether.internal.impl.DefaultOfflineController.checkOffline(DefaultOfflineController.java:72)
	at org.eclipse.aether.internal.impl.Utils.checkOffline(Utils.java:115)
	at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolve(DefaultArtifactResolver.java:387)
	... 29 more
[ERROR] 
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/PluginResolutionException
[ERROR] 
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <goals> -rf :org.eclipse.winery.accountability

@avodonosov
Copy link
Author

When repeating the experiment with fresh maven repo specified through -Dmaven.repo.local=/some/dir//winery-m2-repo, the resolve-dependencies succeeds with the same ERROR and WARNING, but the offline build fails earlier:

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:3.0.0-M4:test (default-test) on project org.eclipse.winery.model.tosca: Execution default-test of goal org.apache.maven.plugins:maven-surefire-plugin:3.0.0-M4:test failed: Plugin org.apache.maven.plugins:maven-surefire-plugin:3.0.0-M4 or one of its dependencies could not be resolved: Cannot access central (https://repo.maven.apache.org/maven2) in offline mode and the artifact org.codehaus.plexus:plexus-utils:jar:1.1 has not been downloaded from it before. -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:3.0.0-M4:test (default-test) on project org.eclipse.winery.model.tosca: Execution default-test of goal org.apache.maven.plugins:maven-surefire-plugin:3.0.0-M4:test failed: Plugin org.apache.maven.plugins:maven-surefire-plugin:3.0.0-M4 or one of its dependencies could not be resolved: Cannot access central (https://repo.maven.apache.org/maven2) in offline mode and the artifact org.codehaus.plexus:plexus-utils:jar:1.1 has not been downloaded from it before.

	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:212)
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
	at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)
	at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80)
	at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)
	at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
	at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:307)
	at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:193)
	at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:106)
	at org.apache.maven.cli.MavenCli.execute(MavenCli.java:863)
	at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:288)
	at org.apache.maven.cli.MavenCli.main(MavenCli.java:199)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
	at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
	at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
	at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
Caused by: org.apache.maven.plugin.PluginExecutionException: Execution default-test of goal org.apache.maven.plugins:maven-surefire-plugin:3.0.0-M4:test failed: Plugin org.apache.maven.plugins:maven-surefire-plugin:3.0.0-M4 or one of its dependencies could not be resolved: Cannot access central (https://repo.maven.apache.org/maven2) in offline mode and the artifact org.codehaus.plexus:plexus-utils:jar:1.1 has not been downloaded from it before.
	at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:106)
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:207)
	... 20 more
Caused by: org.apache.maven.plugin.PluginResolutionException: Plugin org.apache.maven.plugins:maven-surefire-plugin:3.0.0-M4 or one of its dependencies could not be resolved: Cannot access central (https://repo.maven.apache.org/maven2) in offline mode and the artifact org.codehaus.plexus:plexus-utils:jar:1.1 has not been downloaded from it before.
	at org.apache.maven.plugin.internal.DefaultPluginDependenciesResolver.resolveInternal(DefaultPluginDependenciesResolver.java:218)
	at org.apache.maven.plugin.internal.DefaultPluginDependenciesResolver.resolve(DefaultPluginDependenciesResolver.java:149)
	at org.apache.maven.plugin.internal.DefaultMavenPluginManager.createPluginRealm(DefaultMavenPluginManager.java:400)
	at org.apache.maven.plugin.internal.DefaultMavenPluginManager.setupPluginRealm(DefaultMavenPluginManager.java:372)
	at org.apache.maven.plugin.DefaultBuildPluginManager.getPluginRealm(DefaultBuildPluginManager.java:231)
	at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:102)
	... 21 more
Caused by: org.eclipse.aether.resolution.ArtifactResolutionException: Cannot access central (https://repo.maven.apache.org/maven2) in offline mode and the artifact org.codehaus.plexus:plexus-utils:jar:1.1 has not been downloaded from it before.
	at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolve(DefaultArtifactResolver.java:444)
	at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolveArtifacts(DefaultArtifactResolver.java:246)
	at org.eclipse.aether.internal.impl.DefaultRepositorySystem.resolveDependencies(DefaultRepositorySystem.java:367)
	at org.apache.maven.plugin.internal.DefaultPluginDependenciesResolver.resolveInternal(DefaultPluginDependenciesResolver.java:210)
	... 26 more
Caused by: org.eclipse.aether.transfer.ArtifactNotFoundException: Cannot access central (https://repo.maven.apache.org/maven2) in offline mode and the artifact org.codehaus.plexus:plexus-utils:jar:1.1 has not been downloaded from it before.
	at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolve(DefaultArtifactResolver.java:391)
	... 29 more
Caused by: org.eclipse.aether.transfer.RepositoryOfflineException: Cannot access central (https://repo.maven.apache.org/maven2) in offline mode
	at org.eclipse.aether.internal.impl.DefaultOfflineController.checkOffline(DefaultOfflineController.java:72)
	at org.eclipse.aether.internal.impl.Utils.checkOffline(Utils.java:115)
	at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolve(DefaultArtifactResolver.java:387)
	... 29 more
[ERROR] 
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/PluginResolutionException
[ERROR] 
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <goals> -rf :org.eclipse.winery.model.tosca

@avodonosov
Copy link
Author

@Theoderich , thank you for addressing the issue, some problem seems to remain, sorry.

How do you think, why it is so difficult to fetch exactly those dependencies as used during real build? Is it some unfortunate maven architecture?

Ideally the go-offline would execute exactly the same procedure normal maven build uses, instead of replicating the logic.
It would require maven to preform some kind of dry-run - execute everything as in normal build but without actually running build operations. In some sense similary apt-get install --dry-run in Debian.

@Theoderich
Copy link
Collaborator

Unfortunately I know of no way to dry-run the maven build.
I started the go-offline-maven-plugin because the official maven-dependency-plugin:go-offline goal was not even able to handle reactor builds at the time and my pull request fixing the issue was not merged after over a year.
Ideally, the official plugin would be able to go-offline correctly. If there was an easy way to achive this, I am sure the official plugin would use it.

In the end there will never be a "perfect" solution to this problem unless the maven core team addresses it. As it is now, plugins can just dynamicaly load other plugins in their code without ever declaring it in their pom. I see no way to solve that with a plugin

@avodonosov
Copy link
Author

Yes, I faced several problems with the official dependency plugin go-offline functionality, and when wanted to report it to the plugin jira found many similar tickets already reported, one of them suggesting to use the go-offline plugin instead.

@MiguelWeezardo
Copy link

I have identified the issue: The plugin filters duplicate Plugins before fetching. Unfortunately, if a plugin was defined multiple times with different versions, all but the first version was filtered. Different sections of the same plugin where also ignored. In have completely removed the filtering since the performance-impact seems more or less non-existant. The fix is pushed to master. You can try it by building the master and explicitly running the 1.2.8-SNAPSHOT version of the plugin. I will release it to maven-central soon if no further bugs are reported

It seems this issue is not limited to Plugins, but to all dependencies - I have a multimodule project with dependencyManagement defined in root POM and one of the modules overrides some dependency versions.

After running mvn de.qaware.maven:go-offline-maven-plugin:1.2.8:resolve-dependencies my local Maven cache has only one version of the JARs which had overridden versions, instead of two (general and the override) required to build the entire project in offline mode.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants