Deploying jars with classifiers using maven-deploy-plugin fails #515

blueberry · 2016-03-15T03:38:51Z

Hi,

I am trying to deploy a new version of project https://clojars.org/repo/uncomplicate/neanderthal-atlas/

My setup is a bit exotic, since it contains native libraries. I first use maven-nar-plugin to build all artifacts, but, since it produces nar files instead of jar files (only the classifir-ed file) I have to first rename it to jar, and then attach it with build-helper-maven-plugin, and then use maven-deployment-plugin with command line arguments to deploy it to Clojars.

Seems complicated, but it worked. If you look at 0.1.0 in Clojars, both .lar and .jar are there.

However, at some point the classifier-ed library stopped being uploaded. I can see the main jar and pom in Clojars, but there seems to be some issue with maven-metadata.xml.sha1 and maven-metadata.md5 that breaks the upload of other artifacts.

Here is how it looks like in the console, and both pom.xml and dep-pom.xml can be found at http://github.com/uncomplicate/neanderthal-atlas

------------ This is my command:

mvn -e org.apache.maven.plugins:maven-deploy-plugin:2.8.2:deploy-file -Durl=https://clojars.org/repo/ -DrepositoryId=clojars -Dfile=target/neanderthal-atlas-0.2.1-SNAPSHOT.jar -DpomFile=pom-dep.xml -Dfiles=target/neanderthal-atlas-0.2.1-SNAPSHOT-amd64-Linux-gpp-jni.jar -Dclassifiers=amd64-Linux-gpp-jni -Dtypes=jar

------------- This is console output:

tobias · 2016-03-15T11:51:00Z

This is related to the recent changes to support atomic deploys (https://groups.google.com/d/msg/clojure/ThU2hZ3Y3FM/cdGn1Ka2CgAJ), and there are two different failures here:

the rejection of the checksums for maven-metadata.xml
the loss of the classified artifacts. We had another issue with classifiers recently (deploying jars with classifiers appears to be broken #511), and the fix for that may not be enough.

I suspect 2 will be fixed when I fix #514, since I'll probably remove an internal deploy step (see #514 (comment) for details) and just copy the contents of the temp repo to the actual repo, which would take care of the classified artifacts as well.

I'll see if I can recreate both issues locally now.

blueberry · 2016-03-15T12:12:33Z

Thank you. I think I solved the problem by walking around it: I decided to introduce a repackaging step that puts all classified jars into one "superjar" and it seems to work. There is some problem with identifying the snapshot timestamp, but I hope that the release will work properly. Not an ideal solution, but a solution :)

tobias · 2016-03-15T12:30:42Z

Glad you have a workaround at the moment - the snapshot timestamp
problem you are seeing may be #514.

I've been able to recreate here, and don't yet have a solution, but
wanted to note my findings while they are fresh in my mind.

For (1) above, the issue is we are returning a nil response when a
checksum file is uploaded for the non-SNAPSHOT maven-metadata.xml,
and is a simple fix.

For (2), the issue is more complicated: we currently use the upload of
the non-SNAPSHOT maven-metadata.xml as a signal to finalize the
deploy (running validations against it, and then moving the deploy to
the real repo). When deploying normally via mvn, lein, or boot,
maven-metadata.xml and its checksums are always written last, so
it's a good signal to know the deploy is done. But when using
maven-deploy-plugin with multiple files, maven-metadata.xml is
written before the classified jars are uploaded, which means the
deploy is moved to the real repo before the classified jars exist, and
no further move step occurs, since nothing triggers it, so the
classified jars are essentially ignored.

This is actually a big problem for atomic deploys, since if we can't
rely on maven-metadata.xml to always be uploaded last, we can't know
when the deploy is finished. I'll have to give this some thought.

This causes 400's to be returned for those files.

blueberry · 2016-03-15T13:25:49Z

Thank you for working on this. I will probably stay with workaround as a permanent solution for other reasons, but some libraries (usually the ones that are using native binaries) critically depend on classifiers, and require custom deployment steps.

Maven will upload *any* classified artifact after uploading maven-metadata.xml. We use maven-metadata.xml as a trigger to know when to finalize the deploy, moving the contents to the actual repo and to cloudfiles. Before this change, we essentially ignored any files that came in after the finalization. Now, any files that come in after finalization go straight to the actual repo and to cloudfiles. This means that deployments with classified artifacts is no longer atomic - the primary artifact, pom, and maven-metadata.xml will be consistent, but any classified artifacts won't be part of that atomic unit. This commit also switches us from clj-http-lite to clj-http for testing, since we need cookie support to properly test this change, since we need to control the order of deployment, so can't use aether.

tobias · 2016-06-21T02:25:11Z

This is now fixed in production. Closing, but please reopen if you see any issues.

tobias added a commit that referenced this issue Mar 15, 2016

Don't return a nil response for maven-metadata.xml sums [#515]

11b03a4

This causes 400's to be returned for those files.

tobias mentioned this issue Jun 9, 2016

Problem with Clojars dropping / ignoring some artifacts? #532

Closed

tobias closed this as completed Jun 21, 2016

blueberry mentioned this issue Apr 12, 2018

:classifier in exclusions forbidden by Clojars #688

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Deploying jars with classifiers using maven-deploy-plugin fails #515

Deploying jars with classifiers using maven-deploy-plugin fails #515

blueberry commented Mar 15, 2016

tobias commented Mar 15, 2016

blueberry commented Mar 15, 2016

tobias commented Mar 15, 2016

blueberry commented Mar 15, 2016

tobias commented Jun 21, 2016

Deploying jars with classifiers using maven-deploy-plugin fails #515

Deploying jars with classifiers using maven-deploy-plugin fails #515

Comments

blueberry commented Mar 15, 2016

tobias commented Mar 15, 2016

blueberry commented Mar 15, 2016

tobias commented Mar 15, 2016

blueberry commented Mar 15, 2016

tobias commented Jun 21, 2016