Synchronize with central Maven repository

[thboileau]

Initial ticket : http://restlet.tigris.org/issues/show_bug.cgi?id=497

[holyjak]

I'm not really sure why it is necessary that Sonatype supports pulling from the Restlet Maven repo, cannot you just push the built artifact into it during your build process? This should be simple to implement, would keep your Maven repo as the master and would it make possible to use OSS as is, now. (I'm not sure that Sonatype will be willing to do what you want.)

It would be nice to have Restlet in the Maven Central ASAP.

[hoodja]

Just now discovering restlet -- would love to use it to stub out external dependencies, but I've spent the last few hours trying to get a simple maven project that depends on the restlet jars working inside Eclipse.

The extra config required for an alternate repository is a real downer. I would be happy to help automate publication to maven central as part of the build process.

[jlouvel]

Ok, let's move in this direction: pushing to Sonatype repo in addition to our master repository.

Your contribution on this front would be welcome.

[ansell]

How does the removal of pom.xml files in 2.2 in the master branch affect maven support?

bce55ce

[hoodja]

Hi ansell,

This is by no means authoritative, but from what I can see the pom files are generated as part of the build process. So the project does not use maven to accomplish its build, but it does support project that use maven by publishing poms and artifacts into a maven repository.

I'm now working on enhancing that build process so that the artifacts may be published to maven central in addition to the restlet master repo.

Hope this helps.
Jim

[ansell]

That sounds good. I was a little alarmed by the 2.2 draft release notes that seemed to say that Maven was going away.

It will make it possible for me to deploy my restlet-related projects to Maven Central once Restlet is deploying there (unless someone does a Third-Party Artifact Upload, which is impractical for a project as large as Restlet).

[jlouvel]

I confirm that the removal of pom.xml in GitHub is just to facilitate their maintenance and the usage of Eclipse IDE which otherwise tries to process them instead of using the regular Eclipse project files provided.

We still generate those poms and the repo metadata as part of the build process.

Thanks Jim for working on this!

[cstamas]

Pushing to Sonatype (oss.sonatype.org, OSO in short) can be integrated as "after step" too, you don't have to integrate it into your build, and you can just "push" the things after you did your own "homework" on your build.

For example, this is how we maintain and deploy our patched Restlet 1.1 for Nexus needs:
https://github.com/sonatype/restlet1x

In short, we build the Restlet 1.1 "as is", using provided Ant script, and luckily :) the build produces all we need.

For OSO push, the requirements are a bit stricter:
https://docs.sonatype.org/display/Repository/Sonatype+OSS+Maven+Repository+Usage+Guide#SonatypeOSSMavenRepositoryUsageGuide-6.CentralSyncRequirement

This means, you will probably need to update the POMs (ensure required elements like developers and SCM are in place), -sources was already built by restlet 1.1 build so that should not be a problem, would need -javadoc (maybe already built by 2.x build), all files would need SHA1/MD5 hashes (again, already done by restlet 1.1 build), and PGP signatures would need to be provided.

Finally, as for actual "staging" on OSO, you could use Repository Image Upload of Nexus Staging Maven plugin, to upload the locally "prepared" Restlet released artifacts (prepped as a repository).

[cstamas]

Oh, and OSO should work fine with Ant staging tasks too:
https://github.com/sonatype/nexus-ant-tasks/tree/master/nexus-staging-ant-tasks

[jlouvel]

Set milestone to 2.2.0. Let's get this done before the 2.2 release.

[jlouvel]

Let's try to make some progress on this for 2.2 RC2.

[jlouvel]

Statistics from Maven Central repository would still be available:
http://blog.sonatype.com/2010/12/now-available-central-download-statistics-for-oss-projects/#.U020r_ldU7I

[hardyoyo]

Hi, it appears that we have a transitive dependency issue with Apache Solr depending on Restlet, which leads to random build failures for our project (see https://jira.duraspace.org/browse/DS-1921 for more details). We're keeping on eye on #481, and look forward to simply pulling restlet straight from Maven Central.

[jlouvel]

Set milestone to 2.3 M3. @thboileau let's try to push this further finally :)

[jlouvel]

This issue is now part of the 3.0 scope detailed here: https://github.com/restlet/restlet-framework-java/wiki/Road-map-of-version-3.0 We just haven't set the detailled 3.0 milestones in GitHub.

If the uptime of Restlet Maven repository is the real concern, we have made progress on this front and now have a new bigger machine, better protection from attacks and a dedicated ops team that takes care of those aspects.

In the last six months we had an uptime of 99,98% for this repository. You can check on our status page for the history: http://stats.pingdom.com/0dzusgw4evae

[jlouvel]

Another tool mentioned by @glaforge that could help us is Bintray in complement to Sonatype OSS. It can help synchronize to Maven Central:
https://bintray.com/docs/usermanual/uploads/uploads_syncingartifactswithmavencentral.html

[tdiesler]

Not having restlet in central is biting camel-restlet, wildfly-camel and wildfly-swarm.

Caused by: org.eclipse.aether.transfer.ArtifactNotFoundException: Failure to find org.restlet.jee:org.restlet.ext.httpclient:jar:2.3.6 in http://repository.jboss.org/nexus/content/groups/public/ was cached in the local repository, resolution will not be reattempted until the update interval of jboss-public-repository-group has elapsed or updates are forced
    at org.eclipse.aether.internal.impl.DefaultUpdateCheckManager.newException(DefaultUpdateCheckManager.java:231)
    at org.eclipse.aether.internal.impl.DefaultUpdateCheckManager.checkArtifact(DefaultUpdateCheckManager.java:206)
    at org.eclipse.aether.internal.impl.DefaultArtifactResolver.gatherDownloads(DefaultArtifactResolver.java:585)
    at org.eclipse.aether.internal.impl.DefaultArtifactResolver.performDownloads(DefaultArtifactResolver.java:503)

Any chance of fixing this in the near future?

[jlouvel]

@tdiesler Definitely, we are going to solve this issue as soon as possible. I'm going to try to meet with the Bintray team next week at their SwampUP conference and dig more into this Central sync need.

[jlouvel]

To get us started, I've created the issue to get the setup in Sonatype OSS repository:
https://issues.sonatype.org/browse/OSSRH-22534

[jlouvel]

Next steps we'll need to get done are described here:
http://central.sonatype.org/pages/requirements.html

In our case:

• Sign Files with GPG/PGP (see details here: http://central.sonatype.org/pages/working-with-pgp-signatures.html)

Also we'll likely need to:

• Add a dependency on http://maven.restlet.com to make sure special dependencies not in Central are properly resolved

@thboileau Do you think you could take care of those steps sometimes soon?

[jlouvel]

Sonatype team has been pretty reactive. We are now able to push snapshot to their OSS buffer repository, and do the staging and releases steps, assuming we pass the quality checks (signatures esp.).

[thboileau]

here are the required steps before pushing to OSS repository:

• complete the metadata of each artefacts (licenses, developpers, SCM Information)
• sign each artefacts with GPG/PGP

[janstey]

Is this still planned? Would be really great if restlet was available in Maven central.

[jlouvel]

There is still a bit of work to do to make it happen, but @thboileau has already done most of it. I'll let him comment on whether he can make it work for RF v2.3, but this is definitely something we will have for RF v3.

[janstey]

Thanks for the reply @jlouvel. Having RF v2.3 in central would be awesome if possible 👍 Hope @thboileau thinks it's possible! Thanks again.

[rombert]

@jlouvel @thboileau - if I could be of any assistance please let me know. I've done some migrations to Maven Central in the past and am also interested in seeing this done for a project I'm contributing to ( see OAK-6880 - Build fails on a fresh m2 repo ).

[joel-bernstein]

Currently older versions of Apache Solr cannot be built from the main ant-ivy build because of problems resolving the restlet dependencies.

[ivy:retrieve] ==== maven.restlet.org: tried [ivy:retrieve] http://maven.restlet.org/org/restlet/jee/org.restlet.ext.servlet/2.3.0/org.restlet.ext.servlet-2.3.0.pom [ivy:retrieve] -- artifact org.restlet.jee#org.restlet.ext.servlet;2.3.0!org.restlet.ext.servlet.jar: [ivy:retrieve] http://maven.restlet.org/org/restlet/jee/org.restlet.ext.servlet/2.3.0/org.restlet.ext.servlet-2.3.0.jar [ivy:retrieve] ==== sonatype-releases: tried [ivy:retrieve] https://oss.sonatype.org/content/repositories/releases/org/restlet/jee/org.restlet.ext.servlet/2.3.0/org.restlet.ext.servlet-2.3.0.pom [ivy:retrieve] -- artifact org.restlet.jee#org.restlet.ext.servlet;2.3.0!org.restlet.ext.servlet.jar: [ivy:retrieve] https://oss.sonatype.org/content/repositories/releases/org/restlet/jee/org.restlet.ext.servlet/2.3.0/org.restlet.ext.servlet-2.3.0.jar [ivy:retrieve] ==== releases.cloudera.com: tried [ivy:retrieve] http://repository.cloudera.com/content/repositories/releases/org/restlet/jee/org.restlet.ext.servlet/2.3.0/org.restlet.ext.servlet-2.3.0.pom [ivy:retrieve] -- artifact org.restlet.jee#org.restlet.ext.servlet;2.3.0!org.restlet.ext.servlet.jar: [ivy:retrieve] http://repository.cloudera.com/content/repositories/releases/org/restlet/jee/org.restlet.ext.servlet/2.3.0/org.restlet.ext.servlet-2.3.0.jar [ivy:retrieve] ==== working-chinese-mirror: tried [ivy:retrieve] http://uk.maven.org/maven2/org/restlet/jee/org.restlet.ext.servlet/2.3.0/org.restlet.ext.servlet-2.3.0.pom [ivy:retrieve] -- artifact org.restlet.jee#org.restlet.ext.servlet;2.3.0!org.restlet.ext.servlet.jar: [ivy:retrieve] http://uk.maven.org/maven2/org/restlet/jee/org.restlet.ext.servlet/2.3.0/org.restlet.ext.servlet-2.3.0.jar [ivy:retrieve] :::::::::::::::::::::::::::::::::::::::::::::: [ivy:retrieve] :: UNRESOLVED DEPENDENCIES :: [ivy:retrieve] :::::::::::::::::::::::::::::::::::::::::::::: [ivy:retrieve] :: org.restlet.jee#org.restlet;2.3.0: not found [ivy:retrieve] :: org.restlet.jee#org.restlet.ext.servlet;2.3.0: not found [ivy:retrieve] :::::::::::::::::::::::::::::::::::::::::::::: [ivy:retrieve] [ivy:retrieve] :: USE VERBOSE OR DEBUG MESSAGE LEVEL FOR MORE DETAILS

I am working on a different project with a dependency on Solr and it cannot be built from maven:

Failed to read artifact descriptor for org.restlet.jee:org.restlet:jar:2.3.0: Could not transfer artifact org.restlet.jee:org.restlet:pom:2.3.0 from/to maven-restlet (http://maven.restlet.org): Received fatal alert: handshake_failure

I also noticed that now there is a redirect to maven.restlet.talend.com when going to maven.restlet.org? Have the maven dependencies been redirected for branding reasons?

Adjusting the pom files to point to the new URL did not resolve the build issues.

Please advise on both the Apache Solr builds and the maven build failures.

[jlouvel]

Thanks for the report, there is a related issue #1366.

We've recently moved our Maven infrastructure from a custom EC2-based to an S3-based infrastructure as part of the ongoing transition of Restlet to Talend (shared infrastructure).

It seems that the main issue is with the switch from HTTP to HTTPS though rather than the domain name redirection itself. @thboileau and I will keep you updated.

[rombert]

It seems that the main issue is with the switch from HTTP to HTTPS though rather than the domain name redirection itself.

I may miss some historical context, but why not "just" deploy the artifacts to Maven Central? Many projects are doing it, ranging from hobbyist projects to whole foundations with custom staging setups (e.g. the Apache Software Foundation).

Your current setup causes problems for current restlet consumers, and I think many of them (myself included) would welcome a change in the repository setup. If there is anything we as consumers can help with (testing,proposing changes) please let us know.

[jlouvel]

This is definitely the goal, but we historically had some issues with third party libraries especially that were not available in central Maven. With the development of the 2.5 version going on, we should remove all such dependencies which will facilitate this transition to Central and to a full Maven build (vers Ant today).

[rombert]

Great news, thanks @jlouvel !

[jlouvel]

FYI, the redirection from maven.restlet.com to maven.restlet.talend.com now preserves the protocol (HTTP or HTTPS).

We are still working on maven.restlet.org redirection infrastructure, but for now to redirects always to HTTP, so it shouldn't break existing builds anymore.

Let us know if you see remaining issues with your builds.

[breed85]

Hi,

Our build which includes Solr started failing at about 5 CST Jan 16. It last worked successfully 10:51 CST Jan 14.

We get a 403 unauthorized from all URLs under maven.restlet.org. If I navigate to restlet.org I get redirected to https://info.talend.com/

Anything you can do to assist is appreciated. Let me know if you need anything further from me.

[jlouvel]

Hi @breed85 Thanks for the report, I'm looking into this. It's likely related to the deployment of the new redirection from maven.restlet.org to maven.restlet.talend.com.

BTW, the new home of Restlet Framework is https://restlet.talend.com

[jlouvel]

@breed85 We have completed the deployment of our redirections from maven.restlet.org to maven.restlet.talend.com, and from maven.restlet.com to maven.restlet.talend.com, while preserving the protocol (HTTP or HTTPS).

Can you confirm everything works again for you?

[breed85]

@jlouvel I can confirm that the build is no longer failing using the original repository definition. I appreciate your quick response.

Thanks!

[ashrafsarhan]

I fixed this issue by adding the following repo to my maven pom file

<repository>
    <id>maven-restlet</id>
    <name>Restlet repository</name>
    <url>https://maven.restlet.talend.com</url>
</repository>