Migrate build process from Ant to Maven (WIP). #347
Conversation
|
This is awesome. Maybe we can host our own Maven repository for the missing/custom jars? I'd like to see this work complete before we merge, but this is an awesome start. |
|
I would like (somewhat selfishly, I admit) to see code coverage capabilities added to the new build, assuming this is something that can be easily integrated via Maven. I currently run a modified Ant build that uses Emma to measure test coverage with a suite of JMock test fixtures against our custom plugins. I realize that core Openfire does not have much by the way of test fixtures at this point, but perhaps we can begin work toward improving this as well. Agree this is a great start ... nice work! |
|
@tevans: There's an Maven EMMA plugin |
sco0ter
force-pushed
the
maven
branch
8 times, most recently
from
e6dbb2c
to
b1ee7ce
Compare
|
I've updated (rebased) the branch. The Maven structure is: What it can do:
What it can't do:
Todos:
|
|
@tevans @sco0ter - I'm a big fan of cobertura for code coverage in maven. example usage: <plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>cobertura-maven-plugin</artifactId>
<version>${cobertura-maven-plugin.version}</version>
<configuration>
<check>
<haltOnFailure>true</haltOnFailure>
<branchRate>90</branchRate>
<lineRate>90</lineRate>
<packageLineRate>90</packageLineRate>
<packageBranchRate>90</packageBranchRate>
<totalBranchRate>90</totalBranchRate>
<totalLineRate>90</totalLineRate>
</check>
</configuration>
<executions>
<execution>
<goals>
<goal>clean</goal>
<goal>check</goal>
</goals>
</execution>
</executions>
</plugin>It integrates well with Jenkins and I'm sure TravisCI digs it too. |
|
Thanks for picking this up again! Turning the project into Maven will give many benefits! Let's try to avoid having our own Maven repository. Not only is it a hassle to maintain, it will also prevent us from submitting Openfire to a central repository (where it ideally should go, I think). Tinder already is in a central repository, by the way: http://mvnrepository.com/artifact/org.igniterealtime/tinder Not sure what problem you're having there? as for the other library - is that to generate Windows-based installation artifacts? Perhaps we can find a Maven-friendly alternative, or keep that part out of Maven completely. |
|
The problem with Tinder is, that there's only an old version (1.2.3), but Openfire uses 1.3.0-SNAPSHOT. which has a different API. (asBareJid()). About the i4jruntime.jar: I've asked here: https://community.igniterealtime.org/thread/57447 But even if we can solve these two dependency problems, there are like 10-20 jars waiting in the plugins (especially ofmeet, kraken, stun), which uses jars, which are not in Maven Central and are also otherwise hard to find elsewhere. :-( Not sure how to best approach this problem (that's why I've excluded those plugins). I guess we can go the evil system scope way, but in my experience it only causes other problems (like the maven-assembly-plugin doesn't pick them up in the dependencySet). |
|
At the very least, someone should shout at and be generally very unfriendly towards the Tinder project lead. That guy has definitely dropped the ball. As for Kraken - I wonder if any of the transports in there are still functional. It has support for IM-networks that do not exist any longer. Perhaps we should simply drop the entire plugin. I've looked as stun myself - either there was an alternative, or the code is relatively easy to port / replace. We can create our own alternative. OFMeet is the only one that's worrying me. That should definitely be taken care of one way or the other. Might be good to involve @deleolajide ... |
|
Hey, about ofmeet, I haven't checked it lately, but I know it is jitsi based. In the late months we tried to mavenize all of our projects, currently only jigasi is not finished yet, but I'm in a process of doing it. Currently we are using github for our maven repo, but if there is a contribution in our community we can start pushing them to some official repos. Currently github is working fine, as we needed a quick solution to speed up development. But anyway at some point we will hit the size limit of the repo and will need to move :) |
|
I think you will have to exclude ofmeet. It is a fork of Jitsi Videobridge modified for Openfire and enterprise useage as Jitsi Videobridge is biased towards being an external xmpp component to Prosody and depends on a few external services like etherpad, prezzo and callstat.io. The head Jitsi Videobridge source code or jar files will currently not run as an internal XMPP component with Openfire. The implementation is broken somewhere. Unless someone else takes on this task, this is low priority for me right now. |
|
I have removed ofmeet plugin. It is no more a blocker to this PR |
This was referenced by Openfire as dbutil.jar, but because this cannot be referenced via Maven Central, we add it as module for Openfire.
The Maven structure is: -/ --dbutil (used by xmppserver) --i18n (used by starter and xmppserver) --starter (launcher and starter package, startup.jar) --xmppserver (nearly everything in the src dir, openfire.jar) --webadmin (admin plugin) --plugins ---broadcast ---... I've also added Maven plugins to mimic partial behavior of the Ant build (copying dependencies, bin/conf folder, ...) This commit addresses issues OF-546. I reconfigured the pom.xmls to work with the existing directory structure, (instead of default src/main/java), so that the current src folder stays untouched. What it can do: - Run tests successfully - Generate JavaDoc - Assemble most plugin jars (without JSP) - Assemble admin plugin (without JSP) - Generate a source zip/tar.gz for distribution (as the download page offers it currently) - Build the startup.jar and openfire.jar, copy it in the right directory, so that Openfire can be run with `java -jar startup.jar` What it can't do: - Build executables for every platform (e.g. install4j, *.dmg, ...) - Build some plugins (e.g. ofmeet) due to non-Maven dependencies. Todos: - Fix these issues - Move source code to the correct paths, e.g. /src to xmppserver/src/main/java
|
May I suggest having a look at Gradle as a build system? |
|
If looking at Gradle, please be aware of major differences to maven. A good starting point (and at the end with useful links) is http://wiki.apidesign.org/wiki/Gradle |
|
@nelbrecht I think the article you linked to might be a bit anti-Gradle tainted, isn't it? |
|
@netmikey Had to give it a try already. And, yes, I sat in front of that thing asking myself "kill the process or wait even more". Long and repeatedly. That was the practical side. But then again, I will probably just download a binary, not build it myself. Whoever wants to use gradle can do then. Just shared my experience with words of someone who is usually not wrong and quite eloquent. |
|
Using it for years on enterprise-scale projects, this has never happened to me. In fact, for our own Openfire Plugins, I even wrap the whole OF Ant-Build into a Gradle build without the slightest problem. |
|
Please do not hijack this pull request for a Gradle-vs-Maven discussion. Feel free to contribute a Gradle-based approach, but please do so by creating a dedicated pull request. |
|
So that Openfire 4.1 is out the door and his its own 4.1 git branch, what's everybodies thoughts on this PR? I know it is likely poor to do, but my preference would be to break master branch now and guilt any sympathetic devs into whack-a-mole fixing issues that come up. I think this guilt would work for me at least, to attempt to learn more and help with the artifact building procedure... If this goes into a dedicated branch, it seems like the eventual merge back to master would be extremely painful, but perhaps I am wrong... |
I totally agree (though as I'm not able to commit or even submit any PRs in the near future, I don't expect my opinion to carry much weight). IMHO the longer you wait, the longer it will take. Bite the bullet! |
|
So to hint at where I'm coming from on this, we're doing some internal work on Openfire at Surevine currently that will eventually be released into the Open Source builds. I've advised that people use this Maven branch to simplify development patterns. Our security guy would prefer it, as it makes dependency tracking and updates easier. Every one of my customers wants Maven, too - again, it just hugely simplifies development. I'm going to play around with this branch over the New Year, and we'll try and make 4.2 a Maven build ASAP. |
|
A move from Ant to Maven is being prepared for the Spark project. The work done so far is in this branch: https://github.com/igniterealtime/Spark/tree/SPARK-1791_Maven If there is knowledge that we have learned from Mavenizing either project, it would be good to apply that to the other one. I'd love for the people involved in the Openfire Maven process to have a look at what we did so far in the Spark project. Comments? Suggestions? |
|
Nice to see some movement for Maven. I have nothing to add for the Spark project, looks pretty good! |
|
@sco0ter I think there is some appetite to make maven migration happen for the next major Openfire release of 4.2. Are you able to commit some time to help with this effort in the coming weeks/ few months? Thanks for the consideration! |
|
@akrherz Yes I hope so. The hard part with this branch was building the distributables for the different OS. That's where this branch is primarily stuck. |
|
@sco0ter Great, thank you! I have licenses of install4j to share, if you want to work on things there. I think @guusdk did a bunch of stuff with Smack to get it working well with install4j to generate builds there. Otherwise, we have weekly Google Hangouts at 1430 UTC on Fridays, if you'd like to stop by and we could discuss a plan of action with this? |
|
Couldn't you use the maven antrun plugin
<http://maven.apache.org/plugins/maven-antrun-plugin/usage.html> to bring
over anything for which a maven plugin didn't exist? The ant tasks run as
part of the normal maven build depending on what maven phase you bind the
execution too. Each execution can can have tasks directly in the pom or the
recommended way (that would allow sharing among sub-projects) is to invoke
ant tasks in a build.xml. There are ways to pass properties from maven to
ant.
…On Sat, Feb 25, 2017 at 10:10 AM, daryl herzmann ***@***.***> wrote:
@sco0ter <https://github.com/sco0ter> Great, thank you! I have licenses
of install4j to share, if you want to work on things there. I think
@guusdk <https://github.com/guusdk> did a bunch of stuff with Smack to
get it working well with install4j to generate builds
<https://github.com/igniterealtime/Spark/blob/master/distribution/pom.xml#L47>
there. Otherwise, we have weekly Google Hangouts at 1430 UTC on Fridays, if
you'd like to stop by and we could discuss a plan of action with this?
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#347 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/ABfBKXeYRsDkaNPeWefIMHIiZR8t1Xguks5rgER9gaJpZM4GZBny>
.
|
|
No sorry, I am busy at work at that time. @guusdk You can start a thread in the forums for discussion. |
|
Ah, my biggest concern was the move to the standard directory layout - which only now I see was not done, only suggested. I've just rebased the code, and with some additional dependencies, was able to build the project again. Better yet, the Ant script still worked too - which makes a merge of this PR a non-breaking change, I believe. I'm in favor of merging the PR, and start finalizing the Maven work on the master branch. I think that in the immediate future, we should retain both the Ant as well as the Maven build structure (have continuous integration for both), until we have brought Maven up to a "production-ready" quality. After that, we can drop Ant, if it gets in our way (as long as we can keep both, there's some value in that. Given that this PR still "works" after all these months is a good indication that we can actually keep both systems side-by-side for some time, if we would want to). Given that this still is a very useful PR, even after all these months of inactivity, goes to show what an excellent job @sco0ter did! Kudos! |
|
I actually started with the default layout structure and moved the code into different modules and the correct folders (e.g. src/main/java). However, after some rebases I quickly realized, that it won't be sustainable in the long term. Therefore I decided to start with only the pom.xmls without moving .java files. Good to hear, it worked out. The only issue with the non-default layout structure is that my IDE (IntelliJ) can't deal with it very well, i.e. you still have to setup your IDE manually. I agree, that merging to master is probably harmless and like your approach to maintain both Ant and Maven at least short term. |
|
The button was green, so green means good and I pushed it |
|
I like your reasoning. |
|
So, now that we merged, I've been trying to use the structure, and ran into the first issue. I tried to create a plugin that is not part of the Openfire source code. To do that, I copied the pom.xml file of one of the other plugins. This made org.igniterealtime.openfire:plugins be me parent. Things failed, because the build was trying to retrieve the xmppserver artifact with a version number that matched my plugin version number. I dug a little deeper: by having the inheritance structure that we have now, all plugins that would define 'plugins' as its parent would inherit everything from the project. This adds a lot of useful stuff (it makes things work), but you also inherit the things that are not applicable to you. Even stuff like our license, project members etc, are inherited. What's worse: when you invoke maven from within the directory of your plugin, it populates the various ${project.xyz} properties with values that apply to your plugin project. Part of the Openfire maven structure expcects these properties to be filled with the values of the Openfire project (this is why my plugin build tried to look up the org.igniterealtime.openfire.xmppserver dependency with the version number of my plugin (1.0-SNAPSHOT), rather than Openfire's 4.2.0-SNAPSHOT). I've tried to decouple the plugins artifact from the parent artifact (by not defining a parent for the plugins artifact), but that introduces a lot of challenges (as a lot of what is being inherited is needed). I'm not sure if this is the best way to go either. Thoughts? |
|
It strikes me that if developing a plugin that's no part of the Openfire Source then that plugin shouldn't have org.igniterealtime.openfire:plugin as a parent. I think the Maven way to do it would be to simply have a provided dependency on org.igniterealtime.openfire:xmppserver:4.2.0-SNAPSHOT It probably does mean a but of cut'n'paste between the plugins/pom.xml and the /pom.xml with regards to compiling the JSPs and assembling the WAR file. One way around that would be to have a custom Openfire Maven plugin that does all the common work, but that's probably overkill. Perhaps there is a better way. |
|
I tend to agree with you, but the cut/pasting of poms would involve a lot of boilerplate config. I'd prefer to avoid that. A Maven plugin for Openfire isn't something I'd really would like to maintain, but it beats answering user questions five times a day, needing to decipher what's wrong in their plugin pom.xml. I'm interested in the 'better ways' too. :) |
|
There's already the openfire-plugin-assembly-descriptor module which is shared by all plugins. Can't this be used somehow for common stuff? In my company we have multiple Openfire plugins (all without any JSP) which eventually all derive from the company's parent pom and I'd prefer it this way as Greg said. However the plugin descriptor for assembling the plugins into the Openfire structure is nearly identically to Openfire's one. I think ideally the openfire-plugin-assembly-descriptor and the xmppserver module (or interface of it) will be provided by Maven Central at some point with some, so that people only need to reference both, together with a snippet of the maven-assembly-plugin and the jetty-jspc-maven-plugin, which they can use. |
|
In fact, now I've slept on it, I can't help but think that the Openfire bundled plugins should also be moved to a separate source tree. That means that compiling Openfire means that each and every plugin does not have to be compiled, instead the packaging process simply bundles the previously required plugin, fetched as a dependency. A plugin only needs to be be compiled/released when that plugin changes, not every time the core XMPP server changes. That said, I recognised that it's difficult to do whilst maintaining the dual Ant/Maven process. |
|
I've tried to decouple the plugins from the parent project here: #775 (please continue discussing that specific subject there). |
I took Guus' old Maven branch and integrated the pom.xmls into the current master, updating the dependencies (especially Jetty and Mina).
I've also added Maven plugins to mimic partial behavior of the Ant build (copying dependencies, bin/conf folder, ...)
This commit addresses issues OF-546.
I reconfigured the pom.xmls to work with the existing directory structure, (instead of default src/main/java), so that the Ant build keeps working as is.
Build succeeds with "mvn clean install -DskipTests=true".
TODOs:
Please comment on ideas how to proceed from here.