Replacing the current build system (Ant) with Gradle

[Nava2]

Brief Description

Currently, we use Apache Ant which does the job as we need it. Unfortunately, as the project grows and becomes more complex, we are adding complexity and require flows that don't "blend" with Ant's XML build files (e.g. 1, 2, 3). I fear that over time, this system has developed a significant "code-smell" that will degrade over time.

Proposal

I would like to use a more modern build system, Gradle. Gradle is a Groovy-based build tool that allows the full expressiveness of Groovy with the build power of Ant/Maven.

To do this, I would like to do this in parallel with the Ant build. I believe it is "too big too fast" to swap it out without running both side-by-side for a period. Luckily, our current CI tools will allow us to do this as time progresses. Additionally, there would need to be milestones before eventually deprecating the old system (i.e. updating documentation, getting individuals up to speed). I would like to keep simple tasks available, for example: gradle quickTest:BuilderTest would run the cheatsheet equivalent of: ant -DshowJunitOutput=true -Djunit=BuilderTest -f build.umple.xml resetUmpleSelf compile template.test.

This issue is a discussion thread, there would need to be more milestones created and "attainable goals" should this be accepted.

Pros

1. Full access to Groovy
2. Build logic works in dependency graphs (only build a->b->c if necessary)
3. Battle tested by massive projects (e.g. Android)
4. Allows for polygot builds
5. Automatic parallelism in the build chain based on dependency graphs
6. Tight integration with Ivy
7. Acceptable point to improve/change build commands

Cons

1. New tooling, new work flows
2. Will require time running in parallel
3. Groovy is not XML (which everyone knows)
4. Change

Counter build tools

Maven

Maven is another XML build tool, but due to the XML nature of it, I do not see it being a significant benefit over the current Ant implementation. I also have a personal bias against it for being overly complicated.

[vahdat-ab]

I'm also supporting the idea :-).
It's a modern build tool that can improve and simplify our build process. I've tried once before to do some small build parts with Gradle, it was fun and easy especially with the plugin architecture (I stopped because didn't have time to spend on it).
Furthermore, the new generation of students is familiar with Gradle and Maven so the involvement in the future of Umple will be easier and substantial.

For sure, the process must be done in parallel with current build process so it can be tested continuously. When we are done with documentation, we can switch to it.

Since admins may or may not be familiar with Gradle and would like to learn the process, I would suggest to commit small parts and let other active members read the code and understand the whole process. This helps to have expert members in Gradle over time.

I would also like to participate in the development process of new build sytsem.

[TimLethbridge]

Yes, I agree with this. It seems to be the way of the future.

[ahochheiden]

I have chosen this issue as my term project. I will work with Vahdat and Tim to incrementally add portions of our build to Gradle.

As mentioned by Vahdat, the goal is that both Ant and Gradle build systems will work at the same time. Once we're confident in the Gradle build, we will work towards phasing out Ant.

[ahochheiden]

I read through a lot of the Gradle documentation. For our incremental approach, I think the right place to start would be to just import our Ant build into Gradle. (https://docs.gradle.org/current/userguide/ant.html#sec:import_ant_build) From what I understand, this would just wrap our current Ant build, so we could invoke it with Gradle commands and take advantage of some Gradle features.

I'm in the process of getting this to work. The import worked fine, and I can run all our Ant scripts through Gradle, the problem is that they're all failing.

C:\Users\Alex\Documents\GitHub\umple>gradle build
[ant:echo] Using myenv=wlocal
[ant:echo] Last Version: ${last.version}
:build
[ant:taskdef] Could not load definitions from resource org/apache/ivy/ant/antlib.xml. It could not be found.
[ant:taskdef] Could not load definitions from resource net/sf/antcontrib/antcontrib.properties. It could not be found.
:build FAILED

The problem seems to be related to Ivy. I remember that during the Dev Environment setup I did not install Ivy throughChocolatey, it was through Eclipse. I went and installed Ivy through Chocolatey, but that did not resolve this issue.

I'm going to look through our existing Ant scripts to see if I can find the source of these paths, or if they're being added by Gradle instead. If Gradle is adding them, the solution will probably be to add the paths myself. I'll probably ask Vahdat some questions about Ivy in private.

[Nava2]

First off, I'm still lurking these repositories, and I'm sorry I couldn't help directly.

With this issue, I would really really like you to stay away from implementing this in "terms" of Ant. The two builds should run simultaneously side-by-side, but do not "share" between them. The reason for this is that Gradle and Ant follow very different design philosophies and if you mix them you may create inter-dependencies that could never be removed.

For example, with your Ivy problem, Ivy isn't needed with Gradle. You just specify dependencies within the dependencies block and you're done. You will be able to literally copy the "parameters" for each dependency and replace it within these dependency blocks (link).

I believe this issue would also involve changing the layout of the project tree. Because the build scripts are truly just Groovy scripts, we can do a lot more powerful (and less "ant" obnoxious) things with the build chain. For example, checking internet connectivity is much more straight-forward.

Changing the build process creates a lot of opportunity for changing things like project setup (including repository setup (@aforward)). Think about what the "end goal" would be for this outcome and try to design the idea "modern" version of this build tree. Ultimately, because of the flexibility with Gradle, we can "propose future changes," get them working with Gradle and Ant. Replace the Ant build after its working side-by-side and equivalently, then make changes that Ant wouldn't like.

If @vahdat-ab would allow it, I'd happily help with mentoring on this issue. I wanted to do it for a long time, but I've become so busy with my own projects I let this fall to the wayside. Ultimately, you're welcome to ask me questions regardless. 👍 I've actually picked up some Gradle knowledge since opening this months ago.

[aforward]

I concur.

If you implement the "new" in terms of the "old" and the implementation level, then I would argue "why bother". There must be a real pain point that we are addresses (or possibly an "opportunity" point).

That said a few extra points....

1. The work must be done in small increments; please no big bang as the build system is crucial to developers remaining agile about making changes and mitigating regressions

2. Both systems (Ant vs Gradle) should achieve the same high level goals, but as @Nava2 points out don't shoot yourself in the foot by forcing a dependency between the new and old.

Good luck!

[TimLethbridge]

I would be super-happy to have Kevin (aka @Nava2 ) mentor this work.

[vahdat-ab]

@Nava2 There is no issue with that. We are all here to help and work with each other. In fact, this is a great thing we can expect from people who are working in Umple team.
I completely agree with you guys and I have already pointed out this in the response to @ahochheiden.
The reason behind this might come from the fact that reading the ant scripts looks hard and refactoring that makes the process much easier.
However, as @Nava2 said we might end up with some dependencies that are not going to be fixed later.
I completely agree to develop from scratch. It might takes time but it's the best approach.

[Nava2]

@vahdat-ab, and @ahochheiden, why don't we setup some time early next week to chat about this? I'll have some more time then.

From @aforward:

1. The work must be done in small increments; please no big bang as the build system is crucial to developers remaining agile about making changes and mitigating regressions

Nice thing about Gradle, since it's still Groovy, we can have methods that we change the implementation from down the road. Code for big changes, but implement them later (thinking along the lines of separate repository stuff).

[vahdat-ab]

@Nava2 @ahochheiden I agree with that. I'm fully busy Tuesday but we can make it on Wednesday or Thursday. Of course, if you guys are ok with it as well.
I think it will be great to have @TimLethbridge

[ahochheiden]

Tuesday works for me, I'm free between 1:00 PM and 4:00 PM MST. Thursday would have to be before 1:00 PM MST or after 4:30 PM MST.

[Nava2]

Wednesday before 430PM EST and Thursday are open for me this week. What about 1PM EST (11AM MST) on Thursday?

[ahochheiden]

1PM EST on Thursday it is. Do I need to add you on Skype or will Vahdat connect us?

[Nava2]

Hangouts is easier for me.

[vahdat-ab]

I'm fine with both. @Nava2 can you make it on skype? otherwise, we will switch to Hangouts.

[Nava2]

Yup, kevin_brightwell, I think.

[TimLethbridge]

Thanks so much for this. I am 'half way' back from my trip, but will be in the air for this, so I leave it to you. Thanks.

[Nava2]

From our skype call:

Next steps

Create a plan for the milestones listed. Estimate some times, but don't feel too bad about hitting later ones. We are all in new territory with regards to how long this stuff will take (especially since you're new to Groovy!). If you're comfortable with it, I suggest using Githubs project management tools to setup the milestones better than this list here.

https://guides.github.com/features/issues/#filtering

Milestones

• Create a hello world plugin
  • Just figure out how to load a custom plugin
  • https://docs.gradle.org/current/userguide/custom_plugins.html
• Have the plugin load the Umple JAR
  • Use a hard-coded path for now
• Create simple tests for your simple plugin 😄
• Call the compiling interface for the source files to convert to Java
• Enhance the plugin to:
  • allow for specifying other generators (e.g. PHP or Ruby)
  • allow a specification for Umple jar versions or a local JAR path
  • figure out how to specify multiple generations for a single source tree in a build
    • We need this for running the Test Bed

Throughout these milestones, create a "paper trail" of what you're doing. If you run into problems, write it down. When you hit milestones, try to convert the current paper trail into a Wiki document. Keep the Wiki pages in the main Wiki, but as @vahdat-ab suggested in the call, use the other repository for issues.

Additionally, like the umple project, everything should be done in TDD format, make tests for everything as you do it, Red-Green-Refactor!

[TimLethbridge]

I have the credentials to log in to the gradle plugin hub. It seems you have managed to deal with this as I see new updates in the last couple of days.

[AdamBJ]

@TimLethbridge Thanks for checking, I got them from Vahdat

[TimLethbridge]

We have been working on this issue for ages. If it is to be continued we will need a really talented student who can drive it through to completion. The ant builds are complex and continue to evolve meanwhile ...

[AdamBJ]

@TimLethbridge @Nava2 When I finished my UCOSP term working on this (Spring 2017), the Umple Gradle plugin was 90-95% done, and I put together a "proof of concept" build that builds the Umple compiler jar using the Gradle plugin..

My hope is that the next student that comes along can use the Umple Gradle plugin as a "black box", and, using the PoC build as a starting point, begin work on converting the build to Gradle fully. Before that happens, ideally Kevin and I could check off a few (mostly testing related) boxes related to the Umple Gradle plugin. There are a couple open PRs from some work I did over the summer, but I haven't had a chance to respond to Kevin's comments yet as I've been quite busy working through my final semester since September. I can do a bit more work on the plugin over the break, but I think generally speaking it's in pretty good shape.

[TimLethbridge]

PR #1418 is set up to resolve this issue. Also Wiki page https://github.com/umple/umple/wiki/Gradle-Workflow-and-Migration-Document is documentation for it.

[TimLethbridge]

More work in this to be done in #1609