Add Gradle based build #1981

jelovirt · 2015-07-25T17:21:25Z

Replaces Ant+Ivy with a more modern Gradle based build environment. Basic build will remain a similar single command operation, but distribution builds will no longer require separate steps and manual CLASSPATH management, and will too be a single command operation.

This PR retains the original Ant and Ivy files, but in the future Gradle build should completely take over the build infrastructure.

Remaining tasks:

implement PDF2 plugin's compile step and fo.jar generation.

Replaces Ant+Ivy with Gradle based build environment.

jelovirt · 2015-07-25T17:24:29Z

@eerohele any possibility to get a review?

eerohele · 2015-07-27T06:50:36Z

I suspect you'll need to add and commit the gradle/wrapper directory as well?

jelovirt · 2015-07-27T07:12:32Z

@eerohele yes, forgot about that.

jelovirt · 2015-07-27T07:12:35Z

I'm creating dummy Java files for RenderX classes used by PDF2, that way you can build without first copying XEP installation into PDF2 lib. Those empty dummy Java classes are then left out from the fo.jar.

eerohele · 2015-07-27T08:07:22Z

Yeah, the test target fails for me at the moment, presumably because of those missing dummy RenderX classes.

I also wonder whether it might make sense to have a target like this for easily running unit tests:

task unitTest(dependsOn: [buildLocal, test]) << {
    // NOOP
}

My Gradle is a bit rusty, so I'm not sure whether that's the cleanest way of doing it.

eerohele · 2015-07-27T08:10:05Z

Yeah, the test target fails for me at the moment, presumably because of those missing dummy RenderX classes.

Ah, sorry, they actually don't fail if I run the buildLocal target (as I should) before test.

jelovirt · 2015-07-27T09:46:29Z

@eerohele, the test task should work without running buildLocal first, thus there's something I've missed.

Note that this PR is still somewhat work in progress, but I wanted to get more eyes on it while development was still active.

stefan-jung · 2015-07-27T11:22:05Z

I just executed a gradle build and got this trace due to some file-not-found errors (see below).:

[stefan@vaio jelovirt]$ cd dita-ot/
[stefan@vaio dita-ot]$ git checkout feature/gradle
Branch feature/gradle konfiguriert zum Folgen von Remote-Branch feature/gradle von origin.
Zu neuem Branch 'feature/gradle' gewechselt
[stefan@vaio dita-ot]$ clear

[stefan@vaio dita-ot]$ gradle build
:compileJava
Download https://repo1.maven.org/maven2/commons-io/commons-io/2.4/commons-io-2.4.pom
Download https://repo1.maven.org/maven2/org/apache/commons/commons-parent/25/commons-parent-25.pom
Download https://repo1.maven.org/maven2/org/apache/apache/9/apache-9.pom
Download https://repo1.maven.org/maven2/commons-codec/commons-codec/1.9/commons-codec-1.9.pom
Download https://repo1.maven.org/maven2/org/apache/commons/commons-parent/32/commons-parent-32.pom
Download https://repo1.maven.org/maven2/org/apache/apache/13/apache-13.pom
Download https://repo1.maven.org/maven2/xerces/xercesImpl/2.11.0/xercesImpl-2.11.0.pom
Download https://repo1.maven.org/maven2/xml-apis/xml-apis/1.4.01/xml-apis-1.4.01.pom
Download https://repo1.maven.org/maven2/xml-resolver/xml-resolver/1.2/xml-resolver-1.2.pom
Download https://repo1.maven.org/maven2/org/apache/apache/3/apache-3.pom
Download https://repo1.maven.org/maven2/net/sourceforge/saxon/saxon/9.1.0.8/saxon-9.1.0.8.pom
Download https://repo1.maven.org/maven2/com/ibm/icu/icu4j/54.1/icu4j-54.1.pom
Download https://repo1.maven.org/maven2/org/apache/ant/ant/1.9.4/ant-1.9.4.pom
Download https://repo1.maven.org/maven2/org/apache/ant/ant-parent/1.9.4/ant-parent-1.9.4.pom
Download https://repo1.maven.org/maven2/org/apache/ant/ant-launcher/1.9.4/ant-launcher-1.9.4.pom
Download https://repo1.maven.org/maven2/commons-io/commons-io/2.4/commons-io-2.4.jar
Download https://repo1.maven.org/maven2/commons-codec/commons-codec/1.9/commons-codec-1.9.jar
Download https://repo1.maven.org/maven2/xerces/xercesImpl/2.11.0/xercesImpl-2.11.0.jar
Download https://repo1.maven.org/maven2/xml-apis/xml-apis/1.4.01/xml-apis-1.4.01.jar
Download https://repo1.maven.org/maven2/xml-resolver/xml-resolver/1.2/xml-resolver-1.2.jar
Download https://repo1.maven.org/maven2/net/sourceforge/saxon/saxon/9.1.0.8/saxon-9.1.0.8-dom.jar
Download https://repo1.maven.org/maven2/net/sourceforge/saxon/saxon/9.1.0.8/saxon-9.1.0.8.jar
Download https://repo1.maven.org/maven2/com/ibm/icu/icu4j/54.1/icu4j-54.1.jar
Download https://repo1.maven.org/maven2/org/apache/ant/ant/1.9.4/ant-1.9.4.jar
Download https://repo1.maven.org/maven2/org/apache/ant/ant-launcher/1.9.4/ant-launcher-1.9.4.jar
Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
:processResources
:classes
:jar
:assemble
:compileTestJava
Download https://repo1.maven.org/maven2/nu/validator/htmlparser/htmlparser/1.4/htmlparser-1.4.pom
Download https://repo1.maven.org/maven2/junit/junit/4.11/junit-4.11.pom
Download https://repo1.maven.org/maven2/xmlunit/xmlunit/1.6/xmlunit-1.6.pom
Download https://repo1.maven.org/maven2/org/hamcrest/hamcrest-core/1.3/hamcrest-core-1.3.pom
Download https://repo1.maven.org/maven2/org/hamcrest/hamcrest-parent/1.3/hamcrest-parent-1.3.pom
Download https://repo1.maven.org/maven2/nu/validator/htmlparser/htmlparser/1.4/htmlparser-1.4.jar
Download https://repo1.maven.org/maven2/junit/junit/4.11/junit-4.11.jar
Download https://repo1.maven.org/maven2/xmlunit/xmlunit/1.6/xmlunit-1.6.jar
Download https://repo1.maven.org/maven2/org/hamcrest/hamcrest-core/1.3/hamcrest-core-1.3.jar
Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
:processTestResources
:testClasses
:test
Download https://repo1.maven.org/maven2/org/apache/ant/ant-apache-resolver/1.9.4/ant-apache-resolver-1.9.4.pom
Download https://repo1.maven.org/maven2/org/apache/ant/ant-apache-resolver/1.9.4/ant-apache-resolver-1.9.4.jar

org.dita.dost.writer.KeyrefPaserTest > classMethod FAILED
    net.sf.saxon.trans.XPathException at KeyrefPaserTest.java:59
        Caused by: java.io.FileNotFoundException at KeyrefPaserTest.java:59

org.dita.dost.reader.TestGenListModuleReader > testParse FAILED
    java.io.FileNotFoundException at TestGenListModuleReader.java:75

org.dita.dost.reader.MapMetaReaderTest > classMethod FAILED
    net.sf.saxon.trans.XPathException at MapMetaReaderTest.java:50
        Caused by: java.io.FileNotFoundException at MapMetaReaderTest.java:50

org.dita.dost.reader.KeydefFilterTest > testParse FAILED
    java.io.FileNotFoundException at KeydefFilterTest.java:56

org.dita.dost.module.TestGenMapAndTopicListModule > classMethod FAILED
    org.dita.dost.exception.DITAOTException at TestGenMapAndTopicListModule.java:104
        Caused by: java.io.FileNotFoundException at TestGenMapAndTopicListModule.java:104

org.dita.dost.module.DebugAndFilterModuleTest > testGeneratedFiles FAILED
    java.lang.AssertionError at DebugAndFilterModuleTest.java:98

282 tests completed, 6 failed, 7 skipped
:test FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':test'.
> There were failing tests. See the report at: file:///home/stefan/workspace/jelovirt/dita-ot/build/reports/tests/index.html

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.

BUILD FAILED

Total time: 47.264 secs

Failed tests:

DebugAndFilterModuleTest. testGeneratedFiles
java.lang.AssertionError: Error message was thrown: java.io.FileNotFoundException: /home/stefan/workspace/jelovirt/dita-ot/build/dtd/technicalContent/dtd/map.dtd
TestGenMapAndTopicListModule. classMethod
org.dita.dost.exception.DITAOTException: [DOTA069F][FATAL] Input file 'file:/home/stefan/workspace/jelovirt/dita-ot/build/resources/test/TestGenMapAndTopicListModule/src/maps/root-map-01.ditamap' cannot be located or read. Ensure that file was specified properly and that you have permission to access it.
Caused by: java.io.FileNotFoundException: /home/stefan/workspace/jelovirt/dita-ot/build/dtd/technicalContent/dtd/map.dtd
KeydefFilterTest. testParse
java.io.FileNotFoundException: /home/stefan/workspace/jelovirt/dita-ot/build/dtd/technicalContent/dtd/map.dtd
MapMetaReaderTest. classMethod
net.sf.saxon.trans.XPathException: java.io.FileNotFoundException: /home/stefan/workspace/jelovirt/dita-ot/topic.dtd
TestGenListModuleReader. testParse*
java.io.FileNotFoundException: /home/stefan/workspace/jelovirt/dita-ot/build/dtd/technicalContent/dtd/map.dtd
KeyrefPaserTest. classMethod
net.sf.saxon.trans.XPathException: java.io.FileNotFoundException: /home/stefan/workspace/jelovirt/dita-ot/topic.dtd

eerohele · 2015-07-27T12:52:25Z

The problem @xephon2 also encountered occurs because when Gradle runs the test target, the integrate Ant target in integrator.xml hasn't yet been executed, so catalog-dita.xml doesn't exist yet.

Something like this should do the trick:

diff --git a/build.gradle b/build.gradle
index 3443465..8377643 100644
--- a/build.gradle
+++ b/build.gradle
@@ -74,11 +74,15 @@ task copyInstall(type: Copy, dependsOn: initInstall) {
     from (configurations.runtime)
     destinationDir = file("src/main/lib")
 }
-task buildLocal(dependsOn: [copyInstall, ":fo:copyInstall"]) << {
-    description "Build archives and install all plugins with dependencies"
+
+task integrate(dependsOn: initInstall) << {
     ant.ant([antfile: "integrator.xml", dir: file("src/main")])
 }

+task buildLocal(dependsOn: [copyInstall, ":fo:copyInstall", integrate]) << {
+    description "Build archives and install all plugins with dependencies"
+}
+
 // Distribution

 def distTempDir = file("${buildDir}/tmp/dist")
@@ -194,4 +198,8 @@ task distTarGz(type: Tar, dependsOn: [jar, generateDocs]) {
 }
 task dist(dependsOn: [distZip, distTarGz]) << {
     // NOOP
+}
+
+test {
+    dependsOn integrate
 }
\ No newline at end of file

jelovirt · 2015-07-27T13:17:05Z

catalog-dita.xml should not be used by test, it's not a problem with the Gradle build, but rather with the tests themselves.

# Conflicts: # README.markdown

eerohele · 2015-07-27T13:45:08Z

catalog-dita.xml should not be used by test, it's not a problem with the Gradle build, but rather with the tests themselves.

Ah, I see — guess I haven't worked with the unit tests enough to have been aware that they don't use catalog-dita.xml.

So is the answer updating the paths to the DTDs in the unit test DITA files?

jelovirt · 2015-07-27T13:49:40Z

Unit tests should work now without needing to run integration.

eerohele · 2015-07-27T13:52:17Z

Works great for me now.

jelovirt · 2015-07-27T14:00:06Z

Reverted back to FOP 1.1 for compiling fo.jar. I had some dependency issues with FOP 1.1 and 2.0 seemed to work without any issues, but we should still wait for #1958 to upgrade to FOP 2.0

stefan-jung · 2015-07-27T14:22:51Z

I still had issues with gradle buildand 8ff0596

FAILURE: Build failed with an exception.

* What went wrong:
Could not resolve all dependencies for configuration ':fo:compile'.
> Could not find org.apache.avalon.framework:avalon-framework-api:4.2.0.
  Searched in the following locations:
      https://repo1.maven.org/maven2/org/apache/avalon/framework/avalon-framework-api/4.2.0/avalon-framework-api-4.2.0.pom
      https://repo1.maven.org/maven2/org/apache/avalon/framework/avalon-framework-api/4.2.0/avalon-framework-api-4.2.0.jar
  Required by:
      org.dita-ot.pdf:fo:unspecified > org.apache.xmlgraphics:fop:1.1
> Could not find org.apache.avalon.framework:avalon-framework-impl:4.2.0.
  Searched in the following locations:
      https://repo1.maven.org/maven2/org/apache/avalon/framework/avalon-framework-impl/4.2.0/avalon-framework-impl-4.2.0.pom
      https://repo1.maven.org/maven2/org/apache/avalon/framework/avalon-framework-impl/4.2.0/avalon-framework-impl-4.2.0.jar
  Required by:
      org.dita-ot.pdf:fo:unspecified > org.apache.xmlgraphics:fop:1.1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.

BUILD FAILED

Total time: 15.384 secs

I added the following line to the dependencies in build.gradle, now the build is successful:

compile group: 'org.apache.avalon.framework', name: 'avalon-framework-impl', version:'4.3.1'

jelovirt · 2015-07-27T14:26:53Z

@xephon2 pull the latest changes, that's not an issue any more.

stefan-jung · 2015-07-27T14:32:38Z

Oh, I was too slow. You're right. The build is passing.

Add Gradle based build

Local code must be integrated before dist is ran to set up configuration correctly.

Add Gradle based build

440893d

Replaces Ant+Ivy with Gradle based build environment.

jelovirt added the feature New feature or request label Jul 25, 2015

jelovirt self-assigned this Jul 25, 2015

Fix PDF plugin build

da41c76

Add dummy RenderX Java sources to allow compiling PDF2 with XEP support

f937ace

jelovirt added 2 commits July 27, 2015 16:19

Copy Ant launcher JAR without version number

0616090

Merge branch 'develop' into feature/gradle

9df29dc

# Conflicts: # README.markdown

Merge branch 'develop' into feature/gradle

e436b55

Use FOP 1.1 for compiling PDF2

8ff0596

jelovirt added 4 commits July 27, 2015 17:09

Remove redundant FOP compile dependency from PDF

041c492

Update Travis CI configuration to use Gradle

7ec4c7d

Upgrade to container-based Travis CI

c9091e6

Correct Travis CI configuration

9f8ad70

jelovirt added 2 commits July 27, 2015 20:33

Merge branch 'develop' into feature/gradle

f70e6d1

Fix integration tests

1647dbd

jelovirt added a commit that referenced this pull request Jul 27, 2015

Merge pull request #1981 from jelovirt/feature/gradle

bf002c0

Add Gradle based build

jelovirt merged commit bf002c0 into dita-ot:develop Jul 27, 2015

jelovirt deleted the feature/gradle branch July 27, 2015 21:16

jelovirt added a commit that referenced this pull request Jul 28, 2015

Refactor dist to be two step process #1981

f31f49f

Local code must be integrated before dist is ran to set up configuration correctly.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add Gradle based build #1981

Add Gradle based build #1981

jelovirt commented Jul 25, 2015

jelovirt commented Jul 25, 2015

eerohele commented Jul 27, 2015

jelovirt commented Jul 27, 2015

jelovirt commented Jul 27, 2015

eerohele commented Jul 27, 2015

eerohele commented Jul 27, 2015

jelovirt commented Jul 27, 2015

stefan-jung commented Jul 27, 2015

eerohele commented Jul 27, 2015

jelovirt commented Jul 27, 2015

eerohele commented Jul 27, 2015

jelovirt commented Jul 27, 2015

eerohele commented Jul 27, 2015

jelovirt commented Jul 27, 2015

stefan-jung commented Jul 27, 2015

jelovirt commented Jul 27, 2015

stefan-jung commented Jul 27, 2015

Add Gradle based build #1981

Add Gradle based build #1981

Conversation

jelovirt commented Jul 25, 2015

jelovirt commented Jul 25, 2015

eerohele commented Jul 27, 2015

jelovirt commented Jul 27, 2015

jelovirt commented Jul 27, 2015

eerohele commented Jul 27, 2015

eerohele commented Jul 27, 2015

jelovirt commented Jul 27, 2015

stefan-jung commented Jul 27, 2015

eerohele commented Jul 27, 2015

jelovirt commented Jul 27, 2015

eerohele commented Jul 27, 2015

jelovirt commented Jul 27, 2015

eerohele commented Jul 27, 2015

jelovirt commented Jul 27, 2015

stefan-jung commented Jul 27, 2015

jelovirt commented Jul 27, 2015

stefan-jung commented Jul 27, 2015