Overview
The page goes into details about how the build system is configured.
Since 5.5, OMERO decouples many components and uses, for some components, an Gradle-based build. The two overarching repositories are :omero_subs_github_repo_root:omero-build and :omero_subs_github_repo_root:omero-gradle-plugins. See the README of each repository for more details. OMERO still uses an Ant-based build, for some components, with dependency management provided by Ivy. C++ code </developers/Cpp>
is built using Cmake and Python uses the traditional distutils/setuptools tools.
This is an (abbreviated) snapshot of the structure of the filesystem for OMERO:
OMERO_SOURCE_PREFIX
|
|-- build.xml .......................... Top-level build file
|
|-- build.py ........................... Python wrapper to handle OS-specific configuration
|
|-- omero.class ........................ Self-contained Ant launcher
|
|--etc ................................. Configuration folder
| |-- grid ........................... Deployment files folder
| |-- ivysettings.xml ................ Main Ivy configuration file
| |-- hibernate.properties
| |-- build.properties
| |-- logback.xml
| |-- omero.properties
| \-- profiles
|
|-- examples ........................... User examples
|
\components
|
|--<component-name> .................. Each component has this same basic structure.
| |-- build.xml ................... Build file
| |-- ivy.xml ..................... Jar dependencies
| |-- test.xml .................... Test dependencies
| |-- src ......................... Source code
| |-- resources ................... Other files of interest
| |-- test ........................ Test source code and test resources
| \-- target ...................... Build output (deleted on clean)
|
| NOTABLE COMPONENTS
|
|--tools ............................. Other server-components with special build needs.
| |--build.xml .................... Build scripts
| |
| \--<tool-name>
| |--build.xml ............... Build file
| \--ivy.xml ................. Jar dependencies
|
\--antlib ............................ Special component which is not built, but referenced by the build
|
\--resources ..................... Build resources
|--global.xml ................ Global build properties
|--hibernate.xml
|--lifecycle.xml ............. Ivy-related targets
\--version.xml ............... Version properties
Note
User examples are explained under /developers/GettingStarted
Unfortunately, just the above snapshot of the code repository omits some of the most important code. Many megabytes of source code is generated both by our own :dsl_plugin_sourcedir:DSLTask <> as well as by the Ice slice2java
, slice2cpp
, and slice2py
code generators. These take an intermediate representation of the :model_doc:OME-Model <ome-xml/> and generate our . This code is not available in git, but once built, can be found in all the directories named "generated".
./build.py
is a complete replacement for your local ant install. In many cases, you will be fine running ant
. If you have any issues (for example OutOfMemory
) , please use ./build.py
instead. However, only use one or the other; do not mix calls between the two.
The main build targets are defined in the top-level build.xml
file. All available targets can be listed using:
./build.py -p
The build system uses Ivy 2.3.0 as the dependency manager. The general Ivy configuration is defined in a settings file <settings.html>
located under etc/ivysettings.xml
.
In order to determine the transitive closure of all dependencies, Ivy resolves each ivy.xml
file and stores the resolved artifacts in a cache <settings/caches/cache.html>
to speed up other processes. The OMERO build system defines and uses two kinds of caches:
- the local dependencies cache under
lib/cache
is used by most resolvers - Maven resolvers use the Maven cache under
~/.m2/repository
Note
When the Ivy configuration file or the version number is changed, the cache can become stale. Calling ./build.py clean
from the top-level build will delete the content of the local cache.
Resolvers <settings/resolvers.html>
are key to how Ivy functions. Multiple dependency resolvers can be defined fine-grained enough to resolve an individual jar in order to pick up the latest version of any library from a repository <resolver/ibiblio.html>
, a generic URL <resolver/url.html>
or from the local file system <resolver/filesystem.html>
. Since OMERO 5.1.3, the remote repository resolvers are set up to resolve transitive dependencies.
The OMERO build system uses by default a chain resolver <resolver/chain.html>
called omero-resolver
which resolves the following locations in order:
target/repository
which contains most artifacts published by the build system in the install step of the lifecycle- the local dependency repository under
lib/repository
- the local Maven cache under
~/.m2/repository
- the Maven central repository
- the OME artifactory
Bio-Formats dependencies are resolved using a specific chain resolver <resolver/chain.html>
called ome-resolver
which resolves the following locations in order:
- the local Maven cache under
~/.m2/repository
- the OME artifactory
To define its dependencies, each component uses a top-level Ivy file <ivyfile.html>
, ivy.xml
, for the build and optionally another Ivy file, test.xml
, for the tests.
The OMERO build system defines and uses four types of Ivy configurations <ivyfile/configurations.html>
:
- build: defines dependencies to be used for building
- server: defines dependencies to be bundled under
lib/server
- client: defines dependencies to be bundled under
lib/client
- test: defines dependencies to be used for running the tests
While building, most Java components follow the same lifecycle define in lifecycle.xml <components/antlib/resources/lifecycle.xml>
. The default dist target for each component calls each of the following steps in order:
- retrieve:
retrieve <use/retrieve.html>
the resolved dependencies and copy them undertarget/libs
- prepare: prepare various resources (property files,
lib/logback-build.xml
) - generate: copy all resources from the previous step for compilation
- compile: compile the source files into the destination repository
- package-extra: package the sources and the Javadoc into Jar files for publication
- package: package the compiled classes into a Jar file for publication
- install: convert the component Ivy file into a pom file using
makepom <use/makepom.html>
andpublish <use/publish.html>
the component artifacts
Individual components can override the content of this default lifecycle via their build.xml
.
The Ant build alone is not enough to describe all the products which get built. Namely, the builds for the non-Java components stored under components/tools
are a bit more complex. Each tools component installs its artifacts to the tools/target directory which is copied on top of the dist
top-level distribution directory.
The OME project currently uses Jenkins as a continuous integration server available here <>
, so many binary packages can be downloaded without compiling them yourself. See the :devs_doc:Continuous Integration documentation <ci-omero.html> for further details.
The default ant target (build-default
) will build the OMERO system and copy the necessary components for a binary distribution to the dist
directory. Below is a comparison of what is taken from the build, where it is put, and what role it plays in the distribution.
OMERO_SOURCE_PREFIX | OMERO_SOURCE_PREFIX/dist | Comments |
---|---|---|
components/tools/OmeroCpp/lib* | lib/ |
Native shared libraries |
lib/repository/<some> | lib/client & lib/server |
Libraries needed for the build |
etc/ | etc/ |
Configuration |
sql/*.sql |
sql/ |
SQL scripts to prepare the database |
Note
By default, are not built. Use build-all
for that.
These files are then zipped to OMERO.server-<version>.zip via release-zip
Since OMERO 5.1.3, Bio-Formats is decoupled from the OMERO build system which consumes Bio-Formats artifacts from the OME Maven repository via Ivy.
While this decoupling matches most of the development use cases, it is sometimes necessary to work on coupled Bio-Formats and OMERO branches especially during breaking changes of the OME Data Model or the Bio-Formats API.
The general rule for coupled branches is to build each component in their dependency order and use the local Maven repository under ~/.m2/repository
to share artifacts.
From the top-level folder of the Bio-Formats repository,
if necessary, adjust the version of Bio-Formats which will be built, installed locally and consumed by OMERO e.g. for 5.2.0-SNAPSHOT:
$ ./tools/bump_maven_version.py 5.2.0-SNAPSHOT
run the Maven command allowing to build and install the artifacts under the local Maven cache:
$ mvn clean install
From the top-level folder of the OMERO repository,
- in :omero_subs_github_repo_root:omero-model, adjust the version of
ome:formats-gpl
in :model_source:build.gradle to the version chosen for the Bio-Formats build - publish locally the change using
gradle publishToMavenLocal