Guidelines for contributing to Hibernate Search

Contributions from the community are essential in keeping Hibernate Search strong and successful.

This guide focuses on how to contribute back to Hibernate Search using GitHub pull requests.

Legal

All original contributions to Hibernate Search are licensed under the GNU Lesser General Public License (LGPL), version 2.1 or later, or, if another license is specified as governing the file or directory being modified, such other license. The LGPL text is included verbatim in the lgpl.txt file in the root directory of the repository.

All contributions are subject to the Developer Certificate of Origin (DCO). The DCO text is also included verbatim in the dco.txt file in the root directory of the repository.

Contributing a bug report

If you want to see something fixed, but are not comfortable enough to dig into the codebase, you can help us by providing a well-documented bug report:

Open a bug report on our JIRA instance. Make sure to provide enough information, in particular: the code you wrote, the expected result, the result you got instead, the version of your dependencies.
Ideally (and this helps a lot), provide a self-contained test case. We provide test case templates for all Hibernate projects to help you get started: just fork this repository, build your test case and attach it as an archive to a JIRA issue.

Setting up a development environment

Build tools

You will need JDK 17 exactly for the build.

A maven wrapper script is provided at the root of the repository (./mvnw), so you can use that and don't need to care about the required version of Maven (it will be downloaded automatically).

IDE

IntelliJ IDEA

WARNING: Avoid running ./mvnw while IntelliJ IDEA is importing/building, and ideally avoid using Maven from the command line at all while IntelliJ IDEA is open. IntelliJ IDEA's own build might conflict with the Maven build, leaving your working directory in an undetermined state (some classes being generated twice, ...). If you already did that, close IntelliJ IDEA, run ./mvnw clean, and open IntelliJ IDEA again.

You will need to change some settings:

Build, Execution, Deployment > Build Tools > Maven: set Maven home path to Use Maven wrapper
In Project structure, make sure the project JDK is JDK 17.
Set up formatting rules and code style.

Then a few steps will initialize your workspace:

In the "Maven" side panel, click "Reload all Maven projects".
To check your setup, click Build > Rebuild Project. You might get a few errors similar to java: module not found: org.hibernate.search.mapper.orm; those are caused by limitations of IntelliJ IDEA and can be safely ignored. If the build has no other error, your workspace is correctly set up.
If you encounter any problem, that might be caused by the project being half-built before you started. Try again from a clean state: close IntelliJ IDEA, run ./mvnw clean, open IntelliJ IDEA again, and go back to the first step.

Eclipse

Eclipse shouldn't require any particular setup besides formatting rules and code style.

Formatting rules and style conventions

The Hibernate family projects share the same style conventions, and we provide settings for some IDEs to help you follow these conventions. See:

Contributing code

Prerequisites

If you are just getting started with Git, GitHub and/or contributing to Hibernate Search there are a few prerequisite steps:

Make sure you have a Hibernate JIRA account
Make sure you have a GitHub account
Fork the Hibernate Search repository. As discussed in the linked page, this also includes:
- Setting up your local git install
- Cloning your fork

Development environment

Make sure to set up your development environment correctly.

Be especially careful about setting up the formatting rules and code style.

If you built the project at least once (./mvnw clean install), you can very quickly check that you have respected the formatting rules by running Checkstyle:

./mvnw checkstyle:check -fn

Create a topic branch

Create a "topic" branch on which you will work. The convention is to name the branch using the JIRA issue key. If there is not already a JIRA issue covering the work you want to do, create one. Assuming you will be working from the main branch and working on the JIRA HSEARCH-123:

git checkout -b HSEARCH-123 main

Code

See this section for details about the structure of the source code, and this section for how to build the project.

If you need help, feel free to contact us, be it through comments on your JIRA ticket, emails on the mailing list, or directly though our chat: see here for more information.

Commit

Make commits of logical units.
Be sure to start the commit messages with the key of the JIRA issue you are working on. This is how JIRA will pick up the related commits and display them on the JIRA issue.
Avoid formatting changes to existing code as much as possible: they make the intent of your patch less clear.

Prior to committing, if you want to pull in the latest upstream changes (highly appreciated by the way), please use rebasing rather than merging (see instructions below). Merging creates "merge commits" that really muck up the project timeline.

Add the original Hibernate Search repository as a remote repository called upstream:

git remote add upstream https://github.com/hibernate/hibernate-search.git

If you want to rebase your branch on top of the main branch, you can use the following git command:

git pull --rebase upstream main

Check and test your work

Before submitting a pull requests, check your contribution:

Make sure you have added the necessary tests for your changes.
If relevant, make sure you have updated the documentation to match your changes.
Run the relevant tests once again to check that your changes work as expected. No need to run the whole test suite, the Continuous Integration will take care of that.

Note: If you want to run specific tests of the integrationtests/backend/tck module from the IDE, you will need to rely on runner classes to run them in the appropriate context: see org.hibernate.search.integrationtest.backend.elasticsearch.testsupport.util.ElasticsearchTckTestRunner for Elasticsearch, or org.hibernate.search.integrationtest.backend.lucene.testsupport.util.LuceneTckTestRunner for Lucene.

Submit

Push your changes to a topic branch in your fork of the repository.
Initiate a pull request.
Update the JIRA issue, using the "Link to pull request" button to include a link to the created pull request.

Source code structure

The project is split in several Maven modules:

build: Various modules that are mostly useful for the build itself.
backend: The backends, i.e. the modules that provide integration to actual indexing services.
- elasticsearch: A backend that connects to a remote Elasticsearch cluster.
- elasticsearch-aws: Implementation of AWS authentication using request signing for the Elasticsearch backend.
- lucene: A backend that uses an embedded (same JVM) Lucene instance.
build-config: Code-related artifacts like checkstyle and forbiddenapis rules.
distribution: Builds the distribution package.
documentation: The project documentation.
engine: The Hibernate Search engine. This module handles most of the basic integration work (configuration properties, bean instantiation, ...), defines APIs common to every mapper/backend (the Search DSL in particular), and provides the "glue" between mappers and backends.
integrationtest: Integration tests for backends (Elasticsearch, Lucene) and mappers (Hibernate ORM), as well as any other technology Hibernate Search integrates with. Here are some notable sub-directories:
- performance: performance tests.
- showcase/library: a sample application using Hibernate Search in a Spring Boot environment.
jakarta: Modules that take the source code of other modules (e.g. mapper/orm) and transform it to use Jakarta EE instead of Java EE.
mapper: The mappers, i.e. the modules that expose APIs to index and search user entities, and do the work of converting between user entities and documents to be indexed.
- pojo-base: Contains base classes and APIs that are re-used in other POJO-based mapper.
- orm: A mapper for Hibernate ORM entities.
- orm-coordination-outbox-polling: An implementation of coordination of automatic indexing between nodes in the orm mapper (see above) using an outbox, i.e. an event table in the database.
- pojo-standalone: A mapper for POJOs in standalone mode, i.e. without Hibernate ORM. Currently incubating, i.e. backwards-incompatible changes in APIs may happen.
orm6: Modules that take the source code of other modules (e.g. mapper/orm) and transform it to use Hibernate ORM 6 instead of Hibernate ORM 5.x.
util: Various modules containing util classes, both for runtime and for tests.

Building from source

Basic build from the commandline

First, make sure your development environment is correctly set up.

The following command will build Hibernate Search, install it in your local Maven repository, and run unit tests and integration tests.

./mvnw clean install

Note: the produced JARs are compatible with Java 8 and later, regardless of the JDK used to build Hibernate Search.

WARNING: Avoid using other goals unless you know what you're doing, because they may leave your workspace in an undetermined state and lead to strange errors. In particular, ./mvnw compile will not build tests and may skip some post-processing of classes, and ./mvnw package will not install the JARs into your local Maven repository which might be a problem for some of the Maven plugins used in the build. If you did run those commands and are facing strange errors, you'll have to close your IDE then use ./mvnw clean to get back to a clean state.

Building without running tests

To only build Hibernate Search, without running tests, use the following command:

./mvnw clean install -DskipTests

Documentation

The documentation is based on Asciidoctor.

To generate the documentation only, without running tests, use:

./mvnw clean install -pl documentation -am -DskipTests

You can then find the freshly built documentation at the following location:

./documentation/target/dist/

By default only the HTML output is enabled; to also generate the PDF output, enable the documentation-pdf profile:

./mvnw clean install -pl documentation -am -DskipTests -Pdocumentation-pdf

Distribution

To build the distribution bundle, enable the documentation-pdf and dist profiles:

./mvnw clean install -Pdocumentation-pdf,dist

Or if you don't want to run tests:

./mvnw clean install -Pdocumentation-pdf,dist -DskipTests

Other JDKs

To test Hibernate Search against another JDK than the one required for the build, you will need to have both JDKs installed, and then you will need to pass additional properties to Maven.

To test Hibernate Search against JDK 8:

./mvnw clean install -Djava-version.test.release=8 -Djava-version.test.launcher.java_home=/path/to/jdk8

To test Hibernate Search against JDKs other than 8 or the default 17:

./mvnw clean install -Djava-version.test.release=11 -Djava-version.test.compiler.java_home=/path/to/jdk11

Or more simply, if the newer JDK you want to test against is newer than 17 and is your default JDK:

./mvnw clean install -Djava-version.test.release=18

Elasticsearch

The Elasticsearch integration tests run against one single version of Elasticsearch at a time, launching an Elasticsearch server automatically on port 9200 using Docker. You may redefine the distribution/version to use by specifying the properties test.elasticsearch.distribution/test.elasticsearch.version:

./mvnw clean install -Dtest.elasticsearch.distribution=elastic -Dtest.elasticsearch.version=6.0.0

The following distribution options are supported:

elastic - for Elasticsearch distribution
opensearch - for Opensearch distribution

For available versions of Elasticsearch distribution from Elastic see DockerHub. Please note that Elasticsearch distributions starting with version 7.11 are not open-source.

For available versions of OpenSearch distribution see DockerHub.

Alternatively, you can prevent the build from launching an Elasticsearch server automatically and run Elasticsearch-related tests against your own server using the test.elasticsearch.connection.uris property:

./mvnw clean install -Dtest.elasticsearch.connection.uris=http://localhost:9200

If you want to use HTTPS:

./mvnw clean install -Dtest.elasticsearch.connection.uris=https://localhost:9200

If you want to run tests against a different Elasticsearch version (6.x for instance), you will still have to specify the distribution and version:

./mvnw clean install -Dtest.elasticsearch.distribution=elastic -Dtest.elasticsearch.version=6.0.0 \
        -Dtest.elasticsearch.connection.uris=http://localhost:9200

You may also use authentication:

./mvnw clean install -Dtest.elasticsearch.connection.uris=http://localhost:9200 \
        -Dtest.elasticsearch.connection.username=ironman \
        -Dtest.elasticsearch.connection.password=j@rV1s

Also, the elasticsearch integration tests can be executed against an Elasticsearch service on AWS. You will need to execute something along the lines of:

./mvnw clean install -Dtest.elasticsearch.connection.uris=http://<host:port> \
        -Dtest.elasticsearch.connection.aws.signing.enabled=true \
        -Dtest.elasticsearch.connection.aws.region=<Your AWS region ID> \
        -Dtest.elasticsearch.connection.aws.credentials.type=static \
        -Dtest.elasticsearch.connection.aws.credentials.access_key_id=<Your access key ID> \
        -Dtest.elasticsearch.connection.aws.credentials.secret_access_key=<Your secret access key>

Or more simply, if your AWS credentials are already stored in ~/.aws/credentials:

./mvnw clean install -Dtest.elasticsearch.connection.uris=http://<host:port> \
        -Dtest.elasticsearch.connection.aws.signing.enabled=true \
        -Dtest.elasticsearch.connection.aws.region=<Your AWS region ID>

JQAssistant

You can request static analysis and sanity checks with the jqassistant profile. Tests do not need to be run for these checks.

./mvnw clean install -Pjqassistant -DskipTests

To also check cyclic dependencies between packages, use -Djqassistant.groups=default,cycles. Cyclic dependency analysis is costly and may add significant overhead to the build: at least 10 seconds, maybe one minute or more depending on your setup.

./mvnw clean install -Pjqassistant -DskipTests -Djqassistant.groups=default,cycles

You can also inspect the created Neo4j datastore after a build, provided that build had the jqassistant profile enabled:

./mvnw jqassistant:server -Pjqassistant

The Neo4j web UI will be accessible from http://localhost:7474/.

Continuous integration

Continuous integration happens on a self-hosted Jenkins instance at https://ci.hibernate.org.

Several multi-branch pipelines are available.

Main pipeline

https://ci.hibernate.org/job/hibernate-search/

See Jenkinsfile.

This job takes care of:

Primary branch builds
Pull request builds

It executes the build in a default environment, at the very least. For primary branches, it may also re-execute the same build in different environments:

Newer JDKs
Different database vendors (PostgreSQL, Oracle, ...)
Different versions of Elasticsearch/OpenSearch
AWS Elasticsearch/OpenSearch Service

See this section for information on how to execute similar builds from the commandline.

The job can be triggered manually, which is particularly useful to test more environments on a pull request.

Release pipeline