picturesafe-search

An Elasticsearch service wrapper

picturesafe-search is a Java service wrapper for the search engine Elasticsearch.

It allows the fast, flexible and easy integration of Elasticsearch functions into new or existing Java applications.

The following features are included:

• Functions for creating and maintaining Elasticsearch indices

  • Create field definitions

  • Create and delete Elasticsearch indices

  • Add and remove documents

• Query API for creating simple and complex (nested) search queries.

  • Fulltext queries

  • Field queries

  • Complex queries

• Filter builder API for easy implementation of customized search filters

• Aggregation builder API for easy implementation of customized aggregation builders (facets)

Make Elasticsearch queries easy

Elasticsearch is a very powerful search engine, but has a high complexity and requires a long learning curve.

The following example compares the execution of the logical search expression fulltext contains "(test && title)" and count >= 102 sort by id between the Elasticsearch REST API, the Elasticsearch Java High Level REST Client and the picturesafe-search Java service wrapper:

Elasticsearch REST API

POST /picturesafe-search-sample/_search

{
  "query": {
    "bool": {
      "must": [{
        "bool": {
          "filter": [{
            "query_string": {
              "query": "(test && title)",
              "fields": ["fulltext^1.0"]
            }
          }]
        }
      }],
      "filter": [{
        "range": {
          "count": {
            "from": 102,
            "to": null
          }
        }
      }]
    }
  },
  "sort": [{
    "id.keyword": {
      "order": "desc",
      "missing": "_last"
    }
  }]
}

Elasticsearch Java High Level REST Client

SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder()
    .query(QueryBuilders.boolQuery()
    .must(QueryBuilders.queryStringQuery("test && title")
            .field("fulltext")
            .defaultOperator(Operator.AND))
    .filter(QueryBuilders.rangeQuery("count").gte(102)))
    .sort(SortBuilders
            .fieldSort("id.keyword")
            .missing("_last")
            .order(SortOrder.DESC));

SearchRequest searchRequest = new SearchRequest("picturesafe-search-sample");
searchRequest.source(searchSourceBuilder);
SearchResponse searchResponse = new RestClientSearchAction()
    .action(restHighLevelClient, searchRequest);

picturesafe-search Java service wrapper

SearchResult searchResult = elasticsearchService.search("picturesafe-search-sample",
    OperationExpression.and(
        new FulltextExpression("test title"),
        new ValueExpression("count", ValueExpression.Comparison.GE, 102)),
    SearchParameter.builder().sortOptions(SortOption.desc("id")).build());

As you can see, picturesafe-search focuses more on WHAT to search for than on HOW to search it and abstracts some complexity.

Getting started

Only a few steps are necessary to get started with a standard configuration. A complete code example can be found here. If you want to create a Spring Boot application, you can alternatively use the picturesafe-search Starter.

For more information, please see following documentation.

Start Elasticsearch

picturesafe-search requires a running Elasticsearch server from version 7.x.

Local installation of Elasticsearch

For a new application an Elasticsearch server must be installed first:

• Download and unpack the Elasticsearch official distribution.

• Run bin/elasticsearch on Linux or macOS. Run bin\elasticsearch.bat on Windows.

Some features of picturesafe-search (for example sorting documents in a language-specific word order) require the ICU Analysis Plugin for Elasticsearch:

• Run bin/elasticsearch-plugin install analysis-icu on Linux or macOS. Run bin\elasticsearch-plugin install analysis-icu on Windows.

Run Elasticsearch in a Docker container

As an alternative to installing Elasticsearch you can run it in a Docker container. To do this you can use the provided Docker Compose file.

• Install Docker and Docker Compose.

• Clone the picturesafe-search GitHub repository.

• Run docker-compose -f src/test/docker/docker-compose.yml up -d from the project directory to start Elasticsearch.

• To stop Elasticsearch run docker-compose -f src/test/docker/docker-compose.yml stop from the project directory.

Include java library

Add the current version of the picturesafe-search library to your project.

Maven dependency

<dependency>
    <groupId>de.picturesafe.search</groupId>
    <artifactId>picturesafe-search</artifactId>
    <version>3.6.0-SNAPSHOT</version>
</dependency>

Configuration

Configuration bean

Implement a configuration class that imports the DefaultElasticConfiguration.class. This configuration can be extended later.

The following example defines three fields for the Elasticsearch index:

• Field 'id' (Elasticsearch type integer, sortable)

• Field 'fulltext' (Elasticsearch type text)

• Field 'title' (Elasticsearch type text, within fulltext, aggregatable, sortable)

Spring configuration

@Configuration
@ComponentScan(basePackages = {"de.picturesafe.search.elasticsearch"})
@Import({DefaultElasticConfiguration.class})
public class Config {

    @Bean
    List<FieldConfiguration> fieldConfigurations() {
        return Arrays.asList(
            FieldConfiguration.ID_FIELD,
            FieldConfiguration.FULLTEXT_FIELD,
            StandardFieldConfiguration.builder(
                    "title", ElasticsearchType.TEXT).copyToFulltext(true).sortable(true).build(),
            StandardFieldConfiguration.builder(
                    "count", ElasticsearchType.INTEGER).sortable(true).build()
        );
    }
}

Configuration properties

Add a file elasticsearch.properties to the classpath of your application and define the following key:

Property file

elasticsearch.index.alias=picturesafe-search-sample

This configuration can be extended later, see src/main/resources/elasticsearch.template.properties.

Service implementation

Inject the SingleIndexElasticsearchService and implement an expression-based search:

• Create an Elasticsearch index with alias

• Add some documents to the index

• Create an OperationExpression with two terms

• Run the search query

• Delete the Elasticsearch index

If you want to implement searches for more than one index, please use ElasticsearchService instead of SingleIndexElasticsearchService.

Spring service implementation

@Component
@ComponentScan
public class GettingStarted {

    private static final Logger LOGGER = LoggerFactory.getLogger(GettingStarted.class);

    @Autowired
    private SingleIndexElasticsearchService singleIndexElasticsearchService;

    public static void main(String[] args) {
        try (AnnotationConfigApplicationContext ctx
                = new AnnotationConfigApplicationContext(GettingStarted.class)) {
            final GettingStarted gettingStarted = ctx.getBean(GettingStarted.class);
            gettingStarted.run();
        }
    }

    private void run() {
        try {
            singleIndexElasticsearchService.createIndexWithAlias();

            singleIndexElasticsearchService
                    .addToIndex(DataChangeProcessingMode.BLOCKING, Arrays.asList(
                    DocumentBuilder.id(1).put("title", "This is a test title")
                        .put("count", 101).build(),
                    DocumentBuilder.id(2).put("title", "This is another test title")
                        .put("count", 102).build(),
                    DocumentBuilder.id(3).put("title", "This is one more test title")
                        .put("count", 103).build()
            ));

            final Expression expression = OperationExpression.and(
                    new FulltextExpression("test title"),
                    new ValueExpression("count", ValueExpression.Comparison.GE, 102));

            final SearchResult searchResult = singleIndexElasticsearchService
                .search(expression, SearchParameter.DEFAULT);

            LOGGER.info(searchResult.toString());
        } finally {
            singleIndexElasticsearchService.deleteIndexWithAlias();
        }
    }
}

With implementations of the picturesafe-search Expression-Interface complex terms of different search conditions can be easily defined.

Here are some examples:

Simple fulltext search

Expression expression = new FulltextExpression("test title");

Simple field search

Expression expression = new ValueExpression("title", "test");

Simple field search with comparison operator

Expression expression = new ValueExpression("count", ValueExpression.Comparison.GE, 102);

Search with two terms

Expression expression = OperationExpression.and(
        new FulltextExpression("test title"),
        new ValueExpression("count", ValueExpression.Comparison.GE, 102));

In addition there are further expressions like InExpression, MustNotExpression, RangeValueExpression, DayExpression, more

Building picturesafe-search

If you want to build picturesafe-search yourself there are two prerequisites:

JDK

You need to have installed a Java Development Kit. The picturesafe-search project is currently developed using Java 8, but has also been tested on Java 11.

Note when using Java 11:
There is a JavaDoc related bug which has not been fixed in Adopt or Corretto OpenJDK at the moment. If you are using OpenJDK 11 and you are facing a build error like

Failed to execute goal org.apache.maven.plugins:maven-javadoc-plugin:3.2.0:jar (attach-javadocs) on project picturesafe-search: MavenReportException: Error while generating Javadoc:

[ERROR] Exit code: 1 - javadoc: error - The code being documented uses packages in the unnamed module, but the packages defined in https://docs.oracle.com/en/java/javase/11/docs/api/ are in named modules.

, please skip generating JavaDoc until the fix has become part of the OpenJDK build you are using.

Skipping the JavaDoc generation:

mvn -Dmaven.javadoc.skip=true install

Alternatively you could use the OpenJDK 11 reference build provided by Oracle, which has the fix included.

Side note on java modules:
We are not able to provide a module-info.java at the moment, because we are using the Elasticsearch high level rest client which has the monolithic elasticsearch.jar as dependency. The elasticsearch.jar has no module-info and it makes auto module detection impossible because of its internal structure. Please see this issue for details.

Apache Maven

You also need to have installed Apache Maven version 3.6.

Build

Change to the project directory and run the following command in your shell:

mvn install