Skip to content

DATAES-931 - Add query support for geo shape queries. #542

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Oct 21, 2020
Merged

DATAES-931 - Add query support for geo shape queries. #542

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 6 additions & 4 deletions src/main/asciidoc/reference/elasticsearch-new.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,12 @@
[[new-features.4-1-0]]
== New in Spring Data Elasticsearch 4.1

* Upgrade to Elasticsearch 7.9.2
* Improved API for alias management
* Introduction of `ReactiveIndexOperations` for index management
* Index templates support
* Uses Spring 5.3.
* Upgrade to Elasticsearch 7.9.2.
* Improved API for alias management.
* Introduction of `ReactiveIndexOperations` for index management.
* Index templates support.
* Support for Geo-shape data with GeoJson.

[[new-features.4-0-0]]
== New in Spring Data Elasticsearch 4.0
Expand Down
67 changes: 46 additions & 21 deletions src/main/asciidoc/reference/elasticsearch-object-mapping.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,17 +3,19 @@

Spring Data Elasticsearch Object Mapping is the process that maps a Java object - the domain entity - into the JSON representation that is stored in Elasticsearch and back.

Earlier versions of Spring Data Elasticsearch used a Jackson based conversion, Spring Data Elasticsearch 3.2.x introduced the <<elasticsearch.mapping.meta-model>>. As of version 4.0 only the Meta Object Mapping is used, the Jackson based mapper is not available anymore and the `MappingElasticsearchConverter` is used.
Earlier versions of Spring Data Elasticsearch used a Jackson based conversion, Spring Data Elasticsearch 3.2.x introduced the <<elasticsearch.mapping.meta-model>>.
As of version 4.0 only the Meta Object Mapping is used, the Jackson based mapper is not available anymore and the `MappingElasticsearchConverter` is used.

The main reasons for the removal of the Jackson based mapper are:

* Custom mappings of fields needed to be done with annotations like `@JsonFormat` or `@JsonInclude`. This often caused problems when the same object was used in different JSON based datastores or sent over a JSON based API.
* Custom field types and formats also need to be stored into the Elasticsearch index mappings. The Jackson based annotations did not fully provide all the information that is necessary to represent the types of Elasticsearch.
* Custom mappings of fields needed to be done with annotations like `@JsonFormat` or `@JsonInclude`.
This often caused problems when the same object was used in different JSON based datastores or sent over a JSON based API.
* Custom field types and formats also need to be stored into the Elasticsearch index mappings.
The Jackson based annotations did not fully provide all the information that is necessary to represent the types of Elasticsearch.
* Fields must be mapped not only when converting from and to entities, but also in query argument, returned data and on other places.

Using the `MappingElasticsearchConverter` now covers all these cases.


[[elasticsearch.mapping.meta-model]]
== Meta Model Object Mapping

Expand All @@ -23,33 +25,48 @@ This allows to register `Converter` instances for specific domain type mapping.
[[elasticsearch.mapping.meta-model.annotations]]
=== Mapping Annotation Overview

The `MappingElasticsearchConverter` uses metadata to drive the mapping of objects to documents. The metadata is taken from the entity's properties which can be annotated.
The `MappingElasticsearchConverter` uses metadata to drive the mapping of objects to documents.
The metadata is taken from the entity's properties which can be annotated.

The following annotations are available:

* `@Document`: Applied at the class level to indicate this class is a candidate for mapping to the database. The most important attributes are:
** `indexName`: the name of the index to store this entity in. This can contain a SpEL template expression like `"log-#{T(java.time.LocalDate).now().toString()}"`
** `type`: [line-through]#the mapping type. If not set, the lowercased simple name of the class is used.# (deprecated since version 4.0)
* `@Document`: Applied at the class level to indicate this class is a candidate for mapping to the database.
The most important attributes are:
** `indexName`: the name of the index to store this entity in.
This can contain a SpEL template expression like `"log-#{T(java.time.LocalDate).now().toString()}"`
** `type`: [line-through]#the mapping type.
If not set, the lowercased simple name of the class is used.# (deprecated since version 4.0)
** `shards`: the number of shards for the index.
** `replicas`: the number of replicas for the index.
** `refreshIntervall`: Refresh interval for the index. Used for index creation. Default value is _"1s"_.
** `indexStoreType`: Index storage type for the index. Used for index creation. Default value is _"fs"_.
** `createIndex`: flag whether to create an index on repository bootstrapping. Default value is _true_. See <<elasticsearch.repositories.autocreation>>
** `versionType`: Configuration of version management. Default value is _EXTERNAL_.
** `refreshIntervall`: Refresh interval for the index.
Used for index creation.
Default value is _"1s"_.
** `indexStoreType`: Index storage type for the index.
Used for index creation.
Default value is _"fs"_.
** `createIndex`: flag whether to create an index on repository bootstrapping.
Default value is _true_.
See <<elasticsearch.repositories.autocreation>>
** `versionType`: Configuration of version management.
Default value is _EXTERNAL_.

* `@Id`: Applied at the field level to mark the field used for identity purpose.
* `@Transient`: By default all fields are mapped to the document when it is stored or retrieved, this annotation excludes the field.
* `@PersistenceConstructor`: Marks a given constructor - even a package protected one - to use when instantiating the object from the database. Constructor arguments are mapped by name to the key values in the retrieved Document.
* `@PersistenceConstructor`: Marks a given constructor - even a package protected one - to use when instantiating the object from the database.
Constructor arguments are mapped by name to the key values in the retrieved Document.
* `@Field`: Applied at the field level and defines properties of the field, most of the attributes map to the respective https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping.html[Elasticsearch Mapping] definitions (the following list is not complete, check the annotation Javadoc for a complete reference):
** `name`: The name of the field as it will be represented in the Elasticsearch document, if not set, the Java field name is used.
** `type`: the field type, can be one of _Text, Keyword, Long, Integer, Short, Byte, Double, Float, Half_Float, Scaled_Float, Date, Date_Nanos, Boolean, Binary, Integer_Range, Float_Range, Long_Range, Double_Range, Date_Range, Ip_Range, Object, Nested, Ip, TokenCount, Percolator, Flattened, Search_As_You_Type_. See https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-types.html[Elasticsearch Mapping Types]
** `type`: the field type, can be one of _Text, Keyword, Long, Integer, Short, Byte, Double, Float, Half_Float, Scaled_Float, Date, Date_Nanos, Boolean, Binary, Integer_Range, Float_Range, Long_Range, Double_Range, Date_Range, Ip_Range, Object, Nested, Ip, TokenCount, Percolator, Flattened, Search_As_You_Type_.
See https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-types.html[Elasticsearch Mapping Types]
** `format` and `pattern` definitions for the _Date_ type. `format` must be defined for date types.
** `store`: Flag whether the original field value should be store in Elasticsearch, default value is _false_.
** `analyzer`, `searchAnalyzer`, `normalizer` for specifying custom analyzers and normalizer.
* `@GeoPoint`: marks a field as _geo_point_ datatype. Can be omitted if the field is an instance of the `GeoPoint` class.
* `@GeoPoint`: marks a field as _geo_point_ datatype.
Can be omitted if the field is an instance of the `GeoPoint` class.

NOTE: Properties that derive from `TemporalAccessor` must either have a `@Field` annotation of type `FieldType.Date` or a custom converter must be registered for this type. +
If you are using a custom date format, you need to use _uuuu_ for the year instead of _yyyy_. This is due to a https://www.elastic.co/guide/en/elasticsearch/reference/current/migrate-to-java-time.html#java-time-migration-incompatible-date-formats[change in Elasticsearch 7].
If you are using a custom date format, you need to use _uuuu_ for the year instead of _yyyy_.
This is due to a https://www.elastic.co/guide/en/elasticsearch/reference/current/migrate-to-java-time.html#java-time-migration-incompatible-date-formats[change in Elasticsearch 7].

The mapping metadata infrastructure is defined in a separate spring-data-commons project that is technology agnostic.

Expand All @@ -72,6 +89,7 @@ public class Person { <1>
String lastname;
}
----

[source,json]
----
{
Expand All @@ -84,10 +102,10 @@ public class Person { <1>
<1> By default the domain types class name is used for the type hint.
====

Type hints can be configured to hold custom information. Use the `@TypeAlias` annotation to do so.
Type hints can be configured to hold custom information.
Use the `@TypeAlias` annotation to do so.

NOTE: Make sure to add types with `@TypeAlias` to the initial entity set (`AbstractElasticsearchConfiguration#getInitialEntitySet`)
to already have entity information available when first reading data from the store.
NOTE: Make sure to add types with `@TypeAlias` to the initial entity set (`AbstractElasticsearchConfiguration#getInitialEntitySet`) to already have entity information available when first reading data from the store.

.Type Hints with Alias
====
Expand All @@ -100,6 +118,7 @@ public class Person {
// ...
}
----

[source,json]
----
{
Expand All @@ -126,6 +145,7 @@ public class Address {
Point location;
}
----

[source,json]
----
{
Expand All @@ -138,8 +158,9 @@ public class Address {

==== GeoJson Types

Spring Data Elasticsearch supports the GeoJson types by providing an interface `GeoJson` and implementations for the different geometries. They are mapped to Elasticsearch
documents according to the GeoJson specification. The corresponding properties of the entity are specified as `geo_shape` when the index mappings is written.
Spring Data Elasticsearch supports the GeoJson types by providing an interface `GeoJson` and implementations for the different geometries.
They are mapped to Elasticsearch documents according to the GeoJson specification.
The corresponding properties of the entity are specified in the index mappings as `geo_shape` when the index mappings is written. (check the https://www.elastic.co/guide/en/elasticsearch/reference/current/geo-shape.html[Elasticsearch documentation] as well)

.GeoJson types
====
Expand All @@ -151,6 +172,7 @@ public class Address {
GeoJsonPoint location;
}
----

[source,json]
----
{
Expand Down Expand Up @@ -190,6 +212,7 @@ public class Person {

}
----

[source,json]
----
{
Expand Down Expand Up @@ -217,6 +240,7 @@ public class Person {

}
----

[source,json]
----
{
Expand Down Expand Up @@ -283,6 +307,7 @@ public class Config extends AbstractElasticsearchConfiguration {
}
}
----

[source,json]
----
{
Expand Down
Loading