Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[NodeSearchBundle] Elastic search version 6 support #1875

Merged
merged 2 commits into from
Mar 29, 2018
Merged

[NodeSearchBundle] Elastic search version 6 support #1875

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
4d896d9
[NodeSearchBundle] Elastic search version 6 support
Mar 20, 2018
888c7e6
[NodeSearchBundle] Elastic search version 6 support
Mar 26, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
2 changes: 1 addition & 1 deletion composer.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@
"twig/extensions": "~1.0",
"egulias/email-validator": "~1.2",
"box/spout": "^2.5",
"ruflin/elastica": "^5.1",
"ruflin/elastica": "^5.1|^6.0",
"behat/transliterator": "~1.2",
"defuse/php-encryption": "v2.1.0"
},
Expand Down
33 changes: 27 additions & 6 deletions src/Kunstmaan/NodeSearchBundle/DependencyInjection/Configuration.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
use Symfony\Component\Config\Definition\Builder\ArrayNodeDefinition;
use Symfony\Component\Config\Definition\Builder\TreeBuilder;
use Symfony\Component\Config\Definition\ConfigurationInterface;
use Kunstmaan\NodeSearchBundle\Helper\ElasticSearchUtil;

/**
* This is the class that validates and merges configuration from your app/config files
Expand All @@ -28,16 +29,36 @@ public function getConfigTreeBuilder()
/** @var ArrayNodeDefinition $properties */
$properties = $rootNode->children()->arrayNode('mapping')->useAttributeAsKey('name')->prototype('array');

$properties->children()->scalarNode('type')->beforeNormalization()->ifNotInArray($types = [
'string', 'token_count', 'text', 'keyword',
$types = [
'token_count', 'text', 'keyword',
'float', 'double', 'byte', 'short', 'integer', 'long',
'date',
'boolean',
'binary',
])->thenInvalid('type must be one of: ' . implode(', ', $types));
$properties->children()->scalarNode('index')->beforeNormalization()->ifNotInArray(['analyzed', 'not_analyzed', 'no'])
->thenInvalid("index must be one of: analyzed, not_analyzed, no");
$properties->children()->booleanNode('include_in_all');
];
if (!ElasticSearchUtil::useVersion6()) {
$types[] = 'string';
}

$properties->children()->scalarNode('type')->beforeNormalization()->ifNotInArray($types)->thenInvalid('type must be one of: ' . implode(', ', $types));

if (ElasticSearchUtil::useVersion6()) {
$properties->children()->booleanNode('fielddata');
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was reading the https://www.elastic.co/guide/en/elasticsearch/reference/current/fielddata.html article and like they say, it's not a good idea to use this fielddata element because it will cause a lot of memory issues. We should use https://www.elastic.co/guide/en/elasticsearch/reference/current/doc-values.html.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, using doc_values seems a good alternative, however, I don't think we should prohibit it. We are not using fielddata, but allowing that it is used as a mapping value. I'll add the doc_values as well.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alright, super

$properties->children()->booleanNode('doc_values');
$properties->children()
->scalarNode('index')
->beforeNormalization()
->ifNotInArray(['true', 'false', true, false])
->thenInvalid("index must be one of: true, false");
} else {
$properties->children()
->scalarNode('index')
->beforeNormalization()
->ifNotInArray(['analyzed', 'not_analyzed', 'no'])
->thenInvalid("index must be one of: analyzed, not_analyzed, no");
$properties->children()->booleanNode('include_in_all');
}

$properties->children()->booleanNode('store');
$properties->children()->floatNode('boost');
$properties->children()->scalarNode('null_value');
Expand Down
164 changes: 102 additions & 62 deletions src/Kunstmaan/NodeSearchBundle/DependencyInjection/KunstmaanNodeSearchExtension.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@
use Symfony\Component\DependencyInjection\Extension\PrependExtensionInterface;
use Symfony\Component\DependencyInjection\Loader;
use Symfony\Component\HttpKernel\DependencyInjection\Extension;
use Kunstmaan\NodeSearchBundle\Helper\ElasticSearchUtil;

/**
* This is the class that loads and manages your bundle configuration
Expand Down Expand Up @@ -48,67 +49,106 @@ public function load(array $configs, ContainerBuilder $container)
*/
public function prepend(ContainerBuilder $container)
{
$container->prependExtensionConfig('kunstmaan_node_search', [
'mapping' => [
'root_id' => [
'type' => 'integer',
'include_in_all' => false,
'index' => 'not_analyzed'
],
'node_id' => [
'type' => 'integer',
'include_in_all' => false,
'index' => 'not_analyzed'
],
'nodetranslation_id' => [
'type' => 'integer',
'include_in_all' => false,
'index' => 'not_analyzed'
],
'nodeversion_id' => [
'type' => 'integer',
'include_in_all' => false,
'index' => 'not_analyzed'
],
'title' => [
'type' => 'string',
'include_in_all' => true
],
'slug' => [
'type' => 'string',
'include_in_all' => false,
'index' => 'not_analyzed'
],
'type' => [
'type' => 'string',
'include_in_all' => false,
'index' => 'not_analyzed'
],
'page_class' => [
'type' => 'string',
'include_in_all' => false,
'index' => 'not_analyzed'
],
'content' => [
'type' => 'string',
'include_in_all' => true
],
'created' => [
'type' => 'date',
'include_in_all' => false,
'index' => 'not_analyzed'
],
'updated' => [
'type' => 'date',
'include_in_all' => false,
'index' => 'not_analyzed'
],
'view_roles' => [
'type' => 'string',
'include_in_all' => true,
'index' => 'not_analyzed',
],
]
]);
if (ElasticSearchUtil::useVersion6()) {
$mapping = [
'mapping' => [
'root_id' => [
'type' => 'integer',
],
'node_id' => [
'type' => 'integer',
],
'nodetranslation_id' => [
'type' => 'integer',
],
'nodeversion_id' => [
'type' => 'integer',
],
'title' => [
'type' => 'text',
],
'slug' => [
'type' => 'text',
],
'type' => [
'type' => 'keyword',
],
'page_class' => [
'type' => 'keyword',
],
'content' => [
'type' => 'text',
],
'view_roles' => [
'type' => 'keyword',
],
]
];
} else {
$mapping = [
'mapping' => [
'root_id' => [
'type' => 'integer',
'include_in_all' => false,
'index' => 'not_analyzed'
],
'node_id' => [
'type' => 'integer',
'include_in_all' => false,
'index' => 'not_analyzed'
],
'nodetranslation_id' => [
'type' => 'integer',
'include_in_all' => false,
'index' => 'not_analyzed'
],
'nodeversion_id' => [
'type' => 'integer',
'include_in_all' => false,
'index' => 'not_analyzed'
],
'title' => [
'type' => 'string',
'include_in_all' => true
],
'slug' => [
'type' => 'string',
'include_in_all' => false,
'index' => 'not_analyzed'
],
'type' => [
'type' => 'string',
'include_in_all' => false,
'index' => 'not_analyzed'
],
'page_class' => [
'type' => 'string',
'include_in_all' => false,
'index' => 'not_analyzed'
],
'content' => [
'type' => 'string',
'include_in_all' => true
],
'created' => [
'type' => 'date',
'include_in_all' => false,
'index' => 'not_analyzed'
],
'updated' => [
'type' => 'date',
'include_in_all' => false,
'index' => 'not_analyzed'
],
'view_roles' => [
'type' => 'string',
'include_in_all' => true,
'index' => 'not_analyzed',
],
]
];
}

$container->prependExtensionConfig('kunstmaan_node_search', $mapping);
}
}
17 changes: 17 additions & 0 deletions src/Kunstmaan/NodeSearchBundle/Helper/ElasticSearchUtil.php
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
<?php

namespace Kunstmaan\NodeSearchBundle\Helper;

/**
* Class ElasticSearchUtil
*/
final class ElasticSearchUtil
{
/**
* @return bool
*/
public static function useVersion6()
{
return (PHP_MAJOR_VERSION == 7 && !class_exists('\Elastica\Tool\CrossIndex'));
}
}
50 changes: 50 additions & 0 deletions src/Kunstmaan/NodeSearchBundle/Resources/doc/ElasticSearch.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
# Elastic search support

We have added support for Elastic search up to version 6.x.

## Dynamic configuration
For the mapping and configuration we detect which version of the library your project is using.
Dependencies for using the configuration of version 6 is that your project uses PHP 7 and ruflin/elastica >= 6.0.

## Difference between versions

The biggest change in version 6, is that Elastica has dropped support for the "_all" meta field.
(see https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-all-field.html)

When using version 6.x, then you will no longer be able to use the "include_in_all" and "index => 'not_analysed".
use 'index => false' instead. Also some of the index types have been changed and some fields are no longer indexed (like the 'created' and 'updated' fields).

### Example

Some examples of the mapping that have changed (for the kunstmaan_node_search extension):

```
'node_id' => [
'type' => 'integer',
'include_in_all' => false,
'index' => 'not_analyzed'
],
...
'view_roles' => [
'type' => 'string',
'include_in_all' => true,
'index' => 'not_analyzed',
],
```

in


```
'node_id' => [
'type' => 'integer',
],
...
'view_roles' => [
'type' => 'keyword',
],
```

### Link to elastic search documentation

https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping.html