Permalink
Browse files

Elasticsearch 6.X and 5.x implementation (#51)

* Added version property to viewmodel

* Add basic elasticsearch 5.X db implementation

* Improved connect, clear and clearAll

* Fix empty object issue on update

* Fix eslint duplicaed mapping key issue

* remove duplicate condition eslint

* Upsert functionallity

* null check on errors

* fix eslint dependencies and behaviour

* elasticsearch 6 preparation

* Further optimizations and sharding settings

* Added dbType property to repository ( if not custom )

* Rename dbType to repositoryType

* Elasticsearch settings support, and better indexing

* Update elasticsearch dev dependency ( 13.x.x )

* Full test coverage for elasticsearch6 implementation ( supports Elasticsearch 5.x and 6.x )

* Update travis tests to elasticsearch 5.5.1

* Add sudo to travis so Elasticsearch can be started properly

* Remove uneeded IDE and NPM 5 files from repository

* Rename const to var for node.js 0.12 compatibility

* Syntax fixes to support node.js 0.12

* Update README.md

* Extended readme

* Add non brealing settings possibility for mongodb implementation

* Improved settings name and handling, allowing multiple implementation settings in one definition

* Improve README, document settings

* Fix mongodb settings

* version comment fix, new model is with null version
  • Loading branch information...
nanov authored and adrai committed Aug 8, 2017
1 parent fa0c5f5 commit a0547c5257f3acbe6909f79b8f5b9291a45fd19f
Showing with 822 additions and 64 deletions.
  1. +3 −16 .eslintrc
  2. +12 −2 .travis.yml
  3. +69 −0 README.md
  4. +1 −0 lib/base.js
  5. +409 −0 lib/databases/elasticsearch6.js
  6. +4 −2 lib/databases/mongodb.js
  7. +5 −1 lib/viewmodel.js
  8. +3 −2 package.json
  9. +96 −11 test/repositoryReadTest.js
  10. +220 −30 test/repositoryWriteTest.js
View
@@ -1,5 +1,8 @@
---
plugins:
- no-only-in-tests
env:
browser: true
node: true
@@ -11,7 +14,6 @@ globals:
describeSaga: false
describeEvent: false
describeCommand: false
before: false
it: false
xit: false
window : false
@@ -24,7 +26,6 @@ globals:
describeChapter: false
describeStep: false
document : false
window: false
File : false
FormData: false
QCodeDecoder: false
@@ -79,17 +80,3 @@ rules:
eol-last: 0
no-trailing-spaces: 0
indent: 0
# REACT
react/jsx-no-undef: 2
react/jsx-uses-vars: 1
react/jsx-quotes: 1
react/jsx-uses-react: 1
react/no-did-mount-set-state: 1
react/no-did-update-set-state: 1
react/prop-types: 1
react/react-in-jsx-scope: 1
react/self-closing-comp: 1
react/no-multi-comp: 0
react/wrap-multilines: 1
react/display-name: 0
View
@@ -1,9 +1,19 @@
sudo: false
sudo: true
# Elasticsearch 5 requires Java 8
# https://www.peterbe.com/plog/elasticsearch-5-in-travis-ci
addons:
apt:
packages:
- oracle-java8-set-default
before_install:
- curl -O https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-5.1.1.deb && sudo dpkg -i --force-confnew elasticsearch-5.1.1.deb && sudo service elasticsearch start
services:
- mongodb
- couchdb
- redis-server
- elasticsearch
language: node_js
node_js:
View
@@ -199,18 +199,86 @@ For mongodb you can define indexes for performance boosts in find function.
var dummyRepo = repository.extend({
collectionName: 'dummy',
// like that
indexes: [
'profileId',
// or:
{ profileId: 1 },
// or:
{ index: {profileId: 1}, options: {} }
]
// or like that
repositorySettings : {
mongodb: {
indexes: [ // same as above
'profileId',
// or:
{ profileId: 1 },
// or:
{ index: {profileId: 1}, options: {} }
]
}
}
});
## redis
The find function does ignore the query argument and always fetches all items in the collection.
## Elasticsearch >= 5.X
Use the 'elasticsearch6' type for Elasticsearch versions 5.X and 6.X.
The find queries are not mongoDb compatible as the rest of the implementations due to the uneeded overhead and complexity of converting between both formats.
For find queries with elasticsearch6 use elasticsearch [native elastic Query DSL](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl.html);
repository.find( onlyTheQueryClause, otherBodyOptions, callback);
```javascript
repository.find(
{
range : {
age : {
gte : 10,
lte : 20
}
}
),
{
from: 0,
size: 10,
sort: { age: 'asc' }
},
function(error, results) {
});
```
Additionaly for elasticsearch6 the number of shards, number of replicas, the refresh behaivour on index and the mappings on index create can be addtionaly defined to optimize performace.
```javascript
var dummyRepo = repository.extend({
collectionName: 'dummy',
repositorySettings: {
elasticsearch6: {
refresh: 'wait_for', // optional, refresh behaviour on index, default is true ( ie. force index refresh ) https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-refresh.html
waitForActiveShards: 2 // optional, defaults to 1 ( ie. wait only for primary ) https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-create-index.html#create-index-wait-for-active-shards
index: { // optional applied on index create, https://www.elastic.co/guide/en/elasticsearch/reference/6.x/indices-create-index.html
settings : { // will be merged with the default ones,
number_of_shards: 3, // optional defaults to 1,
number_of_replicas: 1 // optional defaults to 0,
},
mappings : { // optiona will be merged with the default ones,
properties: { // specific properties to not be handled by dynamic mapper
title: {
type: "text"
}
}
}
}
}
}
});
```
# [Release notes](https://github.com/adrai/node-viewmodel/blob/master/releasenotes.md)
@@ -225,6 +293,7 @@ Currently these databases are supported:
6. azuretable ([azure-storage](https://github.com/Azure/azure-storage-node))
7. documentdb ([documentdb](https://github.com/Azure/azure-documentdb-node), [doqmentdb](https://github.com/a8m/doqmentdb))
8. elasticsearch ([elasticsearch] (https://github.com/elastic/elasticsearch-js))
9. elasticsearch6 ([elasticsearch] (https://github.com/elastic/elasticsearch-js)) - for Elasticsearch 5.x and 6.x
## own db implementation
You can use your own db implementation by extending this...
View
@@ -13,6 +13,7 @@ var util = require('util'),
*/
function Repository(options) {
options = options || {};
this.repositoryType = (typeof(options.type) === 'string') ? options.type : null;
EventEmitter.call(this);
}
Oops, something went wrong.

0 comments on commit a0547c5

Please sign in to comment.