diff --git a/CHANGELOG.md b/CHANGELOG.md index 4386bc4..6b511f2 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,3 +6,17 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/) and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.html). ## [Unreleased] +### Added +- Solr DB Driver Package +- `endpoint` helper +- `filters` helper +- `query` helper +- `request` helper +- `response` helper +- `schema` helper +- `utils` helper +- `insert` and `multiInsert` methods +- `remove` and `multiRemove` methods +- `get` and `getTotals` methods +- `distinct` method +- `createSchemas`, `updateSchemas` and `createCore` methods \ No newline at end of file diff --git a/README.md b/README.md index d433ede..64c0051 100644 --- a/README.md +++ b/README.md @@ -1,4 +1,4 @@ -# solr +# Solr [![Build Status](https://travis-ci.org/janis-commerce/solr.svg?branch=master)](https://travis-ci.org/janis-commerce/solr) [![Coverage Status](https://coveralls.io/repos/github/janis-commerce/solr/badge.svg?branch=master)](https://coveralls.io/github/janis-commerce/solr?branch=master) @@ -10,13 +10,473 @@ Apache Solr Driver npm install @janiscommerce/solr ``` +## Models +Whenever the `Model` type is mentioned in this document, it refers to an instance of [@janiscommerce/model](https://www.npmjs.com/package/@janiscommerce/model). + +This is used to configure which collection should be used, which unique indexes it has, among other stuff. + ## API +### `new Solr(config)` +Constructs the Solr driver instance, connected with the `config` object. + +**Config properties** + +- url `String` (required): Solr URL +- core `String` (required): Solr Core +- user `String` (optional): Auth user +- password `String` (optional but required if an user is specified): Auth password + +**Config usage** +```js +{ + url: 'http://localhost:8983', + core: 'some-core', + user: 'some-user', + password: 'some-password' +} +``` + +### ***async*** `insert(model, item)` +Inserts one document in a solr core + +- model: `Model`: A model instance +- item: `Object`: The item to save in the solr core + +- Resolves: `String`: The *ID* of the inserted item +- Rejects: `SolrError` When something bad occurs + +### ***async*** `multiInsert(model, items)` +Inserts multiple documents in a solr core + +- model: `Model`: A model instance +- item: `Array`: The items to save in the solr core + +- Resolves: `Array`: Items inserted +- Rejects: `SolrError` When something bad occurs + +### ***async*** `distict(model, [parameters])` +Searches distinct values of a property in a solr core + +- model: `Model`: A model instance +- parameters: `Object` (optional): The query parameters. Default: {}. It only accepts `key` (the field name to get distinct values from), and `filters` -- described below in `get()` method. + +- Resolves `Array`: An array of documents +- Rejects `SolrError`: When something bad occurs + +### ***async*** `get(model, [parameters])` +Searches documents in a solr core + +- model `Model`: A model instance +- parameters: `Object` (optional): The query parameters. Default: `{}` + +- Resolves: `Array`: An array of documents +- Rejects: `SolrError` When something bad occurs + +**Available parameters (all of them are optional)** + +- order `Object`: Sets the sorting criteria of the matched documents, for example: `{ myField: 'asc', myOtherField: 'desc' }` +- limit `Number`: Sets the page size when fetching documents. Default: `500` +- page `Number`: Sets the current page to retrieve. +- filters `Object`: Sets the criteria to match documents. + +Parameters example: +```js +{ + limit: 1000, // Default 500 + page: 2, + order: { + itemField: 'asc' + }, + filters: { + itemField: 'foobar', + otherItemField: { + type: 'notEqual', + value: 'foobar' + }, + anotherItemField: ['foo', 'bar'] + } +} +``` + +#### Filters + +The filters have a simpler structure what raw solr filters, in order to simplify it's usage. + +**Single valued filters** + +These filters sets a criteria using the specified `type` and `value`: +```js +{ + filters: { + myField: {type: 'notEqual', value: 'foobar'} + } +} +``` + +**Multivalued filters** + +These filters are an array of multiple filters that sets the criterias using an *OR* operation with the specified `type` and `value` +```js +{ + filters: { + myField: ['foobar', { type: 'notEqual', value: 'barfoo' }] + } +} +``` + +**Unsupported filters** + +Due Solr limitations, some filters are unsupported, like Array/Object filters: +```js +{ + filters: { + myField: { type: 'equal', value: [1,2,3] }, + myOtherField: { type: 'notEqual', value: { some: 'thing' } } // Also you can use nested filters instead + } +} +``` + +See [nested filters](#nested%20filters) to see how to filter by object properties and sub properties. + +**Filter types** + +The filter types can be defined in the model static getter `fields` like this: +```js +class MyModel extends Model { + + static get fields(){ + + return { + myField: { + type: 'greaterOrEqual' + } + } + } +} +``` + +It can also be overriden in each query like this: +```js +solr.get(myModel, { + filters: { + myField: { + type: 'lesserOrEqual', + value: 10 + } + } +}) +``` + +The following table shows all the supported filter types: + +| Type | Solr equivalence | +| -------------- | ------------------ | +| equal | field:"value" | +| notEqual | -field:"value" | +| greater | field:{value TO *} | +| greaterOrEqual | field:[value TO *] | +| lesser | field:{* TO value} | +| lesserOrEqual | field:[* TO value] | + +If the type isn't defined in the model or in the query, it defaults to `equal`. + +**Internal field names** + +The name of a filter and the field that it will match can differ. To archieve that, you must declare it in the model static getter `fields`: + +```js +class MyModel extends Model { + + static get fields() { + + return { + externalFieldName: { + field: 'internalFieldName' + } + } + } +} +``` + +#### Nested filters +If you want to filter by fields inside objects, you can use nested filters. For example: +```js +{ + /* Sample document to match + { + id: 'some-id', + someField: { + foo: 'bar' + } + } + */ + + solr.get(myModel, { + filters: { + 'someField.foo': 'bar' + } + }); +} +``` + +### ***async*** `getTotals(model)` +Gets information about the quantity of documents matched by the last call to the `get()` method. + +- model: `Model`: A model instance used for the query. **IMPORTANT**: This must be the same instance. + +- Resolves: `Object`: An object containing the totalizers +- Rejects: `SolrError`: When something bad occurs + +Return example: +```js +{ + total: 140, + pageSize: 60, + pages: 3, + page: 1 +} +``` + +If no query was executed before, it will just return the `total` and `pages` properties with a value of zero. + +### ***async*** `remove(model, item)` +Removes a document in a solr core + +- model: `Model`: A model instance +- item: `Object`: The item to be removed + +- Rejects `SolrError`: When something bad occurs + +### ***async*** `multiRemove(model, filters)` +Removes one or more documents in a solr core + +- model: `Model`: A model instance +- filters: `Object`: Filters criteria to match documents + +- Rejects `SolrError`: When something bad occurs + +### ***async*** `createSchema(model, core)` +Build the fields schema using the schema defined in the model static getter `schema`. + +- model `Model`: A model instance +- core `String`: The core where the field schema will created. Default: `core` value from instance config. + +- Rejects: `SolrError`: When something bad occurs + +- **IMPORTANT**: + - This method must be executed before any operation, otherwise Solr will set all new fields as an `array` of `strings`. + - This method can't replace or delete already existing fields schema in Solr. + +If you need details about how to define the fields schema in the model, see the schema apart [below](#schema) + +### ***async*** `updateSchema(model)` +Update the fields schema by replacing the current fields schema in Solr with the defined in the model static getter `schema`. +**IMPORTANT**: This method can't create or delete already existing fields schema in Solr. + +- model `Model`: A model instance + +- Rejects: `SolrError`: When something bad occurs + +If you need details about how to define the fields schema in the model, see the schema apart [below](#schema) + +#### Fields schema + +The fields schema are required by Solr to use the correct data types on every field, by default, Solr sets new fields as an `array` of `strings`, even in objects and his properties. + +**Field types** + +The field types can be defined in the model static getter `schema` like this: +```js +class MyModel extends Model { + + static get schema(){ + + return { + myStringField: true, // Default type string + myNumberString: { type: 'number' }, + myArrayOfStrings: { type: ['string'] }, + myObject: { + property: 'string' + } + } + } +} +``` + +The following table shows all the supported field types: + +| Type | Solr equivalence | +| -------------- | ---------------- | +| string | string | +| boolean | boolean | +| date | pdate | +| number | pint | +| float | pfloat | +| double | pdouble | +| long | plong | + +In case of an array field, the Solr equivalence will be the same for each field, only will set the `multiValued` property to `true`. + +If the type isn't defined, it defaults to `string`. + +**Single valued field** + +These are the most common fields, only stores a single value: +```js +{ + myField: { type: 'number' }, + myOtherField: { type: 'date' } +} +``` + +**Multi valued fields (Array)** + +These fields stores an array with multiple values **of the same type**: +```js +{ + myField: { type: ['string'] }, // Array of strings + myOtherField: { type: ['number'] } // Array of numbers +} +``` + +**Objects (JSON)** + +These fields stores an object with multiple properties that **can be of different types**: +```js +{ + myField: { + type: { + property: 'string', + subProperty: { + property: ['string'] + }, + otherProperty: 'date' + } + } +} +``` + +**IMPORTANT**: Due Solr compatibility issues, the object fields will be created **internally** as single fields like this: +```js +// Schema +{ + myField: { + type: { + property: 'string', + otherProperty: 'number', + subProperty: { + property: ['float'] + } + } + } +} + +// Formatted for Solr +{ + 'myField.property': { type: 'string' }, + 'myField.otherProperty': { type: 'number' }, + 'myField.subProperty.property': { type: ['float'] } +} +``` + +It will show as a **full object** on `get` operations. + +### ***async*** `createCore(model, name)` +Creates a new core in the Solr URL, then build the fields schema for that core. + +- model `Model`: A model instance +- name `String`: The name for the core to create + +- Rejects: `SolrError`: When something bad occurs + +## Errors + +The errors are informed with a `SolrError`. +This object has a code that can be useful for debugging or error handling. +The codes are the following: + +| Code | Description | +|------|-------------------------- | +| 1 | Invalid connection config | +| 2 | Invalid model | +| 3 | Invalid filter type | +| 4 | Invalid filter value | +| 5 | Unsupported filter | +| 6 | Request failed | +| 7 | Internal Solr error | +| 8 | Invalid parameters | ## Usage ```js const Solr = require('@janiscommerce/solr'); -``` +const Model = require('./myModel'); + +const solr = new Solr({ + url: 'localhost:8983' +}); + +const model = new Model(); + +(async () => { + + //Insert + await solr.insert(model, { + id: 'some-id', + name: 'test' + }); + // > 'some-id' + + // multiInsert + await solr.multiInsert(model, [ + { id: 'some-id', name: 'test 1' }, + { id: 'other-id', name: 'test 2' }, + { id: 'another-id', name: 'test 3' } + ]); + // > + /* + [ + { id: 'some-id', name: 'test 1' }, + { id: 'other-id', name: 'test 2' }, + { id: 'another-id', name: 'test 3' } + ] + */ + + // get + await solr.get(model, {}); + // > [...] // Every document in the solr core, up to 500 documents (by default) + + await solr.get(model, { filters: { id: 'some-id' } }); + // > [{ id: 'some-id', name: 'test' }] + + await solr.get(model, { limit: 10, page: 2, filters: { name: 'foo' } }); + // > The second page of 10 documents matching name equals to 'foo'. + + await solr.get(model, { order: { id: 'desc' } }); + // > [...] Every document in the solr core, ordered by descending id, up to 500 documents (by default) + + // getTotals + await solr.getTotals(model); + // > { page: 1, limit: 500, pages: 1, total: 4 } + + // distinct + await solr.distinct(model, { key: 'fieldName', filters: { someField: true } }); + // > ['some-value', 'other-value'] + + // remove + await solr.remove(model, { id: 'some-id', field: 'value' }); + + // multiRemove + await solr.multiRemove(model, { field: { type: 'greater', value: 10 } }); + + // createSchema + await solr.createSchema(model); + + // updateSchema + await solr.updateSchema(model); -## Examples + // createCore + await solr.createCore(model, 'new-core'); +})(); +``` \ No newline at end of file diff --git a/index.js b/index.js deleted file mode 100644 index 4819e25..0000000 --- a/index.js +++ /dev/null @@ -1,5 +0,0 @@ -'use strict'; - -const { Solr } = require('./lib'); - -module.exports = Solr; diff --git a/lib/config-validator.js b/lib/config-validator.js new file mode 100644 index 0000000..388f204 --- /dev/null +++ b/lib/config-validator.js @@ -0,0 +1,40 @@ +'use strict'; + +const { struct } = require('superstruct'); + +const SolrError = require('./solr-error'); + +const configStruct = struct.partial({ + url: 'string', + core: 'string', + user: 'string?', + password: 'string?' +}); + +class ConfigValidator { + + /** + * Validates the received config struct + * @param {Object} config the config to validate + * @returns {Object} received config object + * @throws if the struct is invalid + */ + static validate(config) { + + try { + + const validConfig = configStruct(config); + + if((validConfig.user && !validConfig.password) || (validConfig.password && !validConfig.user)) + throw new Error(`Password required for user '${validConfig.user}'`); + + return validConfig; + + } catch(err) { + err.message = `Error validating connection config: ${err.message}`; + throw new SolrError(err, SolrError.codes.INVALID_CONFIG); + } + } +} + +module.exports = ConfigValidator; diff --git a/lib/helpers/endpoint.js b/lib/helpers/endpoint.js new file mode 100644 index 0000000..2198ff9 --- /dev/null +++ b/lib/helpers/endpoint.js @@ -0,0 +1,35 @@ +'use strict'; + +const path = require('path'); + +const ENDPOINT_BASE = '{{url}}/solr'; + +class Endpoint { + + static get presets() { + + return { + get: '{{core}}/query', + update: '{{core}}/update/json/docs?commit=true', + updateCommands: '{{core}}/update?commit=true', + schema: '{{core}}/schema' + }; + } + + /** + * Creates a Solr endpoint + * @param {String} endpoint The endpoint to create + * @param {String} url The solr url + * @param {String} core The solr core name + * @param {Object} replacements The replacements to apply to the endpoint + */ + static create(endpoint, url, core, replacements) { + return this._buildEndpoint(path.join(ENDPOINT_BASE, endpoint), { url, core, ...replacements }); + } + + static _buildEndpoint(url, replacements) { + return Object.entries(replacements).reduce((endpoint, [key, value]) => endpoint.replace(`{{${key}}}`, value), url); + } +} + +module.exports = Endpoint; diff --git a/lib/helpers/filters.js b/lib/helpers/filters.js new file mode 100644 index 0000000..877f19b --- /dev/null +++ b/lib/helpers/filters.js @@ -0,0 +1,127 @@ +'use strict'; + +const { isObject } = require('./utils'); + +const SolrError = require('../solr-error'); + +const DEFAULT_FILTER_TYPE = 'equal'; + +class Filters { + + /** + * Builds the filters for Solr legibility + * @param {Object} filters The filters + * @param {Object} modelFields The model fields + */ + static build(filters, modelFields) { + + const filtersGroup = this._parseFilterGroup(filters, modelFields); + + const filtersByType = this._formatByType(filtersGroup); + + return Object.values(filtersByType); + } + + static _parseFilterGroup(filterGroup, modelFields) { + + return Object.entries(filterGroup).reduce((parsedFilterGroup, [filterName, filterData]) => { + + const modelField = modelFields && filterName in modelFields ? modelFields[filterName] : {}; + + const filterKey = modelField.field || filterName; + + const filterValues = this._getFilterObjects(filterData, modelField); + + parsedFilterGroup[filterKey] = filterKey in parsedFilterGroup ? [...parsedFilterGroup[filterKey], ...filterValues] : filterValues; + + return parsedFilterGroup; + + }, {}); + } + + static _getFilterObjects(filterData, modelField) { + + if(!Array.isArray(filterData)) + filterData = [filterData]; + + return filterData.map(filterObject => { + + if(!isObject(filterObject)) + filterObject = { value: filterObject }; + + const { value } = filterObject; + + if(isObject(value) || Array.isArray(value)) + throw new SolrError('Invalid filters: JSON/Array filters are not supported by Solr.', SolrError.codes.UNSUPPORTED_FILTER); + + const type = filterObject.type || modelField.type || DEFAULT_FILTER_TYPE; + + if(!value) + throw new SolrError(`Invalid filters: Missing value for filter type ${type}.`, SolrError.codes.INVALID_FILTER_VALUE); + + return { type, value }; + }); + } + + static _formatByType(filtersGroup) { + + return Object.entries(filtersGroup).reduce((filtersByType, [field, filterData]) => { + + const filterByType = filterData.reduce((filters, { type, value }) => { + + const formatter = this._formatters[type]; + + if(!formatter) + throw new SolrError(`'${type}' is not a valid or supported filter type.`, SolrError.codes.INVALID_FILTER_TYPE); + + const formattedFilter = formatter(field, value); + + return filters ? `${filters} OR ${formattedFilter}` : formattedFilter; + + }, ''); + + filtersByType[field] = filterByType; + + return filtersByType; + + }, {}); + } + + static get _formatters() { + + return { + equal: this._formatEq.bind(this), + notEqual: this._formatNe.bind(this), + greater: this._formatGt, + greaterOrEqual: this._formatGte, + lesser: this._formatLt, + lesserOrEqual: this._formatLte + }; + } + + static _formatEq(field, value) { + return `${field}:"${value}"`; + } + + static _formatNe(field, value) { + return `-${field}:"${value}"`; + } + + static _formatGt(field, value) { + return `${field}:{${value} TO *}`; + } + + static _formatGte(field, value) { + return `${field}:[${value} TO *]`; + } + + static _formatLt(field, value) { + return `${field}:{* TO ${value}}`; + } + + static _formatLte(field, value) { + return `${field}:[* TO ${value}]`; + } +} + +module.exports = Filters; diff --git a/lib/helpers/query.js b/lib/helpers/query.js new file mode 100644 index 0000000..ff1208e --- /dev/null +++ b/lib/helpers/query.js @@ -0,0 +1,68 @@ +'use strict'; + +const SolrError = require('../solr-error'); + +const Filters = require('./filters'); + +class Query { + + static get(params) { + + const { limit, page, fields } = params; + + const filters = params.filters ? { filter: Filters.build(params.filters, fields) } : {}; + const order = params.order ? { sort: this._getSorting(params.order) } : {}; + + return { + query: '*:*', + offset: (page * limit) - limit, + limit, + ...filters, + ...order + }; + } + + static delete(params) { + + const { fields } = params; + + const filters = params.filters ? Filters.build(params.filters, fields) : []; + + const query = filters.reduce((stringQuery, terms) => (stringQuery ? `${stringQuery} AND ${terms}` : terms), ''); + + return query ? { delete: { query } } : {}; + } + + static distinct(params) { + + const { key, fields } = params; + + if(typeof key !== 'string') + throw new SolrError(`Distinct key must be a string, received: ${typeof key}.`, SolrError.codes.INVALID_PARAMETERS); + + const filters = params.filters ? { filter: Filters.build(params.filters, fields) } : {}; + + return { + query: '*:*', + fields: key, + params: { + group: true, + 'group.field': key + }, + ...filters + }; + } + + static _getSorting(order) { + + return Object.entries(order).reduce((sortings, [field, term]) => { + + const sort = `${field} ${term}`; + + return sortings ? `${sortings}, ${sort}` : sort; + + }, ''); + } +} + +module.exports = Query; diff --git a/lib/helpers/request.js b/lib/helpers/request.js new file mode 100644 index 0000000..7025f0e --- /dev/null +++ b/lib/helpers/request.js @@ -0,0 +1,72 @@ +'use strict'; + +const { promisify } = require('util'); + +const request = promisify(require('request')); + +const { base64 } = require('./utils'); + +const SolrError = require('../solr-error'); + +class Request { + + /** + * Make an http request with method GET to the specified endpoint with the received request body and headers + * @param {String} endpoint the endpoint to make the request + * @param {Object|Array} requestBody The request body + * @param {Object} headers The request headers + * @returns {Object|Array|null} with the JSON response or null if the request ended with 4XX code + * @throws when the response code is 5XX + */ + static async get(endpoint, requestBody, auth, headers) { + const httpRequest = this._buildHttpRequest(endpoint, requestBody, auth, headers); + return this._makeRequest(httpRequest, 'GET'); + } + + /** + * Make an http request with method POST to the specified endpoint with the received request body and headers + * @param {String} endpoint the endpoint to make the request + * @param {Object|Array} requestBody The request body + * @param {Object} headers The request headers + * @returns {Object|Array|null} with the JSON response or null if the request ended with 4XX code + * @throws when the response code is 5XX + */ + static async post(endpoint, requestBody, auth, headers) { + const httpRequest = this._buildHttpRequest(endpoint, requestBody, auth, headers); + return this._makeRequest(httpRequest, 'POST'); + } + + static _buildHttpRequest(endpoint, body, auth, headers) { + + let credentials; + + if(auth.user && auth.password) + credentials = `Basic ${base64(`${auth.user}:${auth.password}`)}`; + + const httpRequest = { + url: endpoint, + headers: { + ...headers, + 'Content-Type': 'application/json' + }, + body: JSON.stringify(body) + }; + + if(credentials) + httpRequest.headers.Authorization = credentials; + + return httpRequest; + } + + static async _makeRequest(httpRequest, method) { + + const { statusCode, statusMessage, body } = await request({ ...httpRequest, method }); + + if(statusCode >= 400) + throw new SolrError(`[${statusCode}] (${statusMessage}): ${body}`, SolrError.codes.REQUEST_FAILED); + + return JSON.parse(body); + } +} + +module.exports = Request; diff --git a/lib/helpers/response.js b/lib/helpers/response.js new file mode 100644 index 0000000..23b81c8 --- /dev/null +++ b/lib/helpers/response.js @@ -0,0 +1,94 @@ +'use strict'; + +const merge = require('lodash.merge'); + +const { inspect } = require('util'); + +const { superstruct } = require('superstruct'); + +const { isObject } = require('./utils'); + +const SolrError = require('../solr-error'); + +const IGNORED_FIELDS = ['_version_']; + +const struct = superstruct({ + types: { + equalToZero: value => value === 0 + } +}); + +class Response { + + static format(response) { + + return response.map(item => { + + // Remove solr ignored fields + IGNORED_FIELDS.forEach(ignoredField => delete item[ignoredField]); + + return Object.entries(item).reduce((formattedItem, [field, value]) => { + + if(!field.includes('.')) + return { ...formattedItem, [field]: value }; + + return merge(formattedItem, this._buildJsonField(field, value)); + + }, {}); + }); + } + + static validate(res, terms = {}) { + + const responseStruct = struct.partial( + + this._buildNestedStruct({ + responseHeader: { status: 'equalToZero' }, + ...terms + }) + ); + + try { + + responseStruct(res); + + } catch(err) { + throw new SolrError(`Invalid Solr response: ${err.message} ${inspect(res)}`, SolrError.codes.INTERNAL_SOLR_ERROR); + } + } + + static _buildNestedStruct(terms) { + + return Object.entries(terms).reduce((structs, [field, term]) => { + + if(!isObject(term)) + return { ...structs, [field]: term }; + + return { ...structs, [field]: struct.partial(this._buildNestedStruct(term)) }; + + }, {}); + } + + static _buildJsonField(field, value) { + + const fields = field.split('.'); + + const [rootField] = fields; + + // Need to reverse the properties array to iterate first by the property that will have the item value + const properties = fields.splice(1).reverse(); + + return properties.reduce((item, property, i) => { + + if(i === 0) + item[rootField][property] = value; + else + item[rootField] = { [property]: item[rootField] }; + + return item; + + }, { [rootField]: {} }); + } +} + +module.exports = Response; diff --git a/lib/helpers/schema.js b/lib/helpers/schema.js new file mode 100644 index 0000000..6c43a69 --- /dev/null +++ b/lib/helpers/schema.js @@ -0,0 +1,78 @@ +'use strict'; + +const { isObject } = require('./utils'); + +const FIELD_TYPES = { + string: 'string', + boolean: 'boolean', + date: 'pdate', + number: 'pint', + float: 'pfloat', + double: 'pdouble', + long: 'plong' +}; + +class Schema { + + /** + * Builds a schema query for Solr + * @param {String} method The method to use for building the schema api query + * @param {Object} schemas The model schemas + */ + static buildQuery(method, schemas) { + + const builtSchemas = Object.entries(schemas).reduce((fields, [field, schema]) => { + + fields.push(...this._buildSchema(field, schema)); + + return fields; + + }, []); + + return { [`${method}-field`]: builtSchemas }; + } + + static _buildSchema(field, schema) { + + const { type } = schema; + + if(Array.isArray(type)) + return this._buildArraySchema(field, type); + + if(isObject(type)) + return this._buildObjectSchema(field, type); + + return [{ + name: field, + type: this._getType(type) + }]; + } + + static _buildArraySchema(field, fieldType) { + + const [type] = fieldType; + + return [{ + name: field, + type: this._getType(type), + multiValued: true + }]; + } + + static _buildObjectSchema(field, type) { + + return Object.entries(type).reduce((schemas, [property, schema]) => { + + schemas.push(...this._buildSchema(`${field}.${property}`, { type: schema })); + + return schemas; + + }, []); + } + + static _getType(type) { + return FIELD_TYPES[type] || FIELD_TYPES.string; + } +} + +module.exports = Schema; diff --git a/lib/helpers/utils.js b/lib/helpers/utils.js new file mode 100644 index 0000000..2607538 --- /dev/null +++ b/lib/helpers/utils.js @@ -0,0 +1,18 @@ +'use strict'; + +/** + * Validates if the received object is an JSON object and not an array + * @param {Object} object The object to validate + */ +const isObject = object => (object !== null && typeof object === 'object' && !Array.isArray(object)); + +/** + * Encodes the received value into a base64 string + * @param {String} value value + */ +const base64 = value => (Buffer.from(value).toString('base64')); + +module.exports = { + isObject, + base64 +}; diff --git a/lib/index.js b/lib/index.js deleted file mode 100644 index f8caa24..0000000 --- a/lib/index.js +++ /dev/null @@ -1,9 +0,0 @@ -'use strict'; - -const Solr = require('./solr'); -const SolrError = require('./solr-error'); - -module.exports = { - Solr, - SolrError -}; diff --git a/lib/solr-error.js b/lib/solr-error.js index ba5d907..dcd3e14 100644 --- a/lib/solr-error.js +++ b/lib/solr-error.js @@ -5,7 +5,14 @@ class SolrError extends Error { static get codes() { return { - // your errors here... + INVALID_CONFIG: 1, + INVALID_MODEL: 2, + INVALID_FILTER_TYPE: 3, + INVALID_FILTER_VALUE: 4, + UNSUPPORTED_FILTER: 5, + REQUEST_FAILED: 6, + INTERNAL_SOLR_ERROR: 7, + INVALID_PARAMETERS: 8 }; } diff --git a/lib/solr.js b/lib/solr.js index 718ca7f..970e935 100644 --- a/lib/solr.js +++ b/lib/solr.js @@ -1,11 +1,367 @@ 'use strict'; +const UUID = require('uuid/v4'); + +const { isObject } = require('./helpers/utils'); + const SolrError = require('./solr-error'); +const ConfigValidator = require('./config-validator'); + +const Response = require('./helpers/response'); +const Endpoint = require('./helpers/endpoint'); +const Request = require('./helpers/request'); +const Schema = require('./helpers/schema'); +const Query = require('./helpers/query'); + +const DEFAULT_LIMIT = 500; + class Solr { - // package code... + constructor(config) { + this._config = ConfigValidator.validate(config); + } + + /** + * Inserts an item into Solr + * @param {Model} model Model instance (not used but required for @janiscommerce/model compatibilty) + * @param {Object} item The item to insert + * @returns {String} The ID of the inserted item + * @throws When something goes wrong + * @example + * await insert(model, { field: 'value' }); + * // Expected result + * 'f429592c-8318-4507-a2ea-1b7fc388162a' + */ + async insert(model, item) { + + if(!isObject(item)) + throw new SolrError('Invalid item: Should be an object, also not an array.', SolrError.codes.INVALID_PARAMETERS); + + item = this._prepareItem(item); + + const endpoint = Endpoint.create(Endpoint.presets.update, this._url, this._core); + + const res = await Request.post(endpoint, item, this._auth); + + Response.validate(res); + + return item.id; + } + + /** + * Inserts multiple items into Solr + * @param {Model} model Model instance (not used but required for @janiscommerce/model compatibilty) + * @param {Array.} items The items to insert + * @returns {Array.} The inserted items + * @throws When something goes wrong + * @example + * await multiInsert(model, [ + * { field: 'value' }, + * { field: 'other value' } + * ]); + * // Expected result + * [ + * { id: '699ff4c9-ec23-44ab-adb6-36f19b1712f6', field: 'value' }, + * { id: 'f429592c-8318-4507-a2ea-1b7fc388162a', field: 'other value' } + * ] + */ + async multiInsert(model, items) { + + if(!Array.isArray(items)) + throw new SolrError('Invalid items: Should be an array.', SolrError.codes.INVALID_PARAMETERS); + + items = items.map(this._prepareItem); + + const endpoint = Endpoint.create(Endpoint.presets.update, this._url, this._core); + + const res = await Request.post(endpoint, items, this._auth); + + Response.validate(res); + + return items; + } + + /** + * Get data from Solr database + * @param {Model} model Model instance + * @param {Object} params Get parameters (limit, filters, order, page) + * @returns {Array.} Solr get result + * @throws When something goes wrong + * @example + * await get(model, { + * limit: 10, + * page: 2, + * order: { field: 'asc' }, + * filters: { + * field: { type: 'greater', value: 32 } + * } + * }); + * // Expected result + * [ + * { + * id: '699ff4c9-ec23-44ab-adb6-36f19b1712f6' + * field: 34 + * }, + * { + * id: 'f429592c-8318-4507-a2ea-1b7fc388162a', + * field: 33 + * } + * ] + */ + async get(model, params = {}) { + + this._validateModel(model); + + const { fields } = model.constructor; + + const endpoint = Endpoint.create(Endpoint.presets.get, this._url, this._core); + + const page = params.page || 1; + const limit = params.limit || DEFAULT_LIMIT; + + const query = Query.get({ ...params, page, limit, fields }); + + const res = await Request.get(endpoint, query, this._auth); + + Response.validate(res, { + response: { docs: ['object'] } + }); + + const { docs } = res.response; + + model.lastQueryHasResults = !!docs.length; + model.totalsParams = { page, limit, query }; + + return Response.format(docs); + } + + /** + * Get the paginated totals from the latest get query + * @param {Model} model Model instance + * @returns {Object} total, page size, pages and page from the results + */ + async getTotals(model) { + + this._validateModel(model); + + if(!model.lastQueryHasResults) + return { total: 0, pages: 0 }; + + const { page, limit, query } = model.totalsParams; + + const endpoint = Endpoint.create(Endpoint.presets.get, this._url, this._core); + + const res = await Request.get(endpoint, query, this._auth); + + Response.validate(res, { + response: { numFound: 'number' } + }); + + const { numFound } = res.response; + + return { + total: numFound, + pageSize: limit, + pages: Math.ceil(numFound / limit), + page + }; + } + + /** + * Removes an item from the database + * @param {Model} model Model instance + * @param {Object} item The item to delete + * @throws When something goes wrong + * @example + * await remove(model, { id: 'some-id', value: 'some-value' }); + */ + async remove(model, item) { + + if(!isObject(item)) + throw new SolrError('Invalid item: Should be an object, also not an array.', SolrError.codes.INVALID_PARAMETERS); + + if(!item.id) + throw new SolrError('Invalid item: Should have an ID.', SolrError.codes.INVALID_PARAMETERS); + + const endpoint = Endpoint.create(Endpoint.presets.updateCommands, this._url, this._core); + + const res = await Request.post(endpoint, { delete: { id: item.id } }, this._auth); + + Response.validate(res); + } + + /** + * Multi remove items from the database + * @param {Model} model Model instance + * @param {Object} filters solr filters + * @throws When something goes wrong + * @example + * await multiRemove(model, { myField: { type: 'greater', value: 10 } }); + */ + async multiRemove(model, filters) { + + this._validateModel(model); + + const { fields } = model.constructor; + + const endpoint = Endpoint.create(Endpoint.presets.updateCommands, this._url, this._core); + + const query = Query.delete({ filters, fields }); + + const res = await Request.post(endpoint, query, this._auth); + + Response.validate(res); + } + + /** + * Get distinct values of a field + * @param {Model} model Model instance + * @param {Object} params parameters (key and filters) + * @return {Array} results + * @example + * await distinct(model, { key: 'myField', { filters: { field: { type: 'lesser', value: 32 } } } }) + * // Expected result + * ['some data', 'other data'] + */ + async distinct(model, params = {}) { + + this._validateModel(model); + + const { fields } = model.constructor; + + const endpoint = Endpoint.create(Endpoint.presets.get, this._url, this._core); + + const query = Query.distinct({ ...params, fields }); + + const res = await Request.get(endpoint, query, this._auth); + + Response.validate(res, { + grouped: { [params.key]: { groups: ['object'] } } + }); + + const { groups } = res.grouped[params.key]; + + return groups.map(({ doclist }) => { + + // When grouping by specific field, the doclist.docs array will be always with one item + const [value] = Object.values(doclist.docs[0]); + + return value; + }); + } + + /** + * Create the fields schema with the specified field types + * @param {Model} model Model instance + * @param {string} core The core name where create the schemas (Default: core from config) + * @throws When something goes wrong + */ + async createSchema(model, core = this._core) { + + this._validateModel(model); + + const { schema } = model.constructor; + + if(!schema) + return; + + const query = Schema.buildQuery('add', schema); + + const endpoint = Endpoint.create(Endpoint.presets.schema, this._url, core); + + const res = await Request.post(endpoint, query, this._auth); + + Response.validate(res); + } + + /** + * Update the existing fields schema with the specified field types + * @param {Model} model Model instance + * @throws When something goes wrong + */ + async updateSchema(model) { + + this._validateModel(model); + + const { schema } = model.constructor; + + if(!schema) + return; + + const query = Schema.buildQuery('replace', schema); + const endpoint = Endpoint.create(Endpoint.presets.schema, this._url, this._core); + + const res = await Request.post(endpoint, query, this._auth); + + Response.validate(res); + } + + /** + * Create a new core into Solr URL then create the fields schema. + * @param {Model} model Model instance + * @param {string} name The name for the new core to create + */ + async createCore(model, name) { + + try { + + this._validateModel(model); + + } catch(err) { + // Should not explode when the model is invalid, also must not create any core. + return; + } + + const endpoint = Endpoint.create('admin/cores?action=CREATE&name={{name}}&configSet=_default', this._url, null, { name }); + + const res = await Request.post(endpoint, null, this._auth); + + Response.validate(res); + + return this.createSchema(model, name); + } + + _prepareItem(item) { + + return { + id: UUID(), + ...item + }; + } + + _validateModel(model) { + + if(!model) + throw new SolrError('Invalid or empty model.', SolrError.codes.INVALID_MODEL); + + const { fields, schema } = model.constructor; + + if(fields && !isObject(fields)) + throw new SolrError('Invalid model: Fields should be an object, also not an array.', SolrError.codes.INVALID_MODEL); + + if(schema && !isObject(schema)) + throw new SolrError('Invalid model: Schemas should be an object, also not an array.', SolrError.codes.INVALID_MODEL); + } + + get _url() { + return this._config.url; + } + + get _core() { + return this._config.core; + } + + get _auth() { + + if(!this._config.user) + return {}; + return { + user: this._config.user, + password: this._config.password + }; + } } module.exports = Solr; diff --git a/package-lock.json b/package-lock.json index 30ff4bf..9704f62 100644 --- a/package-lock.json +++ b/package-lock.json @@ -168,7 +168,6 @@ "version": "6.11.0", "resolved": "https://registry.npmjs.org/ajv/-/ajv-6.11.0.tgz", "integrity": "sha512-nCprB/0syFYy9fVYU1ox1l2KN8S9I+tziH8D4zdZuLT3N6RMlGSGt5FSTpAiHB/Whv8Qs1cWHma1aMKZyaHRKA==", - "dev": true, "requires": { "fast-deep-equal": "^3.1.1", "fast-json-stable-stringify": "^2.0.0", @@ -233,18 +232,54 @@ "es-abstract": "^1.17.0-next.1" } }, + "asn1": { + "version": "0.2.4", + "resolved": "https://registry.npmjs.org/asn1/-/asn1-0.2.4.tgz", + "integrity": "sha512-jxwzQpLQjSmWXgwaCZE9Nz+glAG01yF1QnWgbhGwHI5A6FRIEY6IVqtHhIepHqI7/kyEyQEagBC5mBEFlIYvdg==", + "requires": { + "safer-buffer": "~2.1.0" + } + }, + "assert-plus": { + "version": "1.0.0", + "resolved": "https://registry.npmjs.org/assert-plus/-/assert-plus-1.0.0.tgz", + "integrity": "sha1-8S4PPF13sLHN2RRpQuTpbB5N1SU=" + }, "astral-regex": { "version": "1.0.0", "resolved": "https://registry.npmjs.org/astral-regex/-/astral-regex-1.0.0.tgz", "integrity": "sha512-+Ryf6g3BKoRc7jfp7ad8tM4TtMiaWvbF/1/sQcZPkkS7ag3D5nMBCe2UfOTONtAkaG0tO0ij3C5Lwmf1EiyjHg==", "dev": true }, + "asynckit": { + "version": "0.4.0", + "resolved": "https://registry.npmjs.org/asynckit/-/asynckit-0.4.0.tgz", + "integrity": "sha1-x57Zf380y48robyXkLzDZkdLS3k=" + }, + "aws-sign2": { + "version": "0.7.0", + "resolved": "https://registry.npmjs.org/aws-sign2/-/aws-sign2-0.7.0.tgz", + "integrity": "sha1-tG6JCTSpWR8tL2+G1+ap8bP+dqg=" + }, + "aws4": { + "version": "1.9.1", + "resolved": "https://registry.npmjs.org/aws4/-/aws4-1.9.1.tgz", + "integrity": "sha512-wMHVg2EOHaMRxbzgFJ9gtjOOCrI80OHLG14rxi28XwOW8ux6IiEbRCGGGqCtdAIg4FQCbW20k9RsT4y3gJlFug==" + }, "balanced-match": { "version": "1.0.0", "resolved": "https://registry.npmjs.org/balanced-match/-/balanced-match-1.0.0.tgz", "integrity": "sha1-ibTRmasr7kneFk6gK4nORi1xt2c=", "dev": true }, + "bcrypt-pbkdf": { + "version": "1.0.2", + "resolved": "https://registry.npmjs.org/bcrypt-pbkdf/-/bcrypt-pbkdf-1.0.2.tgz", + "integrity": "sha1-pDAdOJtqQ/m2f/PKEaP2Y342Dp4=", + "requires": { + "tweetnacl": "^0.14.3" + } + }, "brace-expansion": { "version": "1.1.11", "resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-1.1.11.tgz", @@ -293,6 +328,11 @@ "integrity": "sha512-P8BjAsXvZS+VIDUI11hHCQEv74YT67YUi5JJFNWIqL235sBmjX4+qx9Muvls5ivyNENctx46xQLQ3aTuE7ssaQ==", "dev": true }, + "caseless": { + "version": "0.12.0", + "resolved": "https://registry.npmjs.org/caseless/-/caseless-0.12.0.tgz", + "integrity": "sha1-G2gcIf+EAzyCZUMJBolCDRhxUdw=" + }, "chalk": { "version": "2.4.2", "resolved": "https://registry.npmjs.org/chalk/-/chalk-2.4.2.tgz", @@ -331,6 +371,17 @@ "integrity": "sha1-/xnt6Kml5XkyQUewwR8PvLq+1jk=", "dev": true }, + "clone-deep": { + "version": "2.0.2", + "resolved": "https://registry.npmjs.org/clone-deep/-/clone-deep-2.0.2.tgz", + "integrity": "sha512-SZegPTKjCgpQH63E+eN6mVEEPdQBOUzjyJm5Pora4lrwWRFS8I0QAxV/KD6vV/i0WuijHZWQC1fMsPEdxfdVCQ==", + "requires": { + "for-own": "^1.0.0", + "is-plain-object": "^2.0.4", + "kind-of": "^6.0.0", + "shallow-clone": "^1.0.0" + } + }, "color-convert": { "version": "1.9.3", "resolved": "https://registry.npmjs.org/color-convert/-/color-convert-1.9.3.tgz", @@ -346,6 +397,14 @@ "integrity": "sha1-p9BVi9icQveV3UIyj3QIMcpTvCU=", "dev": true }, + "combined-stream": { + "version": "1.0.8", + "resolved": "https://registry.npmjs.org/combined-stream/-/combined-stream-1.0.8.tgz", + "integrity": "sha512-FQN4MRfuJeHf7cBbBMJFXhKSDq+2kAArBlmRBvcvFE5BB1HZKXtSFASDhdlz9zOYwxh8lDdnvmMOe/+5cdoEdg==", + "requires": { + "delayed-stream": "~1.0.0" + } + }, "commander": { "version": "2.15.1", "resolved": "https://registry.npmjs.org/commander/-/commander-2.15.1.tgz", @@ -370,6 +429,11 @@ "integrity": "sha1-/ozxhP9mcLa67wGp1IYaXL7EEgo=", "dev": true }, + "core-util-is": { + "version": "1.0.2", + "resolved": "https://registry.npmjs.org/core-util-is/-/core-util-is-1.0.2.tgz", + "integrity": "sha1-tf1UIgqivFq1eqtxQMlAdUUDwac=" + }, "cosmiconfig": { "version": "5.2.1", "resolved": "https://registry.npmjs.org/cosmiconfig/-/cosmiconfig-5.2.1.tgz", @@ -423,6 +487,14 @@ "which": "^1.2.9" } }, + "dashdash": { + "version": "1.14.1", + "resolved": "https://registry.npmjs.org/dashdash/-/dashdash-1.14.1.tgz", + "integrity": "sha1-hTz6D3y+L+1d4gMmuN1YEDX24vA=", + "requires": { + "assert-plus": "^1.0.0" + } + }, "debug": { "version": "4.1.1", "resolved": "https://registry.npmjs.org/debug/-/debug-4.1.1.tgz", @@ -447,6 +519,11 @@ "object-keys": "^1.0.12" } }, + "delayed-stream": { + "version": "1.0.0", + "resolved": "https://registry.npmjs.org/delayed-stream/-/delayed-stream-1.0.0.tgz", + "integrity": "sha1-3zrhmayt+31ECqrgsp4icrJOxhk=" + }, "diff": { "version": "3.5.0", "resolved": "https://registry.npmjs.org/diff/-/diff-3.5.0.tgz", @@ -462,6 +539,15 @@ "esutils": "^2.0.2" } }, + "ecc-jsbn": { + "version": "0.1.2", + "resolved": "https://registry.npmjs.org/ecc-jsbn/-/ecc-jsbn-0.1.2.tgz", + "integrity": "sha1-OoOpBOVDUyh4dMVkt1SThoSamMk=", + "requires": { + "jsbn": "~0.1.0", + "safer-buffer": "^2.1.0" + } + }, "emoji-regex": { "version": "7.0.3", "resolved": "https://registry.npmjs.org/emoji-regex/-/emoji-regex-7.0.3.tgz", @@ -765,6 +851,11 @@ "strip-eof": "^1.0.0" } }, + "extend": { + "version": "3.0.2", + "resolved": "https://registry.npmjs.org/extend/-/extend-3.0.2.tgz", + "integrity": "sha512-fjquC59cD7CyW6urNXK0FBufkZcoiGG80wTuPujX590cB5Ttln20E2UB4S/WARVqhXffZl2LNgS+gQdPIIim/g==" + }, "external-editor": { "version": "3.1.0", "resolved": "https://registry.npmjs.org/external-editor/-/external-editor-3.1.0.tgz", @@ -776,17 +867,20 @@ "tmp": "^0.0.33" } }, + "extsprintf": { + "version": "1.3.0", + "resolved": "https://registry.npmjs.org/extsprintf/-/extsprintf-1.3.0.tgz", + "integrity": "sha1-lpGEQOMEGnpBT4xS48V06zw+HgU=" + }, "fast-deep-equal": { "version": "3.1.1", "resolved": "https://registry.npmjs.org/fast-deep-equal/-/fast-deep-equal-3.1.1.tgz", - "integrity": "sha512-8UEa58QDLauDNfpbrX55Q9jrGHThw2ZMdOky5Gl1CDtVeJDPVrG4Jxx1N8jw2gkWaff5UUuX1KJd+9zGe2B+ZA==", - "dev": true + "integrity": "sha512-8UEa58QDLauDNfpbrX55Q9jrGHThw2ZMdOky5Gl1CDtVeJDPVrG4Jxx1N8jw2gkWaff5UUuX1KJd+9zGe2B+ZA==" }, "fast-json-stable-stringify": { "version": "2.1.0", "resolved": "https://registry.npmjs.org/fast-json-stable-stringify/-/fast-json-stable-stringify-2.1.0.tgz", - "integrity": "sha512-lhd/wF+Lk98HZoTCtlVraHtfh5XYijIjalXck7saUtuanSDyLMxnHhSXEDJqHxD7msR8D0uCmqlkwjCV8xvwHw==", - "dev": true + "integrity": "sha512-lhd/wF+Lk98HZoTCtlVraHtfh5XYijIjalXck7saUtuanSDyLMxnHhSXEDJqHxD7msR8D0uCmqlkwjCV8xvwHw==" }, "fast-levenshtein": { "version": "2.0.6", @@ -838,6 +932,34 @@ "integrity": "sha512-a1hQMktqW9Nmqr5aktAux3JMNqaucxGcjtjWnZLHX7yyPCmlSV3M54nGYbqT8K+0GhF3NBgmJCc3ma+WOgX8Jg==", "dev": true }, + "for-in": { + "version": "1.0.2", + "resolved": "https://registry.npmjs.org/for-in/-/for-in-1.0.2.tgz", + "integrity": "sha1-gQaNKVqBQuwKxybG4iAMMPttXoA=" + }, + "for-own": { + "version": "1.0.0", + "resolved": "https://registry.npmjs.org/for-own/-/for-own-1.0.0.tgz", + "integrity": "sha1-xjMy9BXO3EsE2/5wz4NklMU8tEs=", + "requires": { + "for-in": "^1.0.1" + } + }, + "forever-agent": { + "version": "0.6.1", + "resolved": "https://registry.npmjs.org/forever-agent/-/forever-agent-0.6.1.tgz", + "integrity": "sha1-+8cfDEGt6zf5bFd60e1C2P2sypE=" + }, + "form-data": { + "version": "2.3.3", + "resolved": "https://registry.npmjs.org/form-data/-/form-data-2.3.3.tgz", + "integrity": "sha512-1lLKB2Mu3aGP1Q/2eCOx0fNbRMe7XdwktwOruhfqqd0rIJWwN4Dh+E3hrPSlDCXnSR7UtZ1N38rVXm+6+MEhJQ==", + "requires": { + "asynckit": "^0.4.0", + "combined-stream": "^1.0.6", + "mime-types": "^2.1.12" + } + }, "fs.realpath": { "version": "1.0.0", "resolved": "https://registry.npmjs.org/fs.realpath/-/fs.realpath-1.0.0.tgz", @@ -871,6 +993,14 @@ "pump": "^3.0.0" } }, + "getpass": { + "version": "0.1.7", + "resolved": "https://registry.npmjs.org/getpass/-/getpass-0.1.7.tgz", + "integrity": "sha1-Xv+OPmhNVprkyysSgmBOi6YhSfo=", + "requires": { + "assert-plus": "^1.0.0" + } + }, "glob": { "version": "7.1.6", "resolved": "https://registry.npmjs.org/glob/-/glob-7.1.6.tgz", @@ -903,6 +1033,20 @@ "integrity": "sha512-qBr4OuELkhPenW6goKVXiv47US3clb3/IbuWF9KNKEijAy9oeHxU9IgzjvJhHkUzhaj7rOUD7+YGWqUjLp5oSA==", "dev": true }, + "har-schema": { + "version": "2.0.0", + "resolved": "https://registry.npmjs.org/har-schema/-/har-schema-2.0.0.tgz", + "integrity": "sha1-qUwiJOvKwEeCoNkDVSHyRzW37JI=" + }, + "har-validator": { + "version": "5.1.3", + "resolved": "https://registry.npmjs.org/har-validator/-/har-validator-5.1.3.tgz", + "integrity": "sha512-sNvOCzEQNr/qrvJgc3UG/kD4QtlHycrzwS+6mfTrrSq97BvaYcPZZI1ZSqGSPR73Cxn4LKTD4PttRwfU7jWq5g==", + "requires": { + "ajv": "^6.5.5", + "har-schema": "^2.0.0" + } + }, "has": { "version": "1.0.3", "resolved": "https://registry.npmjs.org/has/-/has-1.0.3.tgz", @@ -936,6 +1080,16 @@ "integrity": "sha512-kssjab8CvdXfcXMXVcvsXum4Hwdq9XGtRD3TteMEvEbq0LXyiNQr6AprqKqfeaDXze7SxWvRxdpwE6ku7ikLkg==", "dev": true }, + "http-signature": { + "version": "1.2.0", + "resolved": "https://registry.npmjs.org/http-signature/-/http-signature-1.2.0.tgz", + "integrity": "sha1-muzZJRFHcvPZW2WmCruPfBj7rOE=", + "requires": { + "assert-plus": "^1.0.0", + "jsprim": "^1.2.2", + "sshpk": "^1.7.0" + } + }, "husky": { "version": "2.7.0", "resolved": "https://registry.npmjs.org/husky/-/husky-2.7.0.tgz", @@ -1186,12 +1340,25 @@ "integrity": "sha1-YTObbyR1/Hcv2cnYP1yFddwVSuE=", "dev": true }, + "is-extendable": { + "version": "0.1.1", + "resolved": "https://registry.npmjs.org/is-extendable/-/is-extendable-0.1.1.tgz", + "integrity": "sha1-YrEQ4omkcUGOPsNqYX1HLjAd/Ik=" + }, "is-fullwidth-code-point": { "version": "2.0.0", "resolved": "https://registry.npmjs.org/is-fullwidth-code-point/-/is-fullwidth-code-point-2.0.0.tgz", "integrity": "sha1-o7MKXE8ZkYMWeqq5O+764937ZU8=", "dev": true }, + "is-plain-object": { + "version": "2.0.4", + "resolved": "https://registry.npmjs.org/is-plain-object/-/is-plain-object-2.0.4.tgz", + "integrity": "sha512-h5PpgXkWitc38BBMYawTYMWJHFZJVnBquFE57xFpjB8pJFiF6gZ+bU+WyI/yqXiFR5mdLsgYNaPe8uao6Uv9Og==", + "requires": { + "isobject": "^3.0.1" + } + }, "is-promise": { "version": "2.1.0", "resolved": "https://registry.npmjs.org/is-promise/-/is-promise-2.1.0.tgz", @@ -1228,6 +1395,11 @@ "has-symbols": "^1.0.1" } }, + "is-typedarray": { + "version": "1.0.0", + "resolved": "https://registry.npmjs.org/is-typedarray/-/is-typedarray-1.0.0.tgz", + "integrity": "sha1-5HnICFjfDBsR3dppQPlgEfzaSpo=" + }, "isarray": { "version": "1.0.0", "resolved": "https://registry.npmjs.org/isarray/-/isarray-1.0.0.tgz", @@ -1240,6 +1412,16 @@ "integrity": "sha1-6PvzdNxVb/iUehDcsFctYz8s+hA=", "dev": true }, + "isobject": { + "version": "3.0.1", + "resolved": "https://registry.npmjs.org/isobject/-/isobject-3.0.1.tgz", + "integrity": "sha1-TkMekrEalzFjaqH5yNHMvP2reN8=" + }, + "isstream": { + "version": "0.1.2", + "resolved": "https://registry.npmjs.org/isstream/-/isstream-0.1.2.tgz", + "integrity": "sha1-R+Y/evVa+m+S4VAOaQ64uFKcCZo=" + }, "istanbul-lib-coverage": { "version": "2.0.5", "resolved": "https://registry.npmjs.org/istanbul-lib-coverage/-/istanbul-lib-coverage-2.0.5.tgz", @@ -1285,6 +1467,11 @@ "esprima": "^4.0.0" } }, + "jsbn": { + "version": "0.1.1", + "resolved": "https://registry.npmjs.org/jsbn/-/jsbn-0.1.1.tgz", + "integrity": "sha1-peZUwuWi3rXyAdls77yoDA7y9RM=" + }, "jsesc": { "version": "2.5.2", "resolved": "https://registry.npmjs.org/jsesc/-/jsesc-2.5.2.tgz", @@ -1297,11 +1484,15 @@ "integrity": "sha512-mrqyZKfX5EhL7hvqcV6WG1yYjnjeuYDzDhhcAAUrq8Po85NBQBJP+ZDUT75qZQ98IkUoBqdkExkukOU7Ts2wrw==", "dev": true }, + "json-schema": { + "version": "0.2.3", + "resolved": "https://registry.npmjs.org/json-schema/-/json-schema-0.2.3.tgz", + "integrity": "sha1-tIDIkuWaLwWVTOcnvT8qTogvnhM=" + }, "json-schema-traverse": { "version": "0.4.1", "resolved": "https://registry.npmjs.org/json-schema-traverse/-/json-schema-traverse-0.4.1.tgz", - "integrity": "sha512-xbbCH5dCYU5T8LcEhhuh7HJ88HXuW3qsI3Y0zOZFKfZEHcpWiHU/Jxzk629Brsab/mMiHQti9wMP+845RPe3Vg==", - "dev": true + "integrity": "sha512-xbbCH5dCYU5T8LcEhhuh7HJ88HXuW3qsI3Y0zOZFKfZEHcpWiHU/Jxzk629Brsab/mMiHQti9wMP+845RPe3Vg==" }, "json-stable-stringify-without-jsonify": { "version": "1.0.1", @@ -1309,12 +1500,33 @@ "integrity": "sha1-nbe1lJatPzz+8wp1FC0tkwrXJlE=", "dev": true }, + "json-stringify-safe": { + "version": "5.0.1", + "resolved": "https://registry.npmjs.org/json-stringify-safe/-/json-stringify-safe-5.0.1.tgz", + "integrity": "sha1-Epai1Y/UXxmg9s4B1lcB4sc1tus=" + }, + "jsprim": { + "version": "1.4.1", + "resolved": "https://registry.npmjs.org/jsprim/-/jsprim-1.4.1.tgz", + "integrity": "sha1-MT5mvB5cwG5Di8G3SZwuXFastqI=", + "requires": { + "assert-plus": "1.0.0", + "extsprintf": "1.3.0", + "json-schema": "0.2.3", + "verror": "1.10.0" + } + }, "just-extend": { "version": "4.0.2", "resolved": "https://registry.npmjs.org/just-extend/-/just-extend-4.0.2.tgz", "integrity": "sha512-FrLwOgm+iXrPV+5zDU6Jqu4gCRXbWEQg2O3SKONsWE4w7AXFRkryS53bpWdaL9cNol+AmR3AEYz6kn+o0fCPnw==", "dev": true }, + "kind-of": { + "version": "6.0.3", + "resolved": "https://registry.npmjs.org/kind-of/-/kind-of-6.0.3.tgz", + "integrity": "sha512-dcS1ul+9tmeD95T+x28/ehLgd9mENa3LsvDTtzm3vyBEO7RPptvAD+t44WVXaUjTBRcrpFeFlC8WCruUR456hw==" + }, "levn": { "version": "0.3.0", "resolved": "https://registry.npmjs.org/levn/-/levn-0.3.0.tgz", @@ -1359,12 +1571,30 @@ "integrity": "sha512-8xOcRHvCjnocdS5cpwXQXVzmmh5e5+saE2QGoeQmbKmRS6J3VQppPOIt0MnmE+4xlZoumy0GPG0D0MVIQbNA1A==", "dev": true }, + "lodash.merge": { + "version": "4.6.2", + "resolved": "https://registry.npmjs.org/lodash.merge/-/lodash.merge-4.6.2.tgz", + "integrity": "sha512-0KpjqXRVvrYyCsX1swR/XTK0va6VQkQM6MNo7PqW77ByjAhoARA8EfrP1N4+KlKj8YS0ZUCtRT/YUuhyYDujIQ==" + }, "lolex": { "version": "4.2.0", "resolved": "https://registry.npmjs.org/lolex/-/lolex-4.2.0.tgz", "integrity": "sha512-gKO5uExCXvSm6zbF562EvM+rd1kQDnB9AZBbiQVzf1ZmdDpxUSvpnAaVOP83N/31mRK8Ml8/VE8DMvsAZQ+7wg==", "dev": true }, + "mime-db": { + "version": "1.43.0", + "resolved": "https://registry.npmjs.org/mime-db/-/mime-db-1.43.0.tgz", + "integrity": "sha512-+5dsGEEovYbT8UY9yD7eE4XTc4UwJ1jBYlgaQQF38ENsKR3wj/8q8RFZrF9WIZpB2V1ArTVFUva8sAul1NzRzQ==" + }, + "mime-types": { + "version": "2.1.26", + "resolved": "https://registry.npmjs.org/mime-types/-/mime-types-2.1.26.tgz", + "integrity": "sha512-01paPWYgLrkqAyrlDorC1uDwl2p3qZT7yl806vW7DvDoxwXi46jsjFbg+WdwotBIk6/MbEhO/dh5aZ5sNj/dWQ==", + "requires": { + "mime-db": "1.43.0" + } + }, "mimic-fn": { "version": "1.2.0", "resolved": "https://registry.npmjs.org/mimic-fn/-/mimic-fn-1.2.0.tgz", @@ -1386,6 +1616,22 @@ "integrity": "sha1-hX/Kv8M5fSYluCKCYuhqp6ARsF0=", "dev": true }, + "mixin-object": { + "version": "2.0.1", + "resolved": "https://registry.npmjs.org/mixin-object/-/mixin-object-2.0.1.tgz", + "integrity": "sha1-T7lJRB2rGCVA8f4DW6YOGUel5X4=", + "requires": { + "for-in": "^0.1.3", + "is-extendable": "^0.1.1" + }, + "dependencies": { + "for-in": { + "version": "0.1.8", + "resolved": "https://registry.npmjs.org/for-in/-/for-in-0.1.8.tgz", + "integrity": "sha1-2Hc5COMSVhCZUrH9ubP6hn0ndeE=" + } + } + }, "mkdirp": { "version": "0.5.1", "resolved": "https://registry.npmjs.org/mkdirp/-/mkdirp-0.5.1.tgz", @@ -1502,6 +1748,19 @@ } } }, + "nock": { + "version": "11.7.2", + "resolved": "https://registry.npmjs.org/nock/-/nock-11.7.2.tgz", + "integrity": "sha512-7swr5bL1xBZ5FctyubjxEVySXOSebyqcL7Vy1bx1nS9IUqQWj81cmKjVKJLr8fHhtzI1MV8nyCdENA/cGcY1+Q==", + "dev": true, + "requires": { + "debug": "^4.1.0", + "json-stringify-safe": "^5.0.1", + "lodash": "^4.17.13", + "mkdirp": "^0.5.0", + "propagate": "^2.0.0" + } + }, "normalize-package-data": { "version": "2.5.0", "resolved": "https://registry.npmjs.org/normalize-package-data/-/normalize-package-data-2.5.0.tgz", @@ -2557,6 +2816,11 @@ } } }, + "oauth-sign": { + "version": "0.9.0", + "resolved": "https://registry.npmjs.org/oauth-sign/-/oauth-sign-0.9.0.tgz", + "integrity": "sha512-fexhUFFPTGV8ybAtSIGbV6gOkSv8UtRbDBnAyLQw4QPKkgNlsH2ByPGtMUqdWkos6YCRmAqViwgZrJc/mRDzZQ==" + }, "object-inspect": { "version": "1.7.0", "resolved": "https://registry.npmjs.org/object-inspect/-/object-inspect-1.7.0.tgz", @@ -2747,6 +3011,11 @@ "pify": "^2.0.0" } }, + "performance-now": { + "version": "2.1.0", + "resolved": "https://registry.npmjs.org/performance-now/-/performance-now-2.1.0.tgz", + "integrity": "sha1-Ywn04OX6kT7BxpMHrjZLSzd8nns=" + }, "pify": { "version": "2.3.0", "resolved": "https://registry.npmjs.org/pify/-/pify-2.3.0.tgz", @@ -2783,6 +3052,17 @@ "integrity": "sha512-7PiHtLll5LdnKIMw100I+8xJXR5gW2QwWYkT6iJva0bXitZKa/XMrSbdmg3r2Xnaidz9Qumd0VPaMrZlF9V9sA==", "dev": true }, + "propagate": { + "version": "2.0.1", + "resolved": "https://registry.npmjs.org/propagate/-/propagate-2.0.1.tgz", + "integrity": "sha512-vGrhOavPSTz4QVNuBNdcNXePNdNMaO1xj9yBeH1ScQPjk/rhg9sSlCXPhMkFuaNNW/syTvYqsnbIJxMBfRbbag==", + "dev": true + }, + "psl": { + "version": "1.7.0", + "resolved": "https://registry.npmjs.org/psl/-/psl-1.7.0.tgz", + "integrity": "sha512-5NsSEDv8zY70ScRnOTn7bK7eanl2MvFrOrS/R6x+dBt5g1ghnj9Zv90kO8GwT8gxcu2ANyFprnFYB85IogIJOQ==" + }, "pump": { "version": "3.0.0", "resolved": "https://registry.npmjs.org/pump/-/pump-3.0.0.tgz", @@ -2796,8 +3076,12 @@ "punycode": { "version": "2.1.1", "resolved": "https://registry.npmjs.org/punycode/-/punycode-2.1.1.tgz", - "integrity": "sha512-XRsRjdf+j5ml+y/6GKHPZbrF/8p2Yga0JPtdqTIY2Xe5ohJPD9saDJJLPvp9+NSBprVvevdXZybnj2cv8OEd0A==", - "dev": true + "integrity": "sha512-XRsRjdf+j5ml+y/6GKHPZbrF/8p2Yga0JPtdqTIY2Xe5ohJPD9saDJJLPvp9+NSBprVvevdXZybnj2cv8OEd0A==" + }, + "qs": { + "version": "6.5.2", + "resolved": "https://registry.npmjs.org/qs/-/qs-6.5.2.tgz", + "integrity": "sha512-N5ZAX4/LxJmF+7wN74pUD6qAh9/wnvdQcjq9TZjevvXzSUo7bfmw91saqMjzGS2xq91/odN2dW/WOl7qQHNDGA==" }, "read-pkg": { "version": "2.0.0", @@ -2826,6 +3110,33 @@ "integrity": "sha512-lv0M6+TkDVniA3aD1Eg0DVpfU/booSu7Eev3TDO/mZKHBfVjgCGTV4t4buppESEYDtkArYFOxTJWv6S5C+iaNw==", "dev": true }, + "request": { + "version": "2.88.0", + "resolved": "https://registry.npmjs.org/request/-/request-2.88.0.tgz", + "integrity": "sha512-NAqBSrijGLZdM0WZNsInLJpkJokL72XYjUpnB0iwsRgxh7dB6COrHnTBNwN0E+lHDAJzu7kLAkDeY08z2/A0hg==", + "requires": { + "aws-sign2": "~0.7.0", + "aws4": "^1.8.0", + "caseless": "~0.12.0", + "combined-stream": "~1.0.6", + "extend": "~3.0.2", + "forever-agent": "~0.6.1", + "form-data": "~2.3.2", + "har-validator": "~5.1.0", + "http-signature": "~1.2.0", + "is-typedarray": "~1.0.0", + "isstream": "~0.1.2", + "json-stringify-safe": "~5.0.1", + "mime-types": "~2.1.19", + "oauth-sign": "~0.9.0", + "performance-now": "^2.1.0", + "qs": "~6.5.2", + "safe-buffer": "^5.1.2", + "tough-cookie": "~2.4.3", + "tunnel-agent": "^0.6.0", + "uuid": "^3.3.2" + } + }, "resolve": { "version": "1.15.0", "resolved": "https://registry.npmjs.org/resolve/-/resolve-1.15.0.tgz", @@ -2884,11 +3195,15 @@ "tslib": "^1.9.0" } }, + "safe-buffer": { + "version": "5.2.0", + "resolved": "https://registry.npmjs.org/safe-buffer/-/safe-buffer-5.2.0.tgz", + "integrity": "sha512-fZEwUGbVl7kouZs1jCdMLdt95hdIv0ZeHg6L7qPeciMZhZ+/gdesW4wgTARkrFWEpspjEATAzUGPG8N2jJiwbg==" + }, "safer-buffer": { "version": "2.1.2", "resolved": "https://registry.npmjs.org/safer-buffer/-/safer-buffer-2.1.2.tgz", - "integrity": "sha512-YZo3K82SD7Riyi0E1EQPojLz7kpepnSQI9IyPbHHg1XXXevb5dJI7tpyN2ADxGcQbHG7vcyRHk0cbwqcQriUtg==", - "dev": true + "integrity": "sha512-YZo3K82SD7Riyi0E1EQPojLz7kpepnSQI9IyPbHHg1XXXevb5dJI7tpyN2ADxGcQbHG7vcyRHk0cbwqcQriUtg==" }, "semver": { "version": "5.7.1", @@ -2902,6 +3217,23 @@ "integrity": "sha1-De4hahyUGrN+nvsXiPavxf9VN/w=", "dev": true }, + "shallow-clone": { + "version": "1.0.0", + "resolved": "https://registry.npmjs.org/shallow-clone/-/shallow-clone-1.0.0.tgz", + "integrity": "sha512-oeXreoKR/SyNJtRJMAKPDSvd28OqEwG4eR/xc856cRGBII7gX9lvAqDxusPm0846z/w/hWYjI1NpKwJ00NHzRA==", + "requires": { + "is-extendable": "^0.1.1", + "kind-of": "^5.0.0", + "mixin-object": "^2.0.1" + }, + "dependencies": { + "kind-of": { + "version": "5.1.0", + "resolved": "https://registry.npmjs.org/kind-of/-/kind-of-5.1.0.tgz", + "integrity": "sha512-NGEErnH6F2vUuXDh+OlbcKW7/wOcfdRHaZ7VWtqCztfHri/++YKmP51OdWeGPuqCOba6kk2OTe5d02VmTB80Pw==" + } + } + }, "shebang-command": { "version": "1.2.0", "resolved": "https://registry.npmjs.org/shebang-command/-/shebang-command-1.2.0.tgz", @@ -2999,6 +3331,22 @@ "integrity": "sha1-BOaSb2YolTVPPdAVIDYzuFcpfiw=", "dev": true }, + "sshpk": { + "version": "1.16.1", + "resolved": "https://registry.npmjs.org/sshpk/-/sshpk-1.16.1.tgz", + "integrity": "sha512-HXXqVUq7+pcKeLqqZj6mHFUMvXtOJt1uoUx09pFW6011inTMxqI8BA8PM95myrIyyKwdnzjdFjLiE6KBPVtJIg==", + "requires": { + "asn1": "~0.2.3", + "assert-plus": "^1.0.0", + "bcrypt-pbkdf": "^1.0.0", + "dashdash": "^1.12.0", + "ecc-jsbn": "~0.1.1", + "getpass": "^0.1.1", + "jsbn": "~0.1.0", + "safer-buffer": "^2.0.2", + "tweetnacl": "~0.14.0" + } + }, "string-width": { "version": "2.1.1", "resolved": "https://registry.npmjs.org/string-width/-/string-width-2.1.1.tgz", @@ -3056,6 +3404,15 @@ "integrity": "sha1-PFMZQukIwml8DsNEhYwobHygpgo=", "dev": true }, + "superstruct": { + "version": "0.6.1", + "resolved": "https://registry.npmjs.org/superstruct/-/superstruct-0.6.1.tgz", + "integrity": "sha512-LDbOKL5sNbOJ00Q36iYRhSexKIptZje0/mhNznnz04wT9CmsPDZg/K/UV1dgYuCwNMuOBHTbVROZsGB9EhhK4w==", + "requires": { + "clone-deep": "^2.0.1", + "kind-of": "^6.0.1" + } + }, "supports-color": { "version": "5.5.0", "resolved": "https://registry.npmjs.org/supports-color/-/supports-color-5.5.0.tgz", @@ -3132,12 +3489,41 @@ "integrity": "sha1-3F5pjL0HkmW8c+A3doGk5Og/YW4=", "dev": true }, + "tough-cookie": { + "version": "2.4.3", + "resolved": "https://registry.npmjs.org/tough-cookie/-/tough-cookie-2.4.3.tgz", + "integrity": "sha512-Q5srk/4vDM54WJsJio3XNn6K2sCG+CQ8G5Wz6bZhRZoAe/+TxjWB/GlFAnYEbkYVlON9FMk/fE3h2RLpPXo4lQ==", + "requires": { + "psl": "^1.1.24", + "punycode": "^1.4.1" + }, + "dependencies": { + "punycode": { + "version": "1.4.1", + "resolved": "https://registry.npmjs.org/punycode/-/punycode-1.4.1.tgz", + "integrity": "sha1-wNWmOycYgArY4esPpSachN1BhF4=" + } + } + }, "tslib": { "version": "1.10.0", "resolved": "https://registry.npmjs.org/tslib/-/tslib-1.10.0.tgz", "integrity": "sha512-qOebF53frne81cf0S9B41ByenJ3/IuH8yJKngAX35CmiZySA0khhkovshKK+jGCaMnVomla7gVlIcc3EvKPbTQ==", "dev": true }, + "tunnel-agent": { + "version": "0.6.0", + "resolved": "https://registry.npmjs.org/tunnel-agent/-/tunnel-agent-0.6.0.tgz", + "integrity": "sha1-J6XeoGs2sEoKmWZ3SykIaPD8QP0=", + "requires": { + "safe-buffer": "^5.0.1" + } + }, + "tweetnacl": { + "version": "0.14.5", + "resolved": "https://registry.npmjs.org/tweetnacl/-/tweetnacl-0.14.5.tgz", + "integrity": "sha1-WuaBd/GS1EViadEIr6k/+HQ/T2Q=" + }, "type-check": { "version": "0.3.2", "resolved": "https://registry.npmjs.org/type-check/-/type-check-0.3.2.tgz", @@ -3163,11 +3549,15 @@ "version": "4.2.2", "resolved": "https://registry.npmjs.org/uri-js/-/uri-js-4.2.2.tgz", "integrity": "sha512-KY9Frmirql91X2Qgjry0Wd4Y+YTdrdZheS8TFwvkbLWf/G5KNJDCh6pKL5OZctEW4+0Baa5idK2ZQuELRwPznQ==", - "dev": true, "requires": { "punycode": "^2.1.0" } }, + "uuid": { + "version": "3.4.0", + "resolved": "https://registry.npmjs.org/uuid/-/uuid-3.4.0.tgz", + "integrity": "sha512-HjSDRw6gZE5JMggctHBcjVak08+KEVhSIiDzFnT9S9aegmp85S/bReBVTb4QTFaRNptJ9kuYaNhnbNEOkbKb/A==" + }, "validate-npm-package-license": { "version": "3.0.4", "resolved": "https://registry.npmjs.org/validate-npm-package-license/-/validate-npm-package-license-3.0.4.tgz", @@ -3178,6 +3568,16 @@ "spdx-expression-parse": "^3.0.0" } }, + "verror": { + "version": "1.10.0", + "resolved": "https://registry.npmjs.org/verror/-/verror-1.10.0.tgz", + "integrity": "sha1-OhBcoXBTr1XW4nDB+CiGguGNpAA=", + "requires": { + "assert-plus": "^1.0.0", + "core-util-is": "1.0.2", + "extsprintf": "^1.2.0" + } + }, "which": { "version": "1.3.1", "resolved": "https://registry.npmjs.org/which/-/which-1.3.1.tgz", diff --git a/package.json b/package.json index 8fb0934..76f39a5 100644 --- a/package.json +++ b/package.json @@ -2,13 +2,13 @@ "name": "@janiscommerce/solr", "version": "1.0.0", "description": "Apache Solr Driver", - "main": "index.js", + "main": "./lib/solr.js", "scripts": { "test": "export TEST_ENV=true; mocha --exit -R nyan --recursive tests/", "test-ci": "nyc --reporter=html --reporter=text mocha --recursive tests/", "watch-test": "export TEST_ENV=true; mocha --exit -R nyan -w --recursive tests/", "coverage": "nyc npm test", - "lint": "eslint index.js lib/ tests/" + "lint": "eslint lib/ tests/" }, "repository": { "type": "git", @@ -18,11 +18,12 @@ "license": "ISC", "homepage": "https://github.com/janis-commerce/solr.git#readme", "devDependencies": { - "husky": "^2.4.1", "eslint": "^5.16.0", "eslint-config-airbnb-base": "^13.1.0", "eslint-plugin-import": "^2.17.3", + "husky": "^2.4.1", "mocha": "^5.2.0", + "nock": "^11.7.2", "nyc": "^13.1.0", "sinon": "^7.3.2" }, @@ -31,5 +32,11 @@ ], "directories": { "test": "tests" + }, + "dependencies": { + "lodash.merge": "^4.6.2", + "request": "^2.88.0", + "superstruct": "0.6.1", + "uuid": "^3.4.0" } } diff --git a/tests/helpers/filters.js b/tests/helpers/filters.js new file mode 100644 index 0000000..e6821e0 --- /dev/null +++ b/tests/helpers/filters.js @@ -0,0 +1,88 @@ +'use strict'; + +const assert = require('assert'); + +const SolrError = require('../../lib/solr-error'); + +const Filters = require('../../lib/helpers/filters'); + +describe('Helpers', () => { + + describe('Filters', () => { + + describe('build()', () => { + + const modelFields = { + customField: { + field: 'equal', + type: 'notEqual' + }, + customDefault: { + type: 'greaterOrEqual' + } + }; + + it('Should build the filters for the received fields and model fields', () => { + + const filters = Filters.build({ + + equalDefault: 'something', + customDefault: 32, + equal: { type: 'equal', value: 'something' }, + notEqual: { type: 'notEqual', value: 'something' }, + greater: { type: 'greater', value: 10 }, + greaterOrEqual: { type: 'greaterOrEqual', value: 10 }, + lesser: { type: 'lesser', value: 10 }, + lesserOrEqual: { type: 'lesserOrEqual', value: 10 }, + multiFilters: ['some', 'other', { type: 'greater', value: 5 }], + customField: { value: 'foobar' } + + }, modelFields); + + assert.deepStrictEqual(filters, [ + 'equalDefault:"something"', + 'customDefault:[32 TO *]', + 'equal:"something" OR -equal:"foobar"', + '-notEqual:"something"', + 'greater:{10 TO *}', + 'greaterOrEqual:[10 TO *]', + 'lesser:{* TO 10}', + 'lesserOrEqual:[* TO 10]', + 'multiFilters:"some" OR multiFilters:"other" OR multiFilters:{5 TO *}' + ]); + }); + + [ + + { field: { type: 'equal', value: ['array'] } }, + { field: { type: 'equal', value: { an: 'object' } } } + + ].forEach(filters => { + + it('Should throw when the received filters is not supported', () => { + + assert.throws(() => Filters.build(filters), { + name: 'SolrError', + code: SolrError.codes.UNSUPPORTED_FILTER + }); + }); + }); + + it('Should throw when the received filter doesn\'t have a value', () => { + + assert.throws(() => Filters.build({ field: { type: 'equal' } }), { + name: 'SolrError', + code: SolrError.codes.INVALID_FILTER_VALUE + }); + }); + + it('Should throw when the received filter have an unknown or unsupported type', () => { + + assert.throws(() => Filters.build({ field: { type: 'unknown', value: 'some' } }), { + name: 'SolrError', + code: SolrError.codes.INVALID_FILTER_TYPE + }); + }); + }); + }); +}); diff --git a/tests/solr-test.js b/tests/solr-test.js index 28636fb..d2fe4f5 100644 --- a/tests/solr-test.js +++ b/tests/solr-test.js @@ -1,15 +1,982 @@ 'use strict'; -const assert = require('assert'); - +const assert = require('assert'); + +const nock = require('nock'); + const sandbox = require('sinon').createSandbox(); -const Solr = require('./../index'); +const { base64 } = require('../lib/helpers/utils'); + +const Solr = require('../lib/solr'); -const SolrError = require('./../lib/solr-error') +const SolrError = require('../lib/solr-error'); describe('Solr', () => { - // your tests here... + afterEach(() => { + sandbox.restore(); + nock.cleanAll(); + }); + + class FakeModel { + + static get fields() { + return { + some: { + type: 'notEqual' + }, + myCustomField: { + field: 'date', + type: 'greaterOrEqual' + } + }; + } + + static get schema() { + return { + string: true, + number: { type: 'number' }, + float: { type: 'float' }, + double: { type: 'double' }, + long: { type: 'long' }, + boolean: { type: 'boolean' }, + date: { type: 'date' }, + array: { type: ['string'] }, + object: { + type: { + property: 'string', + subproperty: { + property: ['number'] + } + } + } + }; + } + } + + const builtSchemas = [ + { + name: 'string', + type: 'string' + }, + { + name: 'number', + type: 'pint' + }, + { + name: 'float', + type: 'pfloat' + }, + { + name: 'double', + type: 'pdouble' + }, + { + name: 'long', + type: 'plong' + }, + { + name: 'boolean', + type: 'boolean' + }, + { + name: 'date', + type: 'pdate' + }, + { + name: 'array', + type: 'string', + multiValued: true + }, + { + name: 'object.property', + type: 'string' + }, + { + name: 'object.subproperty.property', + type: 'pint', + multiValued: true + } + ]; + + const host = 'http://some-host.com'; + + const endpoints = { + update: '/solr/some-core/update/json/docs?commit=true', + updateCommands: '/solr/some-core/update?commit=true', + get: '/solr/some-core/query', + schema: '/solr/some-core/schema' + }; + + const model = new FakeModel(); + + const solr = new Solr({ + url: host, + core: 'some-core' + }); + + describe('constructor', () => { + + [ + + null, + undefined, + 1, + 'string', + ['array'], + { invalid: 'config' }, + { url: ['not a string'], core: 'valid' }, + { url: 'valid', core: ['not a string'] }, + { url: 'valid', core: 'valid', user: ['not a string'], password: 'valid' }, + { url: 'valid', core: 'valid', user: ['not a string'], password: ['not a string'] }, + { url: 'valid', core: 'valid', user: 'valid', password: ['not a string'] }, + { url: 'valid', core: 'valid', user: 'valid' }, + { url: 'valid', core: 'valid', password: 'valid' } + + ].forEach(config => { + + it('Should throw when the received config is invalid', async () => { + assert.throws(() => new Solr(config), { + name: 'SolrError', + code: SolrError.codes.INVALID_CONFIG + }); + }); + }); + + }); + + describe('insert()', () => { + + const item = { + id: 'some-id', + some: 'data' + }; + + it('Should call Solr POST api to insert the received item', async () => { + + const request = nock(host) + .post(endpoints.update, item) + .reply(200, { + responseHeader: { + status: 0 + } + }); + + const result = await solr.insert(model, item); + + assert.deepEqual(result, item.id); + request.done(); + }); + + [ + + null, + undefined, + 'string', + 1, + ['array'] + + ].forEach(invalidItem => { + + it('Should throw when the received item is not an object', async () => { + + await assert.rejects(solr.insert(model, invalidItem), { + name: 'SolrError', + code: SolrError.codes.INVALID_PARAMETERS + }); + }); + }); + + it('Should throw when the Solr response code is bigger or equal than 400', async () => { + + const request = nock(host) + .post(endpoints.update, item) + .reply(400, { + responseHeader: { + status: 400 + } + }); + + await assert.rejects(solr.insert(model, item), { + name: 'SolrError', + code: SolrError.codes.REQUEST_FAILED + }); + + request.done(); + }); + + it('Should throw when the Solr response is invalid', async () => { + + const request = nock(host) + .post(endpoints.update) + .reply(200, { + responseHeader: { + status: 1 + } + }); + + await assert.rejects(solr.insert(model, item), { + name: 'SolrError', + code: SolrError.codes.INTERNAL_SOLR_ERROR + }); + + request.done(); + }); + }); + + describe('multiInsert()', () => { + + const items = [ + { + id: 'some-id', + some: 'data' + }, + { + id: 'other-id', + other: 'data' + } + ]; + + it('Should call Solr POST api to insert the received items', async () => { + + const request = nock(host) + .post(endpoints.update, items) + .reply(200, { + responseHeader: { + status: 0 + } + }); + + const result = await solr.multiInsert(model, items); + + assert.deepStrictEqual(result, items); + request.done(); + }); + + [ + null, + undefined, + 'string', + 1, + { not: 'an array' } + + ].forEach(invalidItem => { + + it('Should throw when the received items are not an array', async () => { + + await assert.rejects(solr.multiInsert(model, invalidItem), { + name: 'SolrError', + code: SolrError.codes.INVALID_PARAMETERS + }); + }); + }); + + it('Should throw when the Solr response code is bigger or equal than 400', async () => { + + const request = nock(host) + .post(endpoints.update, items) + .reply(400, { + responseHeader: { + status: 400 + } + }); + + await assert.rejects(solr.multiInsert(model, items), { + name: 'SolrError', + code: SolrError.codes.REQUEST_FAILED + }); + + request.done(); + }); + + it('Should throw when the Solr response is invalid', async () => { + + const request = nock(host) + .post(endpoints.update) + .reply(200, { + responseHeader: { + status: 1 + } + }); + + await assert.rejects(solr.multiInsert(model, items), { + name: 'SolrError', + code: SolrError.codes.INTERNAL_SOLR_ERROR + }); + + request.done(); + }); + }); + + describe('get()', () => { + + it('Should call Solr GET api to get the items and format it correctly', async () => { + + const request = nock(host) + .get(endpoints.get, { + query: '*:*', + offset: 0, + limit: 500 + }) + .reply(200, { + responseHeader: { + status: 0 + }, + response: { + docs: [ + { + id: 'some-id', + some: 'data', + 'object.property': 'some-property', + 'object.subproperty.property': [1, 2, 3], + _version_: 1122111221 + } + ] + } + }); + + const result = await solr.get(model); + + assert.deepStrictEqual(result, [ + { + id: 'some-id', + some: 'data', + object: { + property: 'some-property', + subproperty: { + property: [1, 2, 3] + } + } + } + ]); + + request.done(); + }); + + it('Should call Solr GET api to get the items with the specified params', async () => { + + const request = nock(host) + .get(endpoints.get, { + query: '*:*', + offset: 5, + limit: 5, + filter: [ + 'id:"some-id"', + '-some:"data"', + 'date:[10 TO *] OR date:[11 TO *] OR date:[12 TO *]' + ], + sort: 'id asc, some desc' + }) + .reply(200, { + responseHeader: { + status: 0 + }, + response: { + docs: [ + { + id: 'some-id', + some: 'data' + } + ] + } + }); + + const result = await solr.get(model, { + limit: 5, + page: 2, + order: { id: 'asc', some: 'desc' }, + filters: { + id: 'some-id', + some: 'data', + myCustomField: [10, 11, 12] + } + }); + + assert.deepStrictEqual(result, [ + { + id: 'some-id', + some: 'data' + } + ]); + + request.done(); + }); + + it('Should call Solr GET api to get the items using auth credentials', async () => { + + const authorizedSolr = new Solr({ + url: host, + core: 'some-core', + user: 'some-user', + password: 'some-password' + }); + + const expectedCredentials = `Basic ${base64('some-user:some-password')}`; + + const request = nock(host, { reqheaders: { Authorization: expectedCredentials } }) + .get(endpoints.get, { + query: '*:*', + offset: 0, + limit: 500 + }) + .reply(200, { + responseHeader: { + status: 0 + }, + response: { + docs: [ + { + id: 'some-id', + some: 'data', + 'object.property': 'some-property', + 'object.subproperty.property': [1, 2, 3], + _version_: 1122111221 + } + ] + } + }); + + const result = await authorizedSolr.get(model); + + assert.deepStrictEqual(result, [ + { + id: 'some-id', + some: 'data', + object: { + property: 'some-property', + subproperty: { + property: [1, 2, 3] + } + } + } + ]); + + request.done(); + }); + + it('Should throw when the Solr response code is bigger or equal than 400', async () => { + + const request = nock(host) + .get(endpoints.get) + .reply(400, { + responseHeader: { + status: 400 + } + }); + + await assert.rejects(solr.get(model), { + name: 'SolrError', + code: SolrError.codes.REQUEST_FAILED + }); + + request.done(); + }); + + it('Should throw when the Solr response is invalid', async () => { + + const request = nock(host) + .get(endpoints.get) + .reply(200, { + responseHeader: { + status: 0 + }, + response: {} + }); + + await assert.rejects(solr.get(model), { + name: 'SolrError', + code: SolrError.codes.INTERNAL_SOLR_ERROR + }); + + request.done(); + }); + + it('Should throw when the received model is invalid', async () => { + + sandbox.stub(FakeModel, 'fields') + .get(() => ['not an object']); + + await assert.rejects(solr.get(model), { + name: 'SolrError', + code: SolrError.codes.INVALID_MODEL + }); + }); + }); + + describe('getTotals()', () => { + + afterEach(() => { + delete model.lastQueryHasResults; + delete model.totalsParams; + }); + + it('Should call Solr GET api to get the items count', async () => { + + const request = nock(host) + .get(endpoints.get, { + query: '*:*', + limit: 500, + offset: 0 + }) + .reply(200, { + responseHeader: { + status: 0 + }, + response: { + numFound: 10, + docs: Array(10).fill({ item: 'some-item' }) + } + }) + .persist(); + + await solr.get(model); + + const result = await solr.getTotals(model); + + assert.deepStrictEqual(result, { + total: 10, + pageSize: 500, + pages: 1, + page: 1 + }); + + request.done(); + }); + + it('Should return the default empty results when get can \'t find any item', async () => { + + const request = nock(host) + .get(endpoints.get, { + query: '*:*', + limit: 500, + offset: 0 + }) + .reply(200, { + responseHeader: { + status: 0 + }, + response: { + numFound: 0, + docs: [] + } + }); + + await solr.get(model); + + const result = await solr.getTotals(model); + + assert.deepStrictEqual(result, { total: 0, pages: 0 }); + + request.done(); + }); + + it('Should return the default empty results when get was not called', async () => { + const result = await solr.getTotals(model); + assert.deepStrictEqual(result, { total: 0, pages: 0 }); + }); + }); + + describe('remove()', () => { + + const item = { + id: 'some-id', + some: 'data' + }; + + it('Should call Solr POST api to delete the received item', async () => { + + const request = nock(host) + .post(endpoints.updateCommands, { + delete: { id: item.id } + }) + .reply(200, { + responseHeader: { + status: 0 + } + }); + + await assert.doesNotReject(solr.remove(model, item)); + + request.done(); + }); + + [ + + null, + undefined, + 'string', + 1, + ['array'], + { item: 'without id' } + + ].forEach(invalidItem => { + + it('Should throw when the received item is not an object or not have ID', async () => { + + await assert.rejects(solr.remove(model, invalidItem), { + name: 'SolrError', + code: SolrError.codes.INVALID_PARAMETERS + }); + }); + }); + + it('Should throw when the Solr response code is bigger or equal than 400', async () => { + + const request = nock(host) + .post(endpoints.updateCommands, { + delete: { id: item.id } + }) + .reply(400, { + responseHeader: { + status: 400 + } + }); + + await assert.rejects(solr.remove(model, item), { + name: 'SolrError', + code: SolrError.codes.REQUEST_FAILED + }); + + request.done(); + }); + + it('Should throw when the Solr response is invalid', async () => { + + const request = nock(host) + .post(endpoints.updateCommands, { + delete: { id: item.id } + }) + .reply(200, { + responseHeader: { + status: 1 + } + }); + + await assert.rejects(solr.remove(model, item), { + name: 'SolrError', + code: SolrError.codes.INTERNAL_SOLR_ERROR + }); + + request.done(); + }); + }); + + describe('multiRemove()', () => { + + it('Should call Solr POST api to delete by the received filters', async () => { + + const request = nock(host) + .post(endpoints.updateCommands, { + delete: { query: 'field:"value" AND otherField:[* TO 10]' } + }) + .reply(200, { + responseHeader: { + status: 0 + } + }); + + await assert.doesNotReject(solr.multiRemove(model, { field: 'value', otherField: { type: 'lesserOrEqual', value: 10 } })); + + request.done(); + }); + + it('Should throw when the Solr response code is bigger or equal than 400', async () => { + + const request = nock(host) + .post(endpoints.updateCommands, {}) + .reply(400, { + responseHeader: { + status: 400 + } + }); + + await assert.rejects(solr.multiRemove(model), { + name: 'SolrError', + code: SolrError.codes.REQUEST_FAILED + }); + + request.done(); + }); + + it('Should throw when the Solr response is invalid', async () => { + + const request = nock(host) + .post(endpoints.updateCommands, { + delete: { query: 'field:"value"' } + }) + .reply(200, { + responseHeader: { + status: 1 + } + }); + + await assert.rejects(solr.multiRemove(model, { field: 'value' }), { + name: 'SolrError', + code: SolrError.codes.INTERNAL_SOLR_ERROR + }); + + request.done(); + }); + + it('Should throw when the received model is invalid', async () => { + + await assert.rejects(solr.multiRemove(), { + name: 'SolrError', + code: SolrError.codes.INVALID_MODEL + }); + }); + }); + + describe('distinct()', () => { + + it('Should call Solr GET api to get the items distinct and format it correctly', async () => { + + const request = nock(host) + .get(endpoints.get, { + query: '*:*', + fields: 'someField', + params: { + group: true, + 'group.field': 'someField' + }, + filter: ['otherField:"true"'] + }) + .reply(200, { + responseHeader: { + status: 0 + }, + grouped: { + someField: { + groups: [ + { + doclist: { docs: [{ someField: 'some' }] } + }, + { + doclist: { docs: [{ someField: 'other' }] } + } + ] + } + } + }); + + const result = await solr.distinct(model, { key: 'someField', filters: { otherField: true } }); + + assert.deepStrictEqual(result, ['some', 'other']); + + request.done(); + }); + + it('Should throw when the received key is not a string or not exists', async () => { + + await assert.rejects(solr.distinct(model, { key: ['not a string'] }), { + name: 'SolrError', + code: SolrError.codes.INVALID_PARAMETERS + }); + }); + + it('Should throw when the Solr response code is bigger or equal than 400', async () => { + + const request = nock(host) + .get(endpoints.get, { + query: '*:*', + fields: 'someField', + params: { + group: true, + 'group.field': 'someField' + } + }) + .reply(400, { + responseHeader: { + status: 400 + } + }); + + await assert.rejects(solr.distinct(model, { key: 'someField' }), { + name: 'SolrError', + code: SolrError.codes.REQUEST_FAILED + }); + + request.done(); + }); + + it('Should throw when the Solr response is invalid', async () => { + + const request = nock(host) + .get(endpoints.get, { + query: '*:*', + fields: 'someField', + params: { + group: true, + 'group.field': 'someField' + } + }) + .reply(200, { + responseHeader: { + status: 1 + } + }); + + await assert.rejects(solr.distinct(model, { key: 'someField' }), { + name: 'SolrError', + code: SolrError.codes.INTERNAL_SOLR_ERROR + }); + + request.done(); + }); + + it('Should throw when the received model is invalid', async () => { + + await assert.rejects(solr.distinct(), { + name: 'SolrError', + code: SolrError.codes.INVALID_MODEL + }); + }); + }); + + describe('createSchema()', () => { + + it('Should call Solr POST api to create the schema using core name from config', async () => { + + const request = nock(host) + .post(endpoints.schema, { + 'add-field': builtSchemas + }) + .reply(200, { + responseHeader: { + status: 0 + } + }); + + await assert.doesNotReject(solr.createSchema(model)); + + request.done(); + }); + + it('Should call Solr POST api to create the schema using the specified core name', async () => { + + const request = nock(host) + .post('/solr/foobar-core/schema', { + 'add-field': builtSchemas + }) + .reply(200, { + responseHeader: { + status: 0 + } + }); + + await assert.doesNotReject(solr.createSchema(model, 'foobar-core')); + + request.done(); + }); + + it('Should return an empty object when there are no schema in the model', async () => { + + sandbox.stub(FakeModel, 'schema') + .get(() => undefined); + + const request = nock(host) + .post(endpoints.schema) + .reply(200); + + await assert.doesNotReject(solr.createSchema(model)); + + assert.deepEqual(request.isDone(), false); + }); + + it('Should throw when the model is invalid', async () => { + + sandbox.stub(FakeModel, 'schema') + .get(() => ['not an object']); + + await assert.rejects(solr.createSchema(model), { + name: 'SolrError', + code: SolrError.codes.INVALID_MODEL + }); + }); + }); + + describe('updateSchema()', () => { + + it('Should call Solr POST api to create the schema', async () => { + + const request = nock(host) + .post(endpoints.schema, { + 'replace-field': builtSchemas + }) + .reply(200, { + responseHeader: { + status: 0 + } + }); + + await assert.doesNotReject(solr.updateSchema(model)); + + request.done(); + }); + + it('Should return an empty object when there are no schema in the model', async () => { + + sandbox.stub(FakeModel, 'schema') + .get(() => undefined); + + const request = nock(host) + .post(endpoints.schema) + .reply(200); + + await assert.doesNotReject(solr.updateSchema(model)); + + assert.deepEqual(request.isDone(), false); + }); + + it('Should throw when the model is invalid', async () => { + + await assert.rejects(solr.updateSchema(), { + name: 'SolrError', + code: SolrError.codes.INVALID_MODEL + }); + }); + }); + + describe('createCore()', () => { + + it('Should create a core into the Solr URL', async () => { + + sandbox.stub(Solr.prototype, 'createSchema') + .returns(); + + const request = nock(host) + .post('/solr/admin/cores?action=CREATE&name=new-core&configSet=_default') + .reply(200, { + responseHeader: { + status: 0 + } + }); + + await assert.doesNotReject(solr.createCore(model, 'new-core')); + + request.done(); + sandbox.assert.calledOnce(Solr.prototype.createSchema); + sandbox.assert.calledWithExactly(Solr.prototype.createSchema, model, 'new-core'); + }); + + it('Should not reject when the received model is invalid', async () => { + + sandbox.stub(Solr.prototype, 'createSchema') + .returns(); + + const request = nock(host) + .post('/solr/admin/cores?action=CREATE&name=new-core&configSet=_default') + .reply(200, { + responseHeader: { + status: 0 + } + }); + + await assert.doesNotReject(solr.createCore(null, 'new-core')); + assert.deepEqual(request.isDone(), false); + sandbox.assert.notCalled(Solr.prototype.createSchema); + }); + }); });