Solr adapter for Sails.js and Waterline
JavaScript Other
Latest commit 0f7ca65 Jan 21, 2017 @sajov committed on GitHub Merge pull request #3 from marcbachmann/update-uuid
Update uuid to version 3.0.0
Permalink
Failed to load latest commit information.
api finalize release Jan 2, 2016
bin exceed sold startup memory Jan 4, 2016
lib chore(package): update uuid to version 3.0.0 Nov 23, 2016
test add mocha opts Jan 10, 2016
.coveralls.yml no message Jan 3, 2016
.editorconfig init Nov 27, 2015
.eslintrc init Nov 27, 2015
.gitignore init Nov 27, 2015
.travis.yml no message Jan 3, 2016
CONTRIBUTING.md no message Nov 28, 2015
Makefile make kickstart Jan 5, 2016
README.md update README Jan 5, 2016
package.json chore(package): update uuid to version 3.0.0 Nov 23, 2016
solrAdapter.js init Nov 27, 2015

README.md

sails-solr

Provides easy access to solr from Sails.js & Waterline.

This module is a Sails/Waterline adapter, an early implementation of a rapidly-developing, tool-agnostic data standard. Its goal is to provide a set of declarative interfaces, conventions, and best-practices for integrating with all sorts of data sources.

The main goal is a simple usage and integration of a full managed Solr.

Build Status Coverage Status Dependency Status

NPM

Features

  • Waterline interface support for Semantic, Migratable and Queryable
  • Suggester and Spell checking for Autocomplete as Model Method and Shadow Route Action
  • Catalog with Layerd Navigation (Range and Facet Filter) as Model Method and Shadow Route Action
  • Solr Config API full solrconfig management
    • add, update SearchComponents
    • add, update RequestHandler
  • Solr Schema API full schema management
    • add, update fieldTypes
    • add, update Fields
    • add, update dynamicFields
    • add, update copyFields
  • Multi Model at single Core
  • Raw solr-hyperquest-client access

Installation

To install this adapter, run:

$ npm install sails-solr

Getting started with sails-solr

To install/start solr if you not have one running

make kickstart

Note: not recommended for production systems! Solr installation Tomcat for more

Configuring Connections

Add the solr configuration to the config/connections.js file. The basic options are as follows:

module.exports.connections = {
  solrConnectionOne: {
    module : 'sails-solr',
    host: 'localhost',
    port: 8983,
    core: 'schemaless',
    schema: true,
    migrate: 'drop'
  }
};

Note: you can define multiple solr connections/cores. By default sails-solr will run multiple models inside one core manageCores. Connection Options

Configuring Models

And then change default model configuration to the config/models.js:

module.exports.models = {
  connection: 'solrConnectionOne',
  attributes: {
    name:'string'
    ...
  }
};

Note: you can add more model based configuartion Model Options / Connection Options

Usage

create a user:

  User.create({name:'foo'},console)

find a user:

  User.find({name:'foo'},console);
  User.findOne({name:'foo'},console);
  User.findByName('foo',console);

Note: See Waterline Documentation Query Language and Query Methods

Special Adapter Interfaces

Autocompleter

search suggestion and spellchecked phrase. Known as "Did You Mean: foo?"

  // as sails request  see hooks and blueprint
  // http://localhost:1337/user/suggest/foa

  // in node
  User.suggest('foa', console);

  //response
  {
  "responseHeader": {
    "status": 0,
    "QTime": 1
  },
  "spellcheck": {
    "suggestions": [
      "foa",
      {
        "numFound": 1,
        "startOffset": 0,
        "endOffset": 9,
        "origFreq": 0,
        "suggestion": [
          {
            "word": "foo",
            "freq": 1
          }
        ]
      }
    ],
    "correctlySpelled": false,
    "collations": [
      "collation",
      "foo"
    ]
  },
  "suggest": {
    "suggest": {
      "foa": {
        "numFound": 2,
        "suggestions": [
          {
            "term": "foo",
            "weight": 0,
            "payload": ""
          },{
            "term": "foo bar",
            "weight": 0,
            "payload": ""
          }
        ]
      }
    }
  }
}

Layerd Navigation

Well known as filter. facet for strings and min,max,avg for ìnteger to build Options and Range Slider Elemets. Query Methods

  // as sails request  see hooks and blueprint
  // http://localhost:1337/user/catalog/?name="*"&limit=3&sort=age asc&skip=0

  // in node
  User.catalog({name:'foo'},console);

  //response
  {
  "responseHeader":{
    "status":0,
    "QTime":1,
    "params":{
      "q":"*:*",
      "indent":"true",
      "stats":"true",
      "sort":"percent asc",
      "rows":"100",
      "wt":"json",
      "stats.field":["age",
        "percent"]}},
  "response":{"numFound":13,"start":0,"docs":[
      {
        "name":"foo",
        "age":10,
        "color":"blue",
        "createdAt":"2015-12-30T23:32:24.755Z",
        "updatedAt":"2015-12-30T23:32:24.755Z",
        "id":"612bb75f-be0f-496b-ba51-8e79ee786c50"},
      {
        "name":"bar",
        "age":20,
        "color":"yellow",
        "createdAt":"2015-12-30T23:15:09.859Z",
        "updatedAt":"2015-12-30T23:15:09.859Z",
        "id":"517a4917-b3b8-4ea0-a3fd-acd41497b6e0"},
      {
        "name":"john",
        "age":30,
        "color":"black",
        "createdAt":"2015-12-30T23:15:10.859Z",
        "updatedAt":"2015-12-30T23:15:10.859Z",
        "id":"515a4917-b3b8-4ea0-a3fd-acd4149432fd"},
  },
  "facet_counts": {
    "facet_queries": {},
    "facet_fields": {
        "name": [
        {
          "blue":1},
        {
          "yellow":1},
        {
          "black":1},
      ]
    },
    "facet_dates": {},
    "facet_ranges": {},
    "facet_intervals": {},
    "facet_heatmaps": {}
  },
  "stats":{
    "stats_fields":{
      "age":{
        "min":10.0,
        "max":30.0,
        "count":3,
        "missing":0,
        "sum":60.0,
        "sumOfSquares":100,
        "mean":20,
        "stddev":10}}}}

Supported Waterline Interfaces

Type Methods Build
Semantic create, createEach, find, count, update, destroy Build Status
Migratable define, describe, drop, alter, addAttributes, remove, attributes, addIndex, removeIndex Build Status
Queryable where, limit, sort, skip, select Build Status

Note: See Waterline Documentation

Special Adapter Interfaces

Type Methods Build
Suggest suggest. Return on Object with suggestions and spellecked the requestet term or phrase Build Status
Catalog catalog. Return an Object with matching results and Layered Navigation as facet and stats Build Status

Advanced Configuration

Connection Options

Params Default Description
host 'localhost'
port '8983'
core 'schemaless'
solrconfig object set properties, add and update searchcomponent and requesthandler. Config API
manageCores true create cores if not exists CoreAdmin
schema true allow migrate drop, alter schema manage schema
single false force manageCores to create a core for each model
fieldTypeMap fieldTypes Field Type Map
useFqParam true force query mapping as fq=name:foo param
schemaDefaultFieldTypes {}
debugAdapter false
debugCollection false
debugQuery false
debugSolr false

Model Options

{
  attributes: {
    first_name: {
      type:'string'
      // Overwrite per Field
      schemaDefaultFieldAttributes: {
        indexed: true,
        type: 'text_de'
      }
    }
  }
  // Overwrite per Model
  schemaDefaultFieldAttributes: {
    indexed: false
  }
}

Field Type Map

The following table represents mappings between Sails/Waterline model data types and Solr field types:

Sails/Waterline Type Solr Type
string text_general
text text_general
binary text_general
integer int
float float
date date
time date
datetime date
boolean boolean
binary text_general
array text_general
json text_general
point point

Note: You can even define your custom mapping as fieldTypeMap: inside connection settings and as model option. If you want a field type explicit mapping use fieldType as additional fieldTypeMapattribute

Solr default field attributes

The following table represents Solr field attributes:

Solr Field Attributes Default
name newField
type text_general
indexed true
stored true
docValues false
sortMissingFirst false
sortMissingLast false
multiValued false
omitNorms true
omitTermFreqAndPositions false
omitPositions false
termVectors true
termPositions false
termOffsets false
termPayloads false
required false
dynamicField false
json text_general

Note: You can even define your custom field attribute default as schemaDefaultFieldAttributes: inside connection settings and as model option. If you want a field attribute explicit you can add this attribute as an additional option inside the Model attribute settings

Running the tests

$ npm test

TODO:

  • more test
  • documentation
  • cleanup and refactoring
  • build an e-commerce like demo application with autocomplete and layerd navigation

More Resources

License

MIT