Table of Contents generated with DocToc
http://npm-stat.vorba.ch/charts.html?package=search-index
Search-index is a search index module for Node.js. Think "node version of Lucene, but much simpler".
Search-index allows you to perform free-text search over structured or unstructured data, and return a resultset ordered by relevance.
Search-index is built with the soooperfast levelUP module, and the very useful Natural module.
The Point of Search-Index is to simplify set up and operation of an search engine. Search-index is essentially free from configuration- the index is dynamic and morphs into the structure that you require automatically, based on the documents that it is fed.
Search-index is in an alpha stage- meaning that it has been known to work quite well, but edge cases and portability may be challenging. Query-result is robust. See known issues and performance tips below.
#The Forage Search Engine
Search-index is currently the index powering the Forage search engine.
#Features
- Full text search
- Stopword removal
- Faceting
- Filtering
- Fielded search
- Field weighting
- Relevance weighting (tf-idf)
- Paging (offset and resultset length)
- Teasers
#Installation
Releases are listed here. Generally you will want the most recent one.
The easiest way to include search-index in your project is by using npm
npm install search-index
The module can then be initialised by including the line
si = require('search-index');
at the top of your app.
#API
##si.indexData([,callback])
Returns metadata about the state of the index. Metadata is accrued incrementally, so it is vulnerable to corruption. At any time metadata can be corrected by running the computationally demanding calibrate function.
si.indexData(function(msg) {
console.log(msg);
});##si.calibrate([,callback])
Iterate through entire index and count everything up. Tf-idf calculations are most precise when indexData is up to date. Only needs to be called if index has been altered externally, or if key collisions occur (mostly theoretical). A persistant running tally is kept by search-index which can be seen in the file search-index.json.
si.calibrate(function(msg) {
console.log(msg);
});##si.indexPeek(start, stop [,callback])
Take a look at the raw index. Start is the start point and stop is the stop point. All keys in between will be returned. For debug purposes.
si.indexPeek(req.query['start'], req.query['stop'], function(msg) {
console.log(msg);
});##si.deleteDoc(req.body.docID [,callback])
Delete the document and all associated index entries.
si.deleteDoc(documentID, function(msg) {
console.log(msg);
});##si.search(query, [,callback])
Queries the search index
si.search(query, function(msg) {
res.send(msg);
});...where query is an object similar to (see Query Parameters for more info):
{
"query": [
"usa"
],
"offset": "0",
"pageSize": "20",
"facets": [
"places",
"organisations"
],
"weight": {
"title": [
"10"
]
},
"filter": {
"places": [
"usa"
]
}
}##si.index(batch, batchName, filters, [,callback])
Insets document into the index
si.index(batch, batchName, filters, function(msg) {
res.send(msg);
});Where batch is a JSON sequence named batchName containing one or more documents in a
format similar to:
{
'doc1':{
'title':'A really interesting document',
'body':'This is a really interesting document',
'metadata':['red', 'potato']
},
'doc2':{
'title':'Another interesting document',
'body':'This is another really interesting document that is a bit different',
'metadata':['yellow', 'potato']
}
}...and filters is an array of field names that may be contained
in the document that the index will use for building filters. A filter
field must always be an array of single String tokens, for example ['metadata','places']
#Query Parameters
##query
A free text string containing one or many tokens. Search everything with ´´´´*´´´
"query": [
"usa"
]##searchFields
Fields to search in. Query will only search in the fields specified here.
"searchFields": [
"title"
]##offset
The starting point in the resultset of the results returned. Useful for paging
"offset": 0##pageSize
The maximum number of results in the resultset that can be
returned. Counted from offset
"pageSize": 20##facets
Allows faceted navigation, the parameter is an array of fields. If no
facets is sent, all possible facets are returned.
"facets": [
"topics",
"places"
]##teaser
Creates a field that shows where the search terms exist in the given field. For example, a teaser field could be generated from the document field body
"teaser": "body"##weight
Sets a factor by which the score of a field should be weighted. Useful for building custom relevancy models
"weight": {
"body": [
"10"
],
"title": [
"5"
]
}##filter
Used to return a resultset on a facet.
"filter": {
"places": [
"ussr"
]
}#License
Search-index is released under the MIT license:
Copyright (c) 2013 Fergus McDowall
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.