A search engine based on Node.js and LevelDB
JavaScript Shell HTML
Switch branches/tags
Clone or download
Permalink
Failed to load latest commit information.
bin latest Apr 11, 2017
docs latest Apr 11, 2017
lib some improvements to classification Apr 26, 2017
test some improvements to classification Apr 26, 2017
.gitignore doc and git ignore Nov 8, 2016
.travis.yml add testing for node 6.x Nov 4, 2016
README.md latest Apr 11, 2017
app.json more heroku stuff Aug 25, 2015
package.json some improvements to classification Apr 26, 2017

README.md

NPM version NPM downloads MIT License Build Status Join the chat at https://gitter.im/fergiemcdowall/search-index

      ___           ___           ___           ___           ___      
     /\__\         /\  \         /\  \         /\  \         /\__\     
    /::|  |       /::\  \       /::\  \       /::\  \       /:/  /     
   /:|:|  |      /:/\:\  \     /:/\:\  \     /:/\:\  \     /:/__/      
  /:/|:|  |__   /:/  \:\  \   /::\~\:\  \   /:/  \:\  \   /::\  \ ___  
 /:/ |:| /\__\ /:/__/ \:\__\ /:/\:\ \:\__\ /:/__/ \:\__\ /:/\:\  /\__\ 
 \/__|:|/:/  / \:\  \ /:/  / \/_|::\/:/  / \:\  \  \/__/ \/__\:\/:/  / 
     |:/:/  /   \:\  /:/  /     |:|::/  /   \:\  \            \::/  /  
     |::/  /     \:\/:/  /      |:|\/__/     \:\  \           /:/  /   
     /:/  /       \::/  /       |:|  |        \:\__\         /:/  /    
     \/__/         \/__/         \|__|         \/__/         \/__/     

Deploy

npm install norch and then start with norch

or programatically:

require('norch')(options, function(err, norch) {
  // Norch server started on http://localhost:3030 (or the specified host/port)
})

Put stuff in

curl -X POST -d @myData.json http://localhost:3030/add (where myData.json is a newline separated file of JSON objects)

Search for hits (uses search-index's API)

http://localhost:3030/search?q={"query":[{"AND":{"*":["usa"]}}]}

(http://localhost:3030/search returns everything)

Make autosuggest

http://localhost:3030/matcher?q=usa

Export, import, and replicate an index

# create a snapshot on the server (available under /latestSnapshot)
curl -X POST http://localhost:3030/snapshot
# get latest snapshot
curl -X GET http://anotherIndex:3030/latestSnapshot > export.json
# replicate an export file into a new index on another server
curl -X POST -d @export.json http://someOtherServer:3030/import

API

Endpoint Method Response Typical Use Case
/add POST status code Add documents to the index
/availableFields GET stream Discover the name of fields which can be searched in
/buckets GET stream Aggregate documents on ranges of metadata
/categorize GET stream Aggregate documents on single metadata values
/concurrentAdd POST status code For when more than one source is adding documents to the index at the same time
/createSnapshot POST status code Create a snapshot of the index
/delete DELETE status code Remove documents from index
/docCount GET object Counts total document in index
/flush DELETE status code Remove all documents from index
/get GET stream Get documents by ID
/import POST file Import/merge an existing index into this one
/latestSnapshot GET file Download the latest index snapshot
/listSnapshots GET file See list of snapshots
/match GET stream Match by linguistic similarity- autosuggest, autocomplete
/search GET stream Search in the index
/totalHits GET object Show number of hits that a given query returns

The Norch API docs are here. Norch is essentially an http wrapper around search-index.

About Norch

Norch.js is an experimental search engine built with Node.js and search-index featuring, Full text search, Stopword removal, aggregation Matching (Autosuggest), Phrase search, Fielded search, Field weighting, Relevance weighting (tf-idf), Paging (offset and resultset length)

Logging

On Linux and OSX. Install bunyan, tail the log-file and pipe to bunyan.

Install bunyan:

npm install -g bunyan

Tail log-file:

tail -f log-info.log |bunyan

Mailing list: norchjs@googlegroups.com - subscribe by sending an email to norchjs+subscribe@googlegroups.com

License

MIT, Copyright (c) 2013-16 Fergus McDowall