Skip to content


Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Comparing changes

Choose two branches to see what's changed or to start a new pull request. If you need to, you can also compare across forks.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also compare across forks.
Checking mergeability… Don't worry, you can still create the pull request.
This comparison is big! We're only showing the most recent 250 commits
Commits on Jun 24, 2014
@fergiemcdowall renamed indexData method to getIndexMetadat 2c1273e
@fergiemcdowall Update 2529f23
@fergiemcdowall Update
added ToC
@fergiemcdowall improved indexing functionality, if doc has same ID as existing doc, …
…existing doc is deleted and replaced
Commits on Jul 10, 2014
@fergiemcdowall renamed (simplified) some API calls cdc4326
@fergiemcdowall trying to fix jasmine on travis 9735db0
@fergiemcdowall added tests for matchers a8bab2e
@fergiemcdowall Update .travis.yml f471db2
@fergiemcdowall readme 7eed3f2
@fergiemcdowall doctoc 3b1f576
@fergiemcdowall Update
fixing markdown
@fergiemcdowall Update 3ce5082
@fergiemcdowall Update dbc1546
@fergiemcdowall Update
more doctocing and mardown formatting
@fergiemcdowall Update dda2404
@fergiemcdowall matcher now uses the same levelDB as index for tidyness, and better p…
@jimkang jimkang Fixed typo in error message. c19da39
Commits on Jul 11, 2014
@jimkang jimkang Updated addDocToIndex to skip parsing JSON if the batch param passed …
…in is already an object.
Commits on Jul 13, 2014
@fergiemcdowall matcher uses TF index 43bfca9
@fergiemcdowall fixed problem with matcher 698bea8
@fergiemcdowall Update 53df49a
@fergiemcdowall testing travis e5c10b7
Commits on Jul 14, 2014
@jimkang jimkang Added test for indexing via si.add() with an object instead of a string. 5fb8c5d
Commits on Jul 15, 2014
@fergiemcdowall fixed bloom filters 4c2ac4f
Commits on Jul 16, 2014
@jimkang jimkang - Added a logger, which is right now just a wrapper around console.lo…
…g that can be silenced for tests. (This is so that tests do not put out so much output that they overwhelm Travis.)

- Updated indexer.js to use logger instead of directly calling console.log.
- Updated npm test to clear si directory before start and to use NODE_ENV=test.
@fergiemcdowall Merge pull request #38 from jimkang/master
Support objects in `indexer.addDocToIndex()`.
Commits on Jul 17, 2014
@fergiemcdowall search-index now accepts javascript objects instead of JSON strings 53ca60b
@fergiemcdowall better logging 2dd90fe
Commits on Jul 18, 2014
@fergiemcdowall less scary message 96267c4
@fergiemcdowall improved performance for single term queries 6aaba2d
@fergiemcdowall fixed bug in tf-idf ranking, renamed metadata function ebeb387
@fergiemcdowall Update
updated TOC
Commits on Jul 22, 2014
@fergiemcdowall added condition for reducing fetch when faceting 0bb0d16
@fergiemcdowall Merge branch 'master' of e0c28a7
@fergiemcdowall version bump e6dca89
Commits on Jul 24, 2014
@fergiemcdowall small bug fix to tf-idf 32816ee
@fergiemcdowall small bug fix to tf-idf 03d707f
@fergiemcdowall tf now stored as array of IDs rather than single integer a3a4cc0
@fergiemcdowall fixed matcher aeab591
@fergiemcdowall make TF sets ordered when inserting doc IDs 8518337
@fergiemcdowall more descriptive function names a504f60
Commits on Jul 25, 2014
@fergiemcdowall tidy up ab52964
@fergiemcdowall new intersection strategy d0cad28
Commits on Jul 28, 2014
@fergiemcdowall improved key structure, split up tests 1cb7151
Commits on Jul 29, 2014
@fergiemcdowall new facet functionality 2aca3ed
Commits on Jul 30, 2014
@fergiemcdowall cleanup and fixed build 6face8a
@fergiemcdowall better faceting f7508c8
Commits on Aug 01, 2014
@fergiemcdowall moved from bloom filters to fast array comparison to detect set inter…
Commits on Aug 02, 2014
@fergiemcdowall unbloomed and fixed some bugs 0acdabf
@fergiemcdowall version bump 04814aa
Commits on Aug 05, 2014
@fergiemcdowall fixed 0-result bug as found by : #40 28a767c
Commits on Aug 09, 2014
@fatihky fatihky fixing some undefined errors 39c2cb9
@fergiemcdowall Merge pull request #41 from fatihky/custom
fixing some undefined errors
@fergiemcdowall bug fixes, threshold for matching, removal of unused files 04653c6
@fergiemcdowall bug fixes, threshold for matching, removal of unused files 5dbac3d
@fergiemcdowall matcher cleanup c7a071d
@fergiemcdowall matcher cleanup af7c3ad
Commits on Aug 10, 2014
@fergiemcdowall added sorting and limiting for facets- TODO update README file a87c940
@fergiemcdowall Update 5500437
@fergiemcdowall TOC f83da24
Commits on Aug 12, 2014
@fergiemcdowall added first half of replicator 1e78bb9
Commits on Aug 15, 2014
@fergiemcdowall new function name 9d26385
Commits on Aug 16, 2014
@fergiemcdowall adding more replication (unstable) 1a20595
Commits on Aug 20, 2014
@fergiemcdowall hacking some replication stuff b990fb0
Commits on Aug 28, 2014
@fergiemcdowall improved replication a00e968
Commits on Aug 29, 2014
@fergiemcdowall added zip compression to replication 9286dae
Commits on Aug 30, 2014
@fergiemcdowall replication 66d35af
@fergiemcdowall replication gives and takes 13e9861
Commits on Sep 01, 2014
@fergiemcdowall replication now more natively nodey streamy with no interstitial file…
… storage
@fergiemcdowall renamed function 95c24be
@fergiemcdowall Update 8776191
@fergiemcdowall Update dd1bae9
@fergiemcdowall Update fddcce1
@fergiemcdowall Update 2824545
@fergiemcdowall Update d970d40
Commits on Sep 04, 2014
@frankrousseau frankrousseau add repository field to NPM manifest
To avoid warnings during installation npm install
Commits on Sep 05, 2014
@fergiemcdowall Merge pull request #42 from frankrousseau/master
Add repository field to NPM manifest
@fergiemcdowall Update package.json
added repository URL
Commits on Sep 07, 2014
@fergiemcdowall fixed typo 5bc4885
@fergiemcdowall added a .gitignore 47c80df
@fergiemcdowall reflect renaming of Forage to Norch b2f8ea0
Commits on Sep 11, 2014
@timosaikkonen timosaikkonen Fixed issue#21: Index Location Configurabilty 5135061
Commits on Sep 12, 2014
@timosaikkonen timosaikkonen Make search-index export a function that takes options instead of exp…
…orting a separate configure function
@timosaikkonen timosaikkonen Fixed issue#45: triggers callback twice when key fre…
…quency is 0
@fergiemcdowall Merge pull request #43 from timosaikkonen/issues/21
Fixed issue#21: Index Location Configurabilty
@fergiemcdowall Merge pull request #46 from timosaikkonen/issues/45
Fixed issue#45: triggers callback twice when key freque...
@timosaikkonen timosaikkonen Fixed bug where require('search-index')(config) returned undefined b2883a7
@timosaikkonen timosaikkonen Tests for configurability dd1fe89
@fergiemcdowall Merge pull request #47 from timosaikkonen/issues/21
Commits on Sep 15, 2014
@fergiemcdowall squashing fstream bug 3f0c8a4
@fergiemcdowall added missing include 2ca734c
Commits on Sep 17, 2014
@fergiemcdowall version bump b3a50e2
Commits on Sep 19, 2014
@fergiemcdowall moving API towards options err standard c4e3aad
@fergiemcdowall updated tests to account for new API 67a87d5
Commits on Sep 22, 2014
@fergiemcdowall altered api for .add 4b3cbdb
Commits on Sep 28, 2014
@fergiemcdowall adding error handling to API 1b0d1fc
@fergiemcdowall added error handling for matcher 8164f94
@fergiemcdowall more error handling 97624e4
@fergiemcdowall testing index structure 8410c4d
Commits on Sep 29, 2014
@fergiemcdowall start of better reverse index 49ec1bc
@fergiemcdowall Update 3a1e599
Commits on Sep 30, 2014
@fergiemcdowall Update 128138e
@fergiemcdowall Update d4a0997
@fergiemcdowall Update f85d2a3
@fergiemcdowall Update e5b91e1
@fergiemcdowall Update e32952f
@fergiemcdowall Update 379ac71
@fergiemcdowall Update 2b0ada9
@fergiemcdowall Update 5156202
@fergiemcdowall Update 04a3d4f
@fergiemcdowall Update 63d8ab5
Commits on Nov 04, 2014
@fergiemcdowall init new branch fa423bf
Commits on Nov 06, 2014
@fergiemcdowall testy testy 551a750
@fergiemcdowall revert test 8759d12
@fergiemcdowall stuff works 452d75b
Commits on Nov 07, 2014
@fergiemcdowall double callback bug squashed, query object added to result 8360c81
Commits on Nov 08, 2014
@fergiemcdowall prototype was causing weirdness in search, changing back to function 48d3086
@fergiemcdowall added possibility for seek cutoff ae71ffd
@fergiemcdowall seekCutOff kicks in automatically if there is only one search term 95ac3c0
@fergiemcdowall remove comment 4d91ad3
@fergiemcdowall fixed offset bug b9f31f7
@fergiemcdowall Merge pull request #54 from fergiemcdowall/smallerindex
Merging the smallerindex branch into master, which creates a smaller index (in bytes) which is useful when transferring files back and forward across a network
Commits on Nov 10, 2014
@fergiemcdowall more stability, added possibility for speed/accuracy tradeoffs 646c5ee
Commits on Nov 11, 2014
@fergiemcdowall removed unused file 3cb725c
@fergiemcdowall updated to use laters reuters package 007b1bf
Commits on Nov 13, 2014
@fergiemcdowall IDs are now placed as fields in documents, if no ID is present then o…
…ne is automatically generated as a timestamp + object hash
@fergiemcdowall small bugfix 7bb75e2
@fergiemcdowall remove all test debris 9a41c5f
@fergiemcdowall squashed race condition (hopefully) 87dc959
@fergiemcdowall Update 1ddc6ed
@fergiemcdowall Update d70752f
Commits on Nov 14, 2014
@fergiemcdowall added winston logging, removed unused files 701120c
@fergiemcdowall removed race condition 7d2a138
@fergiemcdowall added loglevel option 10e24b5
@fergiemcdowall Update
initialisation options
@fergiemcdowall Update f85d9c0
@fergiemcdowall Update 47e2b71
@fergiemcdowall fixed facet sorting 17f6c95
@fergiemcdowall fixed tests 98b0be9
@fergiemcdowall added verification and test for malformed data 72abbfb
@fergiemcdowall fixed fergiemcdowall/norch#49 4efa35b
@fergiemcdowall fixed fergiemcdowall/norch#49 (but this time _really_ fixed it) 91a33e4
@nlaplante nlaplante Add logSilent option to suppress logs c1c16db
@nlaplante nlaplante Add test for logSilent 18e5a87
@nlaplante nlaplante Merge branch 'feature/index-logger' into develop 57d372b
Commits on Nov 18, 2014
@fergiemcdowall added tests for indexing numbers, cleaned up 1d17b3a
@fergiemcdowall bumped all dependency versions, added missing test file 97eccbb
@fergiemcdowall version bump 36ca9ff
@fergiemcdowall Merge pull request #55 from nlaplante/master
Add logSilent option to suppress logs
@fergiemcdowall faster searching on big datasets, better .gitignore, version bump bc1b802
@fergiemcdowall Merge branch 'master' of d6cb979
@nlaplante nlaplante Merge pull request #1 from fergiemcdowall/master
Upstream merge
@nlaplante nlaplante Add documentation for logSilent option 4474cc3
@fergiemcdowall Merge pull request #56 from nlaplante/master
Add documentation for logSilent
Commits on Dec 01, 2014
@fergiemcdowall Update
corrected incorrect docs as per #57
Commits on Dec 03, 2014
@fergiemcdowall renamed directory ec7859d
@fergiemcdowall filtering now working properly again (TODO: extend test coverage) a9cb0e5
Commits on Dec 04, 2014
@fergiemcdowall solved bug where autogeneration of IDs created duplicates, hopefully …
…fixing #58
Commits on Dec 07, 2014
@fergiemcdowall fixed getDoc e5e56bc
@fergiemcdowall better tests for .get 68c1d91
Commits on Dec 09, 2014
@fergiemcdowall Update 285c7e6
@fergiemcdowall Update 946ff5d
Commits on Dec 24, 2014
@anandthakker anandthakker db: switch level dependency to levelup + leveldown
This is a first step towards making search-index fully browserifiable. See #20.

Note that currently version of levelup at the time of this writing depends on
leveldown@0.10 (not 1.0).
@anandthakker anandthakker db: add level.js dependency and a browserify entry point.
search-index has been changed to accept a `db` property in its options, which should
be a valid LevelUP-supported database (like level.js or memdown).

search-index-browser.js is a browserify entry point that creates a level.js db and
passes it in as an option to search-index.
@anandthakker anandthakker browserify: basic build.
The core browserify build is now working.  Try it with:

node_modules/.bin/beefy test/spec/0indexing-spec.js -- -i WNdb -i lapack

(The two -i options are to ignore unused modules
that are conditionally require()'d in NaturalNode,
but not declared as dependnecies and thus not

Still something not quite right with
initializing the database, but getting close!
Commits on Dec 25, 2014
@anandthakker anandthakker fix test references to main module b053a34
@anandthakker anandthakker db config: remove restriction on indexPath && db 3244c25
@anandthakker anandthakker remove unused require()'s
Browserify (when used along with brfs transform) was failing on
these when they were in a comma-separated `var` declaration.
Just remove them since they're not being used.
@anandthakker anandthakker browserify: setup testing in browser w/ jasmine.
Tests still failing in the browser, but they're running.

node_modules/.bin/beefy test/spec/0indexing-spec.js -- -i WNdb -i lapack

And then browse to
Commits on Dec 26, 2014
@anandthakker anandthakker browserify: add working browser example
A basic example showing that indexing and searching work in the browser.  Run using beefy:

`node_modules/.bin/beefy examples/basic.js -- -i WNdb -i lapack`

and then browse to
@fergiemcdowall Merge pull request #59 from anandthakker/master
make search-index module browserifiable
Commits on Jan 14, 2015
@sbiaudet sbiaudet Refactor callbacks and errors fefab63
Commits on Jan 15, 2015
@sbiaudet sbiaudet Fix error module 'should' not found f7eb5a7
@sbiaudet sbiaudet Fix #62 use options indexPath instead of hardcoded name 50a34e6
@fergiemcdowall Merge pull request #63 from nubizsoft/master
Some improvements
@fergiemcdowall Update package.json
Version bump to reflect API changes
@sbiaudet sbiaudet improve doesDocExist 8b535a7
@sbiaudet sbiaudet Refactor incrementallyCalibrate with async parallel support b0ed2f4
@sbiaudet sbiaudet Refactor addDocToIndex with async parallel support e6621fe
@sbiaudet sbiaudet Remove time log 9e0b667
@sbiaudet sbiaudet Do not delete if without Id e16acff
Commits on Jan 16, 2015
@sbiaudet sbiaudet Refactoring the logging system 60f6efd
@fergiemcdowall Merge pull request #64 from nubizsoft/master
Using more async statement
@eklem eklem Changed link to npm-stats
Domain name change, updated link
@fergiemcdowall fixed whitespace, one small fix to test 6 f7b9fd2
@fergiemcdowall Merge branch 'master' of a66659b
@sbiaudet sbiaudet Optimize reduceTF function 42834c7
Commits on Jan 17, 2015
@fergiemcdowall Merge pull request #65 from nubizsoft/master
Optimize reduceTF function
Commits on Jan 23, 2015
@fergiemcdowall trying to fix replication upstream in Norch 4c4e50a
Commits on Jan 27, 2015
@fergiemcdowall stronger instantiation (tests must be rewritten) 23cee47
@fergiemcdowall fixed tests eaba8ca
Commits on Jan 29, 2015
@fergiemcdowall temp fix for logging 23348d6
Commits on Feb 02, 2015
@fergiemcdowall metadata fields now searchable and tests updated 371269f
@fergiemcdowall version bump d610717
Commits on Feb 04, 2015
@fergiemcdowall beefed up tests for fielded search b2d742e
@fergiemcdowall Merge branch 'master' of ca711c1
@fergiemcdowall Update package.json 4f98da7
@fergiemcdowall Update 2cca112
@fergiemcdowall these two hits can appear in any order, so removing them from tests da4cc82
@fergiemcdowall Merge branch 'master' of 9ded182
Commits on Feb 10, 2015
@fergiemcdowall navs can now be tagged as active 6482a46
Commits on Feb 18, 2015
@fergiemcdowall reintroduced wildcard search 861d2e1
@fergiemcdowall Update .travis.yml 9011c00
Commits on Feb 20, 2015
@fergiemcdowall new tests for world-bank-dataset 6876542
@fergiemcdowall test facets 2fdae0c
@fergiemcdowall if a specified facet field isnt an array then it is treated as an arr…
…ay with one element
Commits on Feb 23, 2015
@fergiemcdowall whitespace cleanup, integer padding example 1dd0a80
@fergiemcdowall removed redundant code fab99d8
Commits on Feb 25, 2015
@Holger-Will Holger-Will make names of variables fit in first example 499f0bb
@fergiemcdowall Merge pull request #73 from Holger-Will/patch-1
make names of variables fit in first example
@Holger-Will Holger-Will fix name in matcher example 2875202
@fergiemcdowall Merge pull request #74 from Holger-Will/patch-2
fix name in matcher example
Commits on Mar 05, 2015
@fergiemcdowall developing range filters 46008e4
@fergiemcdowall fix and tidy up 51f20c2
@fergiemcdowall tidy up ce61382
@fergiemcdowall tidying e135839
@fergiemcdowall fixed some problems running in norch 7708de9
Commits on Mar 06, 2015
@fergiemcdowall better control flow 70c995e
@fergiemcdowall tidy up 4a69713
Commits on Mar 07, 2015
@fergiemcdowall small tidy up 2d1c47e
@fergiemcdowall first version of facet ranges 97f041b
@fergiemcdowall added tests for range facets. Version bump 19de480
@fergiemcdowall improved range facets 0d8a600
Commits on Mar 08, 2015
@fergiemcdowall rangeFacet command can now be used to generate normal facets, thereby…
… unifying the facets commands
Commits on Mar 11, 2015
@fergiemcdowall work on facets: unstable commit 3500f37
@fergiemcdowall unstable: improvements to facets 346587e
@fergiemcdowall improved range faceting c99b7a8
Commits on Mar 14, 2015
@fergiemcdowall improved faceting 1e1a659
Commits on Mar 22, 2015
@fergiemcdowall range filters now working, weighting currently broken and commented o…
…ut of tests
@fergiemcdowall weighting fixed 926ff26
@fergiemcdowall clean up a006d26
Commits on Mar 24, 2015
@fergiemcdowall fixed crashy bug when no facets specified d518169
@fergiemcdowall fixing object instantiation (for example 'indexPath not respected #77') 839122a
Commits on Mar 25, 2015
@fergiemcdowall added support for words that contain non-ascii chars b291e17
Commits on Mar 27, 2015
@fergiemcdowall Update dcddf23
@fergiemcdowall Update 90f82f9
@fergiemcdowall Update b80c35b
Commits on Apr 03, 2015
@fergiemcdowall fixed deletion, adding duplicates, emptying from indexes now platform…
… indepentdent
Commits on Apr 04, 2015
@fergiemcdowall Update 520b798
@fergiemcdowall updated TOC d66897f
@fergiemcdowall Update ecadc8c
@fergiemcdowall toc 5410fef
@fergiemcdowall Update dbdb237
@fergiemcdowall Update de53bae
@fergiemcdowall toc ddbd8c9
@fergiemcdowall Update 655c1ae
Commits on Apr 05, 2015
@fergiemcdowall version bump 9413abe
Commits on Apr 10, 2015
@eklem eklem Update 6f585bb
@eklem eklem Fixing JSON issue
Adding missing "," in si.add examples
Commits on Apr 16, 2015
@eklem eklem Changed require after breaking change
`var si = require ('search-index')()`
Fixes #88 and possibly #89