Benchmarks

kristopolous edited this page Jun 15, 2011 · 5 revisions
Clone this wiki locally

These benchmarks illustrate the performance of jOrder compared to:

  • Native search function written specifically to perform a given query. With no calls to external libraries, and generally as few function calls as possible.
  • db.js: JavaScript database by Chris McKenzie. A relatively young project (started in 2011), supports flexible queries with an interface that adapts to the user's coding style.
  • Taffy DB 1.7.3: Another JavaScript database project, by Ian Smith. Somewhat older, seems inactive since mid-2009.
  • jLinq 3.0.1: Extensible, LINQ style JavaScript data querying library by Hugo Bonacci. In jLinq, even complex queries can be formulated in an expressive and human readable form. jLinq's performance has been significantly improved in 2010.

Values represent cumulative running times in milliseconds. There was a one second timeout applied to each benchmark, results over 1000ms are projected using the number of test cycles completed within 1s. The benchmark page is available form the repository ("test" folder).

Updated tests, more details, and charts are available here: http://spreadsheets.google.com/ccc?key=0Avnke0Xj5rgddDRaaUJZRkNLM0tiVURUc0JiWF9lOVE&hl=en. These results also include another database-like implementation named JaDE.

Test

Different environments yield slightly different, but generally similar relative times. The benchmark outlined below represents an average scenario.

Environment

  • CPU: 32 bit Dual Pentium
  • OS: Ubuntu 11.04
  • Browser: Chrome 12
  • Test cycles: 100

Table with 77 rows

Benchmark jOrder 1.2 Iteration db.js Taffy DB 1.7.3 jLinq 3.0.1
Object initialization 244 N/A 446 24 N/A
Search / exact matches (A OR B) 5 2 14 10 404
Search / exact matches (A AND B) 5 2 10 5 264
Range search (BETWEEN) 2 1 10 29 238
Sorting 1 11 36 42 549
Insertion 4 1 7 1 N/A

Small table benchmarks

Table with 1000 rows

Benchmark jOrder 1.2 Iteration db.js Taffy DB 1.7.3 jLinq 3.0.1
Object initialization 2448 N/A 1538 8 N/A
Search / exact matches (A OR B) 1 4 63 127 6250
Range search (BETWEEN) 3 4 128 207 2777
Partial range search (#20 to #40) 1 2 132 203 2941
Free text search (beginning of any word) 3 21 76 95 3030
Sorting 4 117 204 617 14285
Sorting partial (#1 to #20) 1 103 173 629 14285

Big table benchmarks

Conclusion

  1. jOrder beats all referenced methods when it comes to running simple queries on large datasets.
  2. There's still room for improvement in jOrder's 'Object initialization', ie. the indexing process.

The reference libraries used in this test address slightly different aspects of the same problem, while here only one of these aspects is examined: speed. Other factors, such as expressiveness and brevity of queries are not investigated at all. A test like that would certainly yield different results.

I did my best to write the reference benchmarks as efficiently as their respective libraries allow. If you are the developer of one of these libraries, please let me know if there's a more efficient way to run a certain query with your library.