Speed up the search-function

[hbi99]

Create some benchmarks and improve the speed of the search function

[duyhung85]

Thank you for the great work, its very easy to use with Xpath syntax. However, when my json data gets large, JSON.search() getting slow. Any recommendations to speed it up or restructure my Json format for better performance?
Hungnd,

[hbi99]

Hi @duyhung85
For a while, I've been thinking on a solution that might improve the performance. I'll experiment with that the next few days. I've also started thinking in another way of implementing, that will most likely improve, with techniques used in game development context. Though the latter requires more rewrite but I'll look in to that as well.

I'll get back ASAP.

Thanks

[hbi99]

@duyhung85 - I have created a benchmark branch.
https://github.com/hbi99/defiant.js/tree/benchmark

In this branch, I have tested, with Firefox, searches on a JSON structure, consisting of ~600 objects. Before the improvement, searching and finding all objects (XPath: '//*') took about 800ms. After the improvement code, the same search takes about 73-75ms.

I'll explore more ways to make it even faster. In the mean time, it would be super if you can try out this branch, at least in your dev-environment and get back to me with feedback.

Thanks

[duyhung85]

Hi hbi99,

Wow, its lightning speed JSON search Im getting here, like more than 10x faster. Before the update, its taking 500ms to scan my JSON structure (My Json is deep nested), now only 38ms-40ms. Superb! You made my day!
One small problem though, in my code after the JSON search I did an update on the result but its not reference/updated my object.

Thanks, keep up the good work.
Hungnd,

[hbi99]

@duyhung85 - thanks for catching the issue.
Cay you please elaborate the issue with a few lines of code?
So that I can see the symptoms...

Cheers

[duyhung85]

Hi hbi99,

I'm taking your code at defiantjs.com for example, the last one where you search JSON then update the original object. If i run it with the benchmark version (defiant.js-benchmark/dist/defiant-latest.min.js), it returns in console:
Object {active: 1, name: "MacBook Air", price_us: 999, price_se: 6563}
Object {active: 1, name: "MacBook Air", price_us: 999, price_se: 6563}

I think it assigns the first new value to all of the elements, so my object elements are all identical after that.

Cheers,
Hungnd

[hbi99]

@duyhung85 - great catch!

As it turned out, the previous solution was no good...so I made a U-turn and did some re-thinking and re-coding of the search-function as well as "adjacent" functions. The re-write has resulted in even faster search. With my sample JSON structure, the search-performance is down to 64ms from 800ms.

The new approach requires me to re-write the trace-function (does not affect to search performance). The trace-function is used to highlight matches in the XPath Evaluator.

I would appreciate if you find time to try out the new update.

[duyhung85]

Hi hbi99,

Thank you for the hard work, I tried out the new update and yes its even faster than the first attempt now, impressive work. However, the original object is not updated/referenced... Take the previous example, the result should be:
// { active=1, name="MacBook Air", price_us=999, price_se=6563 }
// { active=1, name="MacBook Air", price_us=1299, price_se=8534 }

but i only get the original object as:
// { active=1, name="MacBook Air", price_us=999}
// { active=1, name="MacBook Air", price_us=1299}

The search performance now is impressive though :)

Cheers,
Hungnd

[hbi99]

The latest commit in the benchmark branch should fix it...(though I think this code requires more testing). I'll write more Jasmine-tests for this branch before merging it to master-branch.
If you want ot test it - great :)

[duyhung85]

Its working great and speedy in my app now. Thank you :)

[hbi99]

@duyhung85 Great :)
I would like to recommend that you wait with adding this to your production environment. Please take your time and test it out more and I will do that as well. This branch conveys in a larger code-rewrite and I am cautiously optimistic before all the tests...and I recommend restraints if possible.

Thanks