-
-
Notifications
You must be signed in to change notification settings - Fork 51
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
performance bottlenecks #222
Comments
Hey @travisghansen I would love to get a more deep understanding of the benchmark that you have run. The data sample is the same schema and you run jq (node-jq) vs jsonpath 500 times. I would love to see directly without the NodeJS wrappers, to detect if it's a node-jq wrapper fault and we can investigate further. If that's not the case, there's not much we can do, rather than see where Java vs Python fail at this. If it's our fault tho, I would love to improve that. As well, I would love to try not the same schema, try bigger JSONs and think other kind of tests to have a reasonable comparisation. If we create a solid perf benchmark, I'm happy to debug the NodeJS process to understand it better. Could you open a PR with that benchmark and move from there? Thanks for opening the issue. |
Hey, I'll try to knock something out in bash real quick and see how that performs. I picked the schema above simply because for my use-case I'll be dealing with relatively small datasets but lots of iterations over and over (I'm handling auth scenarios: https://github.com/travisghansen/external-auth-server). I'm also digging in over here: robertaboukhalil/jqkungfu#3 to see if fundamentally the same syscalls are going on etc. Thanks for the help! |
Yeah, 500 iterations in bash is essentially the same as invoked from node through your lib. Probably not much we can do :(
|
|
mmm, sad. Not much we can do. In order to help in your use-case, I recommend to forgot to pipe operations on the shell if you want to get high performance. After reading your project, I don't really get where do you need jq. I would suggest trying with native JS or lodash. Since I always heard that lodash.map is faster than native https://jsperf.com/native-map-vs-lodash-map Hope it helps and open any other issue that you might have! |
@davesnx I intend to use Thanks! |
Description
I'd like to work with this in a project that requires relatively high-performance. I ran some tests with this and the
jsonpath
package to compare simple selects (I'm aware the tools have vastly different capabilities etc).jq was magnitudes slower and I'm not sure if that comes from the exec/shell or if something else is at play. Any tips and/or help maximizing performance would be great.
Thanks!
Test Source
results
The text was updated successfully, but these errors were encountered: