Rethinkdb reads at 40K records per second

[my-alok]

Hi,
I'm stuck in a problem that is, when my rethink db starts reading data, it jumps to the over 40K records per second but even after lowering sails (I'm using sails js application) it keeps it's position (40K records read per second) and keeps CPU usage high (160%)

Please help me if you've any solution for this.
Thanks

[traverseda]

There's not really enough information to be able to help out with that, but I would recommend putting an awaitable sleep call in whatever you have in your javascript looping over data from rethinkdb.

Take a look at this? https://stackoverflow.com/questions/951021/what-is-the-javascript-version-of-sleep

[my-alok]

Hi @traverseda ,
Thanks for your response. I solved the CPU issue but now my application is facing a different one. My application is running a cron and each time it runs, the memory consumption by Rethinkdb increases and it doesn't get released after the execution it causes the server RAM to be filled by Rethink over 90% and server to become unresponsive could you please tell me if this memory leak problem is in my Sails app of rethinkdb because I've read many posts regarding this issue and couldn't find any solution yet.

Thanks again.

[traverseda]

Well if the memory is being used by rethinkdb it can't really be a memory leak in your program.

My guess would be rethinkdb is caching your data, not so much leaking memory. Take a look here. If you still get rethinkdb using 90% memory after adjusting the cache size, there's probably a rethinkdb issue.

Make sure you're using it on a system with at least 1GB of ram though. Rethinkdb takes advantage of memory for performance reasons, so the more you give it the happier it will be. It will want to use as much ram as it can in order to increase performance.

[traverseda]

You should also explain how you solved the CPU issue, for the sake of anyone who might be reading this in the future.

[srh]

That could be a memory leak -- RethinkDB has exhibited that.

[my-alok]

Hi @traverseda ,
I didn't do anything for the CPU issue, just reduced the connection scope and it worked like a charm. However, if you could advise me how can I overcome this memory leak issue. I'm using Rethinkdb version 2.3.6. It's legacy code and being a newbie, I'm not aware of its R.js file and things.

Hi @srh ,
I really could use some help if you could advise me on something to deal with it.

[analytik]

It's very hard to help you if you don't post any more specifics. How big is the database? How many queries are running there? How many reads/updates/writes do you do per second? How fast is the SSD? RAM, CPU? Is your app and database sharing the same server?

It's likely this isn't a memory leak, but a low-spec server doing hard durability writes with a slow SSD. Also, RethinkDB performance degrades significantly on larger tables, and the disk bytes written per document increase as the table grows.