add mechanism to handle huge query that timed out/OOM

[wey-gu]

Introduction

Some queries will end up with OOM and time out, but existing interface cannot help auto kill it

Contents

Related work

[wey-gu]

cc @MuYiYong

[bazingame]

I used nebula-explorer to find three steps expansion of one selected vertice accidentally. After I clicked the button Expand the explorer crashed. I can't ssh into the server and kill the query. I can't do anything just watch the server down.

Here are some metrics about the server during that time:

CPU global usage:

CPU status ratio:

Mem usage:

Mem status:

We think the query should be auto-killed by the nebula system which executed a long time or the memory usage up to a threshold configured. If the user used a wrong query statement or correct query without limitation, it will be very dangerous...

[wey-gu]

Now, this should be mitigated/resolved by memtracker:

• more precise momery quota control #5081
• Memory Tracker #5082

cc @bazingame