feat: add keepAlive to crawler options

[B4nan]

Allows to keep the crawler running even when the queue is empty. Use crawler.teardown() to stop it.

Closes #1436

[szmarczak]

Wouldn't it be better to make the crawlers always on standby? So there would be no need to call crawler.run. Just add requests, and as long as the Node.js event loop is busy then it would keep running.

[B4nan]

Many things happen in the run method, so that would need to stay anyway, only thing I can imagine is using it in the background automatically (ensuring it was ran) from inside methods like addRequests(). Or do you see some easy to do refactor? I would rather not fall into a huge rebuild of this part right now, we are out in stable already.

[vladfrangu]

I was more wondering if this means the crawler will keep running and pool the request queue every so often to see if there's new requests, and run them if so (so you can externally add requests to be handled, and make the crawler be always on)

[szmarczak]

Or do you see some easy to do refactor?

No idea, just asking.

[B4nan]

so you can externally add requests to be handled, and make the crawler be always on

Yes, I believe it should work like that.

cc @metalwarrior665, is there any more into this problem than just what I already did?