feat(basic-crawler): allow configuring the automatic status message

[B4nan]

Adds statusMessageCallback option to all the crawler classes to allow customizing the status message call.

Allows overriding the default status message. The callback needs to call crawler.setStatusMessage() explicitly.
The default status message is provided in the parameters.

const crawler = new CheerioCrawler({
    statusMessageCallback: async (ctx) => {
        return ctx.crawler.setStatusMessage(`this is status message from ${new Date().toISOString()}`, { level: 'INFO' }); // log level defaults to 'DEBUG'
    },
    statusMessageLoggingInterval: 1, // defaults to 10s
    async requestHandler({ $, enqueueLinks, request, log }) {
        // ...
    },
});

[metalwarrior665]

I wonder if these are really needed. If you need to customize it, you can set your own setInterval and disable the Crawlee messages (set huge interval?). Because most likely, if you want to make this good, you would want to make the log level depend on the state of the stats (which is what Crawlee is doing with the high error rate) so setting one on crawler level is not that useful

[B4nan]

True, what about changing the callback so it calls the setStatusMessage()?

[B4nan]

That way you can also disable the status periodic messages by not calling it inside the callback.