Storage client supplied via custom configuration is not respected (includes simple fix PR)

[BenBeattieHood]

Which package is this bug report for? If unsure which one to select, leave blank

@crawlee/basic (BasicCrawler)

Issue description

Using the code below, the crawler will continue to save its data using the storage client configured in global config, therefore in this case, in the folder ./storage rather than './our-custom-folder`:

Code sample

    const crawler = new PlaywrightCrawler({
        async requestHandler({ request, page, enqueueLinks, log }) {
             // ...
        }
    },
    new Configuration({
        storageClient: new MemoryStorage({
            localDataDirectory: 'our-custom-folder',
        }),
    }));

Package version

3.5.2

Node.js version

18

Operating system

N/A

Apify platform

• Tick me if you encountered this issue on the Apify platform

I have tested this on the next release

Yes

Other context

A simple fix for this is available here: BenBeattieHood#1

[B4nan]

Thanks, can you open the PR against this repository instead of your fork?

[BenBeattieHood]

Sorry, I don't have permission to create a branch on this repo. Can you help?

[B4nan]

You don't need any permissions to create a PR from your fork, that's the standard way, no need to create a branch on our end.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/creating-a-pull-request-from-a-fork

Btw there is one more place that will need the same adjustment, in the BasicCrawler.run method there is another RequestQueue.open() call:

https://github.com/apify/crawlee/blob/master/packages/basic-crawler/src/internals/basic-crawler.ts#L771