fix(dispatcher): clean up closed stream tasks

[reallav0]

Summary

• cancel and await active crawl tasks when a MemoryAdaptiveDispatcher stream closes
• discard URLs queued by the closed stream before the dispatcher can be reused
• await memory-monitor cancellation and add a deterministic regression test

Root cause

run_urls_stream() created per-URL tasks but its finally block only cancelled the memory monitor. Closing or cancelling the generator therefore left active crawls and queued URLs owned by the old stream alive, allowing them to overlap a later crawl on the same dispatcher and browser.

Fixes #2071.

Validation

• dependency-light async regression harness covering active-task cancellation, queue cleanup, session-count restoration, and absence of background tasks
• python -m py_compile crawl4ai/async_dispatcher.py tests/async/test_dispatchers.py
• git diff --check

Focused pytest was not run because pytest and the project runtime dependencies are not installed in this workspace.

[reallav0]

check failed cause no one put the discord webhook in the env

[ntohidi]

@reallav0 Thanks for your contribution :)