[Bug]: ValueError: Deserialization of type 'Logger' is not allowed. Only allowlisted configuration and strategy types can be deserialized.

[arthiondaena]

crawl4ai version

0.8.5

Expected Behavior

Crawl a website with BFSDeepCrawlStrategy.

Current Behavior

Docker container logs

  File "/usr/local/lib/python3.12/site-packages/slowapi/extension.py", line 734, in async_wrapper
    response = await func(*args, **kwargs)  # type: ignore
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/server.py", line 668, in crawl
    crawler_config = CrawlerRunConfig.load(crawl_request.crawler_config)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/crawl4ai/async_configs.py", line 1871, in load
    config = from_serializable_dict(data)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/crawl4ai/async_configs.py", line 288, in from_serializable_dict
    k: from_serializable_dict(v) for k, v in data["params"].items()
       ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/crawl4ai/async_configs.py", line 288, in from_serializable_dict
    k: from_serializable_dict(v) for k, v in data["params"].items()
       ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/crawl4ai/async_configs.py", line 264, in from_serializable_dict
    raise ValueError(
ValueError: Deserialization of type 'Logger' is not allowed. Only allowlisted configuration and strategy types can be deserialized.

Is this reproducible?

Yes

Inputs Causing the Bug

The given below code snippet is generating this request json

{
  "urls": [
    "http://caw.tech"
  ],
  "browser_config": {
    "type": "BrowserConfig",
    "params": {
      "headers": {
        "type": "dict",
        "value": {
          "sec-ch-ua": "\"Chromium\";v=\"116\", \"Not_A Brand\";v=\"8\", \"Google Chrome\";v=\"116\""
        }
      }
    }
  },
  "crawler_config": {
    "type": "CrawlerRunConfig",
    "params": {
      "scraping_strategy": {
        "type": "LXMLWebScrapingStrategy",
        "params": {}
      },
      "table_extraction": {
        "type": "DefaultTableExtraction",
        "params": {}
      },
      "exclude_social_media_domains": [
        "facebook.com",
        "twitter.com",
        "x.com",
        "linkedin.com",
        "instagram.com",
        "pinterest.com",
        "tiktok.com",
        "snapchat.com",
        "reddit.com"
      ],
      "verbose": false,
      "deep_crawl_strategy": {
        "type": "BFSDeepCrawlStrategy",
        "params": {
          "max_depth": 2,
          "max_pages": 10,
          "logger": {
            "type": "Logger",
            "params": {
              "name": "crawl4ai.deep_crawling.bfs_strategy"
            }
          }
        }
      }
    }
  }
}

The request is working if I manually remove the "logger" key from "params".

Steps to Reproduce

Code snippets

async with Crawl4aiDockerClient(base_url=settings.crawl4ai_api_url) as client:
    strategy = BFSDeepCrawlStrategy(
        max_depth=2,
        include_external=False,
        max_pages=10
    )
    result = await client.crawl(
        [url],
        browser_config=BrowserConfig(headless=True),
        crawler_config=CrawlerRunConfig(cache_mode=CacheMode.BYPASS, deep_crawl_strategy=strategy, verbose=False)
    )

OS

Windows 11

Python version

3.12

Browser

No response

Browser version

No response

Error logs & Screenshots (if applicable)

[READY]... ✓ Connected to http://localhost:11235 
[CRAWL]... ℹ Crawling 1 URLs  
[CLOSE]... ℹ Closing client 
Crawl error: Server error 500: Server error '500 Internal Server Error' for url 'http://localhost:11235/crawl'
For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/500

[hafezparast]

Hi @arthiondaena — thanks for the detailed report and reproduction case! A fix has been submitted in PR #1849.

What was wrong: to_serializable_dict() serialized every class instance — including logging.Logger — as {"type": "Logger", "params": {...}}. When the Docker API tried to deserialize this, it hit the security allowlist and raised ValueError.

How it was fixed: Two-sided fix:

1. to_serializable_dict() now skips types not in the allowlist (returns None) — so Logger is never serialized in the first place
2. from_serializable_dict() returns None for unknown types instead of crashing — so older clients that still send "type": "Logger" payloads work too

How to test before it's merged

pip install git+https://github.com/unclecode/crawl4ai.git@fix/maysam-serialize-skip-non-config-1848

Then try your original code with BFSDeepCrawlStrategy via the Docker client.

@unclecode @ntohidi

[ntohidi]

Thank you for the pull request. The fix has already been merged with the develop branch.
We will include it in the next release.

[ntohidi]

Closing this issue, It's in the new release, 0.8.6