[Bug]: crawler.base_config boolean values are silently ignored (regression from #1505)

[bong-u]

crawl4ai version

0.9.2

Expected Behavior

config.yml sets a server-side default for every crawl:

# deploy/docker/config.yml:74-76
crawler:
  base_config:
    simulate_user: true

A POST /crawl request that does not send simulate_user should run with simulate_user=True, i.e. the server default applies.
A request that does send simulate_user should win over the server default — that is the intent of #1505.

Current Behavior

The server default never applies. simulate_user is False on every crawl.

api.py:715 reads "the client didn't send this field" as "the attribute is None or """:

current_value = getattr(crawler_config, key)
if current_value is None or current_value == "":   # api.py:715
    setattr(crawler_config, key, value)

CrawlerRunConfig.simulate_user defaults to False (async_configs.py:1650), and False is neither None nor "", so the guard never passes.
The same goes for every base_config key defaulting to a boolean or a number: magic, override_navigator, check_robots_txt, remove_overlay_elements, page_timeout.
The config-list path at api.py:707 (8995c1b, #1837) copies the guard.

So the stock image ships simulate_user: true (config.yml:74-76, utils.py:63)
but never injects the navigator_overrider script (browser_manager.py:1229-1235) or runs the mouse-move simulation (async_crawler_strategy.py:980-983).

a1950af (#1505) introduced this.
The setattr used to be unconditional and clobbered client-sent values, so reverting brings #1505 back.
After CrawlerRunConfig.load() (api.py:675) nothing tells "omitted" apart from "sent, equal to the default".
That information only exists in the raw request dict

Is this reproducible?

Yes

Inputs Causing the Bug

- URL(s): any, e.g. https://example.com
- Settings used: stock deploy/docker/config.yml, i.e. crawler.base_config.simulate_user: true
- Input data: {"urls": ["https://example.com"]}   # no crawler_config key

Steps to Reproduce

1. Start the stock server image, config.yml untouched.
2. POST the body above to /crawl.
3. Read effective_config in handle_crawl_request (api.py:697-716).
   simulate_user is False.

Code snippets

# The guard in isolation. No server or browser needed.
from crawl4ai import CrawlerRunConfig

cfg = CrawlerRunConfig()                   # client sent no crawler_config
value = getattr(cfg, "simulate_user")      # False, the dataclass default
assert value is None or value == ""        # api.py:715 -> fails, setattr skipped

OS

Linux (Docker image, python:3.12-slim-bookworm)

Python version

3.12

Browser

Chromium (Playwright, headless)

Browser version

No response

Error logs & Screenshots (if applicable)

No error. The server drops the value silently.

[bong-u]

Working on a fix that keys off the raw request payload instead of the deserialized values, will open a PR against develop.