-
Notifications
You must be signed in to change notification settings - Fork 221
how to avoid bot detection playwright
Avoiding bot detection in Playwright means fixing the right layer before you launch
a session, not patching properties after a site has already blocked you. Detection
is decided at a layer most stealth work never reaches. Patching navigator.webdriver
and its neighbours is real, it is worth doing, and it is also the cheapest thing a
detector checks. The durable tells are not values you can redefine from a page
script. They are outputs a real machine produces and a script cannot: what the
engine actually renders, what it can actually decrypt, how a pointer actually moves.
Building toward that from the start, rather than patching properties after a block
shows up, is the difference this page is about.
This page is written for before you launch a session, not after one gets blocked. If you already have a specific site turning you away, the reactive checklist is the faster read. If you want the full five-layer map of why sites block automation at all, that overview is here. This page sits next to both: it is the proactive version, the decisions to make and the code to write before the first request goes out, so you spend effort on the layer that actually decides the outcome instead of the layer that is easiest to patch.
Stealth advice reads as one undifferentiated pile of tips, and that is the first mistake to avoid. It operates at three distinct levels, and each one has a different ceiling:
-
The page. A script runs before the site's own code and redefines properties:
navigator.webdriver, the plugin list, permission queries. - The driver. The automation layer itself stops setting the flags and leaving the bindings that give it away, so there is nothing left on the page to redefine.
- The engine. The browser's own C++ source reports different values, compiled in, through the same code path a normal build uses.
The full breakdown of what each level can and cannot reach is here. The proactive move is to pick a level deliberately, before you write a line of scraping code, rather than discovering the ceiling of level 1 three weeks into a project when a site starts checking something a page script cannot touch.
navigator.webdriver is the property everyone reaches for first, and it is worth
doing correctly rather than skipping. The naive version looks like this:
from playwright.sync_api import sync_playwright
with sync_playwright() as p:
browser = p.firefox.launch()
page = browser.new_page()
page.add_init_script("""
Object.defineProperty(navigator, 'webdriver', { get: () => false });
""")
page.goto("https://example.com")This is a real fix for a real, cheap check, and on a naive detector it works. It is also the ceiling of level 1, for reasons worth knowing before you rely on it:
- A clean, non-automated browser reports
undefinedfor this property, notfalse. Setting it tofalseis a value no real browser sends. - The property is now an own property on the
navigatorinstance instead of an inherited one on the prototype, which is a one-line check. - The getter you wrote is a JavaScript function with its own source, and
Function.prototype.toStringon it does not read[native code].
None of that is a reason to skip the patch. It is a reason not to stop there. The full mechanics, including the timing race between your init script and the page's own code, are written up here.
Even a perfect page-level patch leaves three things exactly as they were, because none of them are JavaScript properties:
- The machine. The GPU string, the installed fonts, the audio device, the screen. A server reports server defaults regardless of what the page claims.
- The handshake. TLS and HTTP/2 are negotiated before any page-level script has had a chance to run.
- The behaviour. How the pointer moves, how a form gets filled, whether the page is ever scrolled.
This is the argument for moving down a level rather than writing a longer init script. A property can be redefined by anyone with a console. An output has to be produced by something that actually has the capability being tested.
A capability check asks the browser to do something rather than report something, which is why a page-level patch cannot pass one. Here is a concrete version you can run yourself against whatever Playwright launches today.
Playwright's managed Chromium, as of the 1.57 default, ships Chrome's codecs. It does not ship Chrome's DRM module. Ask a live page to negotiate a protected session and read the answer directly:
from playwright.sync_api import sync_playwright
with sync_playwright() as p:
browser = p.chromium.launch()
page = browser.new_page()
page.goto("https://example.com")
result = page.evaluate("""
() => navigator.requestMediaKeySystemAccess('com.widevine.alpha', [{
initDataTypes: ['cenc'],
videoCapabilities: [{ contentType: 'video/mp4; codecs="avc1.42E01E"' }],
}]).then(() => 'yes').catch(() => 'no')
""")
print("widevine:", result)On a default managed Chromium this prints no. Setting the user agent to claim
Chrome does not change the answer, because the missing piece is a DRM module, not a
string. The full argument, including why this is a capability rather than a
value and what closes it, is here. No add_init_script
reaches this, because there is no property to redefine. The module either exists in
the binary or it does not.
A patched Firefox has no equivalent split to exploit or defend. There is no widely-used stripped Firefox that automation runs and real visitors do not, so a capability check like this one is not a live front for the engine this project patches. That structural difference, and its real cost (Firefox is a smaller share of traffic to begin with), is covered in full here.
invisible_playwright patches the C++ source of Firefox itself and ships the
compiled binary, driven by stock Playwright with no wrapped subset of its API. The
switch from plain Playwright is two lines:
from invisible_playwright import InvisiblePlaywright
with InvisiblePlaywright(seed=42) as browser:
page = browser.new_page()
page.goto("https://example.com")
page.click("#submit") # mouse arcs to the button on a Bezier curvebrowser here is a real playwright.sync_api.Browser. Every Playwright method
after that line works exactly as documented upstream, including page.evaluate, so
the Widevine check above runs unchanged against it if you want to see the
difference for yourself: a build with no proprietary-versus-open split has nothing
for that particular check to catch, because there is no Firefox that ships the
codecs and a separate Firefox that automation runs.
What the seed buys, specifically: roughly 400 fingerprint fields (GPU, canvas,
audio context, fonts, screen) come back identical on every run with the same
seed, and a fresh, distinct identity on every run without one. That determinism is
what turns "it worked yesterday" into something you can actually bisect when a site
starts blocking - change one thing at a time against a fixed identity instead of
guessing against a new random machine every launch.
A consistent fingerprint sitting under a pointer that teleports between coordinates
is still going to look automated to anything watching behaviour rather than
properties. page.click above does not send the cursor in a straight line or teleport
it to the target; it moves it along a Bezier curve first, because a form filled in
zero simulated motion is its own tell regardless of how good the fingerprint under it
is.
Two things follow from that, worth deciding before a target ever sees your session rather than after:
- Do not layer a second stealth plugin on top of an engine-level fingerprint. Two systems answering the same questions about the machine will not always agree with each other, and the disagreement is easier to find than either alone.
- Space requests out. A perfectly disguised session making requests at machine speed produces a velocity signal that no per-request fingerprint hides.
Before pointing any of this at a target you have not tested against yet:
- Pick the level deliberately (page, driver, or engine) based on what your target actually needs, using the full comparison.
- Log the seed for every session you run against a real target, so a later failure is something you can replay rather than something you have to guess at again.
- Run one of the public suites and read the whole report, not just the verdict, before you assume a level is enough.
- Compare against a stock browser on the same machine, field by field, rather than trusting a single score.
- Decide up front whether your target cares about codec or DRM capability checks; if it does, know which engines have that gap and which do not before you pick one.
Avoiding bot detection proactively means deciding, before the first request, which layer you are actually defending: the page, the driver, or the engine. A page-level patch is fast and worth doing, and its ceiling is real: it cannot touch the machine, the handshake, or a capability check that asks the browser to do something rather than report something. The durable fixes live one or two levels down, in a driver that never sets the tell in the first place or an engine that has nothing to lie about because it is reporting values compiled into the binary. Pick the level your target actually requires, prove it with a capability check rather than a property read, and keep the identity seeded so a failure is something you can reproduce instead of something you have to re-guess.
What is the single most effective way to avoid bot detection in Playwright? There is no single fix, because detection is layered. Patch the obvious properties, then check whether your target reads the machine, the handshake, or behaviour before assuming you are done.
Is patching navigator.webdriver enough? No. It defeats the cheapest check a
detector runs and does nothing for the GPU, fonts, TLS handshake, or a capability
check like DRM negotiation, none of which are JavaScript properties.
Do I need a stealth plugin, a patched driver, or a patched browser? Depends on what your target checks. A plugin fixes obvious property reads. A patched driver removes the automation flags at the source. A patched engine is the only one of the three that reaches the machine and capability checks a page script cannot see.
Can I prove a Chromium-based tool is lying about being Chrome? Yes, in one line: ask it to negotiate a Widevine session. A missing DRM module is missing code, and no user agent string changes the answer.
Does a random fingerprint every run help or hurt? It helps you blend in and hurts you the moment something breaks, because you cannot tell whether the site changed or your fingerprint did. A seeded, reproducible identity turns a failure into something you can bisect.
Should I combine a stealth plugin with a stealth browser? No. Two layers answering the same questions about the same browser will disagree eventually, and the disagreement is its own signal.
- Three ways to make Playwright undetected and why navigator.webdriver is not the tell people think it is, for the level-1 mechanics above.
- Chromium is not Chrome, for the Widevine capability check and the direct tests it was run against, dated 2026-07-30 in that page's own sources.
- Firefox or Chromium for anti-detect automation, for the structural argument and its honest cost.
See also: How to scrape without getting blocked for the full five-layer map this page assumes, the reactive troubleshooting checklist for when a specific site has already turned you away, and how to test whether your browser is detected for the method to run before you trust any of the above.
From the notes of invisible_playwright, a Firefox patched at the C++ level and driven by stock Playwright. The Widevine check above is one we run against our own binary before every release, not just a claim about someone else's.
Documentation
Guides
-
Browser Identity
- navigator.webdriver is not the tell you think it is
- hardwareConcurrency, deviceMemory and storage quota
- Screen size and viewport tells in headless browsers
- Playwright headless vs headed: what detectors see
- Playwright User Agent: Why You Should Not Set It
- Client Hints and Sec-Fetch: headers that must agree
- Codec fingerprinting: canPlayType and MediaCapabilities
- Permissions API: the two answers that must agree
- CSS fingerprinting: what media queries reveal
- What privacy.resistFingerprinting actually does
- speechSynthesis.getVoices() returns an empty array
- Browser extensions are a fingerprint surface
- BFCache and pageshow.persisted under browser automation
- Service workers, storage partitioning and automation
- Web Workers: where page-level fingerprint patches fail
- fake-useragent is archived: what changes and what doesn't
- navigator.buildID and the stale build date tell
- navigator.maxTouchPoints and pointer consistency
- navigator.platform and oscpu on a spoofed OS
- navigator.vendor and productSub: the Firefox tells
- Accept-Language header vs navigator.languages
- window.devicePixelRatio: the pref that spoofs it
- Can you be fingerprinted in incognito mode?
- Is changing the user agent enough to avoid detection?
- Can a website tell you are running on a server?
- Can two devices share a browser fingerprint?
- Does clearing cookies stop fingerprint tracking?
- Color-gamut and HDR media queries as a fingerprint
- Battery API fingerprint: does Firefox expose it?
- Is navigator.connection a fingerprint in Firefox?
- Can the Gamepad API fingerprint or detect a bot?
- Do accelerometer and gyroscope APIs leak on desktop?
- prefers-reduced-motion and other OS-setting tells
- Does storage quota estimate reveal disk size?
- Can scrollbar width reveal my operating system?
-
Canvas, WebGL, Fonts and Audio
- Canvas fingerprint noise: why per-call randomising fails
- Firefox WebGL renderer strings: what ANGLE reports
- WebGL parameters: the numbers are the same on every GPU
- Your renderer string says NVIDIA. Your pixels say software.
- Why headless browsers render different fonts
- How to make Linux and macOS report real Windows fonts
- measureText and TextMetrics as a fingerprinting surface
- AudioContext fingerprinting, and why adding noise backfired
- Canvas and WebGL fingerprints, identical across OSes
- Emoji fingerprinting: why emoji look the same on any OS
- Detecting installed fonts in JavaScript by width
- WebGL shader precision as a fingerprint surface
- AudioContext sampleRate and latency as a fingerprint
- Is WebGPU a browser fingerprint?
-
Network, Proxy and WebRTC
- WebRTC leak with a proxy in Playwright and Selenium
- WebRTC ICE candidate spoofing: the fields that give it away
- Playwright proxy in Python: per-context, and what leaks
- Playwright proxy not working? SOCKS5 auth in Python
- Playwright timezone does not match the proxy IP
- JA3 and JA4: why a TLS fingerprint cannot be patched
- Playwright in Docker: it runs, and still gets blocked
- Web scraping keeps getting blocked with good proxies
- Python web scraping blocked? The TLS fingerprint reason
- SOCKS5 vs HTTP proxy: what each does in the browser
- WebRTC IPv6 leak: why a proxy does not stop it
- HTTP/2 fingerprint: the layer above the TLS handshake
- TLS fingerprint vs User-Agent: the contradiction
- WebRTC has no ICE candidates behind a proxy
- WebRTC IP that matches the proxy exit, by design
- How to check if a proxy leaks your real IP
- about:webrtc: read your real ICE candidates
- Offline timezone resolution from a proxy exit IP
- Residential vs datacenter vs mobile proxies explained
- Sticky vs rotating proxy sessions: which to use
- Does a proxy leak DNS? DoH and DNS leaks explained
- HTTP/3 and QUIC fingerprint: what a site sees
- What is ASN and IP reputation in bot detection?
- What does a mobile carrier IP look like to a site?
- IPv6 vs IPv4: which does your proxy expose?
- Geolocation API vs IP location: keep them consistent
- Does chaining two proxies help avoid detection?
-
The Automation Layer
- Function.prototype.toString and the [native code] check
- The ChromeDriver
cdc_variable, and why renaming it fails - Why an attached debugger makes automation detectable
- Execution context was destroyed, and when it means detection
- Human-like mouse movement: Bezier curves are the easy part
- Why a Playwright upgrade broke 97 of 133 tests overnight
- Playwright persistent profile: what it fixes and breaks
- Why humanized mouse movement can fail on hover()
- Why content_frame() returns None for a cross-origin iframe
- Orphaned Firefox processes on Windows: the killed-runner leak
- Firefox launches but Playwright can't drive it: packaging gap
- Why automating login is riskier than reusing a session
- Playwright new_page vs new_context: the viewport tell
- Playwright dialog and popup handling without a tell
- Playwright download files with Firefox and the tell
- Playwright connect_over_cdp does not work with Firefox
- Playwright mobile emulation on Firefox and isMobile
- Playwright isTrusted: are automated clicks real?
- Playwright set_input_files uploads and the tell
- Can websites detect Playwright? What is actually visible
- Does Playwright Set navigator.webdriver to True?
- Does Playwright Leave Traces a Website Can See?
- Does Playwright Change My Browser Fingerprint?
- Can I Use My Real Browser Profile With Playwright?
- Does Playwright Support Firefox Stealth?
- Is Playwright Firefox Harder to Detect Than Chromium?
- Does Playwright Get Detected on the First Request?
- Why Playwright's bundled Firefox is easy to detect
- ghost-cursor human mouse paths with Playwright
- Stock Playwright, patched Firefox: how they connect
- Intercept and mock network requests with page.route
- Record and replay HTTP traffic with HAR in Playwright
- Record a Playwright trace to debug a failed scrape
- Record a video of a Playwright browser session
- Save and reuse login with storage_state in Playwright
- Read and set cookies in a Playwright context
- Set geolocation and permissions per Playwright context
- Handle HTTP basic auth in Playwright (http_credentials)
- Isolate identities with a browser context per session
- Drag and drop elements in Playwright with drag_to
- When to use an HTTP client vs a real browser
- Migrating from requests + BeautifulSoup to a browser
-
AI Agents and Frameworks
- AI browser agents and stealth: what fits and what does not
- browser-use gets detected: what you can and cannot change
- crawl4ai stealth mode and custom browser engines
- Give a LangChain agent an invisible_playwright browser
- Feed invisible_playwright pages into a RAG index
- Computer-use agents and browser fingerprint detection
- Give an MCP browser server a stealth Firefox engine
- Give each AI agent a reproducible browser identity
- Run parallel browser agents with distinct fingerprints
- Why AI browser agents have their own timing signal
- Running an AI browser agent headless on a server
- Give a browser agent a persistent logged-in session
- smolagents: hand the agent an invisible_playwright tool
- Stagehand and stealth: why a Firefox engine won't drop in
- DOM-reading vs screenshot agents: which stealth helps
- Back a computer-use agent with a real browser engine
- AI agent retry loops trip rate limits, not fingerprints
-
Detectors, Explained
- What bot.sannysoft.com actually checks, row by row
- How CreepJS decides you are lying
- What BotD actually detects, and what it does not
- Why a FingerprintJS visitor ID changes
- reCAPTCHA v3 score: why a fresh browser scores badly
- BrowserLeaks canvas and WebGL hash, explained
- What BrowserLeaks actually tests, surface by surface
- Browser trust scores explained: what the number means
- How do websites detect bots?
- What is a browser fingerprint?
- What data does a website collect about your browser?
- Does a VPN stop browser fingerprinting?
- Do websites know you are using a script?
- How accurate is browser fingerprinting?
- Can a website detect a virtual machine?
- Can websites detect a datacenter or proxy IP?
- getClientRects fingerprinting: subpixel geometry as ID
- Notification.permission as a bot-detection signal
- speechSynthesis voices as a cross-platform fingerprint
- Can a website detect typing by keystroke timing?
- Can a website detect Clipboard API access?
- What are mouse-dynamics behavioural biometrics?
-
Testing and Troubleshooting
- How to test bot detection without a false pass
- Playwright detected as a bot: the checklist to fix it
- Firefox preferences that silently do nothing
- Slow browser launch: a per-request timeout is not a budget
- Playwright screenshot returns noise: readback fix
- Canvas fingerprint changes every run: use a seed
- Playwright TargetClosedError: the causes and the fixes
- Why am I blocked with a clean fingerprint?
- Why Does My Playwright Script Get Blocked?
- Is Playwright headless detectable? What sites check
- Can You Run Playwright Without Being Detected?
- Why Playwright Works Locally but Fails in the Cloud
- Does Playwright Trigger reCAPTCHA More Often?
-
Scraping with Playwright
- How to scrape without getting blocked
- How to scrape a site that blocks headless browsers
- How to scrape infinite scroll pages with Playwright
- How to rotate proxies when scraping with Playwright
- How to scrape data behind a login with Playwright
- How to run Playwright in Docker without getting detected
- How to use invisible_playwright in Docker
- Playwright bot detection: how to avoid it in Python
- How to scrape paginated pages with Playwright
- How to download files with Playwright
- How to upload files with Playwright, and verify it landed
- How to handle cookie consent banners in Playwright
- How to handle popups and modals in Playwright
- How to take full-page screenshots with Playwright
- How to generate a PDF with Playwright and Firefox
- How to wait for content to load in Playwright
- How to retry failed requests when scraping Playwright
- How to scrape pages in parallel with Playwright
- How to rate limit your own Playwright scraper
- How to scrape HTML tables with Playwright
- How to scrape iframe content with Playwright
- How to scrape shadow DOM content with Playwright
- How to capture XHR and API responses in Playwright
- How to scrape geotargeted content with Playwright
- How to scrape real estate listings with Playwright
- How to scrape job postings with Playwright
- How to scrape e-commerce product pages with Playwright
- How to track product prices with Playwright
- How to scrape hotel room prices with Playwright
- How to scrape flight prices with Playwright
- How to scrape classifieds listings with Playwright
- How to scrape vacation rental listings with Playwright
- How to scrape car listings with Playwright
- How to scrape apartment rentals with Playwright
- How to track product stock and restocks with Playwright
- How to scrape location-based store prices with Playwright
- How to scrape flexible-date fare calendars with Playwright
- How to scrape product reviews with Playwright
- How to scrape reviews and ratings with Playwright
- How to scrape news article text with Playwright
- How to scrape business directory listings with Playwright
- How to scrape event and ticket listings with Playwright
- How to scrape restaurant menu data with Playwright
- How to scrape stock and financial data with Playwright
- How to scrape social media profiles with Playwright
- How to scrape forum and community threads with Playwright
- How to scrape image galleries with Playwright
- How to scrape video listings and metadata with Playwright
- How to scrape map-based local results with Playwright
- How to scrape sports scores and stats with Playwright
- How to scrape cryptocurrency prices with Playwright
- How to scrape deals and coupon codes with Playwright
- How to scrape to CSV with Playwright
- How to scrape to JSON Lines with Playwright
- How to scrape into a SQLite database with Playwright
- How to export scraped data to Excel with Playwright
- How to extract JSON-LD structured data with Playwright
- How to extract Open Graph and meta tags with Playwright
- How to extract links and build a crawl frontier in Playwright
- How to scrape RSS and Atom feeds with Playwright
- How to download images in bulk with Playwright
- How to extract clean article text with Playwright
- How to scrape a sitemap.xml with Playwright
- How to scrape into a pandas DataFrame with Playwright
- How to clean scraped prices and dates with Playwright
- Scrape search results by driving a form in Playwright
- Scrape a map-based search with Playwright
- Scrape autocomplete and typeahead inputs with Playwright
- Scrape date-picker calendars with Playwright
- Crawl list pages to detail pages with Playwright
- Scrape lazy-loaded images with Playwright
- Extract data from canvas charts with Playwright
- Scrape a multi-step wizard flow with Playwright
- How to resume an interrupted scrape with Playwright
- Incremental scraping: only new items since last run
- Handle 403 and 429 backoff mid-scrape in Playwright
- Scrape load-more button pages with Playwright
- Scrape nested pagination with Playwright
- Scrape an SPA that changes URL via history API
- Use BeautifulSoup with invisible_playwright
- Run stealth Playwright tests with pytest fixtures
- Run invisible_playwright concurrently with asyncio
- Run invisible_playwright in GitHub Actions CI
- Can you run invisible_playwright serverless?
- Run invisible_playwright in Celery task workers
- Schedule invisible_playwright scrapes with cron
- Run invisible_playwright headful on a server with Xvfb
- Use invisible_playwright in an Airflow DAG
- Combine invisible_playwright with httpx for speed
- Wrap invisible_playwright in a FastAPI service
- Run invisible_playwright in a Jupyter notebook
- Block images to speed up scraping (and when not to)
- Wait for a specific API response in Playwright
Comparisons
- Playwright stealth in Python: three levels that work
- Firefox or Chromium for anti-detect automation
- Chromium is not Chrome, and detectors know the difference
- Playwright stealth vs Camoufox: two patched Firefoxes
- Playwright stealth vs Patchright: driver vs engine
- Playwright stealth vs undetected-chromedriver and nodriver
- playwright-stealth vs a patched engine: page vs browser
- puppeteer-extra-plugin-stealth: unmaintained since 2024
- selenium-stealth hasn't been updated since December 2021
- pyppeteer's own maintainer says to switch to Playwright
- invisible_playwright vs rebrowser-patches: the same CDP fix
- invisible_playwright vs fingerprint-suite: injection vs engine
- invisible_playwright vs playwright-with-fingerprints
- invisible_playwright vs Scrapling
- invisible_playwright vs Ulixee Hero
- invisible_playwright vs SeleniumBase UC Mode
- Splash is unmaintained, and it was never a real browser
- invisible_playwright vs DrissionPage
- WebDriver BiDi vs CDP: does the new protocol hide you
- invisible_playwright vs hrequests
- zendriver vs invisible_playwright: Chrome CDP vs Firefox
- botasaurus vs invisible_playwright: framework vs library
- curl_cffi vs invisible_playwright: TLS client vs browser
- pydoll vs invisible_playwright: CDP without a driver
- selenium-driverless vs invisible_playwright stealth
- puppeteer-real-browser vs invisible_playwright
- Migrating from Selenium to Playwright for stealth
- Migrating from Puppeteer to Playwright for stealth
- undetected-chromedriver vs a patched Firefox browser
- scrapy-playwright vs a patched Firefox for stealth
- playwright-extra stealth plugins vs a patched browser
- tls-client vs a real browser: when TLS is enough
- Anti-detect browser or Playwright stealth: which you need
- undetected-playwright vs a patched Firefox binary
Integrations
- Using invisible_playwright with CodeceptJS
- Using invisible_playwright with Crawlee for Python
- Using invisible_playwright with Crawlee for JavaScript
- Using invisible_playwright with scrapy-playwright
- Using invisible_playwright with Robot Framework Browser
- Cypress, WebdriverIO, TestCafe and Nightwatch integration
- Using invisible_playwright with Microsoft's Playwright MCP
- Using the engine from Go, Java, C#, Ruby and Rust
docs/ source folder