Failed to decode response from marionette with Firefox >= 65

[peymanslh]

System

• Version: 0.24.0
• Platform: Linux
• Firefox: 65, 66, 67, 69 (nightly)
• Selenium: python=3.7 - selenium=3.141.0

Testcase

I run my code with the above configuration and Firefox <= 64 and it works well but with Firefox >= 65 I got this error:
selenium.common.exceptions.WebDriverException: Message: Failed to decode response from marionette

Stacktrace

INFO:root:Current state is 1. Going to calculate the next state given : <class 'selenium.common.exceptions.InvalidSessionIdException'>
INFO:root:Type of inputException: Message: Tried to run command without establishing a connection
ERROR:root:Traceback (most recent call last):
  File "test_google_doc.py", line 70, in test_send_google_doc_fax
    driver.get(env_config.get('gdoc_url'))
  File "/tmp/nlatte/.venv/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 333, in get
    self.execute(Command.GET, {'url': url})
  File "/tmp/nlatte/.venv/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
    self.error_handler.check_response(response)
  File "/tmp/nlatte/.venv/lib/python3.7/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: Failed to decode response from marionette

Trace-level log

1559036657050	mozrunner::runner	INFO	Running command: "/root/firefox/firefox-bin" "-marionette" "--headless" "--window-size=1920,1080" "--log=trace" "--marionette" "-foreground" "-no-remote" "-profile" "/tmp/rust_mozprofile.nPDoVtPKDJqJ"
1559036657051	geckodriver::marionette	DEBUG	Waiting 60s to connect to browser on 127.0.0.1:46511
*** You are running in headless mode.
1559036657472	addons.webextension.screenshots@mozilla.org	WARN	Loading extension 'screenshots@mozilla.org': Reading manifest: Invalid extension permission: mozillaAddons
1559036657473	addons.webextension.screenshots@mozilla.org	WARN	Loading extension 'screenshots@mozilla.org': Reading manifest: Invalid extension permission: resource://pdf.js/
1559036657473	addons.webextension.screenshots@mozilla.org	WARN	Loading extension 'screenshots@mozilla.org': Reading manifest: Invalid extension permission: about:reader*
1559036657640	Marionette	DEBUG	Init aborted (running=false, enabled=true, finalUIStartup=false)
1559036659888	Marionette	DEBUG	Setting recommended pref apz.content_response_timeout to 60000
1559036659888	Marionette	DEBUG	Setting recommended pref browser.contentblocking.introCount to 99
1559036659888	Marionette	DEBUG	Setting recommended pref browser.download.panel.shown to true
1559036659889	Marionette	DEBUG	Setting recommended pref browser.newtabpage.enabled to false
1559036659889	Marionette	DEBUG	Setting recommended pref browser.pagethumbnails.capturing_disabled to true
1559036659890	Marionette	DEBUG	Setting recommended pref browser.search.update to false
1559036659890	Marionette	DEBUG	Setting recommended pref browser.tabs.disableBackgroundZombification to false
1559036659890	Marionette	DEBUG	Setting recommended pref browser.tabs.warnOnCloseOtherTabs to false
1559036659890	Marionette	DEBUG	Setting recommended pref browser.tabs.warnOnOpen to false
1559036659890	Marionette	DEBUG	Setting recommended pref browser.usedOnWindows10.introURL to
1559036659890	Marionette	DEBUG	Setting recommended pref browser.urlbar.suggest.searches to false
1559036659891	Marionette	DEBUG	Setting recommended pref datareporting.policy.dataSubmissionPolicyAccepted to false
1559036659891	Marionette	DEBUG	Setting recommended pref dom.disable_beforeunload to true
1559036659891	Marionette	DEBUG	Setting recommended pref dom.disable_open_during_load to false
1559036659891	Marionette	DEBUG	Setting recommended pref dom.file.createInChild to true
1559036659891	Marionette	DEBUG	Setting recommended pref dom.max_chrome_script_run_time to 0
1559036659900	Marionette	DEBUG	Setting recommended pref dom.max_script_run_time to 0
1559036659900	Marionette	DEBUG	Setting recommended pref extensions.getAddons.cache.enabled to false
1559036659900	Marionette	DEBUG	Setting recommended pref extensions.webservice.discoverURL to http://%(server)s/dummy/discoveryURL
1559036659900	Marionette	DEBUG	Setting recommended pref network.http.prompt-temp-redirect to false
1559036659900	Marionette	DEBUG	Setting recommended pref network.http.speculative-parallel-limit to 0
1559036659901	Marionette	DEBUG	Setting recommended pref security.fileuri.strict_origin_policy to false
1559036659901	Marionette	DEBUG	Setting recommended pref security.notification_enable_delay to 0
1559036659901	Marionette	DEBUG	Setting recommended pref signon.autofillForms to false
1559036659902	Marionette	DEBUG	Setting recommended pref signon.rememberSignons to false
1559036659902	Marionette	DEBUG	Setting recommended pref toolkit.cosmeticAnimations.enabled to false
1559036660104	Marionette	INFO	Listening on port 46511
1559036660104	Marionette	DEBUG	Remote service is active
1559036660157	geckodriver::marionette	DEBUG	Connection established on 127.0.0.1:46511. Waiting for Marionette handshake
1559036660173	Marionette	DEBUG	Accepted connection 0 from 127.0.0.1:38054
1559036660178	geckodriver::marionette	DEBUG	Connected to Marionette
1559036660182	Marionette	DEBUG	0 -> [0,1,"WebDriver:NewSession",{"acceptInsecureCerts":true,"browserName":"firefox"}]
1559036660183	Marionette	WARN	TLS certificate errors will be ignored for this session
1559036660495	Marionette	DEBUG	0 <- [1,1,null,{"sessionId":"bfd607f3-f752-404a-84d6-9e5dd947bb60","capabilities":{"browserName":"firefox","browserVersion":"66.0" ... ozprofile.nPDoVtPKDJqJ","moz:shutdownTimeout":60000,"moz:useNonSpecCompliantPointerOrigin":false,"moz:webdriverClick":true}}]
1559036660512	webdriver::server	DEBUG	<- 200 OK {"value":{"sessionId":"bfd607f3-f752-404a-84d6-9e5dd947bb60","capabilities":{"acceptInsecureCerts":true,"browserName":"firefox","browserVersion":"66.0","moz:accessibilityChecks":false,"moz:geckodriverVersion":"0.24.0","moz:headless":true,"moz:processID":13429,"moz:profile":"/tmp/rust_mozprofile.nPDoVtPKDJqJ","moz:shutdownTimeout":60000,"moz:useNonSpecCompliantPointerOrigin":false,"moz:webdriverClick":true,"pageLoadStrategy":"normal","platformName":"linux","platformVersion":"4.9.0-7-amd64","rotatable":false,"setWindowRect":true,"strictFileInteractability":false,"timeouts":{"implicit":0,"pageLoad":300000,"script":30000},"unhandledPromptBehavior":"dismiss and notify"}}}
1559036660514	webdriver::server	DEBUG	-> POST /session/bfd607f3-f752-404a-84d6-9e5dd947bb60/url {"url": "https://docs.google.com/document/d/[doc_id]/edit"}
1559036660527	Marionette	DEBUG	0 -> [0,2,"WebDriver:Navigate",{"url":"https://docs.google.com/document/d/[doc_id]//edit"}]
out of memory: 0x0000000000000070 bytes requestedout of memory: 0x0000000000000038 bytes requested

Exiting due to channel error.
Exiting due to channel error.
Exiting due to channel error.
1559036751199	webdriver::server	DEBUG	Deleting session
1559036751221	geckodriver::marionette	DEBUG	Browser process stopped: exit code: 11
1559036751287	webdriver::server	DEBUG	<- 500 Internal Server Error {"value":{"error":"unknown error","message":"Failed to decode response from marionette","stacktrace":""}}
1559036751330	webdriver::server	DEBUG	-> GET /session/bfd607f3-f752-404a-84d6-9e5dd947bb60/screenshot
1559036751334	webdriver::server	DEBUG	<- 404 Not Found {"value":{"error":"invalid session id","message":"Tried to run command without establishing a connection","stacktrace":""}}
1559036751349	webdriver::server	DEBUG	-> DELETE /session/bfd607f3-f752-404a-84d6-9e5dd947bb60
1559036751349	webdriver::server	DEBUG	<- 404 Not Found {"value":{"error":"invalid session id","message":"Tried to run command without establishing a connection","stacktrace":""}}

[usergenic]

I should say Firefox v66.0 seems okay for us but v67.0 not. And this is true of geckodriver 0.23.0 and 0.24.0 on Linux.

[whimboo]

@peymanslh are you running inside docker or a local machine? There is a OOM crash of Firefox which is causing this problem. Please check your machine for available memory before and while running the test. Is it low?

Also please check if there is a crash report by following those steps.

[tohaa]

+1
@whimboo issue is reproduced only inside docker

[whimboo]

@tohaa did the link with the proposed steps help to solve the problem for you?

[rafagonc]

@whimboo This bug is related to zombie processes that hangs after driver.quit(), when you open many browsers one task after another, your machine will run out of memory or out of PIDs, that is happening on docker for chromedriver and geckodriver.

[andreastt]

--memory 1024mb --shm-size 2g should resolve the out of memory issue with Docker, which as @rafagonc correctly points out is a configuration problem rather than a Firefox specific issue. (Apart from that it would obviously be great if it used less memory in general.)

I couldn’t find an open bug/issue for the zombie process issue, although I have seen this many times myself. @whimboo, do you know if we track that somewhere or if I should file a new bug?

[whimboo]

Actually this is a dupe of #1559.

[lock]

This issue has been automatically locked since there has not been any recent activity after it was closed. If you have run into an issue you think is related, please open a new issue.