Not recognizing ZIP of profile directory

[titusfortner]

Description

In debugging this issue: SeleniumHQ/selenium#7796
I found that I am unable to send a Base64-encoded ZIP of my profile directory to geckodriver and have it be used. The Ruby code hasn't changed since Jari wrote it 8 years ago; the Capabilities class is just calling this method: https://github.com/SeleniumHQ/selenium/blob/master/rb/lib/selenium/webdriver/common/zipper.rb#L52-L61 which should be doing the right thing.

System

• Version: 0.26
• Platform: macOS
• Firefox: 70.0.1
• Selenium: Ruby 3.142.6

Testcase

profile = "/Users/titusfortner/Library/Application Support/Firefox/Profiles/x6muv71k.default-1540548528938"
ff_profile = Selenium::WebDriver::Firefox::Profile.new(profile)
options = Selenium::WebDriver::Firefox::Options.new(profile: ff_profile)
driver = Selenium::WebDriver.for :firefox, options: options
driver.quit

Trace-level log

I can't look at actual logs because a trace to just create/destroy session is 112MB text file (we send the serialized profile information twice in latest released Selenium!).
I hacked the code to make sure that the right signature is actually being sent, though:

2019-11-19 20:08:55 INFO Selenium    >>> http://127.0.0.1:4447/session | {"desiredCapabilities":{"browserName":"firefox","version":"","platform":"ANY","javascriptEnabled":false,"cssSelectorsEnabled":false,"takesScreenshot":false,"nativeEvents":false,"rotatable":false,"marionette":true,"timeouts":{},"moz:firefoxOptions":{"profile":"<<PROFILE>>"}},"capabilities":{"firstMatch":[{"browserName":"firefox","timeouts":{},"moz:firefoxOptions":{"profile":"<<PROFILE>>"}}]}}

[whimboo]

@titusfortner sorry for the delay here. Could you please reduce the profile size so that a minimal testcase exists? I would love to see a trace log. Especially because we have unit tests which ensure that this is working: https://searchfox.org/mozilla-central/rev/04d8e7629354bab9e6a285183e763410860c5006/testing/geckodriver/src/capabilities.rs#757-772

[chrismcmahon]

Even if there is a way to make this work, my understanding is that it does not work for certificates (and extensions). I need a custom profile with a certificate installed.

[titusfortner]

What is the right way to reduce the size? As far as I know this is the default profile with my only changing what the start page is so I can see at a glance if it is getting used.

[titusfortner]

@whimboo question for you about Firefox profiles in general.
I know the advice at this point is not to use profiles and to use prefs/args, etc instead.

Selenium Python bindings have gone as far as to mark using the profile parameter as deprecated.
I'm curious what plans geckodriver has for this parameter if any. I'd like for all the languages to take the same approach to this functionality, and would appreciate any input from you on what you think would be best..

[whimboo]

So basically the profile capability is not deprecated as of now. Maybe this was a misunderstanding? What I was saying in the past was that the profile class shouldn't be used to set preferences, proxies etc. This can all be done via moz:firefox[prefs] or the proxy capability these days. In some cases you would still have to prepare a profile eg. for a pre-set list of cookies, or security certificates.

Do you still see the original issue as reported here?

[kretschmannj]

Stumbled across this old thread looking for help. Can someone tell me the procedure for creating a Base64-encoded ZIP of a profile directory?

[titusfortner]

@whimboo I dug into this again based on SeleniumHQ/selenium#11028

Part of the problem is that Python is doing unnecessary things to the directory instead of just zipping it.

The main problem is that the profile I've zipped and sent to the driver does not have a user.js file, but the profile that is generated by the driver for use, has a user.js file added, and it includes values that are overwriting what is in the source profile's prefs.js file.

[whimboo]

Interesting. So not having automation specific preferences specified for Marionette could indeed cause quite some problems. Especially around those prefs that need to be set before Firefox is started. Not that long ago I filed https://bugzilla.mozilla.org/show_bug.cgi?id=1868965 to maybe combine these preferences and make it easier for automation clients to start Firefox with the required/recommended preferences correctly set.

@titusfortner: Otherwise which data of the generated profile is not working? Is it a specific feature only? This could be something different.

[whimboo]

@titusfortner do you have any answers to my questions in the last comment? Thanks.