New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Firefox 62 won't work with HTTPS connections #371

Closed
tresf opened this Issue Oct 17, 2018 · 13 comments

Comments

Projects
None yet
2 participants
@tresf
Contributor

tresf commented Oct 17, 2018

Quoting some release notes...

In enterprise environments, AutoConfig is sandboxed to the documented API by default. You can disable the sandbox by setting the preference general.config.sandbox_enabled to false. The long term plan is to remove the ability to turn off the sandboxing. If you need to continue to use more complex AutoConfig scripts, you will need to use Firefox Extended Support Release (ESR).

A temporary workaround is to edit C:\Program Files\Mozilla Firefox\defaults\pref\firefox-prefs.js

  pref('general.config.filename', 'firefox-config.cfg');
+ pref('general.config.sandbox_enabled', false);
  pref('general.config.obscure_value', 0);

We may need to leverage the enterprise cert flag instead. This will be some work to fix.

@tresf tresf added the bug label Oct 17, 2018

@tresf tresf self-assigned this Oct 17, 2018

@tresf

This comment has been minimized.

Contributor

tresf commented Oct 17, 2018

Possible permanent solution for Windows (and eventually other) environments: https://github.com/mozilla/policy-templates/blob/master/README.md#certificates

tresf added a commit that referenced this issue Oct 17, 2018

Allow temporary disabling of Firefox sandbox
Per #371, allows fallback certificate behavior until a proper Firefox policy is installed.
@tresf

This comment has been minimized.

Contributor

tresf commented Oct 18, 2018

Sandbox patch tested:

  • PASS - Windows 10 - Firefox 62
  • PASS - macOS 10.13 - Firefox 62
  • PASS - Ubuntu 14.04 - Firefox 62
  • PASS - Windows XP - Firefox 52 ESR
@tresf

This comment has been minimized.

Contributor

tresf commented Oct 18, 2018

Some boilerplate code for creating the Firefox 62+ policy.json file. This has not yet been integrated and this is only for Windows. Mac will need to leverage a native scripting language to mimic the JSON search/add.

This adds one additional dependency, json2.js to the Windows installers.

Click for code
'use strict';

include('json2.js'); // https://github.com/douglascrockford/JSON-js

var console = { log: function(o) { WScript.Echo(o); } };

var MOZILLA_POLICY = "policies.json";

var policyJSON = read(MOZILLA_POLICY);
var policy = JSON.parse(policyJSON || "{}");

var before = JSON.stringify(policy, null, 2);
var after = before;

if (policy.policies === undefined) {
	policy.policies = {};
}
if (policy.policies.Certificates === undefined) {
	policy.policies.Certificates = {};
}
if (policy.policies.Certificates.ImportEnterpriseRoots === undefined) {
	policy.policies.Certificates.ImportEnterpriseRoots = true;
	console.log("ImportEnterpriseRoots is missing, adding.");
	after = JSON.stringify(policy, null, 2);
	write(MOZILLA_POLICY, after);
} else {
	console.log("ImportEnterpriseRoots exists, skipping.");
}

console.log("\r\nBEFORE:\r\n" + before);
console.log("\r\nAFTER:\r\n" + after);

function write(path, data) {
	var ForWriting = 2;
	var fso = new ActiveXObject("Scripting.FileSystemObject");
	if (!fso.FileExists(path)) {	
		mkdirs(fso.GetParentFolderName(path), fso);
		fso.CreateTextFile(path);
	}
	var file = fso.GetFile(path);
	var stream = file.OpenAsTextStream(ForWriting);
	var data = stream.Write(data);
	stream.Close();
}

function read(path) {
	var data;
	var ForReading = 1;
	var fso = new ActiveXObject("Scripting.FileSystemObject");
	if (fso.FileExists(path)) {
		var file = fso.GetFile(path);
		if (file.Size > 0) {
			var stream = file.OpenAsTextStream(ForReading);
			data = stream.readAll();
			stream.Close();
		}
	}
	return data;
}

function mkdirs(path, fso) {
	if(!fso.FolderExists(path) && path !== "") {
		if (!fso.FolderExists(fso.GetParentFolderName(path))) {
			mkdirs(fso.GetParentFolderName(path), fso);
		}
		fso.CreateFolder(path);
	}
}

function include(path) {
	eval(read(path));
}
@tresf

This comment has been minimized.

Contributor

tresf commented Oct 18, 2018

So I'm investigating ways to manipulate the policy.json file programmatically and although the above example uses JavaScript through the cscript (e.g. vbscript) engine, I'll need a reliable way of doing this on Mac and Linux.

  • For Mac, ruby has built-in JSON parsing capabilities.
  • For Linux, I'm not sure what's available. Ruby isn't included in a base install of Ubuntu 14.04 and Perl doesn't doesn't include JSON support by default (and the JSON-Parse library appears to contain compiled C-code, which I'd like to avoid).

(Long term, we may rewrite our deployment steps using Java code, but that would be better suited for the 2.2 branch).

I'm tagging @dsanders11 for feedback as the use of more powerful tools will help move along #208 but anyone is free to chime in with ideas. Also tagging @lukas-w incase he has some recommendations.

At the time of writing this, Firefox 62 on Windows and Firefox 63 on macOS (not released at time of posting this) will honor the policy.json file, so I believe the 2.0 patch will land as Ruby shim for macOS to meet the timeliness requirements.

Our consultant, Mike Kaply informs us that Firefox 65 is expected to drop the ability to unsandbox the AutoConfig scripts (i.e. d95b739). Ideally, 2.0.8 would ship with a patch that covers both scenarios by leveraging our existing patch as well as the policies.json installation technique.

@tresf

This comment has been minimized.

Contributor

tresf commented Oct 19, 2018

I'm tagging @dsanders11 for feedback as the use of more powerful tools will help move along #208 but anyone is free to chime in with ideas. Also tagging @lukas-w incase he has some recommendations.

Answering my own question... So Ruby was working out well until I found out the JSON library isn't available by default on MacOS 10.8.

Now I'm testing Python. Python 2 vs Python 3 is going to be tricky but the following Operating Systems come with Python JSON parsing baked in making it the lowest common denominator...

  • Ubuntu 14.04
  • Ubuntu 18.04
  • macOS 10.8
  • macOS 10.13
@onurguzel

This comment has been minimized.

Contributor

onurguzel commented Oct 19, 2018

Hello Tres, if you need any help with Python, I am happy to help.

@tresf

This comment has been minimized.

Contributor

tresf commented Oct 19, 2018

@onurguzel sure... Here's the WIP...

Click for code
#!/usr/bin/env python

import os
import sys
import json
import errno

DEFAULT_PATH = '/Applications/Firefox.app/Contents/Resources/distribution/policies.json'
DEFAULT_DATA = '{ "policies": { "Certificates": { "ImportEnterpriseRoots": true } } }'
DEFAULT_OVERWRITE = False

path = sys.argv[1] if len(sys.argv) > 1 else DEFAULT_PATH
merge = json.loads(sys.argv[2]) if len(sys.argv) > 2 else DEFAULT_DATA
overwrite = sys.argv[3].lower() == 'true' if len(sys.argv) > 3 else DEFAULT_OVERWRITE

def mkdir_p(path):
    try:
        os.makedirs(path)
    except OSError as e:  # Python >2.5
        if e.errno == errno.EEXIST and os.path.isdir(path):
            pass
        else:
            raise

def load_json(path):
    data = json.loads("{}")
    if os.path.isfile(path):
        try:
            stream = open(path, "r")
            data = json.load(stream)
            stream.close()
        except ValueError as e:
            print("Warning: Not a valid JSON file: " + path)

    return data

policy = load_json(path)

# Add only missing keys
#{
#  "policies": {
#    "Certificates": {
#      "ImportEnterpriseRoots": true
#    }
#  }
#}

# TODO: loop over merge object instead
if policy.get('policies') == None:
    policy['policies'] = {}

if policy['policies'].get('Certificates') == None:
    policy['policies']['Certificates'] = {}

if policy['policies']['Certificates'].get('ImportEnterpriseRoots') == None:
    policy['policies']['Certificates']['ImportEnterpriseRoots'] = True

mkdir_p(os.path.dirname(path))
stream = open(path, "w+")
stream.write(json.dumps(policy, sort_keys=True, indent=2))
stream.close()

The Linux and Mac installers -- both in shell script -- have a locator script which returns the path to Firefox. From there we'll be invoking a Python script with the following parameters:

  • The full path to the JSON file that resides inside the Firefox folder (the script not need know this)
  • The JSON data/string to inject (or a path to a JSON file if that makes more sense)
  • An overwrite flag e.g. "ImportEnterpriseRoots": false but we provided "ImportEnterpriseRoots": true, we'll leave it alone.

Other requirements...

  • Needs to run on Python 2.7 and Python 3.x
  • This doesn't have to have any Firefox-specific code in it (scalable to other JSON tasks at a later time if needed). Help is greatly appreciated.
@tresf

This comment has been minimized.

Contributor

tresf commented Oct 20, 2018

WIP branch: 2.0...tresf:policy

@tresf

This comment has been minimized.

Contributor

tresf commented Oct 21, 2018

@onurguzel let me know if you have time to help with the above #371 (comment). The urgency is high because reinstalls of QZ Tray using Firefox are breaking. The python script will allow our next release (2.0.9) to survive the removal of the sandboxing feature.

@tresf

This comment has been minimized.

Contributor

tresf commented Oct 22, 2018

@onurguzel I'm asking @bberenz to help with the Python due to the timeliness of this task.

@tresf

This comment has been minimized.

Contributor

tresf commented Oct 25, 2018

Final patch is ready at #373 and should scale well for when Linux adds certificate installation through policies.json (milestoned for Firefox 64).

@tresf tresf closed this in #373 Oct 25, 2018

@tresf tresf added this to the 2.0.8 milestone Oct 25, 2018

@tresf

This comment has been minimized.

Contributor

tresf commented Oct 25, 2018

Testing passes on all Firefox versions.

  • JScript JSON script tested on Windows XP - Windows 10
  • Python JSON script tested on Python 2.7 - Python 3.7 (macOS 10.8 - 10.14, Ubuntu 14.04)

This patch will be available starting with 2.0.8.

Note, this still leaves Linux in the dark as Mozilla hasn't published a technique for that platform. We've been notified that Firefox 64 (63 is the current version at time of posting this) will add a technique for installing certificates on Linux but NOT (and no plans) for trusting the system roots, so another patch will need to land in 2.0.9 BEFORE Firefox 65 is released. 😕 Sorry Linux users!

In the meantime, the sandbox technique as part of this patch will work for Firefox 63 and Firefox 64 on Linux.

@onurguzel

This comment has been minimized.

Contributor

onurguzel commented Oct 25, 2018

Hello again, I was ill for the last few days and I am deeply sorry for being unresponsive. I didn't know this issue was that urgent.

Since there is a merged PR, I won't be committing changes to the repo. Overall, the code seems clean and nice but I wanted to suggest few improvements.

Argument parsing

It might be better to use argparse module for the arguments, because it is self documenting and the recommended way to parse command line arguments. But, it is only available for Python 2.7+ and unfortunately there are still some supported Linux distributions with built-in Python 2.6 which are based on Red Hat Enterprise Linux 6.

Before:

path = sys.argv[1] if len(sys.argv) > 1 else DEFAULT_PATH
merge = json.loads(sys.argv[2]) if len(sys.argv) > 2 else json.loads(DEFAULT_DATA)
overwrite = sys.argv[3].lower() == 'true' if len(sys.argv) > 3 else DEFAULT_OVERWRITE

After:

import argparse

parser = argparse.ArgumentParser()
parser.add_argument('path', default=DEFAULT_PATH, required=False)
parser.add_argument('json_data', default=DEFAULT_DATA, required=False)
parser.add_argument('-o', '--overwrite', action='store_true')  # Default False 
args = parser.parse_args()

# And use as args.path, etc.
policy = load_json(args.path)

File reading/writing

Python community love to work with context managers. Files are easier to work with them, too. There are examples below for reading and writing files.

with open(path, "r") as stream
    data = json.load(stream)
with open(path, "w+") as stream
    json.dump(stream, policy, sort_keys=True, indent=2)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment