New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Encrypted password store #180

Open
claudehohl opened this Issue Oct 11, 2014 · 12 comments

Comments

Projects
None yet
7 participants
@claudehohl
Contributor

claudehohl commented Oct 11, 2014

Auto fill-out login forms.

@The-Compiler The-Compiler self-assigned this Oct 12, 2014

@fiete201

This comment has been minimized.

Collaborator

fiete201 commented Dec 3, 2014

Is it about storing passwords or filling out the forms? I think storing passwords external in gnome-keyring or so would be better. Perhaps there is an API for keyrings to integrate.

@peterlvilim

This comment has been minimized.

Contributor

peterlvilim commented Dec 3, 2014

This is what I use for storing passwords: http://www.passwordstore.org/
It leverages pgp and git and follows the unix philosophy for tools.

I'd agree that the best approach would be an API or some sort of interface to support multiple approaches to key rings. I know the gnome one is popular, but I don't personally use it.

Could also potentially include a light weight password store with the browser if people don't want to use an external one and have an option to integrate with external ones.

I'd be up for looking into this in a couple weeks here when finals finish.

@The-Compiler

This comment has been minimized.

Collaborator

The-Compiler commented Dec 3, 2014

See #30 for some notes about a Python plugin API.

I'm not sure if implementing this in a pluggable way before designing an actual plugin API makes much sense, and I feel like it's too early at the moment to do that already, since I'm still moving much stuff around to clean things up, and I'd lose that freedom when doing plugins.

What might be a solution until then is a command :fill-field (username|email|password|...) <value>. Then a (shell-) userscript could get the password in any way based on $QUTE_URL, and then write those commands into the fifo. Though in the case of :fill-field password ... there are some unfortunate security implications (command history, logging, maybe more?).

Those shell-userscripts are already implemented (:run-userscript) but underdocumented.

@Carpetsmoker

This comment has been minimized.

Contributor

Carpetsmoker commented May 29, 2015

If #334 gets implemented, there's no more need for a special :fill-field command?

There should probably be a special command which takes a command (like Vim's :silent, :tab, etc.) to prevent logging to the history; eg. :nolog :jseval "alert()"

@The-Compiler

This comment has been minimized.

Collaborator

The-Compiler commented May 29, 2015

I think there still is - e.g. getting the username field requires some heuristics for pages which don't have them clearly marked a such - and it makes more sense to have those heuristics in one place, built into qutebrowser.

And I agree with that - but there are other related issues then, e.g. making sure the commands also don't end up in the debug log.

@Carpetsmoker

This comment has been minimized.

Contributor

Carpetsmoker commented May 29, 2015

I would expect the password manager to store the name & value attributes of all the fields it sends; no heuristics required.

@The-Compiler

This comment has been minimized.

Collaborator

The-Compiler commented May 29, 2015

That doesn't really work with modular password storage backends though.

For example I use KeePassX to store my passwords. Others use LastPass or other things.

Now I'd like a KeePassX integration for qutebrowser at some point - qutebrowser only gets username/password from the KeePassX database, now how would it know where to enter that information?

That approach would only work with a password manager built in into qutebrowser, which isn't really a thing I plan to do.

@t-wissmann

This comment has been minimized.

Contributor

t-wissmann commented Aug 14, 2015

I've written a userscript that uses pass as a password store to auto-fill login forms, i.e. forms with at least one password field. See PR #873, suggestions are welcome!

@Elronnd

This comment has been minimized.

Elronnd commented Feb 5, 2017

-1 to gnome-keyring. Many people (myself included) think GNOME is a really bad idea. An in-house solution would probably be sufficient and a lot better.

@The-Compiler

This comment has been minimized.

Collaborator

The-Compiler commented Jul 5, 2017

WIP patch from #875 - as mentioned there, I'd rather have this when there's a plugin API rather than hardcoding a password backend.

From de7a6f0095a584770f1d769944f204a87ce3d258 Mon Sep 17 00:00:00 2001
From: Antoni Boucher <bouanto@zoho.com>
Date: Thu, 13 Aug 2015 19:24:03 -0400
Subject: [PATCH 01/13] First version of form filler.

---
 qutebrowser/app.py                |   5 +-
 qutebrowser/browser/formfiller.py | 155 ++++++++++++++++++++++++++++++++++++++
 2 files changed, 159 insertions(+), 1 deletion(-)
 create mode 100644 qutebrowser/browser/formfiller.py

diff --git a/qutebrowser/app.py b/qutebrowser/app.py
index 6ca23ba21d..267d322853 100644
--- a/qutebrowser/app.py
+++ b/qutebrowser/app.py
@@ -45,7 +45,7 @@
 from qutebrowser.completion.models import instances as completionmodels
 from qutebrowser.commands import cmdutils, runners, cmdexc
 from qutebrowser.config import style, config, websettings, configexc
-from qutebrowser.browser import urlmarks, cookies, cache, adblock, history
+from qutebrowser.browser import urlmarks, cookies, cache, adblock, history, formfiller
 from qutebrowser.browser.network import qutescheme, proxy, networkmanager
 from qutebrowser.mainwindow import mainwindow
 from qutebrowser.misc import readline, ipc, savemanager, sessions, crashsignal
@@ -413,6 +413,9 @@ def _init_modules(args, crash_handler):
     host_blocker = adblock.HostBlocker()
     host_blocker.read_hosts()
     objreg.register('host-blocker', host_blocker)
+    log.init.debug("Initializing form filler...")
+    form_filler = formfiller.FormFiller()
+    objreg.register('form-filler', form_filler)
     log.init.debug("Initializing quickmarks...")
     quickmark_manager = urlmarks.QuickmarkManager(qApp)
     objreg.register('quickmark-manager', quickmark_manager)
diff --git a/qutebrowser/browser/formfiller.py b/qutebrowser/browser/formfiller.py
new file mode 100644
index 0000000000..ae5124b125
--- /dev/null
+++ b/qutebrowser/browser/formfiller.py
@@ -0,0 +1,155 @@
+# vim: ft=python fileencoding=utf-8 sts=4 sw=4 et:
+
+# Copyright 2015 Antoni Boucher (antoyo) <bouanto@zoho.com>
+#
+# This file is part of qutebrowser.
+#
+# qutebrowser is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# qutebrowser is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with qutebrowser.  If not, see <http://www.gnu.org/licenses/>.
+
+import os
+
+from yaml import Dumper, Loader, dump, load
+
+from PyQt5.QtCore import QUrl
+
+from qutebrowser.commands import cmdexc, cmdutils
+from qutebrowser.utils import objreg, standarddir
+
+class FormManager:
+    def load(self, urlstring):
+        raise NotImplemented
+
+    def save(self, urlstring, form_data):
+        raise NotImplemented
+
+class YamlFormManager(FormManager):
+    def __init__(self, filename):
+        self._filename = filename
+
+    def load(self, urlstring):
+        with open(self._filename, "r") as form_file:
+            data = load(form_file)
+        return data[urlstring]
+
+    def save(self, urlstring, form_data):
+        new_data = {
+            urlstring: form_data,
+        }
+        with open(self._filename, "a") as form_file:
+            dump(new_data, form_file)
+
+class FormFiller:
+    def __init__(self):
+        self._filename = os.path.join(standarddir.config(), 'forms.yaml')
+
+    def _find_form(self, elem):
+        form = elem
+        while not form.isNull() and form.tagName() != 'FORM':
+            form = form.parent()
+        return form
+
+    def _find_form_data(self, elem):
+        form = self._find_form(elem)
+        elems = form.findAll('input[type="checkbox"], input[type="text"],'
+                'input[type="password"], textarea')
+        data = []
+        for element in elems:
+            name = element.attribute("name")
+            id = element.attribute("id")
+            checked = element.evaluateJavaScript("this.checked")
+
+            # TODO: support when JavaScript is disabled.
+            if len(id) > 0:
+                selector = "#" + id
+            elif len(name) > 0:
+                selector = "[name=" + name + "]"
+
+            if element.attribute("type") == "checkbox":
+                value = element.evaluateJavaScript("this.checked")
+            else:
+                value = element.evaluateJavaScript("this.value")
+
+            data.append({
+                'selector': selector,
+                'value': value,
+            })
+        return data
+
+    def _get_form_data(self):
+        mainframe = self._get_frame()
+        elem = mainframe.findFirstElement("input:focus")
+        return self._find_form_data(elem)
+
+    def _get_frame(self):
+        tabbed_browser = objreg.get('tabbed-browser', scope='window',
+                                    window=self._win_id)
+        widget = tabbed_browser.currentWidget()
+        if widget is None:
+            raise cmdexc.CommandError("No WebView available yet!")
+        return widget.page().mainFrame()
+
+    def _get_url(self):
+        tabbed_browser = objreg.get('tabbed-browser', scope='window',
+                                    window=self._win_id)
+        return tabbed_browser.current_url().toString(
+            QUrl.FullyEncoded | QUrl.RemovePassword | QUrl.RemoveQuery)
+
+    def _load(self, url):
+        form_manager = YamlFormManager(self._filename)
+        return form_manager.load(url)
+
+    @cmdutils.register(instance='form-filler', win_id='win_id')
+    def load_form(self, win_id):
+        """Load the form from the current URL."""
+        self._win_id = win_id
+
+        url = self._get_url()
+        try:
+            form_data = self._load(url)
+        except (KeyError, FileNotFoundError):
+            raise cmdexc.CommandError("No form for the current URL!")
+
+        mainframe = self._get_frame()
+
+        for data in form_data:
+            elem = mainframe.findFirstElement(data["selector"])
+            if elem.attribute("type") == "checkbox":
+                elem.evaluateJavaScript("this.checked = " + str(data["value"]).lower())
+            else:
+                elem.evaluateJavaScript("this.value = '" + data["value"] + "'")
+
+    @cmdutils.register(instance='form-filler', win_id='win_id')
+    def load_form_submit(self, win_id):
+        """Load the form from the current URL and submit the form."""
+        self.load_form(win_id)
+        mainframe = self._get_frame()
+        elem = mainframe.findFirstElement("input:focus")
+        form = self._find_form(elem)
+        self._submit(form)
+
+    def _save(self, url, form_data):
+        form_manager = YamlFormManager(self._filename)
+        form_manager.save(url, form_data)
+
+    @cmdutils.register(instance='form-filler', win_id='win_id')
+    def save_form(self, win_id):
+        """Save the form from the current URL."""
+        self._win_id = win_id
+
+        form_data = self._get_form_data()
+        url = self._get_url()
+        self._save(url, form_data)
+
+    def _submit(self, form):
+        form.evaluateJavaScript("this.submit()")

From 54b0d4eeae387d360a81b5b92c0c608d7e92300e Mon Sep 17 00:00:00 2001
From: Antoni Boucher <bouanto@zoho.com>
Date: Fri, 14 Aug 2015 20:22:47 -0400
Subject: [PATCH 02/13] Added confirm if the password already exists.

---
 qutebrowser/app.py                |  3 +-
 qutebrowser/browser/formfiller.py | 97 +++++++++++++++++++++++++++++++--------
 2 files changed, 80 insertions(+), 20 deletions(-)

diff --git a/qutebrowser/app.py b/qutebrowser/app.py
index 267d322853..6ada0a6455 100644
--- a/qutebrowser/app.py
+++ b/qutebrowser/app.py
@@ -45,7 +45,8 @@
 from qutebrowser.completion.models import instances as completionmodels
 from qutebrowser.commands import cmdutils, runners, cmdexc
 from qutebrowser.config import style, config, websettings, configexc
-from qutebrowser.browser import urlmarks, cookies, cache, adblock, history, formfiller
+from qutebrowser.browser import (urlmarks, cookies, cache, adblock, history,
+                                 formfiller)
 from qutebrowser.browser.network import qutescheme, proxy, networkmanager
 from qutebrowser.mainwindow import mainwindow
 from qutebrowser.misc import readline, ipc, savemanager, sessions, crashsignal
diff --git a/qutebrowser/browser/formfiller.py b/qutebrowser/browser/formfiller.py
index ae5124b125..302fdff8e3 100644
--- a/qutebrowser/browser/formfiller.py
+++ b/qutebrowser/browser/formfiller.py
@@ -17,61 +17,99 @@
 # You should have received a copy of the GNU General Public License
 # along with qutebrowser.  If not, see <http://www.gnu.org/licenses/>.
 
+"""Form filler."""
+
 import os
 
-from yaml import Dumper, Loader, dump, load
+from yaml import dump, load
 
 from PyQt5.QtCore import QUrl
 
 from qutebrowser.commands import cmdexc, cmdutils
-from qutebrowser.utils import objreg, standarddir
+from qutebrowser.utils import message, objreg, standarddir, usertypes
+
 
 class FormManager:
+
+    """Base abstract class for form storage."""
+
     def load(self, urlstring):
-        raise NotImplemented
+        raise NotImplementedError
 
     def save(self, urlstring, form_data):
-        raise NotImplemented
+        raise NotImplementedError
+
 
 class YamlFormManager(FormManager):
+
+    """YAML form storage."""
+
     def __init__(self, filename):
         self._filename = filename
 
     def load(self, urlstring):
-        with open(self._filename, "r") as form_file:
+        with open(self._filename, "r", encoding="utf-8") as form_file:
             data = load(form_file)
+            if data is None:
+                data = {}
         return data[urlstring]
 
     def save(self, urlstring, form_data):
-        new_data = {
-            urlstring: form_data,
-        }
-        with open(self._filename, "a") as form_file:
-            dump(new_data, form_file)
+        if not os.path.isfile(self._filename):
+            os.mknod(self._filename)
+
+        with open(self._filename, "r+", encoding="utf-8") as form_file:
+            data = load(form_file)
+            if data is None:
+                data = {}
+            data[urlstring] = form_data
+            form_file.seek(0)
+            dump(data, form_file)
+            form_file.truncate()
+
 
 class FormFiller:
+
+    """Save and load form data."""
+
     def __init__(self):
         self._filename = os.path.join(standarddir.config(), 'forms.yaml')
+        self._win_id = 0
 
     def _find_form(self, elem):
+        """Find the form element from the input elem.
+
+        Args:
+            elem: The input element.
+
+        Return:
+            The form element.
+        """
         form = elem
         while not form.isNull() and form.tagName() != 'FORM':
             form = form.parent()
         return form
 
     def _find_form_data(self, elem):
+        """Find the data from the form containing the specified input element.
+
+        Args:
+            elem: The input element.
+
+        Return:
+            The form data.
+        """
         form = self._find_form(elem)
         elems = form.findAll('input[type="checkbox"], input[type="text"],'
-                'input[type="password"], textarea')
+                             'input[type="password"], textarea')
         data = []
         for element in elems:
             name = element.attribute("name")
-            id = element.attribute("id")
-            checked = element.evaluateJavaScript("this.checked")
+            elem_id = element.attribute("id")
 
             # TODO: support when JavaScript is disabled.
-            if len(id) > 0:
-                selector = "#" + id
+            if len(elem_id) > 0:
+                selector = "#" + elem_id
             elif len(name) > 0:
                 selector = "[name=" + name + "]"
 
@@ -87,11 +125,13 @@ def _find_form_data(self, elem):
         return data
 
     def _get_form_data(self):
+        """Get the data from the form containing the focused element."""
         mainframe = self._get_frame()
         elem = mainframe.findFirstElement("input:focus")
         return self._find_form_data(elem)
 
     def _get_frame(self):
+        """Get the current frame."""
         tabbed_browser = objreg.get('tabbed-browser', scope='window',
                                     window=self._win_id)
         widget = tabbed_browser.currentWidget()
@@ -100,6 +140,7 @@ def _get_frame(self):
         return widget.page().mainFrame()
 
     def _get_url(self):
+        """Get the current URL."""
         tabbed_browser = objreg.get('tabbed-browser', scope='window',
                                     window=self._win_id)
         return tabbed_browser.current_url().toString(
@@ -125,7 +166,8 @@ def load_form(self, win_id):
         for data in form_data:
             elem = mainframe.findFirstElement(data["selector"])
             if elem.attribute("type") == "checkbox":
-                elem.evaluateJavaScript("this.checked = " + str(data["value"]).lower())
+                elem.evaluateJavaScript("this.checked = " +
+                                        str(data["value"]).lower())
             else:
                 elem.evaluateJavaScript("this.value = '" + data["value"] + "'")
 
@@ -146,10 +188,27 @@ def _save(self, url, form_data):
     def save_form(self, win_id):
         """Save the form from the current URL."""
         self._win_id = win_id
-
-        form_data = self._get_form_data()
         url = self._get_url()
-        self._save(url, form_data)
+
+        form_exists = True
+        try:
+            form_data = self._load(url)
+        except (KeyError, FileNotFoundError):
+            form_exists = False
+
+        save = False
+        if form_exists:
+            text = ("A password for this page already exists. "
+                    "Do you want to override it?")
+            save = message.ask(self._win_id, text,
+                               usertypes.PromptMode.yesno,
+                               default=True)
+        else:
+            save = True
+
+        if save:
+            form_data = self._get_form_data()
+            self._save(url, form_data)
 
     def _submit(self, form):
         form.evaluateJavaScript("this.submit()")

From 4ff705ec761c2d47623c57e8532ed87889bebac8 Mon Sep 17 00:00:00 2001
From: Antoni Boucher <bouanto@zoho.com>
Date: Fri, 14 Aug 2015 21:50:28 -0400
Subject: [PATCH 03/13] Added pass storage.

---
 qutebrowser/browser/formfiller.py | 82 ++++++++++++++++++++++++++++++++-------
 qutebrowser/config/configdata.py  |  4 ++
 qutebrowser/config/configtypes.py |  7 ++++
 3 files changed, 78 insertions(+), 15 deletions(-)

diff --git a/qutebrowser/browser/formfiller.py b/qutebrowser/browser/formfiller.py
index 302fdff8e3..aa06822751 100644
--- a/qutebrowser/browser/formfiller.py
+++ b/qutebrowser/browser/formfiller.py
@@ -20,12 +20,15 @@
 """Form filler."""
 
 import os
+import shlex
+import subprocess
 
 from yaml import dump, load
 
 from PyQt5.QtCore import QUrl
 
 from qutebrowser.commands import cmdexc, cmdutils
+from qutebrowser.config import config
 from qutebrowser.utils import message, objreg, standarddir, usertypes
 
 
@@ -44,8 +47,8 @@ class YamlFormManager(FormManager):
 
     """YAML form storage."""
 
-    def __init__(self, filename):
-        self._filename = filename
+    def __init__(self):
+        self._filename = os.path.join(standarddir.config(), "forms.yaml")
 
     def load(self, urlstring):
         with open(self._filename, "r", encoding="utf-8") as form_file:
@@ -68,13 +71,61 @@ def save(self, urlstring, form_data):
             form_file.truncate()
 
 
+class PassFormManager(FormManager):
+
+    """Pass form storage."""
+
+    def _exec_pass(self, args, input_data=None):
+        """Exec the pass command with the specified arguments and input.
+
+        Args:
+            args: The command-line arguments to send to pass.
+            input: The text to send to pass stdin.
+
+        Return:
+            The pass output.
+        """
+        args.insert(0, "pass")
+        process = subprocess.Popen(args, stdout=subprocess.PIPE,
+                                   stderr=subprocess.PIPE,
+                                   stdin=subprocess.PIPE)
+        if input_data is not None:
+            input_data = input_data.encode("utf-8")
+
+        (result, error) = process.communicate(input_data)
+        if len(error) > 0:
+            raise KeyError
+        return result
+
+    def _escape_url(self, urlstring):
+        return shlex.quote(urlstring.replace("/", ""))
+
+    def load(self, urlstring):
+        key = self._escape_url(urlstring)
+        yaml_data = self._exec_pass(["qutebrowser/" + key])
+        result = load(yaml_data)
+        if result is None:
+            raise KeyError
+        return result
+
+    def save(self, urlstring, form_data):
+        key = self._escape_url(urlstring)
+        self._exec_pass(["insert", "-m", "qutebrowser/" + key],
+                        dump(form_data))
+
+
 class FormFiller:
 
     """Save and load form data."""
 
     def __init__(self):
-        self._filename = os.path.join(standarddir.config(), 'forms.yaml')
         self._win_id = 0
+        password_storage = config.get("general", "password-storage")
+        storages = {
+            "pass": PassFormManager,
+            "default": YamlFormManager,
+        }
+        self._form_manager = storages[password_storage]()
 
     def _find_form(self, elem):
         """Find the form element from the input elem.
@@ -86,7 +137,7 @@ def _find_form(self, elem):
             The form element.
         """
         form = elem
-        while not form.isNull() and form.tagName() != 'FORM':
+        while not form.isNull() and form.tagName() != "FORM":
             form = form.parent()
         return form
 
@@ -99,6 +150,7 @@ def _find_form_data(self, elem):
         Return:
             The form data.
         """
+        # TODO: have a workaround for when there is no form element.
         form = self._find_form(elem)
         elems = form.findAll('input[type="checkbox"], input[type="text"],'
                              'input[type="password"], textarea')
@@ -119,8 +171,8 @@ def _find_form_data(self, elem):
                 value = element.evaluateJavaScript("this.value")
 
             data.append({
-                'selector': selector,
-                'value': value,
+                "selector": selector,
+                "value": value,
             })
         return data
 
@@ -132,7 +184,7 @@ def _get_form_data(self):
 
     def _get_frame(self):
         """Get the current frame."""
-        tabbed_browser = objreg.get('tabbed-browser', scope='window',
+        tabbed_browser = objreg.get("tabbed-browser", scope="window",
                                     window=self._win_id)
         widget = tabbed_browser.currentWidget()
         if widget is None:
@@ -141,16 +193,15 @@ def _get_frame(self):
 
     def _get_url(self):
         """Get the current URL."""
-        tabbed_browser = objreg.get('tabbed-browser', scope='window',
+        tabbed_browser = objreg.get("tabbed-browser", scope="window",
                                     window=self._win_id)
         return tabbed_browser.current_url().toString(
             QUrl.FullyEncoded | QUrl.RemovePassword | QUrl.RemoveQuery)
 
     def _load(self, url):
-        form_manager = YamlFormManager(self._filename)
-        return form_manager.load(url)
+        return self._form_manager.load(url)
 
-    @cmdutils.register(instance='form-filler', win_id='win_id')
+    @cmdutils.register(instance="form-filler", win_id="win_id")
     def load_form(self, win_id):
         """Load the form from the current URL."""
         self._win_id = win_id
@@ -171,7 +222,7 @@ def load_form(self, win_id):
             else:
                 elem.evaluateJavaScript("this.value = '" + data["value"] + "'")
 
-    @cmdutils.register(instance='form-filler', win_id='win_id')
+    @cmdutils.register(instance="form-filler", win_id="win_id")
     def load_form_submit(self, win_id):
         """Load the form from the current URL and submit the form."""
         self.load_form(win_id)
@@ -181,10 +232,9 @@ def load_form_submit(self, win_id):
         self._submit(form)
 
     def _save(self, url, form_data):
-        form_manager = YamlFormManager(self._filename)
-        form_manager.save(url, form_data)
+        self._form_manager.save(url, form_data)
 
-    @cmdutils.register(instance='form-filler', win_id='win_id')
+    @cmdutils.register(instance="form-filler", win_id="win_id")
     def save_form(self, win_id):
         """Save the form from the current URL."""
         self._win_id = win_id
@@ -211,4 +261,6 @@ def save_form(self, win_id):
             self._save(url, form_data)
 
     def _submit(self, form):
+        # TODO: have a workaround for when submiting the form does not work.
+        # e.g. click on the submit button.
         form.evaluateJavaScript("this.submit()")
diff --git a/qutebrowser/config/configdata.py b/qutebrowser/config/configdata.py
index 5e40bf7e18..769b20f489 100644
--- a/qutebrowser/config/configdata.py
+++ b/qutebrowser/config/configdata.py
@@ -225,6 +225,10 @@ def data(readonly=False):
              "The name of the session to save by default, or empty for the "
              "last loaded session."),
 
+            ('password-storage',
+             SettingValue(typ.PasswordStorage(), 'default'),
+             "The name of the password storage to use."),
+
             readonly=readonly
         )),
 
diff --git a/qutebrowser/config/configtypes.py b/qutebrowser/config/configtypes.py
index 75e05360c3..7ed8b84702 100644
--- a/qutebrowser/config/configtypes.py
+++ b/qutebrowser/config/configtypes.py
@@ -1297,6 +1297,13 @@ def validate(self, value):
             raise configexc.ValidationError(value, "may not start with '_'!")
 
 
+class PasswordStorage(BaseType):
+
+    """The name of a password storage."""
+
+    valid_values = ValidValues("default", "pass")
+
+
 class SelectOnRemove(MappingType):
 
     """Which tab to select when the focused tab is removed."""

From 0ecab1eb5320ee0205f227be207e6bf405d27693 Mon Sep 17 00:00:00 2001
From: Antoni Boucher <bouanto@zoho.com>
Date: Sat, 22 Aug 2015 20:07:18 -0400
Subject: [PATCH 04/13] Refactored the form filler to be a password filler.

Added the possibility to load/save multiple users for a single site.
---
 qutebrowser/app.py                    |   8 +-
 qutebrowser/browser/formfiller.py     | 266 -------------------------
 qutebrowser/browser/passwordfiller.py | 358 ++++++++++++++++++++++++++++++++++
 3 files changed, 362 insertions(+), 270 deletions(-)
 delete mode 100644 qutebrowser/browser/formfiller.py
 create mode 100644 qutebrowser/browser/passwordfiller.py

diff --git a/qutebrowser/app.py b/qutebrowser/app.py
index 6ada0a6455..d8a145f4a3 100644
--- a/qutebrowser/app.py
+++ b/qutebrowser/app.py
@@ -46,7 +46,7 @@
 from qutebrowser.commands import cmdutils, runners, cmdexc
 from qutebrowser.config import style, config, websettings, configexc
 from qutebrowser.browser import (urlmarks, cookies, cache, adblock, history,
-                                 formfiller)
+                                 passwordfiller)
 from qutebrowser.browser.network import qutescheme, proxy, networkmanager
 from qutebrowser.mainwindow import mainwindow
 from qutebrowser.misc import readline, ipc, savemanager, sessions, crashsignal
@@ -414,9 +414,9 @@ def _init_modules(args, crash_handler):
     host_blocker = adblock.HostBlocker()
     host_blocker.read_hosts()
     objreg.register('host-blocker', host_blocker)
-    log.init.debug("Initializing form filler...")
-    form_filler = formfiller.FormFiller()
-    objreg.register('form-filler', form_filler)
+    log.init.debug("Initializing password filler...")
+    password_filler = passwordfiller.PasswordFiller()
+    objreg.register('password-filler', password_filler)
     log.init.debug("Initializing quickmarks...")
     quickmark_manager = urlmarks.QuickmarkManager(qApp)
     objreg.register('quickmark-manager', quickmark_manager)
diff --git a/qutebrowser/browser/formfiller.py b/qutebrowser/browser/formfiller.py
deleted file mode 100644
index aa06822751..0000000000
--- a/qutebrowser/browser/formfiller.py
+++ /dev/null
@@ -1,266 +0,0 @@
-# vim: ft=python fileencoding=utf-8 sts=4 sw=4 et:
-
-# Copyright 2015 Antoni Boucher (antoyo) <bouanto@zoho.com>
-#
-# This file is part of qutebrowser.
-#
-# qutebrowser is free software: you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation, either version 3 of the License, or
-# (at your option) any later version.
-#
-# qutebrowser is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License
-# along with qutebrowser.  If not, see <http://www.gnu.org/licenses/>.
-
-"""Form filler."""
-
-import os
-import shlex
-import subprocess
-
-from yaml import dump, load
-
-from PyQt5.QtCore import QUrl
-
-from qutebrowser.commands import cmdexc, cmdutils
-from qutebrowser.config import config
-from qutebrowser.utils import message, objreg, standarddir, usertypes
-
-
-class FormManager:
-
-    """Base abstract class for form storage."""
-
-    def load(self, urlstring):
-        raise NotImplementedError
-
-    def save(self, urlstring, form_data):
-        raise NotImplementedError
-
-
-class YamlFormManager(FormManager):
-
-    """YAML form storage."""
-
-    def __init__(self):
-        self._filename = os.path.join(standarddir.config(), "forms.yaml")
-
-    def load(self, urlstring):
-        with open(self._filename, "r", encoding="utf-8") as form_file:
-            data = load(form_file)
-            if data is None:
-                data = {}
-        return data[urlstring]
-
-    def save(self, urlstring, form_data):
-        if not os.path.isfile(self._filename):
-            os.mknod(self._filename)
-
-        with open(self._filename, "r+", encoding="utf-8") as form_file:
-            data = load(form_file)
-            if data is None:
-                data = {}
-            data[urlstring] = form_data
-            form_file.seek(0)
-            dump(data, form_file)
-            form_file.truncate()
-
-
-class PassFormManager(FormManager):
-
-    """Pass form storage."""
-
-    def _exec_pass(self, args, input_data=None):
-        """Exec the pass command with the specified arguments and input.
-
-        Args:
-            args: The command-line arguments to send to pass.
-            input: The text to send to pass stdin.
-
-        Return:
-            The pass output.
-        """
-        args.insert(0, "pass")
-        process = subprocess.Popen(args, stdout=subprocess.PIPE,
-                                   stderr=subprocess.PIPE,
-                                   stdin=subprocess.PIPE)
-        if input_data is not None:
-            input_data = input_data.encode("utf-8")
-
-        (result, error) = process.communicate(input_data)
-        if len(error) > 0:
-            raise KeyError
-        return result
-
-    def _escape_url(self, urlstring):
-        return shlex.quote(urlstring.replace("/", ""))
-
-    def load(self, urlstring):
-        key = self._escape_url(urlstring)
-        yaml_data = self._exec_pass(["qutebrowser/" + key])
-        result = load(yaml_data)
-        if result is None:
-            raise KeyError
-        return result
-
-    def save(self, urlstring, form_data):
-        key = self._escape_url(urlstring)
-        self._exec_pass(["insert", "-m", "qutebrowser/" + key],
-                        dump(form_data))
-
-
-class FormFiller:
-
-    """Save and load form data."""
-
-    def __init__(self):
-        self._win_id = 0
-        password_storage = config.get("general", "password-storage")
-        storages = {
-            "pass": PassFormManager,
-            "default": YamlFormManager,
-        }
-        self._form_manager = storages[password_storage]()
-
-    def _find_form(self, elem):
-        """Find the form element from the input elem.
-
-        Args:
-            elem: The input element.
-
-        Return:
-            The form element.
-        """
-        form = elem
-        while not form.isNull() and form.tagName() != "FORM":
-            form = form.parent()
-        return form
-
-    def _find_form_data(self, elem):
-        """Find the data from the form containing the specified input element.
-
-        Args:
-            elem: The input element.
-
-        Return:
-            The form data.
-        """
-        # TODO: have a workaround for when there is no form element.
-        form = self._find_form(elem)
-        elems = form.findAll('input[type="checkbox"], input[type="text"],'
-                             'input[type="password"], textarea')
-        data = []
-        for element in elems:
-            name = element.attribute("name")
-            elem_id = element.attribute("id")
-
-            # TODO: support when JavaScript is disabled.
-            if len(elem_id) > 0:
-                selector = "#" + elem_id
-            elif len(name) > 0:
-                selector = "[name=" + name + "]"
-
-            if element.attribute("type") == "checkbox":
-                value = element.evaluateJavaScript("this.checked")
-            else:
-                value = element.evaluateJavaScript("this.value")
-
-            data.append({
-                "selector": selector,
-                "value": value,
-            })
-        return data
-
-    def _get_form_data(self):
-        """Get the data from the form containing the focused element."""
-        mainframe = self._get_frame()
-        elem = mainframe.findFirstElement("input:focus")
-        return self._find_form_data(elem)
-
-    def _get_frame(self):
-        """Get the current frame."""
-        tabbed_browser = objreg.get("tabbed-browser", scope="window",
-                                    window=self._win_id)
-        widget = tabbed_browser.currentWidget()
-        if widget is None:
-            raise cmdexc.CommandError("No WebView available yet!")
-        return widget.page().mainFrame()
-
-    def _get_url(self):
-        """Get the current URL."""
-        tabbed_browser = objreg.get("tabbed-browser", scope="window",
-                                    window=self._win_id)
-        return tabbed_browser.current_url().toString(
-            QUrl.FullyEncoded | QUrl.RemovePassword | QUrl.RemoveQuery)
-
-    def _load(self, url):
-        return self._form_manager.load(url)
-
-    @cmdutils.register(instance="form-filler", win_id="win_id")
-    def load_form(self, win_id):
-        """Load the form from the current URL."""
-        self._win_id = win_id
-
-        url = self._get_url()
-        try:
-            form_data = self._load(url)
-        except (KeyError, FileNotFoundError):
-            raise cmdexc.CommandError("No form for the current URL!")
-
-        mainframe = self._get_frame()
-
-        for data in form_data:
-            elem = mainframe.findFirstElement(data["selector"])
-            if elem.attribute("type") == "checkbox":
-                elem.evaluateJavaScript("this.checked = " +
-                                        str(data["value"]).lower())
-            else:
-                elem.evaluateJavaScript("this.value = '" + data["value"] + "'")
-
-    @cmdutils.register(instance="form-filler", win_id="win_id")
-    def load_form_submit(self, win_id):
-        """Load the form from the current URL and submit the form."""
-        self.load_form(win_id)
-        mainframe = self._get_frame()
-        elem = mainframe.findFirstElement("input:focus")
-        form = self._find_form(elem)
-        self._submit(form)
-
-    def _save(self, url, form_data):
-        self._form_manager.save(url, form_data)
-
-    @cmdutils.register(instance="form-filler", win_id="win_id")
-    def save_form(self, win_id):
-        """Save the form from the current URL."""
-        self._win_id = win_id
-        url = self._get_url()
-
-        form_exists = True
-        try:
-            form_data = self._load(url)
-        except (KeyError, FileNotFoundError):
-            form_exists = False
-
-        save = False
-        if form_exists:
-            text = ("A password for this page already exists. "
-                    "Do you want to override it?")
-            save = message.ask(self._win_id, text,
-                               usertypes.PromptMode.yesno,
-                               default=True)
-        else:
-            save = True
-
-        if save:
-            form_data = self._get_form_data()
-            self._save(url, form_data)
-
-    def _submit(self, form):
-        # TODO: have a workaround for when submiting the form does not work.
-        # e.g. click on the submit button.
-        form.evaluateJavaScript("this.submit()")
diff --git a/qutebrowser/browser/passwordfiller.py b/qutebrowser/browser/passwordfiller.py
new file mode 100644
index 0000000000..69dc9b03a2
--- /dev/null
+++ b/qutebrowser/browser/passwordfiller.py
@@ -0,0 +1,358 @@
+# vim: ft=python fileencoding=utf-8 sts=4 sw=4 et:
+
+# Copyright 2015 Antoni Boucher (antoyo) <bouanto@zoho.com>
+#
+# This file is part of qutebrowser.
+#
+# qutebrowser is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# qutebrowser is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with qutebrowser.  If not, see <http://www.gnu.org/licenses/>.
+
+"""Form filler."""
+
+import os
+import shlex
+import subprocess
+
+from yaml import dump, load
+
+from PyQt5.QtCore import QUrl
+
+from qutebrowser.commands import cmdexc, cmdutils
+from qutebrowser.config import config
+from qutebrowser.utils import message, objreg, standarddir, usertypes
+
+
+class PasswordManager:
+
+    """Base abstract class for password storage."""
+
+    def load(self, urlstring):
+        raise NotImplementedError
+
+    def save(self, urlstring, password_data):
+        raise NotImplementedError
+
+
+class YamlPasswordManager(PasswordManager):
+
+    """YAML password storage."""
+
+    def __init__(self):
+        self._filename = os.path.join(standarddir.config(), "passwords.yaml")
+
+    def load(self, urlstring):
+        with open(self._filename, "r", encoding="utf-8") as password_file:
+            data = load(password_file)
+            if data is None:
+                data = {}
+        password_data = data[urlstring]
+        return password_data
+
+    def save(self, urlstring, password_data):
+        if not os.path.isfile(self._filename):
+            os.mknod(self._filename)
+
+        with open(self._filename, "r+", encoding="utf-8") as password_file:
+            data = load(password_file)
+            if data is None:
+                data = {}
+            if urlstring not in data:
+                data[urlstring] = {}
+            username = password_data["username"]["value"]
+            data[urlstring][username] = {
+                "password": password_data["password"]["value"],
+            }
+            if "checkbox" in password_data:
+                data[urlstring][username]["checkbox"] = password_data["checkbox"]["value"]
+            password_file.seek(0)
+            dump(data, password_file)
+            password_file.truncate()
+
+
+class PassPasswordManager(PasswordManager):
+
+    # TODO: update to be on par with YamlPasswordManager.
+
+    """Pass password storage."""
+
+    def _exec_pass(self, args, input_data=None):
+        """Exec the pass command with the specified arguments and input.
+
+        Args:
+            args: The command-line arguments to send to pass.
+            input: The text to send to pass stdin.
+
+        Return:
+            The pass output.
+        """
+        args.insert(0, "pass")
+        process = subprocess.Popen(args, stdout=subprocess.PIPE,
+                                   stderr=subprocess.PIPE,
+                                   stdin=subprocess.PIPE)
+        if input_data is not None:
+            input_data = input_data.encode("utf-8")
+
+        (result, error) = process.communicate(input_data)
+        if len(error) > 0:
+            raise KeyError
+        return result
+
+    def _escape_url(self, urlstring):
+        return shlex.quote(urlstring)
+
+    def load(self, urlstring):
+        key = self._escape_url(urlstring)
+        yaml_data = self._exec_pass(["qutebrowser/" + key])
+        result = load(yaml_data)
+        if result is None:
+            raise KeyError
+        return result
+
+    def save(self, urlstring, password_data):
+        key = self._escape_url(urlstring)
+        self._exec_pass(["insert", "-m", "qutebrowser/" + key],
+                        dump(password_data))
+
+
+class PasswordFiller:
+
+    """Save and load login form data."""
+
+    def __init__(self):
+        self._win_id = 0
+        password_storage = config.get("general", "password-storage")
+        storages = {
+            "pass": PassPasswordManager,
+            "default": YamlPasswordManager,
+        }
+        self._password_manager = storages[password_storage]()
+
+    def _choose_username(self, password_data):
+        """Ask to user which username to use.
+        If there is only one username, use this username without asking.
+
+        Return:
+            The username chosen by the user.
+        """
+
+        usernames = list(password_data.keys())
+        answer = None
+        if len(usernames) > 1:
+            text = "Which username:"
+            index = 0
+            for username in usernames:
+                text += " " + str(index) + ". " + username
+                index += 1
+            answer = message.ask(self._win_id, text,
+                               usertypes.PromptMode.text)
+            try:
+                answer = int(answer)
+            except ValueError:
+                answer = None
+        else:
+            answer = 0
+
+        if answer is None or answer >= len(usernames):
+            max_index = len(usernames) - 1
+            raise cmdexc.CommandError("Type a number between 0 and %d." % max_index)
+
+        return usernames[answer]
+
+    def _find_form(self):
+        """Find the login form element.
+
+        Return:
+            The login form element.
+        """
+        frame = self._get_frame()
+        elem = frame.findFirstElement('input[type="password"]');
+        form = elem
+        # TODO: find a workaround for when there is no form element around the
+        # login form.
+        while not form.isNull() and form.tagName() != "FORM":
+            form = form.parent()
+        return form
+
+    def _find_login_form_elements(self):
+        """Find the login form and return the username, password and checkbox
+        elements.
+
+        Return:
+            A dict containing the username, password and checkbox (if present
+            in the form) elements and values.
+        """
+        form = self._find_form()
+
+        elems = form.findAll('input[type="checkbox"], input[type="text"],'
+                             'input[type="password"]')
+        username_element = None
+        password_element = None
+        checkbox_element = None
+        for element in elems:
+            elem_type = element.attribute("type")
+            if elem_type == "checkbox":
+                checkbox_element = element
+            elif elem_type == "password":
+                password_element = element
+            elif elem_type == "text" and username_element is None:
+                username_element = element
+
+        if username_element is None and password_element is None:
+            raise RuntimeError
+
+        data = {
+            'username': {
+                'element': username_element,
+                'value': username_element.evaluateJavaScript("this.value"),
+            },
+            'password': {
+                'element': password_element,
+                'value': password_element.evaluateJavaScript("this.value"),
+            },
+        }
+        if checkbox_element is not None:
+            data["checkbox"] = {
+                "element": checkbox_element,
+                "value": checkbox_element.evaluateJavaScript("this.checked"),
+            }
+
+        return data
+
+    def _get_frame(self):
+        """Get the current frame."""
+        tabbed_browser = objreg.get("tabbed-browser", scope="window",
+                                    window=self._win_id)
+        widget = tabbed_browser.currentWidget()
+        if widget is None:
+            raise cmdexc.CommandError("No WebView available yet!")
+        return widget.page().mainFrame()
+
+    def _get_host(self):
+        """Get the current URL host."""
+        tabbed_browser = objreg.get("tabbed-browser", scope="window",
+                                    window=self._win_id)
+        return tabbed_browser.current_url().host()
+
+    def _load(self, url):
+        return self._password_manager.load(url)
+
+    @cmdutils.register(instance="password-filler", win_id="win_id")
+    def load_password(self, win_id):
+        """Load the password data from the current URL."""
+        self._win_id = win_id
+
+        host = self._get_host()
+        try:
+            password_data = self._load(host)
+        except (KeyError, FileNotFoundError):
+            raise cmdexc.CommandError("No password data for the current URL!")
+
+        mainframe = self._get_frame()
+
+        username = self._choose_username(password_data)
+
+        password = password_data[username]["password"]
+        form_elements = self._find_login_form_elements()
+        self._set_value(form_elements["username"]["element"], username)
+        self._set_value(form_elements["password"]["element"], password)
+
+        if ("checkbox" in form_elements and "checkbox" in
+            password_data[username]):
+            self._set_checkbox_value(form_elements["checkbox"]["element"],
+                                     password_data[username]["checkbox"])
+
+    @cmdutils.register(instance="password-filler", win_id="win_id")
+    def load_password_submit(self, win_id):
+        """Load the password data for the current URL and submit the form."""
+        self.load_password(win_id)
+        mainframe = self._get_frame()
+        elem = mainframe.findFirstElement("input:focus")
+        form = self._find_form()
+        self._submit(form)
+
+    def _password_exists(self, url, username):
+        """Check if a password exists for the current URL and username.
+
+        Args:
+            url: The URL to check.
+            username: The username to check.
+
+        Return:
+            Whether a password exists or not.
+        """
+        password_exists = True
+        try:
+            existing_password_data = self._load(url)
+            password_exists = username in existing_password_data
+        except (KeyError, FileNotFoundError):
+            password_exists = False
+        return password_exists
+
+    def _save(self, url, password_data):
+        """Save the password data for the current URL using the right
+        password manager.
+
+        Args:
+            url: The URL for the password data to save.
+            password_data: The password data.
+        """
+        self._password_manager.save(url, password_data)
+
+    @cmdutils.register(instance="password-filler", win_id="win_id")
+    def save_password(self, win_id):
+        """Save the password for the current URL."""
+        self._win_id = win_id
+        host = self._get_host()
+
+        try:
+            password_data = self._find_login_form_elements()
+        except RuntimeError:
+            raise cmdexc.CommandError("No login form found in the current page!")
+        else:
+            username = password_data["username"]["value"]
+
+            save = False
+            if self._password_exists(host, username):
+                text = ("A password for this page already exists. "
+                        "Do you want to override it?")
+                save = message.ask(self._win_id, text,
+                                   usertypes.PromptMode.yesno,
+                                   default=False)
+            else:
+                save = True
+
+            if save:
+                self._save(host, password_data)
+
+    def _set_checkbox_value(self, element, value):
+        """Set the checkbox element value.
+
+        Args:
+            element: The element to change its value.
+            value: The new value.
+        """
+        element.evaluateJavaScript("this.checked = " + str(value).lower())
+
+    def _set_value(self, element, value):
+        """Set the element value.
+
+        Args:
+            element: The element to change its value.
+            value: The new value.
+        """
+        element.evaluateJavaScript("this.value = '" + value + "'")
+
+    def _submit(self, form):
+        # TODO: have a workaround for when submiting the form does not work.
+        # e.g. click on the submit button.
+        form.evaluateJavaScript("this.submit()")

From cda86b61033bfa23e66029bc92c5c8defac000e2 Mon Sep 17 00:00:00 2001
From: Antoni Boucher <bouanto@zoho.com>
Date: Sun, 23 Aug 2015 10:29:53 -0400
Subject: [PATCH 05/13] Updated the pass-based password manager.

---
 qutebrowser/browser/passwordfiller.py | 114 +++++++++++++++++++++-------------
 1 file changed, 71 insertions(+), 43 deletions(-)

diff --git a/qutebrowser/browser/passwordfiller.py b/qutebrowser/browser/passwordfiller.py
index 69dc9b03a2..6bc397dba4 100644
--- a/qutebrowser/browser/passwordfiller.py
+++ b/qutebrowser/browser/passwordfiller.py
@@ -25,8 +25,6 @@
 
 from yaml import dump, load
 
-from PyQt5.QtCore import QUrl
-
 from qutebrowser.commands import cmdexc, cmdutils
 from qutebrowser.config import config
 from qutebrowser.utils import message, objreg, standarddir, usertypes
@@ -36,7 +34,10 @@ class PasswordManager:
 
     """Base abstract class for password storage."""
 
-    def load(self, urlstring):
+    def get_usernames(self, urlstring):
+        raise NotImplementedError
+
+    def load(self, urlstring, username):
         raise NotImplementedError
 
     def save(self, urlstring, password_data):
@@ -50,13 +51,21 @@ class YamlPasswordManager(PasswordManager):
     def __init__(self):
         self._filename = os.path.join(standarddir.config(), "passwords.yaml")
 
-    def load(self, urlstring):
+    def get_usernames(self, urlstring):
         with open(self._filename, "r", encoding="utf-8") as password_file:
             data = load(password_file)
             if data is None:
                 data = {}
         password_data = data[urlstring]
-        return password_data
+        return list(password_data.keys())
+
+    def load(self, urlstring, username):
+        with open(self._filename, "r", encoding="utf-8") as password_file:
+            data = load(password_file)
+            if data is None:
+                data = {}
+        password_data = data[urlstring]
+        return password_data[username]
 
     def save(self, urlstring, password_data):
         if not os.path.isfile(self._filename):
@@ -73,7 +82,9 @@ def save(self, urlstring, password_data):
                 "password": password_data["password"]["value"],
             }
             if "checkbox" in password_data:
-                data[urlstring][username]["checkbox"] = password_data["checkbox"]["value"]
+                data[urlstring][username]["checkbox"] = (
+                    password_data["checkbox"]["value"]
+                )
             password_file.seek(0)
             dump(data, password_file)
             password_file.truncate()
@@ -81,8 +92,6 @@ def save(self, urlstring, password_data):
 
 class PassPasswordManager(PasswordManager):
 
-    # TODO: update to be on par with YamlPasswordManager.
-
     """Pass password storage."""
 
     def _exec_pass(self, args, input_data=None):
@@ -105,23 +114,45 @@ def _exec_pass(self, args, input_data=None):
         (result, error) = process.communicate(input_data)
         if len(error) > 0:
             raise KeyError
-        return result
+        return result.decode("utf-8")
 
     def _escape_url(self, urlstring):
         return shlex.quote(urlstring)
 
-    def load(self, urlstring):
-        key = self._escape_url(urlstring)
-        yaml_data = self._exec_pass(["qutebrowser/" + key])
-        result = load(yaml_data)
+    def get_usernames(self, host):
+        key = "qutebrowser/" + self._escape_url(host)
+        result = self._exec_pass([key])
+        lines = result.split("\n")[1:]
+        usernames = []
+        for line in lines:
+            if len(line) > 0:
+                username = line[4:]
+                usernames.append(username)
+        return usernames
+
+    def load(self, urlstring, username):
+        key = "qutebrowser/%s/%s" % (self._escape_url(urlstring), username)
+        result = self._exec_pass([key])
         if result is None:
             raise KeyError
-        return result
+        lines = result.split("\n")
+        password_data = {
+            "password": lines[0],
+        }
+
+        if len(lines) > 1:
+            password_data["checkbox"] = lines[1] == "True"
+
+        return password_data
 
     def save(self, urlstring, password_data):
         key = self._escape_url(urlstring)
-        self._exec_pass(["insert", "-m", "qutebrowser/" + key],
-                        dump(password_data))
+        username = password_data["username"]["value"]
+        content = password_data["password"]["value"]
+        if "checkbox" in password_data:
+            content += "\n" + str(password_data["checkbox"]["value"])
+        self._exec_pass(["insert", "-m", "qutebrowser/%s/%s" %
+                        (key, username)], content)
 
 
 class PasswordFiller:
@@ -137,34 +168,36 @@ def __init__(self):
         }
         self._password_manager = storages[password_storage]()
 
-    def _choose_username(self, password_data):
+    def _choose_username(self, urlstring):
         """Ask to user which username to use.
+
         If there is only one username, use this username without asking.
 
         Return:
             The username chosen by the user.
         """
-
-        usernames = list(password_data.keys())
+        usernames = self._password_manager.get_usernames(urlstring)
         answer = None
         if len(usernames) > 1:
             text = "Which username:"
             index = 0
+            # TODO: show on multiple lines.
             for username in usernames:
                 text += " " + str(index) + ". " + username
                 index += 1
             answer = message.ask(self._win_id, text,
-                               usertypes.PromptMode.text)
+                                 usertypes.PromptMode.text)
             try:
                 answer = int(answer)
-            except ValueError:
+            except (TypeError, ValueError):
                 answer = None
         else:
             answer = 0
 
         if answer is None or answer >= len(usernames):
             max_index = len(usernames) - 1
-            raise cmdexc.CommandError("Type a number between 0 and %d." % max_index)
+            raise cmdexc.CommandError("Type a number between 0 and %d." %
+                                      max_index)
 
         return usernames[answer]
 
@@ -175,7 +208,7 @@ def _find_form(self):
             The login form element.
         """
         frame = self._get_frame()
-        elem = frame.findFirstElement('input[type="password"]');
+        elem = frame.findFirstElement('input[type="password"]')
         form = elem
         # TODO: find a workaround for when there is no form element around the
         # login form.
@@ -184,8 +217,7 @@ def _find_form(self):
         return form
 
     def _find_login_form_elements(self):
-        """Find the login form and return the username, password and checkbox
-        elements.
+        """Find the login form.
 
         Return:
             A dict containing the username, password and checkbox (if present
@@ -243,8 +275,8 @@ def _get_host(self):
                                     window=self._win_id)
         return tabbed_browser.current_url().host()
 
-    def _load(self, url):
-        return self._password_manager.load(url)
+    def _load(self, url, username):
+        return self._password_manager.load(url, username)
 
     @cmdutils.register(instance="password-filler", win_id="win_id")
     def load_password(self, win_id):
@@ -252,31 +284,26 @@ def load_password(self, win_id):
         self._win_id = win_id
 
         host = self._get_host()
+
         try:
-            password_data = self._load(host)
+            username = self._choose_username(host)
+            password_data = self._load(host, username)
         except (KeyError, FileNotFoundError):
             raise cmdexc.CommandError("No password data for the current URL!")
 
-        mainframe = self._get_frame()
-
-        username = self._choose_username(password_data)
-
-        password = password_data[username]["password"]
+        password = password_data["password"]
         form_elements = self._find_login_form_elements()
         self._set_value(form_elements["username"]["element"], username)
         self._set_value(form_elements["password"]["element"], password)
 
-        if ("checkbox" in form_elements and "checkbox" in
-            password_data[username]):
+        if "checkbox" in form_elements and "checkbox" in password_data:
             self._set_checkbox_value(form_elements["checkbox"]["element"],
-                                     password_data[username]["checkbox"])
+                                     password_data["checkbox"])
 
     @cmdutils.register(instance="password-filler", win_id="win_id")
     def load_password_submit(self, win_id):
         """Load the password data for the current URL and submit the form."""
         self.load_password(win_id)
-        mainframe = self._get_frame()
-        elem = mainframe.findFirstElement("input:focus")
         form = self._find_form()
         self._submit(form)
 
@@ -292,15 +319,15 @@ def _password_exists(self, url, username):
         """
         password_exists = True
         try:
-            existing_password_data = self._load(url)
-            password_exists = username in existing_password_data
+            self._load(url, username)
         except (KeyError, FileNotFoundError):
             password_exists = False
         return password_exists
 
     def _save(self, url, password_data):
-        """Save the password data for the current URL using the right
-        password manager.
+        """Save the password data for the current URL.
+
+        This uses the right password manager.
 
         Args:
             url: The URL for the password data to save.
@@ -317,7 +344,8 @@ def save_password(self, win_id):
         try:
             password_data = self._find_login_form_elements()
         except RuntimeError:
-            raise cmdexc.CommandError("No login form found in the current page!")
+            raise cmdexc.CommandError(
+                "No login form found in the current page!")
         else:
             username = password_data["username"]["value"]
 

From 3f0f260c355ef5a89e2023162d55208d08c3ba54 Mon Sep 17 00:00:00 2001
From: Antoni Boucher <bouanto@zoho.com>
Date: Sun, 23 Aug 2015 10:44:34 -0400
Subject: [PATCH 06/13] Added missing comments and fixed variable names.

---
 qutebrowser/browser/passwordfiller.py | 97 ++++++++++++++++++++++++-----------
 1 file changed, 66 insertions(+), 31 deletions(-)

diff --git a/qutebrowser/browser/passwordfiller.py b/qutebrowser/browser/passwordfiller.py
index 6bc397dba4..5a67446720 100644
--- a/qutebrowser/browser/passwordfiller.py
+++ b/qutebrowser/browser/passwordfiller.py
@@ -34,13 +34,37 @@ class PasswordManager:
 
     """Base abstract class for password storage."""
 
-    def get_usernames(self, urlstring):
+    def get_usernames(self, host):
+        """Get a list of usernames stored for a specific URL host.
+
+        Args:
+            host: The URL host to check.
+
+        Return:
+            A list of usernames.
+        """
         raise NotImplementedError
 
-    def load(self, urlstring, username):
+    def load(self, host, username):
+        """Load the password data stored for a specific URL host and username.
+
+        Args:
+            host: The URL host.
+            username: The username.
+
+        Return:
+            The password data as a dict containing the password and optionaly
+            the checkbox keys.
+        """
         raise NotImplementedError
 
-    def save(self, urlstring, password_data):
+    def save(self, host, password_data):
+        """Save the password data for the specified host.
+
+        Args:
+            host: The URL host from where the data is coming.
+            password_data: The password data to save.
+        """
         raise NotImplementedError
 
 
@@ -51,23 +75,23 @@ class YamlPasswordManager(PasswordManager):
     def __init__(self):
         self._filename = os.path.join(standarddir.config(), "passwords.yaml")
 
-    def get_usernames(self, urlstring):
+    def get_usernames(self, host):
         with open(self._filename, "r", encoding="utf-8") as password_file:
             data = load(password_file)
             if data is None:
                 data = {}
-        password_data = data[urlstring]
+        password_data = data[host]
         return list(password_data.keys())
 
-    def load(self, urlstring, username):
+    def load(self, host, username):
         with open(self._filename, "r", encoding="utf-8") as password_file:
             data = load(password_file)
             if data is None:
                 data = {}
-        password_data = data[urlstring]
+        password_data = data[host]
         return password_data[username]
 
-    def save(self, urlstring, password_data):
+    def save(self, host, password_data):
         if not os.path.isfile(self._filename):
             os.mknod(self._filename)
 
@@ -75,14 +99,14 @@ def save(self, urlstring, password_data):
             data = load(password_file)
             if data is None:
                 data = {}
-            if urlstring not in data:
-                data[urlstring] = {}
+            if host not in data:
+                data[host] = {}
             username = password_data["username"]["value"]
-            data[urlstring][username] = {
+            data[host][username] = {
                 "password": password_data["password"]["value"],
             }
             if "checkbox" in password_data:
-                data[urlstring][username]["checkbox"] = (
+                data[host][username]["checkbox"] = (
                     password_data["checkbox"]["value"]
                 )
             password_file.seek(0)
@@ -116,11 +140,8 @@ def _exec_pass(self, args, input_data=None):
             raise KeyError
         return result.decode("utf-8")
 
-    def _escape_url(self, urlstring):
-        return shlex.quote(urlstring)
-
     def get_usernames(self, host):
-        key = "qutebrowser/" + self._escape_url(host)
+        key = "qutebrowser/%s" % shlex.quote(host)
         result = self._exec_pass([key])
         lines = result.split("\n")[1:]
         usernames = []
@@ -130,8 +151,8 @@ def get_usernames(self, host):
                 usernames.append(username)
         return usernames
 
-    def load(self, urlstring, username):
-        key = "qutebrowser/%s/%s" % (self._escape_url(urlstring), username)
+    def load(self, host, username):
+        key = "qutebrowser/%s/%s" % (shlex.quote(host), username)
         result = self._exec_pass([key])
         if result is None:
             raise KeyError
@@ -145,8 +166,8 @@ def load(self, urlstring, username):
 
         return password_data
 
-    def save(self, urlstring, password_data):
-        key = self._escape_url(urlstring)
+    def save(self, host, password_data):
+        key = shlex.quote(host)
         username = password_data["username"]["value"]
         content = password_data["password"]["value"]
         if "checkbox" in password_data:
@@ -168,15 +189,19 @@ def __init__(self):
         }
         self._password_manager = storages[password_storage]()
 
-    def _choose_username(self, urlstring):
+    def _choose_username(self, host):
         """Ask to user which username to use.
 
-        If there is only one username, use this username without asking.
+        If there is only one username for the specified host, use this
+        username without asking.
+
+        Args:
+            host: The URL host.
 
         Return:
             The username chosen by the user.
         """
-        usernames = self._password_manager.get_usernames(urlstring)
+        usernames = self._password_manager.get_usernames(host)
         answer = None
         if len(usernames) > 1:
             text = "Which username:"
@@ -275,8 +300,17 @@ def _get_host(self):
                                     window=self._win_id)
         return tabbed_browser.current_url().host()
 
-    def _load(self, url, username):
-        return self._password_manager.load(url, username)
+    def _load(self, host, username):
+        """Load the password data for a specific URL host and username.
+
+        Args:
+            host: The URL host.
+            username: The username.
+
+        Return:
+            The password data.
+        """
+        return self._password_manager.load(host, username)
 
     @cmdutils.register(instance="password-filler", win_id="win_id")
     def load_password(self, win_id):
@@ -307,11 +341,11 @@ def load_password_submit(self, win_id):
         form = self._find_form()
         self._submit(form)
 
-    def _password_exists(self, url, username):
+    def _password_exists(self, host, username):
         """Check if a password exists for the current URL and username.
 
         Args:
-            url: The URL to check.
+            host: The URL host to check.
             username: The username to check.
 
         Return:
@@ -319,21 +353,21 @@ def _password_exists(self, url, username):
         """
         password_exists = True
         try:
-            self._load(url, username)
+            self._load(host, username)
         except (KeyError, FileNotFoundError):
             password_exists = False
         return password_exists
 
-    def _save(self, url, password_data):
+    def _save(self, host, password_data):
         """Save the password data for the current URL.
 
         This uses the right password manager.
 
         Args:
-            url: The URL for the password data to save.
+            host: The URL host for the password data to save.
             password_data: The password data.
         """
-        self._password_manager.save(url, password_data)
+        self._password_manager.save(host, password_data)
 
     @cmdutils.register(instance="password-filler", win_id="win_id")
     def save_password(self, win_id):
@@ -381,6 +415,7 @@ def _set_value(self, element, value):
         element.evaluateJavaScript("this.value = '" + value + "'")
 
     def _submit(self, form):
+        """Submit the specified form."""
         # TODO: have a workaround for when submiting the form does not work.
         # e.g. click on the submit button.
         form.evaluateJavaScript("this.submit()")

From 5cd15a1c4a2e0f068267a433bfea9a3dba38b3ca Mon Sep 17 00:00:00 2001
From: Antoni Boucher <bouanto@zoho.com>
Date: Sun, 23 Aug 2015 19:06:40 -0400
Subject: [PATCH 07/13] Fixed many edge cases in password filler.

---
 qutebrowser/browser/passwordfiller.py | 133 +++++++++++++++++++++++++++++-----
 1 file changed, 114 insertions(+), 19 deletions(-)

diff --git a/qutebrowser/browser/passwordfiller.py b/qutebrowser/browser/passwordfiller.py
index 5a67446720..ba5540a001 100644
--- a/qutebrowser/browser/passwordfiller.py
+++ b/qutebrowser/browser/passwordfiller.py
@@ -17,14 +17,16 @@
 # You should have received a copy of the GNU General Public License
 # along with qutebrowser.  If not, see <http://www.gnu.org/licenses/>.
 
-"""Form filler."""
+"""Password filler."""
 
 import os
 import shlex
 import subprocess
+from urllib.parse import quote_plus, unquote_plus
 
 from yaml import dump, load
 
+from qutebrowser.browser import webelem
 from qutebrowser.commands import cmdexc, cmdutils
 from qutebrowser.config import config
 from qutebrowser.utils import message, objreg, standarddir, usertypes
@@ -147,11 +149,12 @@ def get_usernames(self, host):
         usernames = []
         for line in lines:
             if len(line) > 0:
-                username = line[4:]
+                username = unquote_plus(line[4:])
                 usernames.append(username)
         return usernames
 
     def load(self, host, username):
+        username = quote_plus(username)
         key = "qutebrowser/%s/%s" % (shlex.quote(host), username)
         result = self._exec_pass([key])
         if result is None:
@@ -169,6 +172,7 @@ def load(self, host, username):
     def save(self, host, password_data):
         key = shlex.quote(host)
         username = password_data["username"]["value"]
+        username = quote_plus(username)
         content = password_data["password"]["value"]
         if "checkbox" in password_data:
             content += "\n" + str(password_data["checkbox"]["value"])
@@ -226,20 +230,58 @@ def _choose_username(self, host):
 
         return usernames[answer]
 
+    def _find_best_form(self, login_forms):
+        """Return the best login form."""
+        # Return the form containing data if a form contains data.
+        for form in login_forms:
+            elem = form.findFirst('input[type="password"]')
+            value = elem.evaluateJavaScript("this.value")
+            if len(value) > 0:
+                return form
+
+        # Return the form containing "login" in its attributes.
+        for form in login_forms:
+            attributes = ["class", "id", "name"]
+            for attribute in attributes:
+                if "login" in form.attribute(attribute):
+                    return form
+
+        # Select the form with the fewest number of input fields because
+        # login forms usually contains fewer fields than register forms.
+        min_element_count = 10
+        best_form = None
+        for form in login_forms:
+            elems = form.findAll('input[type="email"], input[type="text"],'
+                                 'input[type="password"], input:not([type])')
+            if elems.count() < min_element_count:
+                min_element_count = elems.count()
+                best_form = form
+
+        return best_form
+
     def _find_form(self):
         """Find the login form element.
 
         Return:
             The login form element.
         """
-        frame = self._get_frame()
-        elem = frame.findFirstElement('input[type="password"]')
-        form = elem
-        # TODO: find a workaround for when there is no form element around the
-        # login form.
-        while not form.isNull() and form.tagName() != "FORM":
-            form = form.parent()
-        return form
+        frames = self._get_frames()
+        login_forms = []
+        for frame in frames:
+            # TODO: fix when there is more than one login form
+            elements = frame.findAllElements('input[type="password"]')
+            for elem in elements:
+                form = elem
+                while not form.isNull() and (form.tagName() != "FORM" and
+                                             form.tagName() != "BODY"):
+                    form = form.parent()
+                password_fields = form.findAll('input[type="password"]')
+                # Return forms with one password field.
+                if password_fields.count() == 1:
+                    login_forms.append(form)
+
+        if len(login_forms) > 0:
+            return self._find_best_form(login_forms)
 
     def _find_login_form_elements(self):
         """Find the login form.
@@ -250,13 +292,26 @@ def _find_login_form_elements(self):
         """
         form = self._find_form()
 
-        elems = form.findAll('input[type="checkbox"], input[type="text"],'
-                             'input[type="password"]')
+        if form is None:
+            raise RuntimeError
+
         username_element = None
         password_element = None
         checkbox_element = None
+
+        # Check for email first as text field may be a captcha.
+        email_fields = form.findAll('input[type="email"]')
+
+        if email_fields.count() > 0:
+            username_element = email_fields[0]
+
+        elems = form.findAll('input:not([type]), input[type="checkbox"],'
+                             'input[type="password"], input[type="text"]')
         for element in elems:
             elem_type = element.attribute("type")
+            if len(elem_type) == 0:
+                elem_type = "text"
+
             if elem_type == "checkbox":
                 checkbox_element = element
             elif elem_type == "password":
@@ -264,7 +319,7 @@ def _find_login_form_elements(self):
             elif elem_type == "text" and username_element is None:
                 username_element = element
 
-        if username_element is None and password_element is None:
+        if username_element is None or password_element is None:
             raise RuntimeError
 
         data = {
@@ -285,14 +340,14 @@ def _find_login_form_elements(self):
 
         return data
 
-    def _get_frame(self):
+    def _get_frames(self):
         """Get the current frame."""
         tabbed_browser = objreg.get("tabbed-browser", scope="window",
                                     window=self._win_id)
         widget = tabbed_browser.currentWidget()
         if widget is None:
             raise cmdexc.CommandError("No WebView available yet!")
-        return widget.page().mainFrame()
+        return webelem.get_child_frames(widget.page().mainFrame())
 
     def _get_host(self):
         """Get the current URL host."""
@@ -326,7 +381,11 @@ def load_password(self, win_id):
             raise cmdexc.CommandError("No password data for the current URL!")
 
         password = password_data["password"]
-        form_elements = self._find_login_form_elements()
+        try:
+            form_elements = self._find_login_form_elements()
+        except RuntimeError:
+            raise cmdexc.CommandError(
+                "No login form found in the current page!")
         self._set_value(form_elements["username"]["element"], username)
         self._set_value(form_elements["password"]["element"], password)
 
@@ -367,6 +426,12 @@ def _save(self, host, password_data):
             host: The URL host for the password data to save.
             password_data: The password data.
         """
+        if len(password_data["username"]["value"]) == 0:
+            raise cmdexc.CommandError(
+                "Enter your username to be able to save your credentials.")
+        if len(password_data["password"]["value"]) == 0:
+            raise cmdexc.CommandError(
+                "Enter your password to be able to save your credentials.")
         self._password_manager.save(host, password_data)
 
     @cmdutils.register(instance="password-filler", win_id="win_id")
@@ -416,6 +481,36 @@ def _set_value(self, element, value):
 
     def _submit(self, form):
         """Submit the specified form."""
-        # TODO: have a workaround for when submiting the form does not work.
-        # e.g. click on the submit button.
-        form.evaluateJavaScript("this.submit()")
+        # Fix for angularjs.
+        elems = form.findAll('input[type="text"], input[type="email"],'
+                             'input[type="password"], input:not([type])')
+
+        js = ("var event = document.createEvent('HTMLEvents');"
+              "event.initEvent('change', false, true);"
+              "this.dispatchEvent(event);")
+        for elem in elems:
+            elem.evaluateJavaScript(js)
+
+        # Fix for forms needing two submission (like Gmail).
+        password_field = form.findFirst('input[type="password"]')
+        password_was_visible = password_field.evaluateJavaScript(
+            "this.offsetWidth > 0 && element.offsetHeight > 0")
+
+        if not password_was_visible:
+            # TODO: find a way of doing this in Python.
+            password_field.evaluateJavaScript("""setTimeout(function() {
+                    var submit_button = document.querySelector('input[type="submit"], button[type="submit"]');
+                    submit_button.click();
+                }, 500);
+                """)
+
+        submit_button = form.findFirst(
+            'input[type="submit"], button[type="submit"]'
+        )
+
+        # Submit with the submit button if it exists.
+        # Otherwise, send the submit event to the form.
+        if submit_button.isNull():
+            form.evaluateJavaScript("this.submit()")
+        else:
+            submit_button.evaluateJavaScript("this.click()")

From c52ae48975c983709883ef01f0f44b367e492050 Mon Sep 17 00:00:00 2001
From: Antoni Boucher <bouanto@zoho.com>
Date: Sun, 23 Aug 2015 20:18:21 -0400
Subject: [PATCH 08/13] Fixed to work when there is more than one unique login
 form.

---
 qutebrowser/browser/passwordfiller.py | 136 ++++++++++++++++++++++++++++------
 1 file changed, 113 insertions(+), 23 deletions(-)

diff --git a/qutebrowser/browser/passwordfiller.py b/qutebrowser/browser/passwordfiller.py
index ba5540a001..f00fea52ed 100644
--- a/qutebrowser/browser/passwordfiller.py
+++ b/qutebrowser/browser/passwordfiller.py
@@ -107,6 +107,13 @@ def save(self, host, password_data):
             data[host][username] = {
                 "password": password_data["password"]["value"],
             }
+
+            # If there is more than one unique form, save the action of the
+            # form that was used when saving the password data.
+            if "form_action" in password_data:
+                data[host][username]["form_action"] = (
+                    password_data["form_action"])
+
             if "checkbox" in password_data:
                 data[host][username]["checkbox"] = (
                     password_data["checkbox"]["value"]
@@ -165,7 +172,13 @@ def load(self, host, username):
         }
 
         if len(lines) > 1:
-            password_data["checkbox"] = lines[1] == "True"
+            if lines[1] == "True" or lines[1] == "False":
+                password_data["checkbox"] = lines[1] == "True"
+            else:
+                password_data["form_action"] = lines[1]
+
+        if len(lines) > 2:
+            password_data["form_action"] = lines[2]
 
         return password_data
 
@@ -176,6 +189,12 @@ def save(self, host, password_data):
         content = password_data["password"]["value"]
         if "checkbox" in password_data:
             content += "\n" + str(password_data["checkbox"]["value"])
+
+        # If there is more than one unique form, save the action of the
+        # form that was used when saving the password data.
+        if "form_action" in password_data:
+            content += "\n" + password_data["form_action"]
+
         self._exec_pass(["insert", "-m", "qutebrowser/%s/%s" %
                         (key, username)], content)
 
@@ -251,24 +270,41 @@ def _find_best_form(self, login_forms):
         min_element_count = 10
         best_form = None
         for form in login_forms:
-            elems = form.findAll('input[type="email"], input[type="text"],'
-                                 'input[type="password"], input:not([type])')
+            elems = self._find_login_form_input(form)
             if elems.count() < min_element_count:
                 min_element_count = elems.count()
                 best_form = form
 
         return best_form
 
-    def _find_form(self):
+    def _find_form(self, form_action):
         """Find the login form element.
 
         Return:
             The login form element.
         """
+        form = None
+        if form_action is not None:
+            form = self._find_form_by_action(form_action)
+        if form is None:
+            login_forms = self._find_login_forms()
+            if len(login_forms) > 0:
+                form = self._find_best_form(login_forms)
+        return form
+
+    def _find_form_by_action(self, form_action):
+        """Find a form by its action."""
+        frames = self._get_frames()
+        for frame in frames:
+            form = frame.findFirstElement('form[action="%s"]' % form_action)
+            if not form.isNull():
+                return form
+
+    def _find_login_forms(self):
+        """Find all the login forms."""
         frames = self._get_frames()
         login_forms = []
         for frame in frames:
-            # TODO: fix when there is more than one login form
             elements = frame.findAllElements('input[type="password"]')
             for elem in elements:
                 form = elem
@@ -279,18 +315,16 @@ def _find_form(self):
                 # Return forms with one password field.
                 if password_fields.count() == 1:
                     login_forms.append(form)
+        return login_forms
 
-        if len(login_forms) > 0:
-            return self._find_best_form(login_forms)
-
-    def _find_login_form_elements(self):
+    def _find_login_form_elements(self, form_action=None):
         """Find the login form.
 
         Return:
             A dict containing the username, password and checkbox (if present
             in the form) elements and values.
         """
-        form = self._find_form()
+        form = self._find_form(form_action)
 
         if form is None:
             raise RuntimeError
@@ -323,14 +357,15 @@ def _find_login_form_elements(self):
             raise RuntimeError
 
         data = {
-            'username': {
-                'element': username_element,
-                'value': username_element.evaluateJavaScript("this.value"),
+            "username": {
+                "element": username_element,
+                "value": username_element.evaluateJavaScript("this.value"),
             },
-            'password': {
-                'element': password_element,
-                'value': password_element.evaluateJavaScript("this.value"),
+            "password": {
+                "element": password_element,
+                "value": password_element.evaluateJavaScript("this.value"),
             },
+            "form_action": form.attribute("action"),
         }
         if checkbox_element is not None:
             data["checkbox"] = {
@@ -340,6 +375,32 @@ def _find_login_form_elements(self):
 
         return data
 
+    def _find_login_form_input(self, form):
+        """Find the input field of a form."""
+        return form.findAll('input[type="email"], input[type="text"],'
+                            'input[type="password"], input:not([type])')
+
+    def _find_unique_forms_count(self):
+        """Get the count of unique forms."""
+        login_forms = self._find_login_forms()
+        unique_forms = set()
+
+        min_element_count = 10
+
+        for form in login_forms:
+            elem_count = self._find_login_form_input(form).count()
+
+            if elem_count < min_element_count:
+                min_element_count = elem_count
+
+        for form in login_forms:
+            elem_count = self._find_login_form_input(form).count()
+
+            if elem_count == min_element_count:
+                unique_forms.add(self._hash_form(form))
+
+        return len(unique_forms)
+
     def _get_frames(self):
         """Get the current frame."""
         tabbed_browser = objreg.get("tabbed-browser", scope="window",
@@ -355,6 +416,17 @@ def _get_host(self):
                                     window=self._win_id)
         return tabbed_browser.current_url().host()
 
+    def _hash_form(self, form):
+        """Compute the hash of a form.
+
+        It consist of the number of fields and their name.
+        """
+        elems = self._find_login_form_input(form)
+        form_hash = str(elems.count())
+        for elem in elems:
+            form_hash += "_" + elem.attribute("name")
+        return form_hash
+
     def _load(self, host, username):
         """Load the password data for a specific URL host and username.
 
@@ -369,20 +441,29 @@ def _load(self, host, username):
 
     @cmdutils.register(instance="password-filler", win_id="win_id")
     def load_password(self, win_id):
-        """Load the password data from the current URL."""
+        """Load the password data from the current URL.
+
+        Return:
+            The form action of the form if it was saved.
+        """
         self._win_id = win_id
 
         host = self._get_host()
 
+        form_action = None
         try:
             username = self._choose_username(host)
             password_data = self._load(host, username)
+
+            unique_forms_count = self._find_unique_forms_count()
+            if unique_forms_count > 1:
+                form_action = password_data["form_action"]
         except (KeyError, FileNotFoundError):
             raise cmdexc.CommandError("No password data for the current URL!")
 
         password = password_data["password"]
         try:
-            form_elements = self._find_login_form_elements()
+            form_elements = self._find_login_form_elements(form_action)
         except RuntimeError:
             raise cmdexc.CommandError(
                 "No login form found in the current page!")
@@ -393,11 +474,13 @@ def load_password(self, win_id):
             self._set_checkbox_value(form_elements["checkbox"]["element"],
                                      password_data["checkbox"])
 
+        return form_action
+
     @cmdutils.register(instance="password-filler", win_id="win_id")
     def load_password_submit(self, win_id):
         """Load the password data for the current URL and submit the form."""
-        self.load_password(win_id)
-        form = self._find_form()
+        form_action = self.load_password(win_id)
+        form = self._find_form(form_action)
         self._submit(form)
 
     def _password_exists(self, host, username):
@@ -446,6 +529,10 @@ def save_password(self, win_id):
             raise cmdexc.CommandError(
                 "No login form found in the current page!")
         else:
+            unique_forms_count = self._find_unique_forms_count()
+            if unique_forms_count < 2:
+                del password_data["form_action"]
+
             username = password_data["username"]["value"]
 
             save = False
@@ -482,8 +569,7 @@ def _set_value(self, element, value):
     def _submit(self, form):
         """Submit the specified form."""
         # Fix for angularjs.
-        elems = form.findAll('input[type="text"], input[type="email"],'
-                             'input[type="password"], input:not([type])')
+        elems = self._find_login_form_input(form)
 
         js = ("var event = document.createEvent('HTMLEvents');"
               "event.initEvent('change', false, true);"
@@ -513,4 +599,8 @@ def _submit(self, form):
         if submit_button.isNull():
             form.evaluateJavaScript("this.submit()")
         else:
-            submit_button.evaluateJavaScript("this.click()")
+            submit_button.evaluateJavaScript("""var self = this;
+                setTimeout(function() {
+                    self.click();
+                }, 500);
+                """)

From 30bb369f663aaa323a2e8950cb958e6065d8590f Mon Sep 17 00:00:00 2001
From: Antoni Boucher <bouanto@zoho.com>
Date: Wed, 16 Sep 2015 19:44:10 -0400
Subject: [PATCH 09/13] Fixed to work with forms containing only a password
 field.

---
 qutebrowser/browser/passwordfiller.py | 66 ++++++++++++++++++++++++++---------
 1 file changed, 49 insertions(+), 17 deletions(-)

diff --git a/qutebrowser/browser/passwordfiller.py b/qutebrowser/browser/passwordfiller.py
index f00fea52ed..e82801b05c 100644
--- a/qutebrowser/browser/passwordfiller.py
+++ b/qutebrowser/browser/passwordfiller.py
@@ -19,6 +19,8 @@
 
 """Password filler."""
 
+# TODO: show a message when the password is saved.
+
 import os
 import shlex
 import subprocess
@@ -31,6 +33,8 @@
 from qutebrowser.config import config
 from qutebrowser.utils import message, objreg, standarddir, usertypes
 
+DEFAULT_USERNAME = "__qutebrowser_no_username__"
+
 
 class PasswordManager:
 
@@ -83,7 +87,9 @@ def get_usernames(self, host):
             if data is None:
                 data = {}
         password_data = data[host]
-        return list(password_data.keys())
+        usernames = list(password_data.keys())
+        usernames.remove(DEFAULT_USERNAME)
+        return usernames
 
     def load(self, host, username):
         with open(self._filename, "r", encoding="utf-8") as password_file:
@@ -103,7 +109,11 @@ def save(self, host, password_data):
                 data = {}
             if host not in data:
                 data[host] = {}
-            username = password_data["username"]["value"]
+            if "username" in password_data:
+                username = password_data["username"]["value"]
+            else:
+                username = DEFAULT_USERNAME
+
             data[host][username] = {
                 "password": password_data["password"]["value"],
             }
@@ -157,7 +167,8 @@ def get_usernames(self, host):
         for line in lines:
             if len(line) > 0:
                 username = unquote_plus(line[4:])
-                usernames.append(username)
+                if username != DEFAULT_USERNAME:
+                    usernames.append(username)
         return usernames
 
     def load(self, host, username):
@@ -184,8 +195,13 @@ def load(self, host, username):
 
     def save(self, host, password_data):
         key = shlex.quote(host)
-        username = password_data["username"]["value"]
-        username = quote_plus(username)
+
+        if "username" in password_data:
+            username = password_data["username"]["value"]
+            username = quote_plus(username)
+        else:
+            username = DEFAULT_USERNAME
+
         content = password_data["password"]["value"]
         if "checkbox" in password_data:
             content += "\n" + str(password_data["checkbox"]["value"])
@@ -353,20 +369,24 @@ def _find_login_form_elements(self, form_action=None):
             elif elem_type == "text" and username_element is None:
                 username_element = element
 
-        if username_element is None or password_element is None:
+        if password_element is None:
+            # TODO: check if this can happen.
             raise RuntimeError
 
         data = {
-            "username": {
-                "element": username_element,
-                "value": username_element.evaluateJavaScript("this.value"),
-            },
             "password": {
                 "element": password_element,
                 "value": password_element.evaluateJavaScript("this.value"),
             },
             "form_action": form.attribute("action"),
         }
+
+        if username_element is not None:
+            data["username"] = {
+                "element": username_element,
+                "value": username_element.evaluateJavaScript("this.value"),
+            }
+
         if checkbox_element is not None:
             data["checkbox"] = {
                 "element": checkbox_element,
@@ -452,12 +472,20 @@ def load_password(self, win_id):
 
         form_action = None
         try:
-            username = self._choose_username(host)
-            password_data = self._load(host, username)
-
             unique_forms_count = self._find_unique_forms_count()
-            if unique_forms_count > 1:
+
+            username = None
+            if unique_forms_count == 1:
+                login_form = self._find_login_forms()[0]
+                elem_count = self._find_login_form_input(login_form).count()
+                if elem_count == 1:
+                    username = DEFAULT_USERNAME
+            elif unique_forms_count > 1:
                 form_action = password_data["form_action"]
+
+            if username is None:
+                username = self._choose_username(host)
+            password_data = self._load(host, username)
         except (KeyError, FileNotFoundError):
             raise cmdexc.CommandError("No password data for the current URL!")
 
@@ -467,7 +495,8 @@ def load_password(self, win_id):
         except RuntimeError:
             raise cmdexc.CommandError(
                 "No login form found in the current page!")
-        self._set_value(form_elements["username"]["element"], username)
+        if "username" in form_elements:
+            self._set_value(form_elements["username"]["element"], username)
         self._set_value(form_elements["password"]["element"], password)
 
         if "checkbox" in form_elements and "checkbox" in password_data:
@@ -509,7 +538,7 @@ def _save(self, host, password_data):
             host: The URL host for the password data to save.
             password_data: The password data.
         """
-        if len(password_data["username"]["value"]) == 0:
+        if "username" in password_data and len(password_data["username"]["value"]) == 0:
             raise cmdexc.CommandError(
                 "Enter your username to be able to save your credentials.")
         if len(password_data["password"]["value"]) == 0:
@@ -533,7 +562,10 @@ def save_password(self, win_id):
             if unique_forms_count < 2:
                 del password_data["form_action"]
 
-            username = password_data["username"]["value"]
+            if "username" in password_data:
+                username = password_data["username"]["value"]
+            else:
+                username = DEFAULT_USERNAME
 
             save = False
             if self._password_exists(host, username):

From 056298c4e16520a376c75d9489780c778eacba8e Mon Sep 17 00:00:00 2001
From: Antoni Boucher <bouanto@zoho.com>
Date: Thu, 17 Sep 2015 16:14:52 -0400
Subject: [PATCH 10/13] Added todo.

---
 qutebrowser/browser/passwordfiller.py | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/qutebrowser/browser/passwordfiller.py b/qutebrowser/browser/passwordfiller.py
index e82801b05c..20658d091a 100644
--- a/qutebrowser/browser/passwordfiller.py
+++ b/qutebrowser/browser/passwordfiller.py
@@ -20,6 +20,8 @@
 """Password filler."""
 
 # TODO: show a message when the password is saved.
+# FIXME: bug in zeste de savoir (auto-submit not working).
+# FIXME: bug in inscription.ssap.usherbrooke.ca (probably because there is an hidden text field).
 
 import os
 import shlex

From 4b8ce1116e9c3ce0cc16e2d80bdcb5306b9b42cd Mon Sep 17 00:00:00 2001
From: Antoni Boucher <bouanto@zoho.com>
Date: Tue, 27 Oct 2015 16:55:43 -0400
Subject: [PATCH 11/13] Fixed password filler.

---
 qutebrowser/browser/passwordfiller.py | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)

diff --git a/qutebrowser/browser/passwordfiller.py b/qutebrowser/browser/passwordfiller.py
index 20658d091a..069bf4d6b3 100644
--- a/qutebrowser/browser/passwordfiller.py
+++ b/qutebrowser/browser/passwordfiller.py
@@ -326,8 +326,8 @@ def _find_login_forms(self):
             elements = frame.findAllElements('input[type="password"]')
             for elem in elements:
                 form = elem
-                while not form.isNull() and (form.tagName() != "FORM" and
-                                             form.tagName() != "BODY"):
+                while not form.isNull() and (form.tagName().lower() != "form" and
+                                             form.tagName().lower() != "body"):
                     form = form.parent()
                 password_fields = form.findAll('input[type="password"]')
                 # Return forms with one password field.
@@ -482,12 +482,13 @@ def load_password(self, win_id):
                 elem_count = self._find_login_form_input(login_form).count()
                 if elem_count == 1:
                     username = DEFAULT_USERNAME
-            elif unique_forms_count > 1:
-                form_action = password_data["form_action"]
 
             if username is None:
                 username = self._choose_username(host)
             password_data = self._load(host, username)
+
+            if unique_forms_count > 1:
+                form_action = password_data["form_action"]
         except (KeyError, FileNotFoundError):
             raise cmdexc.CommandError("No password data for the current URL!")
 

From 3ae263770d6188d6d981e29cda6a84403bf06cfb Mon Sep 17 00:00:00 2001
From: Antoni Boucher <bouanto@zoho.com>
Date: Sat, 31 Oct 2015 12:14:47 -0400
Subject: [PATCH 12/13] Added a command to load the password in the focused
 password field.

---
 qutebrowser/browser/passwordfiller.py | 70 +++++++++++++++++++++++------------
 1 file changed, 46 insertions(+), 24 deletions(-)

diff --git a/qutebrowser/browser/passwordfiller.py b/qutebrowser/browser/passwordfiller.py
index 069bf4d6b3..4d6c3c9aab 100644
--- a/qutebrowser/browser/passwordfiller.py
+++ b/qutebrowser/browser/passwordfiller.py
@@ -20,8 +20,6 @@
 """Password filler."""
 
 # TODO: show a message when the password is saved.
-# FIXME: bug in zeste de savoir (auto-submit not working).
-# FIXME: bug in inscription.ssap.usherbrooke.ca (probably because there is an hidden text field).
 
 import os
 import shlex
@@ -438,6 +436,31 @@ def _get_host(self):
                                     window=self._win_id)
         return tabbed_browser.current_url().host()
 
+    def _get_password_data(self, host):
+        """Get the form action, username and password for the current URL.
+
+        It asks the username if there is more than one."""
+        form_action = None
+        try:
+            unique_forms_count = self._find_unique_forms_count()
+
+            username = None
+            if unique_forms_count == 1:
+                login_form = self._find_login_forms()[0]
+                elem_count = self._find_login_form_input(login_form).count()
+                if elem_count == 1:
+                    username = DEFAULT_USERNAME
+
+            if username is None:
+                username = self._choose_username(host)
+            password_data = self._load(host, username)
+
+            if unique_forms_count > 1:
+                form_action = password_data["form_action"]
+        except (KeyError, FileNotFoundError):
+            raise cmdexc.CommandError("No password data for the current URL!")
+        return (form_action, username, password_data)
+
     def _hash_form(self, form):
         """Compute the hash of a form.
 
@@ -462,6 +485,26 @@ def _load(self, host, username):
         return self._password_manager.load(host, username)
 
     @cmdutils.register(instance="password-filler", win_id="win_id")
+    def load_password_in_current_field(self, win_id):
+        """Load the password in the focused field."""
+        self._win_id = win_id
+        host = self._get_host()
+        (form_action, username, password_data) = self._get_password_data(host)
+        password = password_data["password"]
+
+        tabbed_browser = objreg.get("tabbed-browser", scope="window",
+                                    window=self._win_id)
+        widget = tabbed_browser.currentWidget()
+        frame = widget.page().mainFrame()
+        element = frame.findFirstElement(":focus")
+        if (not element.isNull() and element.tagName().lower() == "input" and
+                element.attribute("type") == "password"):
+            self._set_value(element, password)
+        else:
+            raise cmdexc.CommandError(
+                "Please select a password input field!")
+
+    @cmdutils.register(instance="password-filler", win_id="win_id")
     def load_password(self, win_id):
         """Load the password data from the current URL.
 
@@ -469,29 +512,8 @@ def load_password(self, win_id):
             The form action of the form if it was saved.
         """
         self._win_id = win_id
-
         host = self._get_host()
-
-        form_action = None
-        try:
-            unique_forms_count = self._find_unique_forms_count()
-
-            username = None
-            if unique_forms_count == 1:
-                login_form = self._find_login_forms()[0]
-                elem_count = self._find_login_form_input(login_form).count()
-                if elem_count == 1:
-                    username = DEFAULT_USERNAME
-
-            if username is None:
-                username = self._choose_username(host)
-            password_data = self._load(host, username)
-
-            if unique_forms_count > 1:
-                form_action = password_data["form_action"]
-        except (KeyError, FileNotFoundError):
-            raise cmdexc.CommandError("No password data for the current URL!")
-
+        (form_action, username, password_data) = self._get_password_data(host)
         password = password_data["password"]
         try:
             form_elements = self._find_login_form_elements(form_action)

From c10b289f197d8f8e74bb31ac9b2ee5e1ff441614 Mon Sep 17 00:00:00 2001
From: Antoni Boucher <bouanto@zoho.com>
Date: Sat, 31 Oct 2015 12:17:58 -0400
Subject: [PATCH 13/13] Added a message after saving the password.

---
 qutebrowser/browser/passwordfiller.py | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/qutebrowser/browser/passwordfiller.py b/qutebrowser/browser/passwordfiller.py
index 4d6c3c9aab..c9b648bec0 100644
--- a/qutebrowser/browser/passwordfiller.py
+++ b/qutebrowser/browser/passwordfiller.py
@@ -19,8 +19,6 @@
 
 """Password filler."""
 
-# TODO: show a message when the password is saved.
-
 import os
 import shlex
 import subprocess
@@ -604,6 +602,7 @@ def save_password(self, win_id):
 
             if save:
                 self._save(host, password_data)
+                message.info(self._win_id, "Password saved!")
 
     def _set_checkbox_value(self, element, value):
         """Set the checkbox element value.
@The-Compiler

This comment has been minimized.

Collaborator

The-Compiler commented Feb 4, 2018

FWIW, from #30 by @biinari, for people using LastPass:

I've made a backend to password_fill for lpass: https://gist.github.com/biinari/e5611220f90258552cd85032c9c09021 - it could be improved upon but works reasonably well for starters.

@The-Compiler

This comment has been minimized.

Collaborator

The-Compiler commented Oct 5, 2018

#4071 has some changes to the lastpass-fill userscript with a different way of filling form fields. Also, according to #4071 (comment) it's also possible to get the form field names from lastpass.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment