Rework our Mac OS X integration

[BjarniRunar]

The current Mac OS X package is not really usable. Known issues:

1. The package is unsigned (see #1366)
2. If multiple users launch Mailpile (using user switching), ports conflict (#1139, #877)
3. The current approach of launching a terminal and then spawning a browser confuses users

Item 3 above is the biggie; we are violating the principle of least surprise in a big way. As I understand it, Mac OS X users expect the following:

1. The app launches and can be started/stopped using an icon in the dock at the bottom.
2. When clicked, the dock icon should launch a window with UI elements (there may be more than one)

So that's what we should do. The dock icon can represent the fact that Mailpile is running, clicking it could launch a browser tab (1st iteration?) or a webkit wrapper (2nd iteration?). In addition there should be an option to open a "debug window", which would be the terminal CLI interface. This should not launch by default.

I think implementation-wise, this means we need a custom app which in turn launches Mailpile in the background and coordinates things. This app can also take care of choosing which port to run on (issue #887). If I recall, the screen utility is included on the Mac, so this launcher app could use the same strategy as mailpile-admin.py, of launching Mailpile itself inside a named screen session that the user can then attach to for debug purposes (and provide a context-menu item for doing exactly that).

[NoahAndrews]

I see a few different ways we could take this. One is to make it seem as much like a normal app as possible. When the user launches the app, a webview comes up containing Mailpile. Opening a browser tab when the app is launched would feel pretty jarring. When the app is quit, the server quits with it. Obviously, this isn't good if a user wants to host Mailpile for other machines.

Option two is that we write a native Mac app that has options for stopping and stopping the server, changing the port, opening the debug window, and opening Mailpile in a browser tab. The server would continue in the background even after the app was quit, with an icon in the status bar for easy access to the control panel app. This is probably the most complicated option.

Finally, we could operate like (for instance) the Dropbox app. On first launch, we have some kind of interface to explain what's going on, but after that, the app lives entirely in the status bar. When the user launches the app, the only thing that visibly happens is that an icon is added to the status bar, and they know that they can now go to Mailpile from their browser, or click an option in the status bar menu to open it for them.

Issue number 2 could be a bit tricky for options 2 or 3, since both of those involve accessing Mailpile directly through the normal browser. If we can't depend on having a consistent port number, the user can't bookmark it, and we shouldn't encourage them to access it through the browser. Choosing a random port upon setup would work.

Really this comes down to whether we want the app to act as an app or as a server. How do we get around issue 2 on the Windows side?

P. S. Did you reference the right issue about user switching? Ah, found the right issue. Option 2 from over there ought to work pretty nicely. I'm honestly not sure why there needs to be anything Mac-specific about solving that.

[BjarniRunar]

I referenced the wrong issue wrt ports - fixed!

I think the first option - the webview - is the one to aim for, although I think that we could get away with leaving Mailpile running in the background as long as the dock icon is "lit up" to show it's still running. I think a bunch of apps behave more or less that way already - you close the last window but the app is still "running". Am I wrong?

[BjarniRunar]

Regarding the dropbox-solution, if we have to explain things, then we're failing our job here. We want to match user expectations, not force them to learn that Mailpile is special and weird. Most users wouldn't read an explanation anyway, so we'd have to invest a fair bit of effort into design and visual communication to make sure that actually works. It's harder than it looks!

[NoahAndrews]

You're not wrong about being apps continuing to run after the last window is closed, but that doesn't mean it's a good way to get the functionality we want. For one, I don't like the idea of a background service taking up precious Dock real estate. But the bigger issue I see is that it's too easy to quit. As a user, when I'm done using an app, I quit it. I don't think to leave it open so that it's accessible from other computers. This would be an example of having to explain things.

I think somewhere we're going to need to put an option to run as a background service. If that option is checked, we also put an icon in the status bar, which will remain after the webview-app is quit.

[NoahAndrews]

You've got a great point about matching expectations. It should work just like any other email client, so option 1 it is.

[NoahAndrews]

I'm thinking we may need to use something like Electron or nw.js. I do have some node.js experience. My research indicates that Electron is more actively developed, starts much faster, and results in a somewhat smaller app. Unfortunately, Electron requires OS X 10.8, and nw.js requires OS X 10.7.

One interesting thing about this route is that these are cross-platform solutions. If we like the end result on Mac, we could easily bring the same packaging to the other platforms.

I just found a couple of closed Electron issues that explain how we should be able to get the functionality I talked about before with that combination of status bar (tray) icon and dock launcher.

atom/electron#1154
atom/electron#422

As long as we can find a good way to communicate between the Python code and the node.js code, I think this will be the best way if we want to integrate with the operating system as much as possible.

Any thoughts?

[NoahAndrews]

I've officially got a start on Electron-enhanced Mailpile! http://imgur.com/qYMMyXs

It's not too late to turn back if you have an objection of course.

The next thing I need to know is how to add a command line flag to the mp command without making it available as an option in the interactive interface. Specifically, I need a way to start the server without opening a new browser tab. It seemed like --www was what I wanted, but it turns out that that just changes the socket the server will listen on. I'll let you decide if that deserves an issue or not.

[NoahAndrews]

Perhaps we could also have a download option for old versions of OS X. That package would be generated using platypus, and would use a standard webview (Electron uses Chromium). It would lack the option to run in the background and the host OS integration that Electron provides us with.

[BjarniRunar]

Electron looks amazing, but incredibly bloated for our use case. Instead of shipping a 200k native Mac wrapper, we're going to ship 100 megs, including node.js and Chromium - not counting Mailpile itself which may include gnupg, tor and possibly Python? Yikes. This adds a huge support burden to us upstream, as we'll have to repackage and update every time a security vuln is found and fixed in Chromium, Electron or Node, instead of relying on Apple's OS updates for the OS-provided embeddable webview (Safari).

I'm not sure this is a reasonable thing to do, except perhaps only as a prototype to work out exactly what the "real" thing needs to look like.

[BjarniRunar]

A bunch of discussions happened on IRC today, some of the main take-aways:

We agreed that Electron was maybe too heavy.

Embedding a webview vs. using the browser was debated, our 1st attempt will be to use the browser but give the user control over things; the wrapper app will pop up a window offering the user buttons to view in browser or view the CLI, and a tickbox to make launching in the browser automatic from then on - this should hopefully remove the element of surprise (few apps launch the browser) without adding any inconvenience once the user has ticked the box.

The app will have a dock icon like other apps, closing the launcher/wrapper/intro window won't terminate the app, instead the user must press an explicit Quit button.

The wrapper will launch Mailpile in a screen session, much like the Apache integration script does.

@NoahAndrews made a mock-up, I am pretty sure he has started working on the wrapper app!

[pix2D]

Please consider adding an option to hide the dock icon and have it live in the status bar instead. Dock space is always at a premium on smaller resolutions and since this is effectively something most people would leave on permanently (I think?) it would be nice to have the ability to not have it in the dock.

[BjarniRunar]

@pix2D Thanks for the suggestion. As an option, that would make sense. It can't be the default mode of operation because of user confusion, but having it as an option shouldn't hurt (as long as there's a resonable place to communicate the option itself, which isn't quite a given). I don't consider this high priority, but it would be nice to have.

[BjarniRunar]

Note: work on this issue should happen in the gui-o-matic project, so the results can be re-used by other projects: https://github.com/mailpile/gui-o-matic

[marksev1]

Why is this issue Mac OS specific. Isn't this project crossplatform?

[caesar]

If MailPile is to be familiar to users of existing mail clients (and of existing software in general), it should run as a standalone app. It shouldn't have to be loaded in the browser. Certainly it shouldn't leave a standalone app running and have to use the browser for its UI.
I appreciate that you might want to use the browser in the short term before putting work into developing a standalone UI wrapper. But your long term goal should be for the UI to run in its own process if you want users to be familiar and comfortable with it.

Plus, there is a huge security benefit to avoiding the browser. Personally, I would not trust my PGP keys to a script running in the browser, and I know a lot of security experts have made the same recommendation.

As a side note, while I understand and agree with your concerns about Electron being bloated, it's important to note that it exists. Maybe it would be a good idea to use it for now, to focus your efforts on the unique code in MailPile rather than on creating a new wrapper framework? This can be done at a later date if/when developers have free time for it.

Incidentally, Nylas N1 (based on Electron) is about the same size as Thunderbird. Doesn't mean we can't do better (and we should!), but just a note of comparison that existing mail clients are already pretty big.

[DaLynX]

From my understanding, the keys are never used by a script in the
browser (that would be javascript, I guess?), but directly by
python calls to GPG. No?

Caesar Schinas notifications@github.com wrote:

If MailPile is to be familiar to users of existing mail clients
(and of existing software in general), it should run as a
standalone app. It shouldn't have to be loaded in the browser.
Certainly it shouldn't leave a standalone app running and
have to use the browser for its UI. I appreciate that you might
want to use the browser in the short term before putting work
into developing a standalone UI wrapper. But your long term
goal should be for the UI to run in its own process if you want
users to be familiar and comfortable with it.

Plus, there is a huge security benefit to avoiding the
browser. Personally, I would not trust my PGP keys to a script
running in the browser, and I know a lot of security experts
have made the same recommendation.

As a side note, while I understand and agree with your concerns
about Electron being bloated, it's important to note that it
exists. Maybe it would be a good idea to use it for now, to
focus your efforts on the unique code in MailPile rather than
on creating a new wrapper framework? This can be done at a
later date if/when developers have free time for it.

Incidentally, Nylas N1 (based on Electron) is about the same
size as Thunderbird. Doesn't mean we can't do better (and we
should!), but just a note of comparison that existing mail
clients are already pretty big.

---

You are receiving this because you are subscribed to this
thread. Reply to this email directly or view it on GitHub:
#1514 (comment)