Handle dropped websockets more gracefully for mobile clients

[blitzcode]

iOS Safari seems to drop the websocket connection to the threepenny server rather aggressively. Switching away from the browser, or just going to another tab or even the screen going to sleep can be enough for a disconnect to happen. It would be great if this could be dealt with somewhat more gracefully. Right now, the user is presented with an app that's just 'dead' for seemingly arbitrary reasons. It would be great if we could somehow seamlessly reconnect, or at least have the client side code grey out the page, refresh automatically etc.

[blitzcode]

I've been experimenting a bit. If I modify the onclose handler for the websocket like this

ws.onclose = function (e) {
    Haskell.log("WebSocket closed: %o", e);
    window.location.reload(true);
};

I get something reasonable. Reloading the page connects again when the page would otherwise be stale after the browsers back/forward buttons are used, or after a sleeping phone wakes up. I'm not an experienced web developer and this probably isn't the right solution, but something in the spirit of this would greatly help.

[blitzcode]

If anybody else has this issue, a hack like this sort of fixes it for iOS devices:

if (navigator.userAgent.match(/(iPhone|iPod|iPad)/i))
{
    window.lastAliveTick = (new Date()).getTime();
    function myTimer() {
        var curAliveTick = (new Date()).getTime();
        if (curAliveTick - window.lastAliveTick > 5000)
        {
            //window.alert('stale');
            window.location.reload(true);
        }
        window.lastAliveTick = curAliveTick;
    }
    window.addEventListener('pagehide', function () { window.lastAliveTick = 0; });
    window.addEventListener('unload', function () { window.lastAliveTick = 0; });
    setInterval(myTimer, 1000);
}

I think something like this might unfortunately be the best workaround till an actual fix happens inside the library.

[blitzcode]

Is there any progress / ETA on this? I'd love to unburden my users from being tripped up by this bug, and there's no good workaround I can find. My PR would provide relief from this issue, maybe it's at least acceptable till you find whatever you think a permanent solution would be?

[HeinrichApfelmus]

I intend to tackle the batching issue first, but if you like, I can implement a configurable solution for this first.

[blitzcode]

I guess this one is easier to fix! I'd wait for a stable hackage release in any case.

[blitzcode]

It's been ~6 months, is threepenny basically abandoned / in basic maintenance mode at this point? :-( Issues like this and #131 / #133 are very severe bugs, have no real workaround and cause user visible problems.

[HeinrichApfelmus]

Sorry for taking so long. Threepenny-GUI is not abandoned, but I'm developing this in my free time, which tends to come and go in long stretches; my projects usually go into hibernation during that time. I found a workaround for #131 that should work for you, I will post it there asap.

[blitzcode]

Sure, no hard feeling either way ;-)

I haven't paid much attention to threepenny in quite a while, but I'll release a new version of my threepenny based software before the end of the year and wanted to check in so I can put in any improvements that might been available.

[HeinrichApfelmus]

I'm taking another look at this issue.

As far as I understand, your workaround is to reload the page whenever the browser drops the websocket connection. The main problem with this is that the browser window may contain some "transient state", say the content of a textarea that the user has edited and that has not yet been transferred to the sever when the connection drops. This state will be lost if the page just reloads itself. (For your use case, this is not important, because you only consider server-side state to be relevant, e.g. the light bulbs that actually emit light.) Of course, the transient state is not particularly useful, but it may be important for the user to at least be able to salvage it (e.g. copy & paste somewhere else).

Of course, I can gate your workaround with a configuration option, so that the application author can decide whether reloading the page is ok or not.

The only workaround I could come up with for that one has some very unfortunate side-effects, like being incompatible with the special HTML-app modes that iOS/Android have. With these you could add a threepenny application to the home screen, have it load as its own process, no browser controls etc. It would go a long way in making threepenny based software feel more slick and native on mobile devices. I'd love to support that, especially now that the loading times are so fast...

Could you be more specific on this incompatibility? I have to admit that I'm not very familiar with mobile development, mostly because I did not own a smartphone until a few weeks ago... 😄 A reference to some tutorial or blog post would already help.

[blitzcode]

Ok, great! First, let me sum up why I didn't find a solution outside of threepenny. Basically, all of the DOM events (the 'onleave' or-whatever-it's-called-type-stuff) is extremely browser dependent. Some browsers might kill the websocket if the tab is hidden for a certain amount of time, some if the device goes to sleep, this might depend on whether you're on battery or mains, some trigger certain events in that case, some don't.

The only thing I could find that at least reliably works on some systems is the timer trick. Basically, if the timer wasn't running, chances are the websocket connection has been closed as well. Now, Android & iOS have a web app mode specifically designed for single-page web applications like threepenny. The idea is that all typical browser UI is hidden, everything runs in a separate process from the system's browser, various other tweaks like no text selection, no zooming not controlled by the app etc. That's really neat because you can just add a threepenny application to your home screen and it loads, looks and feels much more like an app rather than a website. The problem is, for some reason, timers get suspended when the user scrolls such a web app on iOS. So the timeout detection breaks and since there's no more refresh button either, the user is stuck. So no support for this with the current threepenny.

I tried to just reload the page in the ondisconnect event for the websocket, and that seems to work reliable on every platform and browser I tried. I'd of course be fine if you made this behavior optional, but I'd say that's not strictly necessary. Basically the only state that a user could extract from a disconnected threepenny application would be text they typed. And I'd argue that losing whatever text you typed is expected behavior from both users and developers. If you type some novel of a forum post and the hit refresh or let the tab be evicted from memory, that text would be gone. Some web site / application developers prevent this by caching the text periodically on the server, which of course you could also implement in any threepenny application. That's what I'd do if my software had client-side user input besides a few dialog boxes with some form elements.

[HeinrichApfelmus]

Android & iOS have a web app mode specifically designed for single-page web applications like threepenny.

Could you point me to a resource on how to convert a web page to an app on my Android phone? I tried to google for this, but was not clever enough to find what you describe. (I found something about PhoneGap/Cordova, NativeScript and other frameworks, but that was not what you meant.)

Basically the only state that a user could extract from a disconnected threepenny application would be text they typed. [..] If you type some novel of a forum post and then hit refresh or let the tab be evicted from memory, that text would be gone.

Well, the difference is that, at least on a desktop, losing the text still requires an explicit action by the user (close the tab), whereas reloading on a dropped websocket connection just deletes the text without warning. But I'm happy to give this some more thought — implemented as an option, this should be fine in any case.

[blitzcode]

I have most experience with iOS, I meant this:

https://developer.apple.com/library/content/documentation/AppleApplications/Reference/SafariWebContent/ConfiguringWebApplications/ConfiguringWebApplications.html

That should be the Android equivalent:

https://developer.chrome.com/multidevice/android/installtohomescreen

Seems to be intended exactly for single page applications like threepenny. A Raspberry Pi + threepenny is a really neat way to get app-like functionality on mobile devices without having to deal with the native development environment (and its lack of Haskell...). Hence my interest in getting this fixed so I can use those special 'web app modes'!

[blitzcode]

Is there any progress on this? This bug has been open for nearly a year now. I know you're working on some more fancy solution, but a perfectly good (and in any case far better than the current situation) fix for this is one line of code. Every day this critical bug remains unfixed it impacts real users. A recent change in FF makes websocket disconnects quite frequent even in a desktop browser, further increasing the issues caused by this. Could we please have any fix at all for this?

[sjakobi]

a perfectly good (and in any case far better than the current situation) fix for this is one line of code.

Could we please have any fix at all for this?

I have too little experience to provide a fix myself, but if @blitzcode makes a PR on top of the v0.7.0.1 tag and @HeinrichApfelmus approves, I can make a bugfix release.

[HeinrichApfelmus]

Sorry for the delay, my free time has taken a deep dive again. I won't be able to work on this until the beginning of April.

That said, I haven't found a good solution so far, so I'm ok with making it a configuration option, named jsForceClientReload or something like that.

@sjakobi This would need to be at least a v0.7.1.0 release, because a new field (the option) is added to a constructor. Actually, I am not sure what the PVP says in this case.

[blitzcode]

I don't think there is a truly great solution. If you reconnect and discard all messages sent during the disconnect, you potentially leave the application in an incoherent / broken state. If you resend all messages from during the disconnect, you potentially send hours worth of messages when a device wakes up from standby, freezing up the server & client. I think at best you could have some heuristic like a reload of the application after some threshold of build up messages?

I think any attempt at a workaround will just be a leaky abstraction. Browsers are fickle, they drop connections, users accidentally close tabs or hit refresh. Anything client-side is always ephemeral. Unless you have full control over client & server, like perhaps with Electron, applications will need to be written with this in mind to be reliable.

I've been getting more complaints about this issue recently as it seems some browsers have changed behavior. I can confirm that at least Firefox on my MacBook is now dropping web socket connections even when just the display goes to sleep. Mobile devices have always been even more aggressive about this.

As for adding a field - I always liked the pattern were the actual ctor is not exported and you have to use/modify a defaultSettings variable. This way new fields with a default can be added without breaking existing code.

[HeinrichApfelmus]

I have now added the aforementioned configuration option in commit 5ccc078 .

@blitzcode Does this improve matters for you? On Android, I do seem to have trouble running Threepenny apps in the first place.

[blitzcode]

Awesome, thanks a bunch!

Did a quick check on iOS/Firefox, seems to work. I'll do some more testing for sure, but I've been running a fork that does basically the same thing and I haven't seen it fail on any OS/browser combination.

I had some issues with threepenny on older Android devices, but I tried with some more recent ones and it seemed to work then. Never had any issues on iPad/iPhone and desktop Safari/FF/Chrome also seemed to work great.

Will you do a hackage release with all the recent useful fixes & improvements?

[HeinrichApfelmus]

Great!

Will you do a hackage release with all the recent useful fixes & improvements?

Yes, that's the plan. I intend to solve all issues with the label "Status: In Progress" for the 0.8 release.

[HeinrichApfelmus]

Working towards Hackage release. I take it that this issue has been solved? If not, please don't hesitate to reopen!

[HeinrichApfelmus]

@blitzcode I have just uploaded the 0.8 release to hackage. 😄🚀

[blitzcode]

Sweet! I'll be able to start switching to & testing with the new release next weekend, great to have all the recent improvement on Hackage. I'll let you know if I find any unexpected things!