New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Mac Sierra, Safari 10, /meta/connect stops sending when browser minimized, client loses connection #699

Open
aquacode opened this Issue Jan 12, 2017 · 21 comments

Comments

Projects
None yet
4 participants
@aquacode

aquacode commented Jan 12, 2017

Using Mac Sierra and Safari 10 to run our cometd web application.

The Javascript web client completes the handshake and initial /meta/connect, then begins sending the periodic /meta/connects to the server using long polling. If I minimize Safari, wait about 2 minutes, then maximize Safari (bring the app back to focus), the web client has gone "offline" due to the last /meta/connect request getting an "Unknown client" error response.

It appears that /meta/connects are being sent after their 30 second long poll response is returned, but for some reason when Safari is minimized, app loses focus, /meta/connects stop sending at the regular rate, and when a /meta/connect is eventually sent, it is passed the 10 second default expected by the server, and seems the server invalidates the session returning an "Unknown client".

Have a look at the attached from Safari's Network tool and notice that I minimized Safari when the /meta/connect request was sent at 365.54s mark of the web app. I then waited the 2 minutes while minimized, then maximized Safari. Notice at the 455.54s mark (about 90 seconds later) a /meta/connect request was made and a response was returned in 78.64ms which contained the "Unknown client" payload. Note the unsubscribe was sent followed by another handshake. The failed /meta/connect was sent at least 20 seconds after the previous one which would give the server the impression that the client disconnected, right?

Why is /meta/connect stopping to send the periodic requests to ensure the server knows the client is still connected when Safari is minimized? I do not detect this using Chrome or Firefox, but my colleagues report that occasionally they have seen it with IE.

2017-01-11 20_41_14-dashboard

@aquacode

This comment has been minimized.

Show comment
Hide comment
@aquacode

aquacode Jan 12, 2017

Additional info:

This can be reproduced using Mac El Capitan and Safari 9.1, although the Safari had to be minimized for at least 7 minutes, then the Cometd /meta/connect requests began to be sent back to the server with an increased spacing, not immediately after the last request finished, causing the Cometd server to invalidate the client's session.

aquacode commented Jan 12, 2017

Additional info:

This can be reproduced using Mac El Capitan and Safari 9.1, although the Safari had to be minimized for at least 7 minutes, then the Cometd /meta/connect requests began to be sent back to the server with an increased spacing, not immediately after the last request finished, causing the Cometd server to invalidate the client's session.

@sbordet

This comment has been minimized.

Show comment
Hide comment
@sbordet

sbordet Jan 12, 2017

Member

@aquacode I don't have Mac equipment, so would be great if you can correlate what you see in the Safari network console with a network traffic capture via Wireshark. Even if it's HTTPS (or perhaps you can also do HTTP ?) we will know if there is network activity after you have minimized the browser.
Does look like a Safari issue though.

Member

sbordet commented Jan 12, 2017

@aquacode I don't have Mac equipment, so would be great if you can correlate what you see in the Safari network console with a network traffic capture via Wireshark. Even if it's HTTPS (or perhaps you can also do HTTP ?) we will know if there is network activity after you have minimized the browser.
Does look like a Safari issue though.

@aquacode

This comment has been minimized.

Show comment
Hide comment
@aquacode

aquacode Jan 12, 2017

FYI we are using 3.0.4

We tested and retested, and also separately verified (in a small little app) that Safari throttles down setTimeout when minimized/losing focus, so from our perspective it appears verified (I can readily reproduce). I have had sessions open on Chrome/IE all day and they don't seem to be experiencing this throttling. I could investigate Firefox as well to see what it does I suppose.
In addition, we have increased our maxInterval, upwards of about 60 seconds, and then observe that the Safari client maintains its connection to the Cometd server. We don't particularly like having to set such a high value, but cursory tests showed that the throttling can be over 30 seconds at times, making the 60 second value reasonable. The time back off 'spacing' doesn't appear to follow any series, but I did not verify this definitively.

Would providing the Wireshark data still be of interest?

What, if anything, do you think Cometd could do to mitigate this (any non-setTimeout option available?), or is configuration the only known, and recommended workaround? Trying to understand where we stand.

aquacode commented Jan 12, 2017

FYI we are using 3.0.4

We tested and retested, and also separately verified (in a small little app) that Safari throttles down setTimeout when minimized/losing focus, so from our perspective it appears verified (I can readily reproduce). I have had sessions open on Chrome/IE all day and they don't seem to be experiencing this throttling. I could investigate Firefox as well to see what it does I suppose.
In addition, we have increased our maxInterval, upwards of about 60 seconds, and then observe that the Safari client maintains its connection to the Cometd server. We don't particularly like having to set such a high value, but cursory tests showed that the throttling can be over 30 seconds at times, making the 60 second value reasonable. The time back off 'spacing' doesn't appear to follow any series, but I did not verify this definitively.

Would providing the Wireshark data still be of interest?

What, if anything, do you think Cometd could do to mitigate this (any non-setTimeout option available?), or is configuration the only known, and recommended workaround? Trying to understand where we stand.

@sbordet

This comment has been minimized.

Show comment
Hide comment
@sbordet

sbordet Jan 12, 2017

Member

Did I understand correctly that you are saying that Safari "delays" running the timer behind window.setTimeout() ?

If you can confirm this via a super-simple setup that just prints on the console from a function triggered by window.setTimeout(), then no, there is no need for network capture.

I could not think of any solution for this... window.setTimeout() is pretty fundamental in CometD, and if it does not work there is not much to do.

Increasing maxInterval looks the right solution to me if Safari "delays" window.setTimeout(); it will have effects on the server, as sessions will continue to accumulate messages until their communication is restored, so expect some memory consumption increase.

Member

sbordet commented Jan 12, 2017

Did I understand correctly that you are saying that Safari "delays" running the timer behind window.setTimeout() ?

If you can confirm this via a super-simple setup that just prints on the console from a function triggered by window.setTimeout(), then no, there is no need for network capture.

I could not think of any solution for this... window.setTimeout() is pretty fundamental in CometD, and if it does not work there is not much to do.

Increasing maxInterval looks the right solution to me if Safari "delays" window.setTimeout(); it will have effects on the server, as sessions will continue to accumulate messages until their communication is restored, so expect some memory consumption increase.

@aquacode

This comment has been minimized.

Show comment
Hide comment
@aquacode

aquacode Jan 19, 2017

It would be good if your team could access Mac hardware, or a virtual Safari instance, so that actual testing could take place for this. As it is now, seems as if Safari losing focus causes an application based on CometD to simply have to re-establish its connection/session all over again.

We switched to web sockets, and this did not solve the issue either: the /meta/connect occurs over the established socket, and Safari appears to be stopping the sending of those requests as well, when the browser is minimized, i.e. loses its focus.

We increased the maxInterval, and it "solved" the problem, but finding the perfect value for that, is challenging, because if we set the value to say, 30 seconds, we have found that sometimes it still gets outside of that time window and server invalidates the user session. It starts to demand a too high maxInterval time.

aquacode commented Jan 19, 2017

It would be good if your team could access Mac hardware, or a virtual Safari instance, so that actual testing could take place for this. As it is now, seems as if Safari losing focus causes an application based on CometD to simply have to re-establish its connection/session all over again.

We switched to web sockets, and this did not solve the issue either: the /meta/connect occurs over the established socket, and Safari appears to be stopping the sending of those requests as well, when the browser is minimized, i.e. loses its focus.

We increased the maxInterval, and it "solved" the problem, but finding the perfect value for that, is challenging, because if we set the value to say, 30 seconds, we have found that sometimes it still gets outside of that time window and server invalidates the user session. It starts to demand a too high maxInterval time.

@sbordet

This comment has been minimized.

Show comment
Hide comment
@sbordet

sbordet Jan 19, 2017

Member

Did you file a bug to Safari ?

Member

sbordet commented Jan 19, 2017

Did you file a bug to Safari ?

@aquacode

This comment has been minimized.

Show comment
Hide comment
@aquacode

aquacode Jan 19, 2017

I've started with a Community posted question to see if anyone provides feedback

aquacode commented Jan 19, 2017

I've started with a Community posted question to see if anyone provides feedback

@sbordet

This comment has been minimized.

Show comment
Hide comment
@sbordet

sbordet Jan 19, 2017

Member

Can you link it here ?

Member

sbordet commented Jan 19, 2017

Can you link it here ?

@aquacode

This comment has been minimized.

Show comment
Hide comment

aquacode commented Jan 19, 2017

@aquacode

This comment has been minimized.

Show comment
Hide comment
@aquacode

This comment has been minimized.

Show comment
Hide comment
@aquacode

aquacode Jan 20, 2017

@sbordet This Stack Overflow question I posted might be of interest to you and your team:
http://stackoverflow.com/questions/41768173/safari-javascript-settimeout-stops-when-minimized/41768617#41768617

Also, we conducted a test using the following code:

$(document).ready(function(){
	i=0;
	setTimeout(addItem,1000);
function addItem(){
	var date = new Date();
	$("#list li:last").after('<li>'+ i++ + ":"+date+'</li>');
	console.log(date);
	setTimeout(addItem,1000);
}
});

And here are the results in Safari:

screen shot 2017-01-20 at 12 30 15 pm

At time 12:16:47 I switched tabs and minimized the browser. You can see that Safari randomly delays the execution sometimes with a delay of over a minute!

Running the same code in Chrome:

2017-01-20 13_47_57-developer tools - file____c__users_michaelb_downloads_test html

The screenshot I attached is of a test that is run over a short period, but I also ran a test of over an hour, and Chrome maintained the interval spacing of 2 seconds when the tab lost focus and the browser was minimized.

aquacode commented Jan 20, 2017

@sbordet This Stack Overflow question I posted might be of interest to you and your team:
http://stackoverflow.com/questions/41768173/safari-javascript-settimeout-stops-when-minimized/41768617#41768617

Also, we conducted a test using the following code:

$(document).ready(function(){
	i=0;
	setTimeout(addItem,1000);
function addItem(){
	var date = new Date();
	$("#list li:last").after('<li>'+ i++ + ":"+date+'</li>');
	console.log(date);
	setTimeout(addItem,1000);
}
});

And here are the results in Safari:

screen shot 2017-01-20 at 12 30 15 pm

At time 12:16:47 I switched tabs and minimized the browser. You can see that Safari randomly delays the execution sometimes with a delay of over a minute!

Running the same code in Chrome:

2017-01-20 13_47_57-developer tools - file____c__users_michaelb_downloads_test html

The screenshot I attached is of a test that is run over a short period, but I also ran a test of over an hour, and Chrome maintained the interval spacing of 2 seconds when the tab lost focus and the browser was minimized.

@sbordet

This comment has been minimized.

Show comment
Hide comment
@sbordet

sbordet Jan 20, 2017

Member

@aquacode all right. To me it's obvious it's a Safari bug. I understand the idea to save battery, but Chrome does it better, and this should be really exposed as a property to configure in the browser Web APIs. Imagine if one uses long polling for a medical application and the browser decide it's more important to save the battery.

But, bitching about how crappy is Safari won't solve this issue.

So let's restart this thread by analyzing the problem first.

You have an application that uses CometD. The user puts Safari in background. Session expires.
Why you want to keep the session ? After all, the user decided to not look at the CometD application anymore. Why would it be surprised that the session is gone ?
I'm basically asking for the use case here, and playing devil's advocate.

Just to give you the full picture, we have had in the past users that explicitly required the behavior of expiring the session while the user was "gone", for example for a lunch break. Iconize the browser, go to lunch (or in vacation for one week), you don't want their session to linger on the server for that much time.

I would like a strong argument in favor of your case. I have a few ideas of how to solve it, but I don't want to influence you. Explain why for you is a problem that the session goes away. Explain what would you do if the user keeps the browser iconized for 2 days, meanwhile using another browser or other tabs. What if you have 10k users do that - will your server memory be exhausted ? And so forth.

Thanks !

Member

sbordet commented Jan 20, 2017

@aquacode all right. To me it's obvious it's a Safari bug. I understand the idea to save battery, but Chrome does it better, and this should be really exposed as a property to configure in the browser Web APIs. Imagine if one uses long polling for a medical application and the browser decide it's more important to save the battery.

But, bitching about how crappy is Safari won't solve this issue.

So let's restart this thread by analyzing the problem first.

You have an application that uses CometD. The user puts Safari in background. Session expires.
Why you want to keep the session ? After all, the user decided to not look at the CometD application anymore. Why would it be surprised that the session is gone ?
I'm basically asking for the use case here, and playing devil's advocate.

Just to give you the full picture, we have had in the past users that explicitly required the behavior of expiring the session while the user was "gone", for example for a lunch break. Iconize the browser, go to lunch (or in vacation for one week), you don't want their session to linger on the server for that much time.

I would like a strong argument in favor of your case. I have a few ideas of how to solve it, but I don't want to influence you. Explain why for you is a problem that the session goes away. Explain what would you do if the user keeps the browser iconized for 2 days, meanwhile using another browser or other tabs. What if you have 10k users do that - will your server memory be exhausted ? And so forth.

Thanks !

@bmorton

This comment has been minimized.

Show comment
Hide comment
@bmorton

bmorton Jan 25, 2017

Does this blog post mean that Chrome is going to start doing the same thing as Safari? http://blog.strml.net/2017/01/chrome-56-now-aggressively-throttles.html

bmorton commented Jan 25, 2017

Does this blog post mean that Chrome is going to start doing the same thing as Safari? http://blog.strml.net/2017/01/chrome-56-now-aggressively-throttles.html

@sbordet

This comment has been minimized.

Show comment
Hide comment
@sbordet

sbordet Jan 25, 2017

Member

@bmorton, the proposed implementation in Chrome is kind of reasonable, with a budget that regenerates. Unless the server is frantically sending messages to the client, typical /meta/connect messages should not consume much CPU and the time between them should be long enough for the budget to regenerate.

Using Service Workers could be an option, but only Firefox and Chrome implement it - no Safari, no IE, no Edge, no mobile.

Apparently the problem of "drains the battery" will be solved soon, so this "feature" of throttling things and complicating the life of web developers seems worrying too much for a problem that won't exist in the near future.

We'll see.

Member

sbordet commented Jan 25, 2017

@bmorton, the proposed implementation in Chrome is kind of reasonable, with a budget that regenerates. Unless the server is frantically sending messages to the client, typical /meta/connect messages should not consume much CPU and the time between them should be long enough for the budget to regenerate.

Using Service Workers could be an option, but only Firefox and Chrome implement it - no Safari, no IE, no Edge, no mobile.

Apparently the problem of "drains the battery" will be solved soon, so this "feature" of throttling things and complicating the life of web developers seems worrying too much for a problem that won't exist in the near future.

We'll see.

@unindented

This comment has been minimized.

Show comment
Hide comment
@unindented

unindented Jan 25, 2017

I've run the following two experiments on Safari and Chrome, which seem to confirm WebWorkers are not throttled. They are supported by all modern browsers (http://caniuse.com/#feat=webworkers), so you could put your reconnection logic in one as a workaround?

Plain setTimeout

<html>
<body>
<script>
function tick() {
  console.log(new Date());
  setTimeout(tick, 1000);
}
tick();
</script>
</body>
</html>

Safari 10.0.3

Throttles aggressively (up to 30s) when window is minimized:

screen shot 2017-01-25 at 11 21 17

Chrome 58.0.2992.0 with throttling enabled

Throttles slightly (up to 3s) when window is minimized:

screen shot 2017-01-25 at 11 31 20

Worker setTimeout

<html>
<body>
<script>
var worker = new Worker("tick.js");
worker.onmessage = function (e) {
  console.log(e.data);
};
</script>
</body>
</html>
// tick.js
function tick() {
  postMessage(new Date());
  setTimeout(tick, 1000);
}
tick();

Safari 10.0.3

No throttling:

screen shot 2017-01-25 at 11 34 07

Chrome 58.0.2992.0 with throttling enabled

No throttling:

screen shot 2017-01-25 at 11 35 52

unindented commented Jan 25, 2017

I've run the following two experiments on Safari and Chrome, which seem to confirm WebWorkers are not throttled. They are supported by all modern browsers (http://caniuse.com/#feat=webworkers), so you could put your reconnection logic in one as a workaround?

Plain setTimeout

<html>
<body>
<script>
function tick() {
  console.log(new Date());
  setTimeout(tick, 1000);
}
tick();
</script>
</body>
</html>

Safari 10.0.3

Throttles aggressively (up to 30s) when window is minimized:

screen shot 2017-01-25 at 11 21 17

Chrome 58.0.2992.0 with throttling enabled

Throttles slightly (up to 3s) when window is minimized:

screen shot 2017-01-25 at 11 31 20

Worker setTimeout

<html>
<body>
<script>
var worker = new Worker("tick.js");
worker.onmessage = function (e) {
  console.log(e.data);
};
</script>
</body>
</html>
// tick.js
function tick() {
  postMessage(new Date());
  setTimeout(tick, 1000);
}
tick();

Safari 10.0.3

No throttling:

screen shot 2017-01-25 at 11 34 07

Chrome 58.0.2992.0 with throttling enabled

No throttling:

screen shot 2017-01-25 at 11 35 52

@sbordet

This comment has been minimized.

Show comment
Hide comment
@sbordet

sbordet Jan 25, 2017

Member

Thanks for trying this and reporting the results !

If this holds true, then I don't understand the scope of the Chrome/Safari "feature". If they were worried about battery consumption, now people will just drain the battery from Web Worker or Service Worker code.

@unindented would you be able to test your code in a Service Worker too ?

Member

sbordet commented Jan 25, 2017

Thanks for trying this and reporting the results !

If this holds true, then I don't understand the scope of the Chrome/Safari "feature". If they were worried about battery consumption, now people will just drain the battery from Web Worker or Service Worker code.

@unindented would you be able to test your code in a Service Worker too ?

@unindented

This comment has been minimized.

Show comment
Hide comment
@unindented

unindented Jan 25, 2017

@sbordet I tried creating a Service Worker with a setTimeout similar to the above, based on this simple test: https://github.com/mdn/sw-test

However, it looks like Service Workers aren't designed to have a long lifespan. The browser activates the SW when your page loads, then terminates it after about a minute. It'll then re-activate it to carry out a specific task (like process a push notification), and then will terminate it again. This answer on StackOverflow seems to confirm these observations: http://stackoverflow.com/questions/29741922/prevent-service-worker-from-automatically-stopping

Also Service Workers aren't close to being implemented in Safari (still under consideration https://webkit.org/status/#specification-service-workers).

unindented commented Jan 25, 2017

@sbordet I tried creating a Service Worker with a setTimeout similar to the above, based on this simple test: https://github.com/mdn/sw-test

However, it looks like Service Workers aren't designed to have a long lifespan. The browser activates the SW when your page loads, then terminates it after about a minute. It'll then re-activate it to carry out a specific task (like process a push notification), and then will terminate it again. This answer on StackOverflow seems to confirm these observations: http://stackoverflow.com/questions/29741922/prevent-service-worker-from-automatically-stopping

Also Service Workers aren't close to being implemented in Safari (still under consideration https://webkit.org/status/#specification-service-workers).

@aquacode

This comment has been minimized.

Show comment
Hide comment
@aquacode

aquacode Jan 25, 2017

@sbordet

Thanks for the attention you have given this issue.

Let me start off by saying that our software team hopes cometd will solve the issue I have described. Our team is willing to help with this as well.

Here follows our requirements:

• Secure real-time message-based collaboration solution targeted at Financial Services and other organizations with strong information control requirements
• Accessible from all popular HTML5-based browsers (Chrome, Firefox, IE, Safari) - this is a customer requirement that we have to satisfy
• Enables the user to stay connected with their associates across a 24-hour business cycle, with our application ready to immediately contact other people in the organization when needed

I have also addressed each of your questions below as well:

Why you want to keep the session ? After all, the user decided to not look at the CometD application anymore. Why would it be surprised that the session is gone ?
If a user isn’t ‘looking’ at an application doesn’t mean they don’t need it anymore. What if I have notifications turned on and want to background the application but continue to be notified when something new happens? I do this with other collaboration applications and expect to be notified when a new message arrives.

Iconize the browser, go to lunch (or in vacation for one week), you don't want their session to linger on the server for that much time.
I agree that we would not want to keep a user logged in forever, but I also don’t want them to lose their session within the period of 24 hours, or maybe even longer, say the user works Monday through Friday and doesn’t feel like logging in multiple times during the week.

Explain what would you do if the user keeps the browser iconized for 2 days, meanwhile using another browser or other tabs. What if you have 10k users do that - will your server memory be exhausted ?
We will definitely have to support the number of users that remain logged in for long periods of time. We are willing to do that. This of course means that our users should not be logged off at the whim of the browser. As an application developer, I’d like a means to keep my user’s session as long as is reasonable. Something way less than forever, but certainly a lot longer than a few minutes.

I hope this helps put things into perspective and frames the context better.

Thanks again for all your help so far.

aquacode commented Jan 25, 2017

@sbordet

Thanks for the attention you have given this issue.

Let me start off by saying that our software team hopes cometd will solve the issue I have described. Our team is willing to help with this as well.

Here follows our requirements:

• Secure real-time message-based collaboration solution targeted at Financial Services and other organizations with strong information control requirements
• Accessible from all popular HTML5-based browsers (Chrome, Firefox, IE, Safari) - this is a customer requirement that we have to satisfy
• Enables the user to stay connected with their associates across a 24-hour business cycle, with our application ready to immediately contact other people in the organization when needed

I have also addressed each of your questions below as well:

Why you want to keep the session ? After all, the user decided to not look at the CometD application anymore. Why would it be surprised that the session is gone ?
If a user isn’t ‘looking’ at an application doesn’t mean they don’t need it anymore. What if I have notifications turned on and want to background the application but continue to be notified when something new happens? I do this with other collaboration applications and expect to be notified when a new message arrives.

Iconize the browser, go to lunch (or in vacation for one week), you don't want their session to linger on the server for that much time.
I agree that we would not want to keep a user logged in forever, but I also don’t want them to lose their session within the period of 24 hours, or maybe even longer, say the user works Monday through Friday and doesn’t feel like logging in multiple times during the week.

Explain what would you do if the user keeps the browser iconized for 2 days, meanwhile using another browser or other tabs. What if you have 10k users do that - will your server memory be exhausted ?
We will definitely have to support the number of users that remain logged in for long periods of time. We are willing to do that. This of course means that our users should not be logged off at the whim of the browser. As an application developer, I’d like a means to keep my user’s session as long as is reasonable. Something way less than forever, but certainly a lot longer than a few minutes.

I hope this helps put things into perspective and frames the context better.

Thanks again for all your help so far.

@sbordet

This comment has been minimized.

Show comment
Hide comment
@sbordet

sbordet Jan 30, 2017

Member

I'm leaning towards (optionally) moving the calls to setTimeout() into a Worker.
Basically the current CometD code will delegate the setTimeout() to the Worker and be called back when the timeout expires, so that it does not get throttled.

Hopefully this would have a very small impact on CometD.

If you have experience on writing Worker code and avoid its downsides, I'm all ears.
I read about the Worker file being cached very aggressively, and requiring additional roundtrip (gah!).

Member

sbordet commented Jan 30, 2017

I'm leaning towards (optionally) moving the calls to setTimeout() into a Worker.
Basically the current CometD code will delegate the setTimeout() to the Worker and be called back when the timeout expires, so that it does not get throttled.

Hopefully this would have a very small impact on CometD.

If you have experience on writing Worker code and avoid its downsides, I'm all ears.
I read about the Worker file being cached very aggressively, and requiring additional roundtrip (gah!).

@sbordet

This comment has been minimized.

Show comment
Hide comment
@aquacode

This comment has been minimized.

Show comment
Hide comment
@aquacode

aquacode Feb 2, 2017

We do have someone versed in writing Workers. However, we are internally discussing our priorities and focus for the near future and will have a response sometime next week.

aquacode commented Feb 2, 2017

We do have someone versed in writing Workers. However, we are internally discussing our priorities and focus for the near future and will have a response sometime next week.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment