Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Users can remain online even when their sessions get disconnected #29

Open
mizzao opened this issue Aug 1, 2014 · 15 comments
Open

Users can remain online even when their sessions get disconnected #29

mizzao opened this issue Aug 1, 2014 · 15 comments
Assignees
Labels

Comments

@mizzao
Copy link
Collaborator

mizzao commented Aug 1, 2014

Currently not sure how to replicate. Could be a multiplexing bug with reloads mixed in.

Observed in TurkServer after several hundred user connections (the 1 session below is the admin):

image

UAs of these "stuck" users seem to be pretty recent browsers, including latest Chrome:

"Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/36.0.1985.125 Safari/537.36", 
"Mozilla/5.0 (Windows NT 6.1; WOW64; rv:30.0) Gecko/20100101 Firefox/30.0", 
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:30.0) Gecko/20100101 Firefox/30.0", 
"Mozilla/5.0 (Windows NT 6.3; WOW64; rv:31.0) Gecko/20100101 Firefox/31.0", 
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/36.0.1985.125 Safari/537.36", 
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:31.0) Gecko/20100101 Firefox/31.0", 
"Mozilla/5.0 (Windows NT 6.1; WOW64; rv:27.0) Gecko/20100101 Firefox/27.0"

These will get cleared when the server restarts, but are generally troublesome for determining when someone has actually gone away.

@mizzao mizzao self-assigned this Aug 1, 2014
@mizzao mizzao added the bug label Aug 7, 2014
@mizzao
Copy link
Collaborator Author

mizzao commented Jan 29, 2015

This may be fixed in Meteor 1.0.3.1 with

Fix inaccurate session statistics and possible multiple invocation of Connection.onClose callbacks.

@mizzao
Copy link
Collaborator Author

mizzao commented Feb 18, 2015

See also #43, which seems to be the opposite of this (users that are actually online when their sessions appear to be gone).

@marcelpanse
Copy link

I'm noticing the same problem using meteor 1.0.3.2 and mizzao:user-status 0.6.4
It is hard to reproduce, seems only to happen with certain users..

The users stay online but idle forever. Is there a way to access the lastActivity timestamp server-side, so I can create a cleanup-cronjob for users that are idle for long periods of time..?

@mizzao
Copy link
Collaborator Author

mizzao commented Mar 16, 2015

Yep, there is a collection of all "connected" sessions that you can inspect (see the docs), as well as the Meteor.users collection itself.

@elie222
Copy link

elie222 commented Jul 15, 2015

This is still an issue on Meteor 1.1.0.2.

I have 49 users online at the moment according to this package, but Google analytics is telling me 15 users are online, and Kadira is telling me 34 sessions/host (I only have 1 server, so that's 1 host right?). It's consistently in this state of having way more users online than there actually are. I'm sure this number of 49 will continue to increase and increase. Some users just seem to stay online forever.

@mizzao
Copy link
Collaborator Author

mizzao commented Jul 15, 2015

GA has no way of knowing if someone is connected to your server, unless you are hitting their API. The online (but idle) state will just remain if someone leaves their computer on with an Internet connection.

On the other hand, the discrepancy with Kadira is interesting. Perhaps @arunoda has figured out the bugs in the Meteor connection API and rolled his own, more robust solution. While in the past this package used its own method of detecting connections, I switched over to just using Meteor because I thought it was stable enough. Or perhaps Kadira tracks online users through API hits as well, and it's the same issue as comparing with GA.

One crude way to tell if the users are actually offline but appear to be online (as opposed to just having left your app open in a browser tab in a running computer) is to try pinging the IPs of these ghost users. It's not super accurate if they are behind a NAT, but might help narrow things down.

I will also note (for future readers) that running this in a cluster is not supported (#21).

@elie222
Copy link

elie222 commented Jul 15, 2015

What do you mean GA has no way of knowing? What are there real time updates
then that show how many people are online and exactly which pages they are
visiting?

On 15 July 2015 at 18:28, Andrew Mao notifications@github.com wrote:

GA has no way of knowing if someone is connected to your server, unless
you are hitting their API. The online (but idle) state will just remain if
someone leaves their computer on with an Internet connection.

On the other hand, the discrepancy with Kadira is interesting. Perhaps
@arunoda https://github.com/arunoda has figured out the bugs in the
Meteor connection API and rolled his own, more robust solution. While in
the past this package used its own method of detecting connections, I
switched over to just using Meteor because I thought it was stable enough.
Or perhaps Kadira tracks online users through API hits as well, and it's
the same issue as comparing with GA.

One crude way to tell if the users are actually offline but appear to be
online (as opposed to just having left your app open in a browser tab in a
running computer) is to try pinging the IPs of these ghost users. It's not
super accurate if they are behind a NAT, but might help narrow things down.

I will also note (for future readers) that running this in a cluster is
not supported (#21
#21).


Reply to this email directly or view it on GitHub
#29 (comment)
.

@mizzao
Copy link
Collaborator Author

mizzao commented Jul 15, 2015

I'm pretty sure that what GA means by "online" is API hits, which presumably on your app are triggered when a new route is navigated. I don't think GA has any sort of live websocket connection like Meteor. Users are probably no longer online when they haven't hit the API after some threshold.

This is a different online than Meteor's definition, which is literally "is there a live connection from the server to this person's computer". That's why we have the idle state tracking.

Feel free to correct me if I misunderstand how GA works.

@elie222
Copy link

elie222 commented Jul 15, 2015

No. I was just interested in understanding the situation better. Thanks for
the explanation
On 15 Jul 2015 6:50 pm, "Andrew Mao" notifications@github.com wrote:

I'm pretty sure that what GA means by "online" is API hits, which
presumably on your app are triggered when a new route is navigated. I don't
think GA has any sort of live websocket connection like Meteor. Users are
probably no longer online when they haven't hit the API after some
threshold.

This is a different online than Meteor's definition, which is literally
"is there a live connection from the server to this person's computer".
That's why we have the idle state tracking.

Feel free to correct me if I misunderstand how GA works.


Reply to this email directly or view it on GitHub
#29 (comment)
.

@abecks
Copy link

abecks commented Aug 15, 2016

I just started noticing this issue in my Meteor application, I have 3 "stuck online" users after having about 50 users come through.

They do indeed clear when the application is restarted, however, I have an observer setup on 'status.online' to trigger events in my application when users go online/offline.

I need a way to check for these "orphaned" online users so I was thinking of using the lastActivity property to check for "online" users with no activity for a certain time period. I'm assuming I need to enable idle monitoring because I don't see lastActivity in my database yet.

Can you think of any other way to check for these stuck users without enabling idle monitoring?

Edit: Here is a workaround to clear out the orphaned online users every 5 minutes:

export const idleUserCheckInterval = Meteor.setInterval(function idleUserCheck() {
  // Grab online users
  Meteor.users.find({ 'status.online': true }).forEach((user) => {
    // Cross reference with UserStatus.connections collection
    const hasConnection = UserStatus.connections.findOne({ userId: user._id });
    if (!hasConnection) {
      Meteor.users.update({ _id: user._id }, {
        $set: {
          'status.online': false,
        },
      });
    }
  });
}, 5 * 60 * 1000);

Will report back on whether this works or not. I have a feeling the orphaned users might still be in the UserStatus.connections collection.

@StorytellerCZ
Copy link
Member

StorytellerCZ commented Oct 21, 2019

Is this still issue for people? #112 ?

@diavrank
Copy link

diavrank commented Nov 10, 2020

Hi, I just got this error with Meteor 1.11.1. I was doing a load test with 1000 concurrent users to login them and when it arrived to 900 clients, it started to take more time, so I stopped the process of load test and I realized that some clients were stuck in online (to be exact, 838 clients).

So, I changed the status.online value to false of a client and I tried to login again and that property changed to true but when I did logout, the status.online keeps in true.

I did that load test in a VM of AWS (using docker with disney/meteor-base image) with the following requirements:

  • 2 vCPUs
  • 3.7 GB RAM
  • NODE_OPTIONS=--max-old-space-size=2048

Here are some docker stats during the load test:

CONTAINER ID        NAME                CPU %               MEM USAGE / LIMIT     MEM %               NET I/O             BLOCK I/O           PIDS
293417e35008        crontab             0.00%               268KiB / 3.699GiB     0.01%               6.34kB / 4.53kB     86MB / 8.19kB       1
4fd21805112f        app                 135.87%             365.3MiB / 3.699GiB   9.64%               2.09GB / 318MB      4.47MB / 0B         11
fd8e9b170f8d        mongo               16.22%              115.8MiB / 3.699GiB   3.06%               312MB / 2.08GB      89.6MB / 104MB      40

CONTAINER ID        NAME                CPU %               MEM USAGE / LIMIT     MEM %               NET I/O             BLOCK I/O           PIDS
293417e35008        crontab             0.00%               268KiB / 3.699GiB     0.01%               6.34kB / 4.53kB     86MB / 8.19kB       1
4fd21805112f        app                 125.74%             449.4MiB / 3.699GiB   11.86%              6.47GB / 696MB      4.47MB / 0B         11
fd8e9b170f8d        mongo               14.19%              122.7MiB / 3.699GiB   3.24%               682MB / 6.46GB      89.6MB / 135MB      40

CONTAINER ID        NAME                CPU %               MEM USAGE / LIMIT     MEM %               NET I/O             BLOCK I/O           PIDS
293417e35008        crontab             0.00%               268KiB / 3.699GiB     0.01%               6.34kB / 4.53kB     86MB / 8.19kB       1
4fd21805112f        app                 85.01%              371.7MiB / 3.699GiB   9.81%               7.34GB / 809MB      4.47MB / 0B         11
fd8e9b170f8d        mongo               35.19%              122.8MiB / 3.699GiB   3.24%               795MB / 7.33GB      89.6MB / 137MB      40

CONTAINER ID        NAME                CPU %               MEM USAGE / LIMIT     MEM %               NET I/O             BLOCK I/O           PIDS
293417e35008        crontab             0.00%               268KiB / 3.699GiB     0.01%               6.34kB / 4.53kB     86MB / 8.19kB       1
4fd21805112f        app                 82.13%              374.7MiB / 3.699GiB   9.89%               8.02GB / 929MB      4.47MB / 0B         11
fd8e9b170f8d        mongo               34.21%              124.3MiB / 3.699GiB   3.28%               913MB / 8.01GB      89.6MB / 139MB      40

CONTAINER ID        NAME                CPU %               MEM USAGE / LIMIT     MEM %               NET I/O             BLOCK I/O           PIDS
293417e35008        crontab             0.00%               268KiB / 3.699GiB     0.01%               6.34kB / 4.53kB     86MB / 8.19kB       1
4fd21805112f        app                 100.23%             350.5MiB / 3.699GiB   9.25%               8.13GB / 943MB      4.47MB / 0B         11
fd8e9b170f8d        mongo               23.86%              124.2MiB / 3.699GiB   3.28%               927MB / 8.12GB      89.6MB / 139MB      40

CONTAINER ID        NAME                CPU %               MEM USAGE / LIMIT     MEM %               NET I/O             BLOCK I/O           PIDS
293417e35008        crontab             0.00%               268KiB / 3.699GiB     0.01%               6.34kB / 4.53kB     86MB / 8.19kB       1
4fd21805112f        app                 66.30%              306.2MiB / 3.699GiB   8.08%               8.36GB / 973MB      4.47MB / 0B         11
fd8e9b170f8d        mongo               47.04%              125.4MiB / 3.699GiB   3.31%               957MB / 8.35GB      89.6MB / 140MB      40

To repair it, I had to run on the database the next command:
db.getCollection('users').update({"status.idle":{$exists:true}},{$set:{'status':{online:false}}},{multi:true})
And after that, I restarted the containers. However, I think that scenario can be helpful to find the cause of this issue :) .

Note: I did the same load test in my computer and this issue didn't happen. I guess it is due to the hardware because I assigned 8GB to app (through NODE_OPTIONS) and it has a processor of core i7 (six cores), ssd, 16gb, mac os.

@copleykj
Copy link
Member

@diavrank maybe give socialize:user-presence a try.

@diavrank
Copy link

Hey @copleykj , I finished the load test (1000 concurrent logins) using socialize:user-presence package and the issue has gone. So, it worked very well.

I share the code of how to adapt mizzao:user-status to socialize:user-presence (to use it as alternative solution in these cases):

UserPresenceConfig.js

import { Meteor } from 'meteor/meteor';
import { User } from 'meteor/socialize:user-model';
import { UserPresence } from 'meteor/socialize:user-presence';
import Utilities from '../../startup/server/Utilities';
import SimpleSchema from 'simpl-schema';

// Schema for the fields where we will store the status data
const StatusSchema = new SimpleSchema({
	status: Object,
	'status.online': { type: Boolean },
	'status.idle': { type: Boolean, optional: true },
	'status.lastLogin': { type: Object, optional: true, blackbox: true }
});

// Add the schema to the existing schema currently attached to the User model
User.attachSchema(StatusSchema);

// If `sessionIds` is undefined this signifies we need a fresh start.
// When a full cleanup is necessary we will unset the status field to show all users as offline
UserPresence.onCleanup(function onCleanup(sessionIds) {
	if (!sessionIds) {
		console.log('cleanup sessionIds executed');
		Meteor.users.update({}, { $set: { 'status.online': false }, $unset: { 'status.idle': true } }, { multi: true });
	}
});

// When a user comes online we set their status to online and set the lastOnline field to the current time
UserPresence.onUserOnline(function onUserOnline(userId, connection) {
	Meteor.users.update(userId, {
		$set: {
			'status.online': true,
			'status.idle': false,
			'status.lastLogin.date': Utilities.currentLocalDate(),
			'status.lastLogin.ipAddr': connection.clientAddress,
			'status.lastLogin.userAgent': connection.httpHeaders['user-agent']
		}
	});
});

// When a user goes idle we'll set their status to indicate this
UserPresence.onUserIdle(function onUserIdle(userId) {
	Meteor.users.update(userId, { $set: { 'status.idle': true } });
});

// When a user goes offline we'll unset their status field to indicate offline status
UserPresence.onUserOffline(function onUserOffline(userId) {
	Meteor.users.update(userId, { $set: { 'status.online': false }, $unset: { 'status.idle': true } });
});

And don't forget to add simple schema to your profile key (so that it does not crash when creating a new user):

User.js

import { Meteor } from 'meteor/meteor';
import { User } from 'meteor/socialize:user-model';
import SimpleSchema from 'simpl-schema';

Meteor.users.allow({
	insert: () => false,
	update: () => false,
	remove: () => false
});

Meteor.users.deny({
	insert: () => true,
	update: () => true,
	remove: () => true
});

const UserProfileSchema = new SimpleSchema({
	profile: {
		type: Object,
		optional: false,
		blackbox: true
	}
});
User.attachSchema(UserProfileSchema);

Note: Don't forget remove mizzao:user-status package.

@copleykj
Copy link
Member

Awesome to hear @diavrank! Thanks for the update and sharing the code so that it might help others in the future.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

7 participants