Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

"Removed by Server" #814

Open
n00b12345 opened this issue Jul 10, 2017 · 15 comments
Open

"Removed by Server" #814

n00b12345 opened this issue Jul 10, 2017 · 15 comments

Comments

@n00b12345
Copy link

@n00b12345 n00b12345 commented Jul 10, 2017

Quite often I notice some of my messages not going through. When I press the "i" button, I see that the OMEMO fingerprint has been "removed by server". I authenticate it again by verifying it and then it works perfectly fine. The same thing repeats quite often (sometimes after the SSL certificate is renewed).

Is this something to do with ChatSecure or the XMPP server software I'm using?

@chrisballinger
Copy link
Member

@chrisballinger chrisballinger commented Jul 18, 2017

@n00b12345
Copy link
Author

@n00b12345 n00b12345 commented Jul 28, 2017

@chrisballinger I'm using ChatSecure with someone who is using Conversations for android. Should that be the root of the problem?

@a11fruitless
Copy link

@a11fruitless a11fruitless commented Aug 7, 2017

I'm seeing similar strange behavior to n00b12345 with only some of my clients. This is on a Prosody server. I am using multiple instances of Gajim and Conversations with another user who is using ChatSecure. All my clients stay permanently online. Then the other user will send ChatSecure to the background where it eventually logs out. Once the user opens ChatSecure back up, the OMEMO Conversations' key is still trusted, but the Gajim keys have been "removed by server." Despite the fact that none of the keys have changed. Once they are trusted again, everything works great, but the process will repeat itself once ChatSecure logs out. This process is pretty repeatable for me right now.

It is strange that n00b12335 sees this for a Conversations client, but I see it for my Gajim clients.

@chrisballinger
Copy link
Member

@chrisballinger chrisballinger commented Aug 7, 2017

Hmmmmm well that should only happen if the keys were removed from the server. Some OMEMO clients still don't update their device list properly and wipe out other (active) devices.

@hschraep
Copy link

@hschraep hschraep commented Sep 26, 2017

Same i have with a colleague he have a iphone with chat secure installed and from time to time he has the problem that some omemo fingerprints was deactivated. After he enable it back all is working again.

We both are connected to the same prosody server. But i have a android connected with conversations but here never would be fingerprints deactivated.

So the question is how we could help to solve the prolbem? If we can provide some logs, tell us how and what you need.

@a11fruitless
Copy link

@a11fruitless a11fruitless commented Jan 13, 2018

I think I can clarify the root issue for myself here (which is admittedly a little different from what I posted earlier). Hopefully this coincides with the other user's experinces as well. I am presently using 4.2.1

The primary issue for me is that if a device is logged out for an extended period of time from the server, the verification of that OMEMO key is forgotten in the ChatSecure client when they log back in finally. I can manually verify it once they log back in and everything again works fine.

This causes problems for desktop clients that may be logged out for a while. If the user also has a phone client that stays online consistently (like Conversations which that key never seems to be unverified by ChatSecure) then the desktop client won't receive messages sent by ChatSecure until that particular key is reverified in ChatSecure.

Even if this issue is related to the server or the other client, it seems strange that ChatSecure wouldn't remember that the key has already been manually verified before and automatically verify the key again. ChatSecure can still receive messages from the other client and they show up with the caution symbol (that client just doesn't receive messages from ChatSecure until the key is reverified).

@chrisballinger
Copy link
Member

@chrisballinger chrisballinger commented Jan 13, 2018

The original intention was to prevent you from forever encrypting to someone's unused keys. When receiving a message it should update the "last seen date" of a key, and then it will untrust a key that hasn't been seen in 60 (? i forget) days. This logic could be improved, or at least the duration could be extended.

"Remove By Server" is a different distinction and it's when the list of keys on the server no longer contains a key. Sometimes servers or clients are buggy and wipe out the list by accident. The logic here could also be improved so previously trusted keys are re-trusted when added back by the server (or you receive a message from them again), but is limited by the current OMEMOTrustLevel states we store in the database model.

typedef NS_ENUM(NSUInteger, OMEMOTrustLevel) {
    OMEMOTrustLevelUntrustedNew = 0,
    OMEMOTrustLevelUntrusted    = 1,
    OMEMOTrustLevelTrustedTofu  = 2,
    OMEMOTrustLevelTrustedUser  = 3,
    /** If the device has been removed from the server */
    OMEMOTrustLevelRemoved  = 4,
};

We'd need to rethink these states a little bit to fix these edge cases.

@jotwewe
Copy link

@jotwewe jotwewe commented Mar 31, 2018

I experienced the same as described in #814 (comment)
I was using conversations 1.23.6, the other guy chatsecure/ios 4.2.1. Both accounts on a prosody 0.10, which never showed similar behaviour with other clients (multiple versions of conversations, dino and recently gajim).
We exchanged omemo-encrypted messages one day (both directions), but ten hours later his messages didn't arrive in conversations anymore (only in my dino client). Maybe conversations was offline during the day and dino was not.
It turned out that the key for my conversations client was marked as untrusted in chatsecure and something like "removed by server" was written beside it.

If I remember correctly, my gajim key (installed after the said message exchange) was also not marked as trusted. What is the intended behaviour of chatsecure regarding new keys?

@iNPUTmice
Copy link

@iNPUTmice iNPUTmice commented Mar 31, 2018

In Conversations a key that is removed from the PEP device list will only appear as inactive (grayed out in Conversations). We won’t send messages to inactive devices but they can become active again when they get re-announced/re-added to the list or when we receive a message from that device.
Inactive keys aren't marked as untrusted. Trust state in Conversations is independent of active/inactive.
Prosody removes all keys on restart. Devices going online one after the other will add themselves to the announcement again.

@jotwewe
Copy link

@jotwewe jotwewe commented Mar 31, 2018

iNPUTmice, thanks for your comment. Indeed the related Prosody server was restarted shortly before the messages from Chatsecure stopped appearing in Conversations and I guess Conversations was offline during this time. I feel iNPUTmice/Conversations#1901 is related in this context.

@Spydar007
Copy link

@Spydar007 Spydar007 commented May 14, 2018

I too am having this problem after almost every message sent with the other person using Conversations for Android. Various keys get "removed" by the server, and I think it is mostly whilst the devices are offline.

@LeoniePhiline
Copy link

@LeoniePhiline LeoniePhiline commented Feb 27, 2019

I can confirm what a11fruitless describes in #814 (comment)

Incidentally, I can probably also confirm jotwewe (#814 (comment)): Gajim's "Server Status" dialog shows that prosody (dismail.de) has an uptime of only 7h, so it must have been restarted just before the message loss occured again.

I am using Conversations and Gajim, the other person is using ChatSecure. Conversations never misses any messages, but Gajim regularly gets cut off from the conversation:
When logging in with Gajim, message Carbons are then not received.
New messages from Conversations got Gajim do then appear in Gajim, but messages from ChatSecure arrive only at Conversations but not at Gajim, until the other person edits the fingerprints list in ChatSecure to manually re-enable the Gajim fingerprint.

It would be very desirable if, like Conversations does, also ChatSecure would automatically re-trust the key Gajim key when it is re-added to the receivers list after having been offline (and after a possible prosody restart?!).

@LeoniePhiline
Copy link

@LeoniePhiline LeoniePhiline commented Feb 27, 2019

Apart from that, thank you so much for your fantastic work!

@mimi89999
Copy link
Contributor

@mimi89999 mimi89999 commented Feb 27, 2019

@LeoniePhiline Please make sure that your server is running Prosody >= 0.11.0 as it is the first version that introduced support for persistent nodes after server restart. In a commit I made recently it XMPPFramework robbiehanson/XMPPFramework@4c060bf , I added setting the persist_items items option to true to make sure that they will be persistent on the server.

@anu12anu12
Copy link

@anu12anu12 anu12anu12 commented May 16, 2020

I am using Mangoose IM. I am using the version (3.6.1)of it. Still for me keys are getting removed by the server in Chatsecure. In Android I am using Conversations. And I am always replacing the keys in Android and maintain one key in Android client. Still no clue. Will taking robbiehanson/XMPPFramework@4c060bf will solve the issue? Do you have any suggestion what changes are required in MangooseIm server.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
10 participants