New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Group chat #1

Closed
abika opened this Issue Jul 23, 2014 · 50 comments

Comments

Projects
None yet
6 participants
@abika
Member

abika commented Jul 23, 2014

See kontalk/androidclient#179 and kontalk/tigase-extension#5.
The basic data model and GUI support this but the real implementation depends on the decisions for the "Kontalk vs. XMPP" protocol specifications.

Can't do anything right now.

@abika abika added the enhancement label Jul 23, 2014

@daniele-athome

This comment has been minimized.

Show comment
Hide comment
@daniele-athome

daniele-athome Jul 23, 2014

Member

There is a whole debate that can be opened here. MUC is a server-assisted (and server-centric) group chat, very bad for end-to-end encryption. Some standards are being drafted during the past few months, but there isn't really something useful yet. I'd like to avoid that, mainly because of the "server-centric" part.

What I have in mind is using some sort of multicast system to deliver messages using a previously agreed shared group key for all the members of the group, with a minimal assistance from the server at most (in theory that should not even needed). What do you think? What else do you have in mind apart from MUC?

Member

daniele-athome commented Jul 23, 2014

There is a whole debate that can be opened here. MUC is a server-assisted (and server-centric) group chat, very bad for end-to-end encryption. Some standards are being drafted during the past few months, but there isn't really something useful yet. I'd like to avoid that, mainly because of the "server-centric" part.

What I have in mind is using some sort of multicast system to deliver messages using a previously agreed shared group key for all the members of the group, with a minimal assistance from the server at most (in theory that should not even needed). What do you think? What else do you have in mind apart from MUC?

@abika

This comment has been minimized.

Show comment
Hide comment
@abika

abika Jul 23, 2014

Member

in short: MUC can't be used. Too complicated and doesn't meet our requirements.

What the user expects:

  • create a group chat and send messages to participants that are currently unavailable (not connected), they will receive the messages later when connecting
  • add/remove participants on the fly

This is common in other mobile messengers and I really believe people won't use Kontalk instead if there are downsides here. (And usability is what makes Kontalk superior to native XMPP clients).

I guess you have read https://whispersystems.org/blog/private-groups/ . It's funny to read, describing a perfect security protocol, stating this can not be applied here, and then going on with a much simpler (and more insecure) approach.

So, as I don't see a real alternative and I'm a friend of the KISS principle anyway, make it real simple: Send each message individual encrypted to each participant.

Later, and specially for sending big content other than simple text, this can be advanced: Send a symmetric encrypted message once, together with (asymmetric) encrypted decryption keys for each participant. The server needs to distribute them.

What is needed right now is only a protocol extension for group-id, -name, -participants (and add/remove tasks).

Member

abika commented Jul 23, 2014

in short: MUC can't be used. Too complicated and doesn't meet our requirements.

What the user expects:

  • create a group chat and send messages to participants that are currently unavailable (not connected), they will receive the messages later when connecting
  • add/remove participants on the fly

This is common in other mobile messengers and I really believe people won't use Kontalk instead if there are downsides here. (And usability is what makes Kontalk superior to native XMPP clients).

I guess you have read https://whispersystems.org/blog/private-groups/ . It's funny to read, describing a perfect security protocol, stating this can not be applied here, and then going on with a much simpler (and more insecure) approach.

So, as I don't see a real alternative and I'm a friend of the KISS principle anyway, make it real simple: Send each message individual encrypted to each participant.

Later, and specially for sending big content other than simple text, this can be advanced: Send a symmetric encrypted message once, together with (asymmetric) encrypted decryption keys for each participant. The server needs to distribute them.

What is needed right now is only a protocol extension for group-id, -name, -participants (and add/remove tasks).

@daniele-athome

This comment has been minimized.

Show comment
Hide comment
@daniele-athome

daniele-athome Jul 24, 2014

Member

I thought about doing some sort of "multiple-unicast" thing as a first step, but I discarded it because I thought it would be replaced by something more robust and safe. Besides, it's an approach and we can use it speed up the inclusion of a group chat feature in Kontalk.
We'll have to handle group security and metadata. Being all client-side (that is, multiple unicast messages), I'd prefer not to let the server handle those things, we shall leave all to the clients (remember: Kontalk is a distributed network - the less we store, the better it scales).

As for security, a first approach would be to generate a random hash identifying the group (generated by the user that creates the group). This hash will be broadcasted to all participants and will be attached to every message intended to the group. I know it's not so "secure", but it's a first approach without using additional keys. What else do you have in mind?

The downside of this approach is that anyone in the group can be the group admin: anyone can invite new users (knowing the group ID) and change the name (the name would be broadcasted with an iq stanza). But I guess it's fine for a "democratic" group chat. This is not IRC :-)

Of course there is no XEP for any of this. I'm really not keen to propose this one to the Council... they have their so unique vision about certain aspects. It shall be a very simple XEP, similar to the <thread/> part in RFC 6121 (although somewhat with different purpose).

Member

daniele-athome commented Jul 24, 2014

I thought about doing some sort of "multiple-unicast" thing as a first step, but I discarded it because I thought it would be replaced by something more robust and safe. Besides, it's an approach and we can use it speed up the inclusion of a group chat feature in Kontalk.
We'll have to handle group security and metadata. Being all client-side (that is, multiple unicast messages), I'd prefer not to let the server handle those things, we shall leave all to the clients (remember: Kontalk is a distributed network - the less we store, the better it scales).

As for security, a first approach would be to generate a random hash identifying the group (generated by the user that creates the group). This hash will be broadcasted to all participants and will be attached to every message intended to the group. I know it's not so "secure", but it's a first approach without using additional keys. What else do you have in mind?

The downside of this approach is that anyone in the group can be the group admin: anyone can invite new users (knowing the group ID) and change the name (the name would be broadcasted with an iq stanza). But I guess it's fine for a "democratic" group chat. This is not IRC :-)

Of course there is no XEP for any of this. I'm really not keen to propose this one to the Council... they have their so unique vision about certain aspects. It shall be a very simple XEP, similar to the <thread/> part in RFC 6121 (although somewhat with different purpose).

@daniele-athome

This comment has been minimized.

Show comment
Hide comment
@daniele-athome

daniele-athome Aug 26, 2014

Member

Going on on my previous comment... I'd like not to add more metadata to the server. Since Kontalk initial implementation of group chats will be more of a "broadcast" approach, there is in fact no need for metadata to be stored by servers.

We need a protocol for clients to "agree" on a conversation (i.e. a group) and start exchanging messages. We can split it into two main parts:

  1. group agreement (opening, joining, parting)
  2. chat messages exchange

The biggest problem

The problem with group chats is that authorization is a bit different: a server must know who is part of a group in order to forward a message.
Imagine this scenario: you want to write to the group. So you send 3 messages because you are 4 in your group (including yourself) to the other users. But one of them has not authorized you. Technically, the message would be silently discarded because of lack of authorization. Instead, it should deliver it nonetheless because it's part of the group chat.
Also by not having metadata on the servers, a group can't have an admin (the creator who can invite or kick out users).

Any idea?

Member

daniele-athome commented Aug 26, 2014

Going on on my previous comment... I'd like not to add more metadata to the server. Since Kontalk initial implementation of group chats will be more of a "broadcast" approach, there is in fact no need for metadata to be stored by servers.

We need a protocol for clients to "agree" on a conversation (i.e. a group) and start exchanging messages. We can split it into two main parts:

  1. group agreement (opening, joining, parting)
  2. chat messages exchange

The biggest problem

The problem with group chats is that authorization is a bit different: a server must know who is part of a group in order to forward a message.
Imagine this scenario: you want to write to the group. So you send 3 messages because you are 4 in your group (including yourself) to the other users. But one of them has not authorized you. Technically, the message would be silently discarded because of lack of authorization. Instead, it should deliver it nonetheless because it's part of the group chat.
Also by not having metadata on the servers, a group can't have an admin (the creator who can invite or kick out users).

Any idea?

@abika

This comment has been minimized.

Show comment
Hide comment
@abika

abika Aug 27, 2014

Member

About the authorization: I actually don't know how the server handles that. You can't send messages to a user that is unknown to you (=not in your roster), correct? One possibility is to postpone sending the specific group messages to this user until the authorization is made(?)
Blocking group messages from a user shout be possible somehow, too.

About the admin: What's the problem if the group creator is the admin? He/she can manage the user in the group by sending the respective commands to all user (including the new/old ones). Commands from other user will be ignored.
This can lead to minor inconsistencies: Alice is admin but Bob (already in the group) invites Charles who doesn't know that Bob isn't admin. Now everybody except Charles knows something is wrong and can handle that (ignore messages from Charles/reply with error message).

That somebody is forwarding messages to someone who isn't in the group can't be prevented but this is never possible.

Member

abika commented Aug 27, 2014

About the authorization: I actually don't know how the server handles that. You can't send messages to a user that is unknown to you (=not in your roster), correct? One possibility is to postpone sending the specific group messages to this user until the authorization is made(?)
Blocking group messages from a user shout be possible somehow, too.

About the admin: What's the problem if the group creator is the admin? He/she can manage the user in the group by sending the respective commands to all user (including the new/old ones). Commands from other user will be ignored.
This can lead to minor inconsistencies: Alice is admin but Bob (already in the group) invites Charles who doesn't know that Bob isn't admin. Now everybody except Charles knows something is wrong and can handle that (ignore messages from Charles/reply with error message).

That somebody is forwarding messages to someone who isn't in the group can't be prevented but this is never possible.

@abika

This comment has been minimized.

Show comment
Hide comment
@abika

abika Aug 27, 2014

Member

Second thought: We agree that in the first version group messages are just normal XMPP messages with a custom group extension (including a chat ID or something)? To make sure were not talking at cross-purposes.

Member

abika commented Aug 27, 2014

Second thought: We agree that in the first version group messages are just normal XMPP messages with a custom group extension (including a chat ID or something)? To make sure were not talking at cross-purposes.

@daniele-athome

This comment has been minimized.

Show comment
Hide comment
@daniele-athome

daniele-athome Aug 27, 2014

Member

About the authorization: I actually don't know how the server handles that. You can't send messages to a user that is unknown to you (=not in your roster), correct? One possibility is to postpone sending the specific group messages to this user until the authorization is made(?)

Authorization is a subscription process just like on XMPP. So what I'm saying here is that if the server doesn't know about groups (because let's say that, hypothetically, in order to save server resources, we don't store anything on the server about groups), Alice hasn't authorized Bob because she never met him in her life, so even if they are in the same group (e.g. a 3rd party that knows both Alice and Bob invited them), they can't communicate until Alice authorizes 1-to-1 chat first.

About the admin: What's the problem if the group creator is the admin?

There is no problem, but as I said before, if the server doesn't know about groups, it doesn't know about group admin either.

Second thought: We agree that in the first version group messages are just normal XMPP messages with a custom group extension (including a chat ID or something)? To make sure were not talking at cross-purposes.

Same authorization problem: Charlie invites Alice and Bob to a group chat, despite Alice doesn't know Bob. Alice isn't authorized to chat with Bob and vice-versa.

It's not that I absolutely don't want to store anything on the server about groups, it's just that I'd like to examine all the possible choices before deciding to store group information on Kontalk servers. The less we know about our users, the better - both for privacy and resource saving.

Member

daniele-athome commented Aug 27, 2014

About the authorization: I actually don't know how the server handles that. You can't send messages to a user that is unknown to you (=not in your roster), correct? One possibility is to postpone sending the specific group messages to this user until the authorization is made(?)

Authorization is a subscription process just like on XMPP. So what I'm saying here is that if the server doesn't know about groups (because let's say that, hypothetically, in order to save server resources, we don't store anything on the server about groups), Alice hasn't authorized Bob because she never met him in her life, so even if they are in the same group (e.g. a 3rd party that knows both Alice and Bob invited them), they can't communicate until Alice authorizes 1-to-1 chat first.

About the admin: What's the problem if the group creator is the admin?

There is no problem, but as I said before, if the server doesn't know about groups, it doesn't know about group admin either.

Second thought: We agree that in the first version group messages are just normal XMPP messages with a custom group extension (including a chat ID or something)? To make sure were not talking at cross-purposes.

Same authorization problem: Charlie invites Alice and Bob to a group chat, despite Alice doesn't know Bob. Alice isn't authorized to chat with Bob and vice-versa.

It's not that I absolutely don't want to store anything on the server about groups, it's just that I'd like to examine all the possible choices before deciding to store group information on Kontalk servers. The less we know about our users, the better - both for privacy and resource saving.

@abika

This comment has been minimized.

Show comment
Hide comment
@abika

abika Aug 27, 2014

Member

I still don't see the need for storing anything on the server regarding groups.

they can't communicate until Alice authorizes 1-to-1 chat first.

Yes, so the client should notice that and ask for authorization. All messages to this user will be saved first and send after authorization is done.
There is no alternative: Maybe I got invited to a group and there is a guy I don't like. I don't want to get his messages or send him any.

Member

abika commented Aug 27, 2014

I still don't see the need for storing anything on the server regarding groups.

they can't communicate until Alice authorizes 1-to-1 chat first.

Yes, so the client should notice that and ask for authorization. All messages to this user will be saved first and send after authorization is done.
There is no alternative: Maybe I got invited to a group and there is a guy I don't like. I don't want to get his messages or send him any.

@abika

This comment has been minimized.

Show comment
Hide comment
@abika

abika Aug 27, 2014

Member

Ok a bigger example: Alice creates a group with Bob and Charles which means she sends two messages:

  • <message from=alice to=bob> <body> Hi there! </body> <kon_group_ext id=1234 member=alice,bob,charles></message>
  • <message from=alice to=charles> <body> Hi there! </body> <kon_group_ext id=1234 member=alice,bob,charles>></message>

Bob and Charles don't know each other and are offline first. But now Bob gets online gets the message from Alice, realizes he doesn't know Charles and requests authorization from him. In the meantime he already sends a reply to the group (again two messages). The first to Alice is send immediately the one to Charles is postponed.

  • <message from=bob to=alice> <body> Hi! Nice to meet you Charles!</body> <kon_group_ext id=1234></message>

After some time also Charles gets online, the message from Alice and authorization is made with Bob in both ways. After that, Bob sends the message already send to Alice to Charles now:

  • <message from=bob to=charles> <body> Hi! Nice to meet you Charles!</body> <kon_group_ext id=1234></message>

Now everything is solved and they talk about cryptography until the end of the world...

Member

abika commented Aug 27, 2014

Ok a bigger example: Alice creates a group with Bob and Charles which means she sends two messages:

  • <message from=alice to=bob> <body> Hi there! </body> <kon_group_ext id=1234 member=alice,bob,charles></message>
  • <message from=alice to=charles> <body> Hi there! </body> <kon_group_ext id=1234 member=alice,bob,charles>></message>

Bob and Charles don't know each other and are offline first. But now Bob gets online gets the message from Alice, realizes he doesn't know Charles and requests authorization from him. In the meantime he already sends a reply to the group (again two messages). The first to Alice is send immediately the one to Charles is postponed.

  • <message from=bob to=alice> <body> Hi! Nice to meet you Charles!</body> <kon_group_ext id=1234></message>

After some time also Charles gets online, the message from Alice and authorization is made with Bob in both ways. After that, Bob sends the message already send to Alice to Charles now:

  • <message from=bob to=charles> <body> Hi! Nice to meet you Charles!</body> <kon_group_ext id=1234></message>

Now everything is solved and they talk about cryptography until the end of the world...

@daniele-athome

This comment has been minimized.

Show comment
Hide comment
@daniele-athome

daniele-athome Aug 28, 2014

Member

Ok then, authorization problem is solved. There will be the need for a custom UI implementation for group chat authorization (i.e. different than the normal authorization workflow - imagine 10 new users in a opening group - 10 authorization requests in the conversation list. No way :-) ). But I can work on that in due time.

The protocol

We'll need something to include:

  • a group ID
  • participants list

The group ID is simply a random sequence of characters used by clients to identify the group (any idea if we should derive it from something about the group? i.e. a hash of something, but I don't know how useful that could be).
The participants list shall be included in every message.

<message from="..." to="...">
  <body>...</body>
  <group xmlns="...">
    <id>...</id>
    <members>
      <jid>...</jid>
      <jid>...</jid>
      <jid>...</jid>
      ...
    </members>
  </group>
</message>

The group part must be added after encryption because the server must be able to read it: it will queue unauthorized messages for a time, waiting for authorization (the normal behaviour for 1-to-1 messages is to silently discard them).

This might actually become a XEP. What do you think? Do you think the Council will approve such blasphemy? :-)

Member

daniele-athome commented Aug 28, 2014

Ok then, authorization problem is solved. There will be the need for a custom UI implementation for group chat authorization (i.e. different than the normal authorization workflow - imagine 10 new users in a opening group - 10 authorization requests in the conversation list. No way :-) ). But I can work on that in due time.

The protocol

We'll need something to include:

  • a group ID
  • participants list

The group ID is simply a random sequence of characters used by clients to identify the group (any idea if we should derive it from something about the group? i.e. a hash of something, but I don't know how useful that could be).
The participants list shall be included in every message.

<message from="..." to="...">
  <body>...</body>
  <group xmlns="...">
    <id>...</id>
    <members>
      <jid>...</jid>
      <jid>...</jid>
      <jid>...</jid>
      ...
    </members>
  </group>
</message>

The group part must be added after encryption because the server must be able to read it: it will queue unauthorized messages for a time, waiting for authorization (the normal behaviour for 1-to-1 messages is to silently discard them).

This might actually become a XEP. What do you think? Do you think the Council will approve such blasphemy? :-)

@abika

This comment has been minimized.

Show comment
Hide comment
@abika

abika Aug 28, 2014

Member

The group part must be added after encryption because the server must be able to read it: it will queue unauthorized messages for a time, waiting for authorization (the normal behaviour for 1-to-1 messages is to silently discard them).

Isn't it better, if the client handles that? Then, the server doesn't need to be modified at all and the incompatibility to standard XMPP is reduced to a minimum.

This might actually become a XEP. What do you think? Do you think the Council will approve such blasphemy? :-)

A simple and easy to implement competitor to the almighty MUC design ( https://community.igniterealtime.org/thread/48020 )?
Seriously: We first have to prove that it works in real-live with a user base. And then, let's see...

Member

abika commented Aug 28, 2014

The group part must be added after encryption because the server must be able to read it: it will queue unauthorized messages for a time, waiting for authorization (the normal behaviour for 1-to-1 messages is to silently discard them).

Isn't it better, if the client handles that? Then, the server doesn't need to be modified at all and the incompatibility to standard XMPP is reduced to a minimum.

This might actually become a XEP. What do you think? Do you think the Council will approve such blasphemy? :-)

A simple and easy to implement competitor to the almighty MUC design ( https://community.igniterealtime.org/thread/48020 )?
Seriously: We first have to prove that it works in real-live with a user base. And then, let's see...

@daniele-athome

This comment has been minimized.

Show comment
Hide comment
@daniele-athome

daniele-athome Aug 29, 2014

Member

Isn't it better, if the client handles that? Then, the server doesn't need to be modified at all and the incompatibility to standard XMPP is reduced to a minimum.

What do you mean? That if a user is authorized later in a group chat, the sender should send the "missing" messages again?

Continuing on my previous topic about unauthorized messages being silently discarded: it's a countermeasure I wrote for spammers/annoying people. But it probably wouldn't be such a bad decision to delay 1-to-1 unauthorized messages for sometime until the user is authorized and then deliver them (or discard them if they expire). This way group chat XML tags can be encrypted too.
By the way, many people were concerned about messages sent in the time span between authorization request sent and authorization request accepted: those messages simply get lost. By changing this behaviour, messages will be simply queued and delivered if authorization process is complete, or discarded after sometime otherwise (something that standard XMPP doesn't do).

Member

daniele-athome commented Aug 29, 2014

Isn't it better, if the client handles that? Then, the server doesn't need to be modified at all and the incompatibility to standard XMPP is reduced to a minimum.

What do you mean? That if a user is authorized later in a group chat, the sender should send the "missing" messages again?

Continuing on my previous topic about unauthorized messages being silently discarded: it's a countermeasure I wrote for spammers/annoying people. But it probably wouldn't be such a bad decision to delay 1-to-1 unauthorized messages for sometime until the user is authorized and then deliver them (or discard them if they expire). This way group chat XML tags can be encrypted too.
By the way, many people were concerned about messages sent in the time span between authorization request sent and authorization request accepted: those messages simply get lost. By changing this behaviour, messages will be simply queued and delivered if authorization process is complete, or discarded after sometime otherwise (something that standard XMPP doesn't do).

@abika

This comment has been minimized.

Show comment
Hide comment
@abika

abika Aug 29, 2014

Member

What do you mean? That if a user is authorized later in a group chat, the sender should send the "missing" messages again?

Exactly. For the reasons you already mentioned, and I like the idea of the client to have control over what/when to send messages (again).

With the option to block arbitrary user is authorization even needed for sending messages? As far as I know the Jabber network works pretty good without this requirement.

BTW, in my opinion it would be a killer feature if it's simply possible to communicate with other Jabber/XMPP clients (unencrypted or after XEP-0191). The potential user base is a lot bigger and it would help popularize both Kontalk and the Jabber Network. When developing the desktop client I already had that at the back of my mind. (discussing that becomes off-topic and is better off in another thread)

Member

abika commented Aug 29, 2014

What do you mean? That if a user is authorized later in a group chat, the sender should send the "missing" messages again?

Exactly. For the reasons you already mentioned, and I like the idea of the client to have control over what/when to send messages (again).

With the option to block arbitrary user is authorization even needed for sending messages? As far as I know the Jabber network works pretty good without this requirement.

BTW, in my opinion it would be a killer feature if it's simply possible to communicate with other Jabber/XMPP clients (unencrypted or after XEP-0191). The potential user base is a lot bigger and it would help popularize both Kontalk and the Jabber Network. When developing the desktop client I already had that at the back of my mind. (discussing that becomes off-topic and is better off in another thread)

@daniele-athome

This comment has been minimized.

Show comment
Hide comment
@daniele-athome

daniele-athome Aug 29, 2014

Member

Exactly. For the reasons you already mentioned, and I like the idea of the client to have control over what/when to send messages (again).

Considering the user experience about invitation, it's really bad for users to see messages sent prior to the invitation not being delivered automatically after the invitation is accepted. In the meantime I've opened a ticket on xmppserver about this (kontalk/xmppserver#103).
Also, as far as I recall, XMPP doesn't regulate message delivery policy in the sense described here.

With the option to block arbitrary user is authorization even needed for sending messages? As far as I know the Jabber network works pretty good without this requirement.

The invitation system follows the standard presence subscription flow. That workflow is used everyday on the Jabber network.
Although having a harasser to spam you before you can even block him can be unsettling.

BTW, in my opinion it would be a killer feature if it's simply possible to communicate with other Jabber/XMPP clients (unencrypted or after XEP-0191). The potential user base is a lot bigger and it would help popularize both Kontalk and the Jabber Network. When developing the desktop client I already had that at the back of my mind. (discussing that becomes off-topic and is better off in another thread)

That's always been in Kontalk plans. The problem is the s2s server module which is still unfinished and needs massive testing. But eventually (I hope before releasing stable 3.0) Kontalk will be part of the XMPP federation.

Member

daniele-athome commented Aug 29, 2014

Exactly. For the reasons you already mentioned, and I like the idea of the client to have control over what/when to send messages (again).

Considering the user experience about invitation, it's really bad for users to see messages sent prior to the invitation not being delivered automatically after the invitation is accepted. In the meantime I've opened a ticket on xmppserver about this (kontalk/xmppserver#103).
Also, as far as I recall, XMPP doesn't regulate message delivery policy in the sense described here.

With the option to block arbitrary user is authorization even needed for sending messages? As far as I know the Jabber network works pretty good without this requirement.

The invitation system follows the standard presence subscription flow. That workflow is used everyday on the Jabber network.
Although having a harasser to spam you before you can even block him can be unsettling.

BTW, in my opinion it would be a killer feature if it's simply possible to communicate with other Jabber/XMPP clients (unencrypted or after XEP-0191). The potential user base is a lot bigger and it would help popularize both Kontalk and the Jabber Network. When developing the desktop client I already had that at the back of my mind. (discussing that becomes off-topic and is better off in another thread)

That's always been in Kontalk plans. The problem is the s2s server module which is still unfinished and needs massive testing. But eventually (I hope before releasing stable 3.0) Kontalk will be part of the XMPP federation.

@abika

This comment has been minimized.

Show comment
Hide comment
@abika

abika Aug 30, 2014

Member

ok, whatever you think works better. But consider that you may want to send a message to someone you are requesting authorization from. So that user knows who you are.

Member

abika commented Aug 30, 2014

ok, whatever you think works better. But consider that you may want to send a message to someone you are requesting authorization from. So that user knows who you are.

@daniele-athome

This comment has been minimized.

Show comment
Hide comment
@daniele-athome

daniele-athome Aug 31, 2014

Member

That's one feature I'd like to include one day. For now the name of the user is displayed (the name you are asked for during registration). I'll open an issue.

Member

daniele-athome commented Aug 31, 2014

That's one feature I'd like to include one day. For now the name of the user is displayed (the name you are asked for during registration). I'll open an issue.

@daniele-athome daniele-athome referenced this issue Jan 23, 2015

Closed

Group chat #5

@abika

This comment has been minimized.

Show comment
Hide comment
@abika

abika Apr 20, 2015

Member

can we clarify some points again?:

  • What should be handled by the server?
    I would say: nothing, right now. We only need to keep in mind that this can be changed in the future: The server gets one message and forwards it to each group member, encrypted individually
  • What protocol specifications do we need?
    Constraints: simple and fail-save. The worst that can happen is that after everything is specified and implemented, this one unthinkable use-case pop ups that breaks everything
    • group ID: A must. A global unique identifier with no further derived meaning - like the message or thread ID
    • adding a list with all participants to each message?
      PRO: reduced protocol, membership information unambiguous for every participant (and the server later)
      CONTRA: overhead in terms of message size and message parsing (suitable with 50+ participants?). Shannon would kill us :~
    • and connected to this: required protocol commands?
      create (maybe),
      delete (no),
      add (member) (depends on above),
      remove (member) (depends on above),
      advanced:
      subject,
      admin

Together with the usage of public/subscriber and blocking specs I still get a headache here. But as long as we keep it simple and consider the group-chat extension only as the interpretation of this message was also sended to... we should be on the safe side (eh, right?).

Member

abika commented Apr 20, 2015

can we clarify some points again?:

  • What should be handled by the server?
    I would say: nothing, right now. We only need to keep in mind that this can be changed in the future: The server gets one message and forwards it to each group member, encrypted individually
  • What protocol specifications do we need?
    Constraints: simple and fail-save. The worst that can happen is that after everything is specified and implemented, this one unthinkable use-case pop ups that breaks everything
    • group ID: A must. A global unique identifier with no further derived meaning - like the message or thread ID
    • adding a list with all participants to each message?
      PRO: reduced protocol, membership information unambiguous for every participant (and the server later)
      CONTRA: overhead in terms of message size and message parsing (suitable with 50+ participants?). Shannon would kill us :~
    • and connected to this: required protocol commands?
      create (maybe),
      delete (no),
      add (member) (depends on above),
      remove (member) (depends on above),
      advanced:
      subject,
      admin

Together with the usage of public/subscriber and blocking specs I still get a headache here. But as long as we keep it simple and consider the group-chat extension only as the interpretation of this message was also sended to... we should be on the safe side (eh, right?).

@daniele-athome

This comment has been minimized.

Show comment
Hide comment
@daniele-athome

daniele-athome Apr 21, 2015

Member

The server gets one message and forwards it to each group member, encrypted individually

Just a note on this: to support encryption, we could use an approach similar to what OpenPGP does: encrypt the message with a symmetric key and then send that key encrypted multiple times (one for each user in the group) with the users' public keys.

This key could be generated by the groups's creator and then sent to all participants (encrypted with each one's public key of course). This new key will be used by the participants to encrypt messages for the group and certify that they are part of the group. This could be used as some sort of an "invitation system" (of course this would mean anyone can invite anyone, but that's fine).

group ID: A must. A global unique identifier with no further derived meaning - like the message or thread ID

We could use an hash of the group public key I described above, but that's just a hint anyway.

adding a list with all participants to each message?

Yes, Shannon will certainly kill us. I don't have other ideas though.
However, by using a shared group key, we'll avoid encrypting the message multiple times for every user (which will increase its size even more).

What do you think?

Member

daniele-athome commented Apr 21, 2015

The server gets one message and forwards it to each group member, encrypted individually

Just a note on this: to support encryption, we could use an approach similar to what OpenPGP does: encrypt the message with a symmetric key and then send that key encrypted multiple times (one for each user in the group) with the users' public keys.

This key could be generated by the groups's creator and then sent to all participants (encrypted with each one's public key of course). This new key will be used by the participants to encrypt messages for the group and certify that they are part of the group. This could be used as some sort of an "invitation system" (of course this would mean anyone can invite anyone, but that's fine).

group ID: A must. A global unique identifier with no further derived meaning - like the message or thread ID

We could use an hash of the group public key I described above, but that's just a hint anyway.

adding a list with all participants to each message?

Yes, Shannon will certainly kill us. I don't have other ideas though.
However, by using a shared group key, we'll avoid encrypting the message multiple times for every user (which will increase its size even more).

What do you think?

@abika

This comment has been minimized.

Show comment
Hide comment
@abika

abika Apr 21, 2015

Member

Just a note on this: to support encryption, we could use an approach similar to what OpenPGP does: encrypt the message with a symmetric key and then send that key encrypted multiple times (one for each user in the group) with the users' public keys.

again some bullet points (correct me if I'm wrong somewhere):

  • this is was the guys from TextSecure are describing as even further optimization, without implementing it
  • it does not involve the server at any point which is a yeah
  • and now imo a big mistake:

This could be used as some sort of an "invitation system" (of course this would mean anyone can invite anyone, but that's fine)

really? There is good reason why other messengers have an admin role here (unfortunately friendship is not transitive). This can also lead to membership conflicts/ambiguity, headache again.
We should only let the creator allow to handle membership administration, and not derive anything from somebody having the group key.

  • adding a list with all participants to each message?
    Why are dedicated add/remove commands not an option? (just asking, still unsure)
Member

abika commented Apr 21, 2015

Just a note on this: to support encryption, we could use an approach similar to what OpenPGP does: encrypt the message with a symmetric key and then send that key encrypted multiple times (one for each user in the group) with the users' public keys.

again some bullet points (correct me if I'm wrong somewhere):

  • this is was the guys from TextSecure are describing as even further optimization, without implementing it
  • it does not involve the server at any point which is a yeah
  • and now imo a big mistake:

This could be used as some sort of an "invitation system" (of course this would mean anyone can invite anyone, but that's fine)

really? There is good reason why other messengers have an admin role here (unfortunately friendship is not transitive). This can also lead to membership conflicts/ambiguity, headache again.
We should only let the creator allow to handle membership administration, and not derive anything from somebody having the group key.

  • adding a list with all participants to each message?
    Why are dedicated add/remove commands not an option? (just asking, still unsure)
@daniele-athome

This comment has been minimized.

Show comment
Hide comment
@daniele-athome

daniele-athome Apr 21, 2015

Member

adding a list with all participants to each message? Why are dedicated add/remove commands not an option? (just asking, still unsure)

Because it would make it just like MUC, wouldn't it? A single server will be responsible for hosting a group, with all its rules (members, admin role, etc.). Unless I'm not fully getting your point.
Isn't add/remove/other_command handled by the server? Unless you want them to be handled by the group creator client, am I getting this right?

Member

daniele-athome commented Apr 21, 2015

adding a list with all participants to each message? Why are dedicated add/remove commands not an option? (just asking, still unsure)

Because it would make it just like MUC, wouldn't it? A single server will be responsible for hosting a group, with all its rules (members, admin role, etc.). Unless I'm not fully getting your point.
Isn't add/remove/other_command handled by the server? Unless you want them to be handled by the group creator client, am I getting this right?

@abika

This comment has been minimized.

Show comment
Hide comment
@abika

abika Apr 21, 2015

Member

Unless you want them to be handled by the group creator client

That's what I mean. Server is not involved, the group creator sends add/remove/... messages to all participants.

Member

abika commented Apr 21, 2015

Unless you want them to be handled by the group creator client

That's what I mean. Server is not involved, the group creator sends add/remove/... messages to all participants.

@daniele-athome

This comment has been minimized.

Show comment
Hide comment
@daniele-athome

daniele-athome Apr 21, 2015

Member

Oh I see. Sorry I wasn't getting that :-)
What about authorization? I mean anyone could send add/remove commands at their will, right?

Member

daniele-athome commented Apr 21, 2015

Oh I see. Sorry I wasn't getting that :-)
What about authorization? I mean anyone could send add/remove commands at their will, right?

@abika

This comment has been minimized.

Show comment
Hide comment
@abika

abika Apr 21, 2015

Member

I mean anyone could send add/remove commands at their will, right?

Yes, but clients should ignore commands not coming from the group creator (we HAVE to rely on the clients, when the server is not handling this). Already discussed above

Member

abika commented Apr 21, 2015

I mean anyone could send add/remove commands at their will, right?

Yes, but clients should ignore commands not coming from the group creator (we HAVE to rely on the clients, when the server is not handling this). Already discussed above

@daniele-athome

This comment has been minimized.

Show comment
Hide comment
@daniele-athome

daniele-athome Apr 22, 2015

Member

Ok then, I will write a wiki page soon with a protocol draft. Let's see if I got it right:

  • message extension: <group> with an ID to every message
  • add/remove commands sent by the admin to all participants, sent as messages (body will include a description displayed by incompatible clients)
  • create command (initial create-and-join command)
  • part command (leave group; an announcement to all other users that I'm leaving the group)
  • since messages are sent individually to each member by each clients, using a shared key doesn't really make sense anymore, does it?

Of course clients supporting this protocol will advertise it through service discovery.

Only thing that fits wrong with this approach is blocking support. Blocking is for JIDs, and JIDs are entities supported and handled by a server. If group chat is handled fully by clients, no JID entity would exist that represents a group. Unless we implement block group feature as "ignore and discard messages from this group". What do you think?

Member

daniele-athome commented Apr 22, 2015

Ok then, I will write a wiki page soon with a protocol draft. Let's see if I got it right:

  • message extension: <group> with an ID to every message
  • add/remove commands sent by the admin to all participants, sent as messages (body will include a description displayed by incompatible clients)
  • create command (initial create-and-join command)
  • part command (leave group; an announcement to all other users that I'm leaving the group)
  • since messages are sent individually to each member by each clients, using a shared key doesn't really make sense anymore, does it?

Of course clients supporting this protocol will advertise it through service discovery.

Only thing that fits wrong with this approach is blocking support. Blocking is for JIDs, and JIDs are entities supported and handled by a server. If group chat is handled fully by clients, no JID entity would exist that represents a group. Unless we implement block group feature as "ignore and discard messages from this group". What do you think?

@abika

This comment has been minimized.

Show comment
Hide comment
@abika

abika May 6, 2015

Member

since messages are sent individually to each member by each clients, using a shared key doesn't really make sense anymore, does it?

No, currently not. But the server could be extended with an (independent?) protocol, so that it gets a list of addresses, one (encrypted) message and simply distributes it.

Everything else looks fine by me.

Unless we implement block group feature as "ignore and discard messages from this group". What do you think?

I would say special blocking of groups is not required. Other messengers don't have/need that, and the current blocking is sufficient: If you don't want get normal messages from another user, you usually also don't wanna get group invitations.

Of course, we need to take care of the case if the group creator or group members are blocked. But the clients should be able to handle that themselves.

Member

abika commented May 6, 2015

since messages are sent individually to each member by each clients, using a shared key doesn't really make sense anymore, does it?

No, currently not. But the server could be extended with an (independent?) protocol, so that it gets a list of addresses, one (encrypted) message and simply distributes it.

Everything else looks fine by me.

Unless we implement block group feature as "ignore and discard messages from this group". What do you think?

I would say special blocking of groups is not required. Other messengers don't have/need that, and the current blocking is sufficient: If you don't want get normal messages from another user, you usually also don't wanna get group invitations.

Of course, we need to take care of the case if the group creator or group members are blocked. But the clients should be able to handle that themselves.

@daniele-athome

This comment has been minimized.

Show comment
Hide comment
@daniele-athome

daniele-athome May 6, 2015

Member

No, currently not. But the server could be extended with an (independent?) protocol, so that it gets a list of addresses, one (encrypted) message and simply distributes it.

Well, shared secrets are always dangerous in one way or another, so if clients have the power here, let's encrypt every message with only the recipient's key.

I'll draft something as soon as I can.

Member

daniele-athome commented May 6, 2015

No, currently not. But the server could be extended with an (independent?) protocol, so that it gets a list of addresses, one (encrypted) message and simply distributes it.

Well, shared secrets are always dangerous in one way or another, so if clients have the power here, let's encrypt every message with only the recipient's key.

I'll draft something as soon as I can.

@abika

This comment has been minimized.

Show comment
Hide comment
@abika

abika May 8, 2015

Member

huh? Wasn't it your idea to use a shared symmetric key? If the group creator initially sends it to all members, all senders can send only their one message (together with the list of recipients/group members) -> great traffic reduction.

But anyway, not that important right now

Member

abika commented May 8, 2015

huh? Wasn't it your idea to use a shared symmetric key? If the group creator initially sends it to all members, all senders can send only their one message (together with the list of recipients/group members) -> great traffic reduction.

But anyway, not that important right now

@daniele-athome

This comment has been minimized.

Show comment
Hide comment
@daniele-athome

daniele-athome May 8, 2015

Member

Sorry, yes actually that comes naturally with OpenPGP encryption (when encrypting to multiple recipients that's exactly what GnuPG does). I was confusing it with something else, never mind :-)

Member

daniele-athome commented May 8, 2015

Sorry, yes actually that comes naturally with OpenPGP encryption (when encrypting to multiple recipients that's exactly what GnuPG does). I was confusing it with something else, never mind :-)

@abika abika added the on hold label Jun 12, 2015

@abika abika modified the milestone: v3.1 Jun 29, 2015

@abika abika removed the on hold label Jun 29, 2015

@daniele-athome

This comment has been minimized.

Show comment
Hide comment
@daniele-athome

daniele-athome Jun 29, 2015

Member

I put down some words about the protocol in a wiki page, absolutely in a very draft status:
https://github.com/kontalk/specs/wiki/Group-chat

I'm a little uncertain about the add user command:

(The group creator|Anyone) can add users by sending a group command:

What if the group creator leaves the group? Does the group become orphan?
Also, are we granting the group creator ban privileges?

Member

daniele-athome commented Jun 29, 2015

I put down some words about the protocol in a wiki page, absolutely in a very draft status:
https://github.com/kontalk/specs/wiki/Group-chat

I'm a little uncertain about the add user command:

(The group creator|Anyone) can add users by sending a group command:

What if the group creator leaves the group? Does the group become orphan?
Also, are we granting the group creator ban privileges?

@Manizuca

This comment has been minimized.

Show comment
Hide comment
@Manizuca

Manizuca Jun 29, 2015

Hi, after reading the discussion i ended with this image of the implementation, maybe i added my own ideas :)

  • Alicia creates a group chat with Bob and Catalina, the client sends creates a group ID and a symetric key, then sends the invitations with the ID to Bob and Catalina. They can accept or reject. Blocking the admin will block the group, as the invitation wouldn't be received
    • The admin will have a pending-invitations list, he should be able to cancel pending-invitations in the members-list. There need to be a invitation time limit
  • When someone accept, he will be added to a members-list and the key is sent with the ID and the list as a personal-message. When the second one accepts the message with the key, the ID and the new list will be sent to him as a personal-message, and the change in the list will be sent by the admin to previous member as a group-message, the group-messages will be signed with the symetric key
    • Options for participants list:
      • The group-messages have a field with the list of receivers, the server send the message to everyone in the list. The participants list is managed by the admin. This can help to also implement kontalk/androidclient#105 (the broadcast will have a list of receivers, and will be signed with the private key of the sender)
      • The server has a list of participants, when there is a change in the list, its informed as a group message to everyone, and also to the server
  • Bob accepts the invitation and then sends a message to the group.
  • Catalina accepts the invitation, and she doesn't receive any message
  • Bob gets offline, and tries to send a new message
  • Alicia adds Daniel and Eva to the group, when they accept he send the change to everywhere as a group-message (and to the server if it has the list), and to Daniel and Eva as personal messages with the key
  • Bob gets online, he must check for a change in the members-list before trying again to send the message
  • Catalina gets offline, she tries to send a 20MiB video (assuming that it would be possible in the future to send big files encrypted)
  • Alicia removes Eva and Bob from the group. She sends a message to both at once with the notification, and send the changes in the list as a group-message to every one left in the group
    • She should be able to delete multiple users at once, to avoid recreating the key multiple times.
    • Eva and bob had the group key, so if they can intercept messages they still can decrypt them. So alicia creates a new key and send it as a personal-message to everyone left in the group.
  • Catalina gets online and receives the new key and members-list, then tries again to send the video
    • I dont know if encrypting big files its something that takes to much resources, maybe we can avoid re-encrypting the video with the new key. For this Catalina should check if the message is bigger than some threshole, if is bigger she should use a message extension and add tho the message the old-key encrypted with her private key and the new-key.
    • What needs to happen if the list is included with the message and there is a new/removed member when uploading the video?
  • All the participants should keep the old key until checking for missing messages and decrypting all of them
  • There must be a time limit of time, e.g. if a user never clicks the "download" option in a image, after the limit the old key would be deleted and the download optiod disabled (I also assume the fileserver has a time limit for things like images)
    • Im not sure how actually works if a user recreates his key, he just doesn't receive all the missing messages encypted before the change?
  • If someone leaves a group, he send a group-message with a notification about the change
    • When the admin receives the change, he creates a new key and send it as personal-messages
  • The admin can change the admin of the group by sending a group-message with the changes.
    • The admin can only make the change if is connected, to prevent trying to set a non-member as admin
    • Someone can only leave the group if is connected, to prevent leaving it being admin
    • When the admin leaves the group, he should be asked for the new admin.

Maybe instead of recreating the key with every participant removed, the admin could have the option to manually recreate in and/or it will be recreated after 5 participants removed (with information about when was created the key)

My question is how would this work with using multiple clients, here can help kontalk/androidclient#250, so the other client know the changes. But if the message was missed (timeouted), there should be some information in the roster that includes a timestamp of the key and participants-list... if the timestamp is different the client can request the key to the admin and himself (if is connected in the other client)

And how should this work when using kontalk and a client that doesnt support this?, i suppose that the server will just send the messages to the client with support, so kontalk don't misses the messages

Manizuca commented Jun 29, 2015

Hi, after reading the discussion i ended with this image of the implementation, maybe i added my own ideas :)

  • Alicia creates a group chat with Bob and Catalina, the client sends creates a group ID and a symetric key, then sends the invitations with the ID to Bob and Catalina. They can accept or reject. Blocking the admin will block the group, as the invitation wouldn't be received
    • The admin will have a pending-invitations list, he should be able to cancel pending-invitations in the members-list. There need to be a invitation time limit
  • When someone accept, he will be added to a members-list and the key is sent with the ID and the list as a personal-message. When the second one accepts the message with the key, the ID and the new list will be sent to him as a personal-message, and the change in the list will be sent by the admin to previous member as a group-message, the group-messages will be signed with the symetric key
    • Options for participants list:
      • The group-messages have a field with the list of receivers, the server send the message to everyone in the list. The participants list is managed by the admin. This can help to also implement kontalk/androidclient#105 (the broadcast will have a list of receivers, and will be signed with the private key of the sender)
      • The server has a list of participants, when there is a change in the list, its informed as a group message to everyone, and also to the server
  • Bob accepts the invitation and then sends a message to the group.
  • Catalina accepts the invitation, and she doesn't receive any message
  • Bob gets offline, and tries to send a new message
  • Alicia adds Daniel and Eva to the group, when they accept he send the change to everywhere as a group-message (and to the server if it has the list), and to Daniel and Eva as personal messages with the key
  • Bob gets online, he must check for a change in the members-list before trying again to send the message
  • Catalina gets offline, she tries to send a 20MiB video (assuming that it would be possible in the future to send big files encrypted)
  • Alicia removes Eva and Bob from the group. She sends a message to both at once with the notification, and send the changes in the list as a group-message to every one left in the group
    • She should be able to delete multiple users at once, to avoid recreating the key multiple times.
    • Eva and bob had the group key, so if they can intercept messages they still can decrypt them. So alicia creates a new key and send it as a personal-message to everyone left in the group.
  • Catalina gets online and receives the new key and members-list, then tries again to send the video
    • I dont know if encrypting big files its something that takes to much resources, maybe we can avoid re-encrypting the video with the new key. For this Catalina should check if the message is bigger than some threshole, if is bigger she should use a message extension and add tho the message the old-key encrypted with her private key and the new-key.
    • What needs to happen if the list is included with the message and there is a new/removed member when uploading the video?
  • All the participants should keep the old key until checking for missing messages and decrypting all of them
  • There must be a time limit of time, e.g. if a user never clicks the "download" option in a image, after the limit the old key would be deleted and the download optiod disabled (I also assume the fileserver has a time limit for things like images)
    • Im not sure how actually works if a user recreates his key, he just doesn't receive all the missing messages encypted before the change?
  • If someone leaves a group, he send a group-message with a notification about the change
    • When the admin receives the change, he creates a new key and send it as personal-messages
  • The admin can change the admin of the group by sending a group-message with the changes.
    • The admin can only make the change if is connected, to prevent trying to set a non-member as admin
    • Someone can only leave the group if is connected, to prevent leaving it being admin
    • When the admin leaves the group, he should be asked for the new admin.

Maybe instead of recreating the key with every participant removed, the admin could have the option to manually recreate in and/or it will be recreated after 5 participants removed (with information about when was created the key)

My question is how would this work with using multiple clients, here can help kontalk/androidclient#250, so the other client know the changes. But if the message was missed (timeouted), there should be some information in the roster that includes a timestamp of the key and participants-list... if the timestamp is different the client can request the key to the admin and himself (if is connected in the other client)

And how should this work when using kontalk and a client that doesnt support this?, i suppose that the server will just send the messages to the client with support, so kontalk don't misses the messages

@daniele-athome

This comment has been minimized.

Show comment
Hide comment
@daniele-athome

daniele-athome Jun 30, 2015

Member

Thanks @Manizuca for your protocol review. However there are a few things that are not allowed.

  • servers don't store any information about groups
  • the key is renewed at every message. As of OpenPGP standard, a new symmetric key is created with every message and it's used to encrypt it. The symmetric key is then encrypted for every user using their public keys and attached to the encrypted message
  • making group chat permissions based solely on encryption makes the protocol incompatible with non-encrypted clients. Encryption should be something on top of the protocol
Member

daniele-athome commented Jun 30, 2015

Thanks @Manizuca for your protocol review. However there are a few things that are not allowed.

  • servers don't store any information about groups
  • the key is renewed at every message. As of OpenPGP standard, a new symmetric key is created with every message and it's used to encrypt it. The symmetric key is then encrypted for every user using their public keys and attached to the encrypted message
  • making group chat permissions based solely on encryption makes the protocol incompatible with non-encrypted clients. Encryption should be something on top of the protocol
@Manizuca

This comment has been minimized.

Show comment
Hide comment
@Manizuca

Manizuca Jun 30, 2015

the key is renewed at every message. As of OpenPGP standard, a new symmetric key is created with every message and it's used to encrypt it. The symmetric key is then encrypted for every user using their public keys and attached to the encrypted message

With big groups there can be a big overhead in the message size, there is no way that the same key can be used with multiple messages without being incompatible with the standard? (I'm not familiar with OpenPGP) E.g. it can be renewed every 10 messages?, or can the key be used for messages in the next 4 hours? (In that case can be a problem if the message with the key arrives later)
If is not possible then the groups need to have a limit of participants

Manizuca commented Jun 30, 2015

the key is renewed at every message. As of OpenPGP standard, a new symmetric key is created with every message and it's used to encrypt it. The symmetric key is then encrypted for every user using their public keys and attached to the encrypted message

With big groups there can be a big overhead in the message size, there is no way that the same key can be used with multiple messages without being incompatible with the standard? (I'm not familiar with OpenPGP) E.g. it can be renewed every 10 messages?, or can the key be used for messages in the next 4 hours? (In that case can be a problem if the message with the key arrives later)
If is not possible then the groups need to have a limit of participants

@abika

This comment has been minimized.

Show comment
Hide comment
@abika

abika Jun 30, 2015

Member

[Finally have some time, sorry for delay].

All messages are multicasted to the other participants through XEP-0033.

Hmm, at first I thought this a really great idea, as it reduces the custom group protocol to a minimum. But there is a problem here: Each client gets the same (unencrypted) recipient list, even it it didn't change. This means unneeded overhead and a security problem (The server needs the list, but not anybody listening between server and receiving client). Also, removing a participant from the group should be possible, see below.

(The group creator|Anyone) can add users by sending a group command:

What if the group creator leaves the group? Does the group become orphan?
Also, are we granting the group creator ban privileges?

I'm still thinking only the group creator should be able to do that. He/she initially created the group and should have the privilege to decide who's in there. Yes, when the creator leaves nobody can add user, but another one can be set up easily. And a group "lives" forever anyway, as there is no way to "destroy" it.

As its only client side this surely can't be enforced. But the whole protocol can be seen only as a behaviour recommendation for the clients - which is totally fine, as long as clients can't force other clients to do unwanted things.

The creator should also be able to remove user. This gets tricky as the removed user need to get a message about this and at the same time everyone remaining in the group needs to know who was removed.

But this is a very good start. The mean idea of implementing a group (almost) only by an ID is brilliant.

@Manizuca : You're raising some good questions. But keep in mind that we need to have simple and strict rules to ensure reliability, usability and security.

  • Implementation is client side, server not involved. And actions like group creation, sending messages, leaving a group should still be possible independent of any receiver/client being online or responding. This means (for example) we cant have special group invitations.
  • For the same reason a group key is not a good idea. The group creator is responsible for key renewal and distribution, until then no new messages can't be exchanged.
  • multiple clients/message loss: very good point, seems like sending the complete group participant list in each message is a MUST to ensure membership consistency(?)
Member

abika commented Jun 30, 2015

[Finally have some time, sorry for delay].

All messages are multicasted to the other participants through XEP-0033.

Hmm, at first I thought this a really great idea, as it reduces the custom group protocol to a minimum. But there is a problem here: Each client gets the same (unencrypted) recipient list, even it it didn't change. This means unneeded overhead and a security problem (The server needs the list, but not anybody listening between server and receiving client). Also, removing a participant from the group should be possible, see below.

(The group creator|Anyone) can add users by sending a group command:

What if the group creator leaves the group? Does the group become orphan?
Also, are we granting the group creator ban privileges?

I'm still thinking only the group creator should be able to do that. He/she initially created the group and should have the privilege to decide who's in there. Yes, when the creator leaves nobody can add user, but another one can be set up easily. And a group "lives" forever anyway, as there is no way to "destroy" it.

As its only client side this surely can't be enforced. But the whole protocol can be seen only as a behaviour recommendation for the clients - which is totally fine, as long as clients can't force other clients to do unwanted things.

The creator should also be able to remove user. This gets tricky as the removed user need to get a message about this and at the same time everyone remaining in the group needs to know who was removed.

But this is a very good start. The mean idea of implementing a group (almost) only by an ID is brilliant.

@Manizuca : You're raising some good questions. But keep in mind that we need to have simple and strict rules to ensure reliability, usability and security.

  • Implementation is client side, server not involved. And actions like group creation, sending messages, leaving a group should still be possible independent of any receiver/client being online or responding. This means (for example) we cant have special group invitations.
  • For the same reason a group key is not a good idea. The group creator is responsible for key renewal and distribution, until then no new messages can't be exchanged.
  • multiple clients/message loss: very good point, seems like sending the complete group participant list in each message is a MUST to ensure membership consistency(?)
@daniele-athome

This comment has been minimized.

Show comment
Hide comment
@daniele-athome

daniele-athome Jul 1, 2015

Member

Hmm, at first I thought this a really great idea, as it reduces the custom group protocol to a minimum. But there is a problem here: Each client gets the same (unencrypted) recipient list, even it it didn't change. This means unneeded overhead and a security problem (The server needs the list, but not anybody listening between server and receiving client). Also, removing a participant from the group should be possible, see below.

I could make the server filter that part from messages containing the group extension (that would mean leaving the <group/> extension unencrypted) intended to clients. We'd be slightly out of standard, but Kontalk group chat is already out of standard, so no big deal :-) Besides, it would be of great help in saving bandwidth. However, outgoing messages will always need the list of participants I'm afraid.

The creator should also be able to remove user. This gets tricky as the removed user need to get a message about this and at the same time everyone remaining in the group needs to know who was removed.

We may do it in either two ways:

  • send something like a kick command to the kicked user and then send a *remove" command to everyone but the excluded indicating the kicked user in the group extension
  • send a remove command to everyone indicating the kicked user in the group extension

Any other idea?

  • multiple clients/message loss: very good point, seems like sending the complete group participant list in each message is a MUST to ensure membership consistency(?)

I completely skipped that part, sorry.
Theorically, if for example a new device registers after creating a group, it would have no way of knowing the participants. We could make the new device ask the old device to provide it group information... I don't know... ideas about this? Even though kontalk/androidclient#250 isn't very well defined yet.

Member

daniele-athome commented Jul 1, 2015

Hmm, at first I thought this a really great idea, as it reduces the custom group protocol to a minimum. But there is a problem here: Each client gets the same (unencrypted) recipient list, even it it didn't change. This means unneeded overhead and a security problem (The server needs the list, but not anybody listening between server and receiving client). Also, removing a participant from the group should be possible, see below.

I could make the server filter that part from messages containing the group extension (that would mean leaving the <group/> extension unencrypted) intended to clients. We'd be slightly out of standard, but Kontalk group chat is already out of standard, so no big deal :-) Besides, it would be of great help in saving bandwidth. However, outgoing messages will always need the list of participants I'm afraid.

The creator should also be able to remove user. This gets tricky as the removed user need to get a message about this and at the same time everyone remaining in the group needs to know who was removed.

We may do it in either two ways:

  • send something like a kick command to the kicked user and then send a *remove" command to everyone but the excluded indicating the kicked user in the group extension
  • send a remove command to everyone indicating the kicked user in the group extension

Any other idea?

  • multiple clients/message loss: very good point, seems like sending the complete group participant list in each message is a MUST to ensure membership consistency(?)

I completely skipped that part, sorry.
Theorically, if for example a new device registers after creating a group, it would have no way of knowing the participants. We could make the new device ask the old device to provide it group information... I don't know... ideas about this? Even though kontalk/androidclient#250 isn't very well defined yet.

@abika

This comment has been minimized.

Show comment
Hide comment
@abika

abika Jul 1, 2015

Member

I could make the server filter that part from messages containing the group extension (that would mean leaving the extension unencrypted) intended to clients. We'd be slightly out of standard, but Kontalk group chat is already out of standard, so no big deal :-) Besides, it would be of great help in saving bandwidth. However, outgoing messages will always need the list of participants I'm afraid.

Well, the problem here is we actually need two lists: One for the server to know each recipient, in plain text. And one for the recipients to know the group member, encrypted. Is it feasible to send two lists (only for management commands)?

Any other idea?

Nope, had also these two in mind. I prefer second one: sending one message to everybody with the removed user labelled.

Another idea: How about adding the JID of the group creator(=admin/owner) to the group ID (in every message): <group xmlns=... owner=alice@kontalk.org id='GROUP-ID'/>
A group is know identified by the tuple (owner,id). This will solve two problems:

  • Whenever a client gets a message with an unknown group (bcause of message loss, or whatever), there is a contact entity, for requesting the group participants. And the client is sure to ask the right one.
  • Group hijacking is impossible: Multiple group member can conspire to claim on of them is the owner, or one member resends the create message to claim ownership... (or other evil stuff I can't even think of).
Member

abika commented Jul 1, 2015

I could make the server filter that part from messages containing the group extension (that would mean leaving the extension unencrypted) intended to clients. We'd be slightly out of standard, but Kontalk group chat is already out of standard, so no big deal :-) Besides, it would be of great help in saving bandwidth. However, outgoing messages will always need the list of participants I'm afraid.

Well, the problem here is we actually need two lists: One for the server to know each recipient, in plain text. And one for the recipients to know the group member, encrypted. Is it feasible to send two lists (only for management commands)?

Any other idea?

Nope, had also these two in mind. I prefer second one: sending one message to everybody with the removed user labelled.

Another idea: How about adding the JID of the group creator(=admin/owner) to the group ID (in every message): <group xmlns=... owner=alice@kontalk.org id='GROUP-ID'/>
A group is know identified by the tuple (owner,id). This will solve two problems:

  • Whenever a client gets a message with an unknown group (bcause of message loss, or whatever), there is a contact entity, for requesting the group participants. And the client is sure to ask the right one.
  • Group hijacking is impossible: Multiple group member can conspire to claim on of them is the owner, or one member resends the create message to claim ownership... (or other evil stuff I can't even think of).
@abika

This comment has been minimized.

Show comment
Hide comment
@abika

abika Sep 2, 2015

Member

[ did you reply by editing my message .) ]

Member

abika commented Sep 2, 2015

[ did you reply by editing my message .) ]

@daniele-athome

This comment has been minimized.

Show comment
Hide comment
@daniele-athome

daniele-athome Sep 2, 2015

Member

Yes. I am sorry. Seriously. I was probably in a great need for rest that day. I'll fix it immediately.

Member

daniele-athome commented Sep 2, 2015

Yes. I am sorry. Seriously. I was probably in a great need for rest that day. I'll fix it immediately.

@daniele-athome

This comment has been minimized.

Show comment
Hide comment
@daniele-athome

daniele-athome Sep 2, 2015

Member

Well, the problem here is we actually need two lists: One for the server to know each recipient, in plain text. And one for the recipients to know the group member, encrypted. Is it feasible to send two lists (only for management commands)?

Well, I guess having duplicate information has no advantage in this case, hasn't it? I mean metadata is leaked anyway because routing needs the participants list.
Of course management commands will have the JID of the involved user (for example when adding or removing) in a XML element included in the encrypted part. Unless we are talking about the same thing here?

Nope, had also these two in mind. I prefer second one: sending one message to everybody with the removed user labelled.

Labelled as in putting it in an inner XML element in the group extension right? What about recipients? I guess also the removed user needs to know he/she is been removed.

Another idea: How about adding the JID of the group creator(=admin/owner) to the group ID (in every message): <group xmlns=... owner=alice@kontalk.org id='GROUP-ID'/>
A group is now identified by the tuple (owner,id). This will solve two problems:

Great idea, I'll add it to the draft spec. Feel free to edit the wiki page of course if you think there's something wrong.

Member

daniele-athome commented Sep 2, 2015

Well, the problem here is we actually need two lists: One for the server to know each recipient, in plain text. And one for the recipients to know the group member, encrypted. Is it feasible to send two lists (only for management commands)?

Well, I guess having duplicate information has no advantage in this case, hasn't it? I mean metadata is leaked anyway because routing needs the participants list.
Of course management commands will have the JID of the involved user (for example when adding or removing) in a XML element included in the encrypted part. Unless we are talking about the same thing here?

Nope, had also these two in mind. I prefer second one: sending one message to everybody with the removed user labelled.

Labelled as in putting it in an inner XML element in the group extension right? What about recipients? I guess also the removed user needs to know he/she is been removed.

Another idea: How about adding the JID of the group creator(=admin/owner) to the group ID (in every message): <group xmlns=... owner=alice@kontalk.org id='GROUP-ID'/>
A group is now identified by the tuple (owner,id). This will solve two problems:

Great idea, I'll add it to the draft spec. Feel free to edit the wiki page of course if you think there's something wrong.

@abika

This comment has been minimized.

Show comment
Hide comment
@abika

abika Sep 18, 2015

Member

As you already noticed, I'm working on the group extension and group chat integration. But don't worry, nothing is considered final. In fact, a lot of different ways to implement this popped up and its the question of best practice now.

  • A group command is added as attribute to the stanza as one message can only contain one operation (like the type attribute for many XMPP stanzas). If there is no command then the stanza is only there for identifying the group.
  • Adding and removing is now combined with a set command. This makes it possible to add and/or remove multiple users with with one command (but I consider this only being a small advantage).
  • Related to that and more important is the problem of adding users with two possibilities:
    1. A message including the added user and a list of all other group users is sent to everybody.
    2. A message with the list of group members is only send to the new user. Everybody else gets another (smaller) message.

The trade-off: sending one/two messages vs. bigger/smaller payload of messages received by unchanged users.
Based on the decision here the protocol will look different.

Other issues:

  • XEP-0033 doesn't mention the id attribute, but I guess as one sent message can only have one ID every receiver will get a message with the same ID. The ID is not really unique anymore, but there shouldn't be any major problems.
  • If a received message can't be encrypted for whatever reason the client can't distinguish if the message was part of a group conversation or a single chat. Quick idea for a workaround: use the group ID also for the thread attribute in a message stanza.

TODO: A subject stanza.

[btw: I don't have rights to edit the wiki in specs]

Member

abika commented Sep 18, 2015

As you already noticed, I'm working on the group extension and group chat integration. But don't worry, nothing is considered final. In fact, a lot of different ways to implement this popped up and its the question of best practice now.

  • A group command is added as attribute to the stanza as one message can only contain one operation (like the type attribute for many XMPP stanzas). If there is no command then the stanza is only there for identifying the group.
  • Adding and removing is now combined with a set command. This makes it possible to add and/or remove multiple users with with one command (but I consider this only being a small advantage).
  • Related to that and more important is the problem of adding users with two possibilities:
    1. A message including the added user and a list of all other group users is sent to everybody.
    2. A message with the list of group members is only send to the new user. Everybody else gets another (smaller) message.

The trade-off: sending one/two messages vs. bigger/smaller payload of messages received by unchanged users.
Based on the decision here the protocol will look different.

Other issues:

  • XEP-0033 doesn't mention the id attribute, but I guess as one sent message can only have one ID every receiver will get a message with the same ID. The ID is not really unique anymore, but there shouldn't be any major problems.
  • If a received message can't be encrypted for whatever reason the client can't distinguish if the message was part of a group conversation or a single chat. Quick idea for a workaround: use the group ID also for the thread attribute in a message stanza.

TODO: A subject stanza.

[btw: I don't have rights to edit the wiki in specs]

@sim6

This comment has been minimized.

Show comment
Hide comment
@sim6

sim6 Oct 2, 2015

Is the development of group chat in the following branch?
https://github.com/kontalk/desktopclient-java/commits/feature-group_chat

sim6 commented Oct 2, 2015

Is the development of group chat in the following branch?
https://github.com/kontalk/desktopclient-java/commits/feature-group_chat

@abika

This comment has been minimized.

Show comment
Hide comment
@abika

abika Oct 3, 2015

Member

yes, but its not finished yet. Everything will be in master when it is at least in a 'proof of concept' state.

And the protocol is not final either.

Member

abika commented Oct 3, 2015

yes, but its not finished yet. Everything will be in master when it is at least in a 'proof of concept' state.

And the protocol is not final either.

@daniele-athome

This comment has been minimized.

Show comment
Hide comment
@daniele-athome

daniele-athome Nov 16, 2015

Member

The trade-off: sending one/two messages vs. bigger/smaller payload of messages received by unchanged users.
Based on the decision here the protocol will look different.

First of all, thanks for your work so far.
Getting to the point: I'd be more keen to adhere to XEP-0033 and not strip the <addresses/> part. Although messages will be bigger, group members will always be available, even if the owner is currently offline (and therefore if a sender doesn't know the members for whatever reason, he/she must wait for the owner to come online). I feel like I'd accept such a overhead in this case. Besides, we can always improve it later in some other way (or just wait for the almighty MUC2).

XEP-0033 doesn't mention the id attribute, but I guess as one sent message can only have one ID every receiver will get a message with the same ID. The ID is not really unique anymore, but there shouldn't be any major problems.

Indeed. Besides, having the same ID for the "same message" in a group could make sense (I mean it's the same message afterall, just broadcasted).

If a received message can't be encrypted for whatever reason the client can't distinguish if the message was part of a group conversation or a single chat. Quick idea for a workaround: use the group ID also for the thread attribute in a message stanza.

I don't think this can be much of an issue. In 99% of cases, when you receive an encrypted messages you can't decrypt it and you're stuck with it (the sender used an old key which is already invalid/non-existant, so the message is pretty much lost). Knowing that an encrypted message is part of a group doesn't really matter.
The other 1% is when a temporary error occur in the Android client (I can't actually think of one right now, I'm just assuming the worse so I never use "100%" :-)


I've seen the GroupExtension code, but I'd bring some modifications:

  • Since we are going to use XEP-0033 also for all messages, add and remove commands are not needed anymore (the client can see which users are new and which are gone by comparing its own list)
  • Because of the same reason, we don't even need a create command anymore. We just send the first message along with the <group/> extension and all recipients will know about the group.
  • In theory, in order to stick to the "always all information" rule, we should also include a <subject/> tag (child to <group/>) in every message. In practice: we should.
  • It'd be more "XMPP/XML-way" if the command would be child to <group/> as a tag instead inside of a command attribute

Replying to your latest email:

How to handle group members that are not in the contact list: A user that gets an
invitation for a new group chat might not know the participants at all and doesn't expect
them to be added as contact. However, this breaks (at least in the desktop client) the
model associations as requesting/accepting/saving a public key requires to have a contact
entity.

The group members are always listed in every message so this should not be a problem, if I understand you correctly.
Unknown contacts will be listed as unknown (using the hash, that's the only information available until an invitation workflow starts - which will make the users exchange their keys, where we can find the username).

Another question is what will happen if a contact that is part of a group chat is
deleted.

That will all depend on the client UI behaviour. I'd revert back to the hash for the sake of consistency. What do you think?


Final note: I've been away from this discussion for a long time, doing other tasks and working on multiple things at the same time. If by any means I've skipped something or some of the stuff I said doesn't make sense at all, please say it :-)

Member

daniele-athome commented Nov 16, 2015

The trade-off: sending one/two messages vs. bigger/smaller payload of messages received by unchanged users.
Based on the decision here the protocol will look different.

First of all, thanks for your work so far.
Getting to the point: I'd be more keen to adhere to XEP-0033 and not strip the <addresses/> part. Although messages will be bigger, group members will always be available, even if the owner is currently offline (and therefore if a sender doesn't know the members for whatever reason, he/she must wait for the owner to come online). I feel like I'd accept such a overhead in this case. Besides, we can always improve it later in some other way (or just wait for the almighty MUC2).

XEP-0033 doesn't mention the id attribute, but I guess as one sent message can only have one ID every receiver will get a message with the same ID. The ID is not really unique anymore, but there shouldn't be any major problems.

Indeed. Besides, having the same ID for the "same message" in a group could make sense (I mean it's the same message afterall, just broadcasted).

If a received message can't be encrypted for whatever reason the client can't distinguish if the message was part of a group conversation or a single chat. Quick idea for a workaround: use the group ID also for the thread attribute in a message stanza.

I don't think this can be much of an issue. In 99% of cases, when you receive an encrypted messages you can't decrypt it and you're stuck with it (the sender used an old key which is already invalid/non-existant, so the message is pretty much lost). Knowing that an encrypted message is part of a group doesn't really matter.
The other 1% is when a temporary error occur in the Android client (I can't actually think of one right now, I'm just assuming the worse so I never use "100%" :-)


I've seen the GroupExtension code, but I'd bring some modifications:

  • Since we are going to use XEP-0033 also for all messages, add and remove commands are not needed anymore (the client can see which users are new and which are gone by comparing its own list)
  • Because of the same reason, we don't even need a create command anymore. We just send the first message along with the <group/> extension and all recipients will know about the group.
  • In theory, in order to stick to the "always all information" rule, we should also include a <subject/> tag (child to <group/>) in every message. In practice: we should.
  • It'd be more "XMPP/XML-way" if the command would be child to <group/> as a tag instead inside of a command attribute

Replying to your latest email:

How to handle group members that are not in the contact list: A user that gets an
invitation for a new group chat might not know the participants at all and doesn't expect
them to be added as contact. However, this breaks (at least in the desktop client) the
model associations as requesting/accepting/saving a public key requires to have a contact
entity.

The group members are always listed in every message so this should not be a problem, if I understand you correctly.
Unknown contacts will be listed as unknown (using the hash, that's the only information available until an invitation workflow starts - which will make the users exchange their keys, where we can find the username).

Another question is what will happen if a contact that is part of a group chat is
deleted.

That will all depend on the client UI behaviour. I'd revert back to the hash for the sake of consistency. What do you think?


Final note: I've been away from this discussion for a long time, doing other tasks and working on multiple things at the same time. If by any means I've skipped something or some of the stuff I said doesn't make sense at all, please say it :-)

@abika

This comment has been minimized.

Show comment
Hide comment
@abika

abika Nov 17, 2015

Member

(For the sake of clarity I only respond to points where there is something important to say / that are not already obsolete).

Getting to the point: I'd be more keen to adhere to XEP-0033 and not strip the <addresses/>
part.

I have to say I don't like that.

  • we are changing and misusing a standard XEP for something that it was not intended to be used for
  • bigger payload for every message (but you pointed that already out yourself)

I propose using 0033 only for sending a group message (with striping) and including the member list again encrypted inside the group extension (with the downside of even bigger payload).

But if you say your design is the way to go, we can do it like that. I still can't recommend it.

Something else:

group members will always be available, even if the owner is currently offline (and therefore if a sender doesn't know the members for whatever reason, he/she must wait for the owner to come online).

There is a problem with authorization here: The group owner is the only one who can add/remove members, right? So, member lists from other group participants can not be trusted, otherwise they will also be able to add/remove members. Therefore the group owner is the only one who can give reliable information and MUST be asked if needed.


  • Since we are going to use XEP-0033 also for all messages, add and remove commands are not needed anymore (the client can see which users are new and which are gone by comparing its own list)

With one exception: removing will require sending an additional message to the removed member with a kick command.

And it could better to exchange explicit "diffs" (add/remove) to avoid ambiguous member affiliation. (You know, clients are not always connected and there is always some latency in transmissions, if two clients send contradicting messages...)
But I'm really not sure about that myself.

  • Because of the same reason, we don't even need a create command anymore. We just send the first message along with the <group/> extension and all recipients will know about the group.

New group participants won't be able to tell if a group was just created or if they were added after some time. But that's not a big deal.


The group members are always listed in every message so this should not be a problem, if I understand you correctly.

Well, my point is this: If a group participant is unknown to me: will it be added to the contact list like an ordinary contact? Will it also be inserted into Androids address book (although the phone number will be missing)? Can the display name be changed?

That will all depend on the client UI behaviour. I'd revert back to the hash for the sake of consistency. What do you think?

Yes, the hashes must used. But if the contacts public key is also deleted, sending encrypted messages to the group won't be possible anymore.

But this is only about implementation design. If you found solutions for these problems, everything is fine.

Member

abika commented Nov 17, 2015

(For the sake of clarity I only respond to points where there is something important to say / that are not already obsolete).

Getting to the point: I'd be more keen to adhere to XEP-0033 and not strip the <addresses/>
part.

I have to say I don't like that.

  • we are changing and misusing a standard XEP for something that it was not intended to be used for
  • bigger payload for every message (but you pointed that already out yourself)

I propose using 0033 only for sending a group message (with striping) and including the member list again encrypted inside the group extension (with the downside of even bigger payload).

But if you say your design is the way to go, we can do it like that. I still can't recommend it.

Something else:

group members will always be available, even if the owner is currently offline (and therefore if a sender doesn't know the members for whatever reason, he/she must wait for the owner to come online).

There is a problem with authorization here: The group owner is the only one who can add/remove members, right? So, member lists from other group participants can not be trusted, otherwise they will also be able to add/remove members. Therefore the group owner is the only one who can give reliable information and MUST be asked if needed.


  • Since we are going to use XEP-0033 also for all messages, add and remove commands are not needed anymore (the client can see which users are new and which are gone by comparing its own list)

With one exception: removing will require sending an additional message to the removed member with a kick command.

And it could better to exchange explicit "diffs" (add/remove) to avoid ambiguous member affiliation. (You know, clients are not always connected and there is always some latency in transmissions, if two clients send contradicting messages...)
But I'm really not sure about that myself.

  • Because of the same reason, we don't even need a create command anymore. We just send the first message along with the <group/> extension and all recipients will know about the group.

New group participants won't be able to tell if a group was just created or if they were added after some time. But that's not a big deal.


The group members are always listed in every message so this should not be a problem, if I understand you correctly.

Well, my point is this: If a group participant is unknown to me: will it be added to the contact list like an ordinary contact? Will it also be inserted into Androids address book (although the phone number will be missing)? Can the display name be changed?

That will all depend on the client UI behaviour. I'd revert back to the hash for the sake of consistency. What do you think?

Yes, the hashes must used. But if the contacts public key is also deleted, sending encrypted messages to the group won't be possible anymore.

But this is only about implementation design. If you found solutions for these problems, everything is fine.

@daniele-athome

This comment has been minimized.

Show comment
Hide comment
@daniele-athome

daniele-athome Nov 22, 2015

Member

I propose using 0033 only for sending a group message (with striping) and including the member list again encrypted inside the group extension (with the downside of even bigger payload).

It might be a good solution afterall... the biggest payload would be only for the initial message; XEP-0033 parts will be stripped out and we already have a list of members inside.

Still... I have reservations even on myself on this because I always think about saving bandwidth and energy so I am conflicted.
However, with the help and patches I've been providing to Tigase (and their continuous efforts and help too), message delivery has now reached (as far as I could have seen for the past few weeks) 100% reliability. So I guess we could save adding the members list to every message.
Worst case scenario, a client reinstalling Kontalk will receive a message about an unknown group that will have members only after the group creator has responded?
(About this specific aspect: what about asking the first sender that comes up online about group members? And then ask for confirmation to the owner as soon as it comes online? I know it seems like a silly idea, but whatever, we are brainstorming :-)

With one exception: removing will require sending an additional message to the removed member with a kick command.

Right, thanks for pointing that out.

And it could better to exchange explicit "diffs" (add/remove) to avoid ambiguous member affiliation. (You know, clients are not always connected and there is always some latency in transmissions, if two clients send contradicting messages...)
But I'm really not sure about that myself.

As I said before, I am conflicted too.
It seems to me we eventually always go back to the same conclusion: having add/remove commands is better than having the members list in every message. Maybe we should really adopt these method and begin trusting server reliability (starting with myself doing it - and probably the fact that I work on the server too makes me think it's unreliable, but I mean we are using a server mainly because it's more reliable than clients, isn't it?).

Well, my point is this: If a group participant is unknown to me: will it be added to the contact list like an ordinary contact? Will it also be inserted into Androids address book (although the phone number will be missing)? Can the display name be changed?

It will happen much like it happens now if you receive a message from an unknown contact.
If you are authorized to chat to each other, you will just see his/her username or the hash if a public key isn't available yet (which should not happen, but that's a problem for another time).
If you are not authorized to chat to each other, you will not receive group messages at all from that user - we talked about this before, we need a serious ad-hoc authorization workflow for groups - people expect that conversations in groups do not need 1-to-1 chat authorization.

Yes, the hashes must used. But if the contacts public key is also deleted, sending encrypted messages to the group won't be possible anymore.

Currently, if you remove a contact from Android contacts, the subscription is kept alive, no unsubscription is done, so technically the key is preserved (the sync process in the Android app also looks for roster entries). You will see the username and the hash though, no phone number. But message exchanging will continue to work.

I'm moving the Group chat wiki page into the specs repository inside an md file. I will grant you commit access to that repo so we can work together on the specs.

Member

daniele-athome commented Nov 22, 2015

I propose using 0033 only for sending a group message (with striping) and including the member list again encrypted inside the group extension (with the downside of even bigger payload).

It might be a good solution afterall... the biggest payload would be only for the initial message; XEP-0033 parts will be stripped out and we already have a list of members inside.

Still... I have reservations even on myself on this because I always think about saving bandwidth and energy so I am conflicted.
However, with the help and patches I've been providing to Tigase (and their continuous efforts and help too), message delivery has now reached (as far as I could have seen for the past few weeks) 100% reliability. So I guess we could save adding the members list to every message.
Worst case scenario, a client reinstalling Kontalk will receive a message about an unknown group that will have members only after the group creator has responded?
(About this specific aspect: what about asking the first sender that comes up online about group members? And then ask for confirmation to the owner as soon as it comes online? I know it seems like a silly idea, but whatever, we are brainstorming :-)

With one exception: removing will require sending an additional message to the removed member with a kick command.

Right, thanks for pointing that out.

And it could better to exchange explicit "diffs" (add/remove) to avoid ambiguous member affiliation. (You know, clients are not always connected and there is always some latency in transmissions, if two clients send contradicting messages...)
But I'm really not sure about that myself.

As I said before, I am conflicted too.
It seems to me we eventually always go back to the same conclusion: having add/remove commands is better than having the members list in every message. Maybe we should really adopt these method and begin trusting server reliability (starting with myself doing it - and probably the fact that I work on the server too makes me think it's unreliable, but I mean we are using a server mainly because it's more reliable than clients, isn't it?).

Well, my point is this: If a group participant is unknown to me: will it be added to the contact list like an ordinary contact? Will it also be inserted into Androids address book (although the phone number will be missing)? Can the display name be changed?

It will happen much like it happens now if you receive a message from an unknown contact.
If you are authorized to chat to each other, you will just see his/her username or the hash if a public key isn't available yet (which should not happen, but that's a problem for another time).
If you are not authorized to chat to each other, you will not receive group messages at all from that user - we talked about this before, we need a serious ad-hoc authorization workflow for groups - people expect that conversations in groups do not need 1-to-1 chat authorization.

Yes, the hashes must used. But if the contacts public key is also deleted, sending encrypted messages to the group won't be possible anymore.

Currently, if you remove a contact from Android contacts, the subscription is kept alive, no unsubscription is done, so technically the key is preserved (the sync process in the Android app also looks for roster entries). You will see the username and the hash though, no phone number. But message exchanging will continue to work.

I'm moving the Group chat wiki page into the specs repository inside an md file. I will grant you commit access to that repo so we can work together on the specs.

@abika

This comment has been minimized.

Show comment
Hide comment
@abika

abika Nov 23, 2015

Member

So I guess we could save adding the members list to every message.
Worst case scenario, a client reinstalling Kontalk will receive a message about an unknown group that will have members only after the group creator has responded?

True, and I think this case is acceptable.

But I just realized that unless message sync between clients (kontalk/androidclient#250) is implemented having multiple clients will break group chat. Clients not connected for some time can never be sure they didn't miss group changes (and must ask the group creator for every group on every re-connect to make sure they are up-to-date).

(About this specific aspect: what about asking the first sender that comes up online about group members? And then ask for confirmation to the owner as soon as it comes online? I know it seems like a silly idea, but whatever, we are brainstorming :-)

This will make it just more complicated. If the requested information is invalid the client could send messages to users not in the group and actual group members will miss messages. (Yes, I'm assuming the worst but it's not unlikely some script-kiddie will exploit this)

As I said before, I am conflicted too.
It seems to me we eventually always go back to the same conclusion: having add/remove commands is better than having the members list in every message. Maybe we should really adopt these method and begin trusting server reliability (starting with myself doing it - and probably the fact that I work on the server too makes me think it's unreliable, but I mean we are using a server mainly because it's more reliable than clients, isn't it?).

If the server is not reliable there is really no way to ensure group consistency - unless the clients constantly ask for the group members (see above). And this is independent from the protocol details so we should focus the other error cases and make handling these as robust as possible.

I'll try adding the use-cases for the problems were discussing here to the specs in the next days.

Member

abika commented Nov 23, 2015

So I guess we could save adding the members list to every message.
Worst case scenario, a client reinstalling Kontalk will receive a message about an unknown group that will have members only after the group creator has responded?

True, and I think this case is acceptable.

But I just realized that unless message sync between clients (kontalk/androidclient#250) is implemented having multiple clients will break group chat. Clients not connected for some time can never be sure they didn't miss group changes (and must ask the group creator for every group on every re-connect to make sure they are up-to-date).

(About this specific aspect: what about asking the first sender that comes up online about group members? And then ask for confirmation to the owner as soon as it comes online? I know it seems like a silly idea, but whatever, we are brainstorming :-)

This will make it just more complicated. If the requested information is invalid the client could send messages to users not in the group and actual group members will miss messages. (Yes, I'm assuming the worst but it's not unlikely some script-kiddie will exploit this)

As I said before, I am conflicted too.
It seems to me we eventually always go back to the same conclusion: having add/remove commands is better than having the members list in every message. Maybe we should really adopt these method and begin trusting server reliability (starting with myself doing it - and probably the fact that I work on the server too makes me think it's unreliable, but I mean we are using a server mainly because it's more reliable than clients, isn't it?).

If the server is not reliable there is really no way to ensure group consistency - unless the clients constantly ask for the group members (see above). And this is independent from the protocol details so we should focus the other error cases and make handling these as robust as possible.

I'll try adding the use-cases for the problems were discussing here to the specs in the next days.

@erdmark

This comment has been minimized.

Show comment
Hide comment
@erdmark

erdmark May 17, 2016

Instead of sending the whole member list in every message just send a hash of it.
When a member is added or removed then send the whole member list.
If somebody does not know all members ask any member or the group admin for the full list.
Because the admin sent the hash first, the hash is reliable. Now you can prove the list is correct by calculating the hash.

erdmark commented May 17, 2016

Instead of sending the whole member list in every message just send a hash of it.
When a member is added or removed then send the whole member list.
If somebody does not know all members ask any member or the group admin for the full list.
Because the admin sent the hash first, the hash is reliable. Now you can prove the list is correct by calculating the hash.

@daniele-athome

This comment has been minimized.

Show comment
Hide comment
@daniele-athome

daniele-athome May 17, 2016

Member

Thanks @erdmark, the protocol has been recently revised to not send the members list on every message in the <group/> element. However, the server needs to know where to deliver the message, so XEP-0033 is used for this purpose (unfortunately there is no other way without storing state on the server).


https://github.com/kontalk/specs/blob/master/group.md#sending-messages

Member

daniele-athome commented May 17, 2016

Thanks @erdmark, the protocol has been recently revised to not send the members list on every message in the <group/> element. However, the server needs to know where to deliver the message, so XEP-0033 is used for this purpose (unfortunately there is no other way without storing state on the server).


https://github.com/kontalk/specs/blob/master/group.md#sending-messages

@webratte

This comment has been minimized.

Show comment
Hide comment
@webratte

webratte Mar 11, 2017

Contributor

Is there a release date planed for Desktopclient with groupchat?

Contributor

webratte commented Mar 11, 2017

Is there a release date planed for Desktopclient with groupchat?

@daniele-athome

This comment has been minimized.

Show comment
Hide comment
@daniele-athome

daniele-athome Mar 13, 2017

Member

IIRC I believe it is already fully implemented. I can't try it right now, @abika can you confirm please so we can close the issue.

Member

daniele-athome commented Mar 13, 2017

IIRC I believe it is already fully implemented. I can't try it right now, @abika can you confirm please so we can close the issue.

@abika

This comment has been minimized.

Show comment
Hide comment
@abika

abika Mar 15, 2017

Member

Just tested again, basic operations (create/leave/add/remove) are working between current versions of Desktop and Android client. There are issues left but I will work on them separately.

A big problem are still the lost messages due to missing message archive. A disconnected client will miss group commands making the groups inconsistent. For this reason I won't enable the creation of groups in the Desktop client, user experience will just be bad.

Member

abika commented Mar 15, 2017

Just tested again, basic operations (create/leave/add/remove) are working between current versions of Desktop and Android client. There are issues left but I will work on them separately.

A big problem are still the lost messages due to missing message archive. A disconnected client will miss group commands making the groups inconsistent. For this reason I won't enable the creation of groups in the Desktop client, user experience will just be bad.

@abika abika closed this Mar 15, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment