New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How to differentiate between posts and private (direct) messages? #196

Closed
annando opened this Issue Apr 18, 2017 · 55 comments

Comments

Projects
None yet
@annando
Copy link

annando commented Apr 18, 2017

Because of the issue diaspora/diaspora#7422 I had a look at the specifications once again.

Traditional decentralized systems like Friendica or Diaspora, but also centralized systems like Twitter or Facebook are knowing two different message types.

The first one is the standard message. It can be addressed to the public or to a group of people or even to a single person alone. This is mostly meant for stuff that you just want to share with other people - and it doesn't really matter if all the receivers see the message at all.

The second one is more like the traditional mail. You not only get a notification about it, but also these messages are collected at a special place so that you can keep an eye on this. These messages can mostly be addressed to a group of people as well.

I haven't found any trace in the specifications on how to implement these two types - but this is crucial for replacing existing protocols.

@cwebber

This comment has been minimized.

Copy link
Collaborator

cwebber commented Apr 18, 2017

We have a Public collection, and if you include that in to or cc, it's a public post! If you don't, it's not a public post! You can see an example of this in the last example of the tutorial I wrote.

Does that answer things?

@annando

This comment has been minimized.

Copy link

annando commented Apr 18, 2017

No. This is not the same. I could share my holiday photos to a bunch of people as a not public post. It would be nice - but not really important - if all of the dedicated receivers would read this kind of post.

On the other hand I have posts of high importance. I want the receivers to not forget about them. Because of this they are even optically separated from the other posts.

Just a real life example: I'm normally receiving more than 1,000 messages per day in the regular stream. I cannot read them all. Additionally I got mentioned about 20 or 30 times a day (which could be implemented with to or cc). It is nice that I get informed about it - but if I don't have the time to react in time, I mostly forget about it - and it doesn't matter that much, if I don't react.

But 3 or 4 times a week I get these messages where the sender really wanted to make sure that I don't miss the message - and I want to have them in a place where I don't forget to answer, even when it takes some days to do so - just like the messages in the email inbox.

It is not enough to have a list of messages where I was addressed directly via to or cc because this happens many times a day and these really important messages would simply flushed away in the notification list.

@cwebber

This comment has been minimized.

Copy link
Collaborator

cwebber commented Apr 18, 2017

Pump.io clients make a distinction between these things depending on whether you are addressed specifically in the post. For example, in Pumpa if someone adds me specifically to the "to" line... even if it's a public post... it shows up in my "Direct" feed. And of course if someone's sending it only to me, it shows up in my Direct feed. So I think that can be handled with what we have.

On the other hand if you're looking for some sort of "prioritization" flagging, that would be an extension?

@annando

This comment has been minimized.

Copy link

annando commented Apr 18, 2017

The "Direct" feed wouldn't suit my needs. I am mentioned too often a day so that this special kind of message would vanish inside of this. (And I wouldn't find this message in a few days again)

With a "priority" flag this could be emulated in some way.

@jaywink

This comment has been minimized.

Copy link

jaywink commented Apr 18, 2017

I totally get what @annando is after here, but still I've always felt these two concepts are overlapping fully on the type of objects. The fact that they are represented differently in the UI and system internally is a different matter. For example just as they are in Diaspora, Friendica, and previously Facebook. I say previously Facebook because there the chat system has replaced the old messaging system completely. This makes sense, since even in Diaspora it is more of a group chat/mail functionality.

Rather than some kind of prioritizing, it could also be an extension of another type of object totally. After all, quite a few need to already be supported. What is a status message for example, a Note or an Article? Depends on text length, says AS2 vocabulary, so in terms of Diaspora for example, both should count as status messages.

Maybe for systems who want to support a kind of internal mail which the conversation feature basically is, an extension of type Conversation for example might be the simplest way to go?

@annando

This comment has been minimized.

Copy link

annando commented Apr 18, 2017

This Conversation type is a good idea.

@cwebber

This comment has been minimized.

Copy link
Collaborator

cwebber commented Apr 18, 2017

We could explore priority flags... of course, we can look to email to see how well used those are, since I think Outlook has them, and they are used by Outlook users... I have no experience with them.

I do worry that priority flags will start to result in Klaxon-like "this is priority!" on most messages, but maybe people can self-regulate.

@cwebber

This comment has been minimized.

Copy link
Collaborator

cwebber commented Apr 18, 2017

Regardless, we won't have time to incubate a priority flag within this cycle. The good news is that AP is extensible, and we'll have the community group to explore extensions, so we can explore it then.

I've added it to https://www.w3.org/wiki/ActivityPub_extensions ... we'll look into it after this round of ActivityPub stuff, when we get to the Community Group.

@cwebber cwebber closed this Apr 18, 2017

@cwebber

This comment has been minimized.

Copy link
Collaborator

cwebber commented Apr 18, 2017

Reopening so we can talk about it in the call

@cwebber cwebber reopened this Apr 18, 2017

@cwebber

This comment has been minimized.

Copy link
Collaborator

cwebber commented Apr 18, 2017

Somehow I missed the conversation about the new type. That could be interesting. Again, would have to be done as an extension at this point, but may be well worth doing.

@sandhawke

This comment has been minimized.

Copy link

sandhawke commented Apr 18, 2017

I'm having some trouble thinking of a clear line between these two types. I can make a post (type 1) or send a direct message (type 2) to a group of the same people (eg my co-workers). Type 1 is a bulletin board post and type 2 is a chat message, but when you add enough functionality, they start to overlap more and more. With type 2, there's an expectation of somewhat faster delivery, and of strict time ordering. I don't agree type type 2 is more important, or has less traffic -- I've been on plenty of group chats that had way more traffic than I read, and some of my co-workers make FB posts and tweets that are very important for me to see.

@strugee

This comment has been minimized.

Copy link

strugee commented Apr 18, 2017

Assuming implementations use Mention it seems like it should be possible to do this without extensions? E.g.:

  1. Is the post addressed to Public? If so it's a public post
  2. Is the post addressed to only the user? If not it's not a direct message
  3. Is the post related to a Mention activity? If so it's a mention not a direct message
  4. The post is a direct message

Not sure I've thought through everything, but I'm pretty sure this can be done.

@sandhawke I think the difference is in how the sender expects the message to be presented to the receiver. Of course you can find exceptions for either but in general I think "direct messages" tend to be more important/personally relevant than general stream messages

@strugee

This comment has been minimized.

Copy link

strugee commented Apr 18, 2017

Or maybe we could just end up with an algorithm as simple as:

  1. Is the activity addressed to a Collection (like e.g. Followers) or Public? If so it's a stream post
  2. The post is a direct message
@annando

This comment has been minimized.

Copy link

annando commented Apr 18, 2017

@strugee this won't work for Friendica and would make problems with Diaspora as well, I guess. On Diaspora you can chose the aspects you want to post to. On Friendica you can even select and deselect single receivers - additionally on Diaspora the direct messages can be addressed to a group of people.

There is the need for a flag of some kind. And if the standard doesn't contain it, there are two options:

  1. Implement some self designed extension
  2. Wait with the implementation until the extension is standardized

Although there might be some overlapping, it is no option to remove the direct messages from existing systems. Users won't accept the removal of existing functionality.

@cwebber

This comment has been minimized.

Copy link
Collaborator

cwebber commented Apr 18, 2017

Okay, we talked about this today on the call... I've removed the postponed field. There's at minimum some discussion to be had about what it means to have a "direct" message, and whether the standard already handles that.

I think we're also planning on talking about this on next week's call.

@sandhawke

This comment has been minimized.

Copy link

sandhawke commented Apr 19, 2017

@annando, can you explain what this flag would mean, in general, not just in a particular system? Like, can you define it so that all of us would agree, looking at some new system, whether it produces and consumes the flag correctly?

@annando

This comment has been minimized.

Copy link

annando commented Apr 19, 2017

@sandhawke I tried so in several comments above. Maybe this wasn't understandable due to my limited English knowledge or just because I'm bad in explaining things. Maybe others can explain it better.

@sandhawke

This comment has been minimized.

Copy link

sandhawke commented Apr 19, 2017

Sorry, on re-reading I see my message sounds critical, @annando, when I was trying to be helpful. Let me try to propose some text which i think covers what you're saying, and then you can tell me where I'm getting it wrong.


The delivery mode property allows systems to distinguish between two prevailing metaphors for communication. Within AP, both delivery modes have the same capabilities, but between systems which have different user interfaces for the two modes, this flag is used to communicate the appropriate mode.

Delivery mode post appeals to intuitions about communicating by putting content where the recipient can see it, such as pinning it to a wall. In contrast, delivery mode direct is like handing someone the content. Since you could put a message on a wall where only one person is allowed to see it, and is very likely to see it, post can be used for 1-1 messages. Meanwhile, since you could make many copies of a message and hand them out to everyone you see, post-like behavior is possible using the direct mode. Nonetheless, the distinction appears to be useful, since some popular systems implement both and keep them distinct.

In practice, direct delivery tends to differ from post delivery in these ways:

  • more strictly and obviously time-ordered (no 'algorithmic feed')
  • more feedback to the author about whether it's been seen
  • more likely an indicator to the recipient that a message is being composed
  • more likely a smaller audience, often just one person
  • more likely only the target audience is allowed to see the message
  • more likely to be important, time-sensitive, and require a response

Systems which offer users two delivery modes that align well with these metaphors MUST encode the delivery mode in the deliveryMode flag during content creation and MUST use this flag to determine which mode to use in delivering content to users. Other systems MAY use the flag in a manner which is approximately consistent with this meaning.

On receiving content without a delivery-mode flag, systems are free to use their own logic for sorting into the appropriate mode.


Is that kind of what folks are thinking here?

@annando

This comment has been minimized.

Copy link

annando commented Apr 19, 2017

@sandhawke my words weren't intended to sound aggressive. That's one of the problems when one (me) isn't talking in his own language.

I have to take some time to understand your text (some words are unknown to me or sound strange in the context). Then I will answer to this as well.

@annando

This comment has been minimized.

Copy link

annando commented Apr 20, 2017

Now, after translating some of the missing words and guessing the meaning of others, I guess that you hit the nail right on the head with this definition.

@Flaburgan

This comment has been minimized.

Copy link

Flaburgan commented Apr 20, 2017

Hi guys. The last definition sounds pretty well.
Also, ++ to

I think the difference is in how the sender expects the message to be presented to the receiver.

Conversation are what we call private messages in forum, twitter, etc.

If it helps, here is how diaspora* is currently working (reminder, the spec are here. There is the post entity, and the conversation entity. The major differences are:

  • Audience:
    • A post can only be posted to a group of contact (called Aspect in the diaspora language) or publicly. Of course, you can have a group with only one contact, but still, the post, from a user point of view, target a group. Also, there is no symmetrical relationship in diaspora*, so you can add anyone you want to a group of contacts, and send a post there, this post will be accessible to the person even if it never interacted with the sender.
    • On the other side, a conversation targets contacts. That means there is no predefined group, each new conversation has its specific audience. Note that, unlike Facebook, a conversation is not grouped by the targeted audience. I think on Facebook, when you write to the same persons than before, the new message comes at the end of the previous conversation. On diaspora*, it is not the case, you create a new conversation with a new subject, the two conversations live separately, and you can post on the old conversation in the future.
      Also, at the moment, you can only target a contact if he also shares with you. I don't know if this behavior will be kept (see diaspora/diaspora#4992) but it is the way it works at the moment, so this is deeply different than the post feature.
  • Reaction:
    • The post entity can be reshared if it is public, it can also be commented (with a comment entity). A comment can then also be posted on the previous comment, or still on the post. Meaning you can have threaded comment, like this:
Post
- Comment 1
- Comment 2
| - Comment 2.1
- Comment 3
| - Comment 3.1
| - Comment 3.2
|  | - Comment 3.2.1
- Comment 4
  • The conversation entity only have messages, (and starts itself with a message, the first of the conversation). They are all displayed under the other message. A conversation can only look like that:
Conversation
- Initial message (posted with the conversation)
- Message 2
- Message 3
- Message 4
- Message 5

There is no reaction to conversation messages (no like, no threaded answer, no reshare of course, no attached pictures...). I don't say it shouldn't have some, I'm just describing the actual feature in diaspora*.

  • Presentation: As said above in this issue, the two features are presented very differently.
    • A post is included in the stream. It is very difficult to go back to an old post. The easiest way is to remember who posted it (so initiated the discussion), go to his profile, and scroll down. A post is something kind of temporary.
    • A conversation is here to stay. Here is how it looks in diaspora*:

      You access it with the mail icon in the header, and it lives completely separately from the stream. All the conversation you participated to can easily be accessible, in a unique place.

Do you have more questions about the diaspora* current features?

Pinging @SuperTux88 and @cmrd-senya to be sure I didn't say wrong things about the protocol, it's not the part I know well.

@SuperTux88

This comment has been minimized.

Copy link

SuperTux88 commented Apr 20, 2017

Just a short answer to Flas comment (didn't read the rest): The protocol doesn't know aspects, so posts are sent to single recipients too (aspects are only a UX thing).

The main difference between a (private) post and a conversation is: For the conversation the recipients are visible to all other recipients, for a post you only see that it's "private", but not who else can see it. Comments of a post are relayed through the author of the post (because it's the only point where all recipients are known), for a conversation messages are sent directly to all recipients, because they are known to everybody involved.

@wilkie

This comment has been minimized.

Copy link

wilkie commented Apr 20, 2017

Some technical highlights in the specifications that are worth noting: Public and private audiences are disambiguated via bto/bcc vs to/cc. The bcc and bto targets are removed from the messages as they are sent to that actor's inbox. If your note has cc or to it is a public message. If your note has bcc or bto it is a private message. bto is a private direct message. to is a public direct message / reply. bcc is a private multicast message (an identical message sent to multiple people) where nobody can see who it was sent to. cc is a public conversation.

See https://www.w3.org/TR/activitystreams-vocabulary/#audienceTargeting

On the receiving side, the object will not have bcc or bto as they must be removed by the sender. How do you tell the difference between public and private messages? Messages that were private would have no targeting. They would be messages without a to field, referred to as silent messages. In ActivityPub, there is an interesting aside that says non-targeted messages should be treated as private as a precaution:

See https://www.w3.org/TR/activitypub/#delivery

Are you tagged in an Object that doesn't have a to or audience, then that's a private message. Private messages are more important by social convention, so that's a good way to get people's attention. They should go into a section of a UI that separates them from the normal firehose.

@wilkie

This comment has been minimized.

Copy link

wilkie commented Apr 20, 2017

Personally, I think the wording could be stronger. That note about private activities should be part of the spec as a must, considering its use involves some implied security by the sender, but I'd settle for a should. In the end, to represents who the Object is visible to. If there is no to field, then the Object is invisible to all but the person who created it or received it. That way, no matter what cool extension you come up with, you know implementations that don't understand it will at least pedantically consider it private if that's what you intend.

@cwebber

This comment has been minimized.

Copy link
Collaborator

cwebber commented May 30, 2017

@astronouth7303 I think that's a valid issue but a bit out of scope here; let's assume that sufficient authorization has already been done. The real question here is how to convey intent of "where" this should be represented.

@cwebber

This comment has been minimized.

Copy link
Collaborator

cwebber commented May 30, 2017

This issue is getting a bit hard to follow already, but btw one solution that I'm pretty sure Evan proposed on a previous call (but I can't find it in the minutes??) is to have another endpoint, like inbox, which is for direct messages. However, that also adds its own set of challenges because now you have another federation feature some instances may support and some may not.

@cwebber cwebber closed this May 30, 2017

@cwebber

This comment has been minimized.

Copy link
Collaborator

cwebber commented May 30, 2017

(Augh, both close and this comment were mistakes.)

@cwebber cwebber reopened this May 30, 2017

@cwebber

This comment has been minimized.

Copy link
Collaborator

cwebber commented May 30, 2017

Note that Pump.io already does this, and I think this may be the answer: inbox is still used for federation, but multiple streams can be presented to the user in terms of inbox / the "major" feed (which is all the main posts and comments and etc) / the "minor" feed (which includes a lot more flotsam such as each individual favorite post, etc) / the "direct" feed (which are posts that mention you directly).

Every server can choose to make its own "filtered inbox-consumer endpoints" like this. In fact, pump.io will probably need extensions for these terms, since clients will probably want to support that.

I'd suggest that this whole thread can be resolved by the following:

  1. Servers can provide their own filtered inboxes, such as plain ol' inbox but also directMessages. These can be added as an extension, and servers are free to decide how to filter into them however they like.
  2. If you'd like to help sort these in some sort of way, a notification flag can be added to messages, as an extension. For example, we could have "directMessage": true and that should be able to help inform a server appropriately. Such a flag can also be an extension.

I think that wraps up this whole thing and provides a path forward. From there, I'd like to bring this to the community group.

What do people think?

@annando

This comment has been minimized.

Copy link

annando commented May 30, 2017

This "directMessage": true would be the easiest way since this wouldn't cause any problems between servers that support this and others that doesn't support this. But some rules should be applied as well.

"directMessage": true does only makes sense with posts to a limited set of users. And it should be noted that this "directMessage": true only needs to be apply to the starting post. This has the following background: I'm writing such a direct message from some server that supports it to another server that doesn't support it. There someone answers. Now the post doesn't contain the "directMessage": true - but should - of course - be arranged as an answer to the first post.

On the other hand it would be cool to have some possibility to fetch the capabilities of a profile. Then one could detect if the other server supports direct messages at all.

@cwebber

This comment has been minimized.

Copy link
Collaborator

cwebber commented May 30, 2017

@annando Great, glad to hear the "directMessage": true route sounds good to you. :)

While I don't think it's within the realm of handling within ActivityPub itself (at least within the time period we have), fetching capabilities of a profile would be great. I'm not sure if this would be an ActivityPub extension or a reuse of some Interface Description Language that exists, but it seems worthy of exploration in the community group.

@wilkie

This comment has been minimized.

Copy link

wilkie commented May 31, 2017

You need a large disclaimer that direct messages are not private if you do it with a flag. Because non-conforming implementations are going to render these posts publicly.

You do need an extension that adds a route (secondary inbox) for strictly private messages, generally. So that non-conforming implementations just 404 instead of trying too hard to render them.

@nightpool

This comment has been minimized.

Copy link

nightpool commented May 31, 2017

@wilkie this is about distinguishing between direct message and other types of private posts, I think? My understanding is that the audience specifier pretty easily handles post visibility, and people just want a specific way to call out a "direct" message

@wilkie

This comment has been minimized.

Copy link

wilkie commented May 31, 2017

@nightpool this whole thing doesn't make much sense, then. if direct messages are intended to be private (and not simply public messages directed at a single person), and the spec ignores privacy, then you have to have a separate route; otherwise non-conforming implementations will render them as public messages.

@nightpool

This comment has been minimized.

Copy link

nightpool commented May 31, 2017

@wilkie sorry, to rephrase that: My understanding is that what people proposing is that direct messages are a subset of the existing private messages (messages that use bto/bcc or that don't have any targeting).

@wilkie

This comment has been minimized.

Copy link

wilkie commented May 31, 2017

@nightpool Ah. Fair enough. It seems important to point out, then, that the spec, including the audience tracking sections of AS2, do not suggest such messages are actually private.

I have historically had a hard time seeing how access control is described in either spec, but people seem to speak very confidently that it is somehow there. At least, I don't see enough there to convince me that private messages are actually spec'd at all. Implementations that don't have private message support are completely free to render all incoming messages as public messages from what I can tell.

Not requiring access control in the spec seems very appropriate! But it should mean that extensions that do talk about private messages (implying access control) need a mechanism that hard-fails on non-conforming implementations. (therefore: a separate route)

I also like the bto/bcc to denote a private message, but implementations are only recommended to make such messages private. Which means a private messaging extension must create a brand new mechanism.

Basically: direct message flag is fine if direct messages are not private messages. Private messages are (justifiably) not defined, as far as I can tell, in ActivityPub or ActivityStreams. Or, admittedly equally likely, I simply have no idea what a "private message" is.

@cwebber

This comment has been minimized.

Copy link
Collaborator

cwebber commented Jun 1, 2017

Let's leave out the term "private message" indeed and just refer to "direct message" then.

@wilkie Yes, I think this isn't any promise of further protection to try to keep things private. Either you have an ACL inferred by the addressing, or you don't, and hopefully you do. All the directMessage flag thing will do is really provide information to expose things differently in the user interface. I agree it's confusing (and this thread has gotten so long it's hard to keep track of such things anyway.)

@astronouth7303

This comment has been minimized.

Copy link

astronouth7303 commented Jun 1, 2017

re: ACL by IP address: Everyone on Google Cloud is put through Google's URL Fetch service. AWS Lambda could be on any IP. A great many autoscaling setups could have a number of IPs, most likely from a shared pool.

Basically: Using the IP address to infer provider is fairly risky, and should only serve as a corroborating signal.

@cwebber

This comment has been minimized.

Copy link
Collaborator

cwebber commented Jun 1, 2017

@astronouth7303 Was anyone suggesting an ACL by IP address??? I think we were talking about an ACL that applied to authenticated actors. I would assume that any implementation that did authentication by IP address is an authentication written by a developer that needs their developer badge revoked.

@cwebber

This comment has been minimized.

Copy link
Collaborator

cwebber commented Jun 1, 2017

Oh, you saw "ACL by addressing" and misread I think. We were talking about ActivityStreams addressing, which is to say the to, cc, bcc, and bto fields on an AS2 object.

@strugee

This comment has been minimized.

Copy link

strugee commented Jun 5, 2017

FWIW, I've added a couple things here to https://www.w3.org/wiki/ActivityPub_extensions.

@geppy

This comment has been minimized.

Copy link

geppy commented Jun 7, 2017

There's an informal SocialCG/SWICG consensus supporting expressing intent to target a subinbox via a flag/property.

@nightpool

This comment has been minimized.

Copy link

nightpool commented Jun 7, 2017

The consensus from the SocialCG Meeting today was that we should either use a different note type (where fail-fast is desired) or a directMessage flag (where compatibility is desired) for server-to-server communications. Then servers can sort into this into different inboxes per their own discretion and their clients' preferences, and we should have a thing in the client-server protocol to support this.

@cwebber

This comment has been minimized.

Copy link
Collaborator

cwebber commented Jul 25, 2017

So, this is something that should be done. Given that we're going to do it as either a new type or new property, that's extension territory.

Would someone like to take the initiative of starting this as an AS2 extension?

@annando

This comment has been minimized.

Copy link

annando commented Oct 25, 2017

Does the closing of this issue mean that it was done or that it won't be done at all?

@rhiaro

This comment has been minimized.

Copy link
Member

rhiaro commented Oct 25, 2017

Oops that was an accident.

@rhiaro rhiaro reopened this Oct 25, 2017

@rhiaro

This comment has been minimized.

Copy link
Member

rhiaro commented Oct 25, 2017

Oh wait no it wasn't. I was closing postponed issues to clear space. But having re-read the last couple of messages, I'm confused about its postponed status. @cwebber is this something that needs opening in the CG instead?

@cwebber

This comment has been minimized.

Copy link
Collaborator

cwebber commented Oct 28, 2017

It should move to the CG. We've agreed that the ActivityPub spec will not take on changes that are vocabulary in a way that isn't strictly the protocol itself at this point.

A few options, maybe one or all as next steps:

Interested parties are encouraged to continue this by taking one or more of the above steps! So that is to say: it's something to pursue, but not on this tracker now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment