New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add the CHATHISTORY extension specification. #292

Open
wants to merge 29 commits into
base: master
from

Conversation

Projects
None yet
9 participants
@MuffinMedic
Contributor

MuffinMedic commented Jan 2, 2017

@jwheare

This comment has been minimized.

Show comment
Hide comment
@jwheare

jwheare Jan 2, 2017

Member

Interesting spec. Can you elaborate why msgid is required?

Member

jwheare commented Jan 2, 2017

Interesting spec. Can you elaborate why msgid is required?

@prawnsalad

This comment has been minimized.

Show comment
Hide comment
@prawnsalad

prawnsalad Jan 3, 2017

As mentioned on IRC I've been getting this idea into the next version of Kiwi IRC. Something that stands out during implementation:

No acknowledgement by the server of the command is required other then returning the requested content.

It would be tidier if this was required to send an empty batch if there is no content to be returned. Just so the client doesn't sit there waiting for nothing.

prawnsalad commented Jan 3, 2017

As mentioned on IRC I've been getting this idea into the next version of Kiwi IRC. Something that stands out during implementation:

No acknowledgement by the server of the command is required other then returning the requested content.

It would be tidier if this was required to send an empty batch if there is no content to be returned. Just so the client doesn't sit there waiting for nothing.

@MuffinMedic

This comment has been minimized.

Show comment
Hide comment
@MuffinMedic

MuffinMedic Jan 3, 2017

Contributor

@jwheare The draft/msgid is to allow for usage of the draft/reply tag in servers where multiple users may have the same scrollback content and a user wishes to reply to an older message. The draft/msgid would be that which the server originally sent to clients. If a user replies to a scrollback message, other users with the same history can scrollback as well and see the original message replied to.

@prawnsalad Agreed & updated.

Contributor

MuffinMedic commented Jan 3, 2017

@jwheare The draft/msgid is to allow for usage of the draft/reply tag in servers where multiple users may have the same scrollback content and a user wishes to reply to an older message. The draft/msgid would be that which the server originally sent to clients. If a user replies to a scrollback message, other users with the same history can scrollback as well and see the original message replied to.

@prawnsalad Agreed & updated.

@dequis

This comment has been minimized.

Show comment
Hide comment
@dequis

dequis Jan 3, 2017

Contributor

Thanks for taking the time to write a spec for this!

This should probably be together with the other extensions, not just a batch type. In fact, I don't see a strong reason for this to use a new batch type, if it's just chathistory plus event replay.

Please read the previous discussion on why event replay wasn't added as part of the chathistory batch: #156 - the short version is that the way this spec handles it isn't enough.

to allow for usage of the draft/reply tag

While I agree that sounds desirable and that all messages from history should have an ID (also helps with deduplication), there's no need to make that into a requirement.

Contributor

dequis commented Jan 3, 2017

Thanks for taking the time to write a spec for this!

This should probably be together with the other extensions, not just a batch type. In fact, I don't see a strong reason for this to use a new batch type, if it's just chathistory plus event replay.

Please read the previous discussion on why event replay wasn't added as part of the chathistory batch: #156 - the short version is that the way this spec handles it isn't enough.

to allow for usage of the draft/reply tag

While I agree that sounds desirable and that all messages from history should have an ID (also helps with deduplication), there's no need to make that into a requirement.

@MuffinMedic

This comment has been minimized.

Show comment
Hide comment
@MuffinMedic

MuffinMedic Jan 3, 2017

Contributor

@dequis Do you suggest the extension be implemented as an RPL_ISUPPORT 005 command or as the scrollback capability? The original implementation I had was as a CAP but initial feedback suggested to simply send the 005 numeric instead.

While I agree that sounds desirable and that all messages from history should have an ID (also helps with deduplication), there's no need to make that into a requirement.

Agreed. Updated to make draft/msgid optional but recommended.

Contributor

MuffinMedic commented Jan 3, 2017

@dequis Do you suggest the extension be implemented as an RPL_ISUPPORT 005 command or as the scrollback capability? The original implementation I had was as a CAP but initial feedback suggested to simply send the 005 numeric instead.

While I agree that sounds desirable and that all messages from history should have an ID (also helps with deduplication), there's no need to make that into a requirement.

Agreed. Updated to make draft/msgid optional but recommended.

@dequis

This comment has been minimized.

Show comment
Hide comment
@dequis

dequis Jan 3, 2017

Contributor

Yeah just an ISUPPORT token is fine for this.

Contributor

dequis commented Jan 3, 2017

Yeah just an ISUPPORT token is fine for this.

@MuffinMedic MuffinMedic changed the title from Add the SCROLLBACK batch type specification. to Add the SCROLLBACK extension specification. Jan 3, 2017

@prawnsalad

This comment has been minimized.

Show comment
Hide comment
@prawnsalad

prawnsalad Jan 3, 2017

The changes to use chathistory makes it more confusing now. Sending the SCROLLBACK command to get a chathistory batch type now uses 2 different terms to refer to the same thing - scrollback.

Would there be anything stopping us from using a CHATHISTORY command instead to keep things consistent?

prawnsalad commented Jan 3, 2017

The changes to use chathistory makes it more confusing now. Sending the SCROLLBACK command to get a chathistory batch type now uses 2 different terms to refer to the same thing - scrollback.

Would there be anything stopping us from using a CHATHISTORY command instead to keep things consistent?

@MuffinMedic

This comment has been minimized.

Show comment
Hide comment
@MuffinMedic

MuffinMedic Jan 3, 2017

Contributor

@prawnsalad Fixed in d261046 - Use chathistory as both extension and batch type for consistency.

Contributor

MuffinMedic commented Jan 3, 2017

@prawnsalad Fixed in d261046 - Use chathistory as both extension and batch type for consistency.

@SaberUK

This comment has been minimized.

Show comment
Hide comment
@SaberUK

SaberUK Jan 3, 2017

Contributor

I'm wondering if for the purpose of this specification we might be better off introducing a new 'HISTORY' batch type which works similar to chathistory but for all messages instead of just PRIVMSG/NOTICE. The chathistory batch was only really intended for lightweight stuff like ZNC scrollback and InspIRCd's chathistory module.

Contributor

SaberUK commented Jan 3, 2017

I'm wondering if for the purpose of this specification we might be better off introducing a new 'HISTORY' batch type which works similar to chathistory but for all messages instead of just PRIVMSG/NOTICE. The chathistory batch was only really intended for lightweight stuff like ZNC scrollback and InspIRCd's chathistory module.

@MuffinMedic MuffinMedic changed the title from Add the SCROLLBACK extension specification. to Add the CHATHISTORY extension specification. Jan 3, 2017

@dequis

This comment has been minimized.

Show comment
Hide comment
@dequis

dequis Jan 3, 2017

Contributor

a new 'HISTORY' batch type which works similar to chathistory but for all messages instead of just PRIVMSG/NOTICE

A new batch type will have the same problems the chathistory batch had. Batch types by themselves can't introduce the sort of behavior changes we need to stop (for example) a QUIT from the past making someone quit in the present.

We haven't talked much about how to do the event replay thing, maybe it's worth doing it in a new issue. But the ideas i have in mind could reasonably be added retroactively to chathistory.

Contributor

dequis commented Jan 3, 2017

a new 'HISTORY' batch type which works similar to chathistory but for all messages instead of just PRIVMSG/NOTICE

A new batch type will have the same problems the chathistory batch had. Batch types by themselves can't introduce the sort of behavior changes we need to stop (for example) a QUIT from the past making someone quit in the present.

We haven't talked much about how to do the event replay thing, maybe it's worth doing it in a new issue. But the ideas i have in mind could reasonably be added retroactively to chathistory.

@Zarthus

This comment has been minimized.

Show comment
Hide comment
@Zarthus

Zarthus Jan 4, 2017

I have some serious concerns about how this is going to be properly handled security wise.

I think for bouncers this'll be great, but for an ircd I can think of many ways in which this can be abused, and i'm not sure to which point it would be a violation of privacy and where the line'll be drawn. Logs would be very unlikely to have the state of a channel knowledge.

Some of the below points don't necessarily need to be mentioned in the spec, but I'm curious on how some things would be handled nevertheless.


I would consider getting some numerics for errors (to which point it is relevant to IRCv3): What should happen if I do a /CHATHISTORY #channel-i'm-not-in or /CHATHISTORY #secret-channel? Personally, I think access should be rejected, but the specification gives no guidance on how to do this.


  • User1 sends STATUSMSG(@) to #coolchannel
  • User1 ops User2 in #coolchannel
  • User2 does a /CHATHISTORY #coolchannel

Should they receive the opnotice they were not opped for?
I think it'll be hard to keep track of every single edge case.


  • User1 accidentally pastes their password, email, or address. While they change it, do we want to permit a way to delete chathistory? Should the user be able to do it for their own messages?
    • This doesn't have to be the user themselves, plenty of doxxing attempts happen on IRC.

  • Some channels have confidentiality settings (think +mz, or +qz on charybdis)
  • opped User1 talks with deopped User{2,3,4}
  • Spam stops. -mz gets set.
  • Should User2 be able to request chathistory for the logs of when +mz was enabled?

  • coolserver.coolnework.net splits from hub.coolnetwork.net
  • User{1,2,3} chat on coolserver.* whilst split, User{4,5,6} chat on hub.*
  • coolserver.* rejoins hub.*
  • User1 wants to view the chat on hub.*'s chat, do they have to reconnect to hub to see it?

With that, an optional server parameter might be worth considering. Although with different specifications (like server-time) some sort of resync between messages for split servers could also exist in the future.


  • Services are down. My channel is SET RESTRICTED.
  • User2 joins my channel, and requests a CHATHISTORY.

With the above points, I really think that for a bouncer oriented specification, it'll be great. And while I'd like to keep the option for ircds to implement it, I have a hard time seeing how they would cover some of the problems. I imagine some sort of opt-in mechanic could work, but this comment is already getting quite long and it's a bit too much outside of the IRCv3 realm of influence.

Zarthus commented Jan 4, 2017

I have some serious concerns about how this is going to be properly handled security wise.

I think for bouncers this'll be great, but for an ircd I can think of many ways in which this can be abused, and i'm not sure to which point it would be a violation of privacy and where the line'll be drawn. Logs would be very unlikely to have the state of a channel knowledge.

Some of the below points don't necessarily need to be mentioned in the spec, but I'm curious on how some things would be handled nevertheless.


I would consider getting some numerics for errors (to which point it is relevant to IRCv3): What should happen if I do a /CHATHISTORY #channel-i'm-not-in or /CHATHISTORY #secret-channel? Personally, I think access should be rejected, but the specification gives no guidance on how to do this.


  • User1 sends STATUSMSG(@) to #coolchannel
  • User1 ops User2 in #coolchannel
  • User2 does a /CHATHISTORY #coolchannel

Should they receive the opnotice they were not opped for?
I think it'll be hard to keep track of every single edge case.


  • User1 accidentally pastes their password, email, or address. While they change it, do we want to permit a way to delete chathistory? Should the user be able to do it for their own messages?
    • This doesn't have to be the user themselves, plenty of doxxing attempts happen on IRC.

  • Some channels have confidentiality settings (think +mz, or +qz on charybdis)
  • opped User1 talks with deopped User{2,3,4}
  • Spam stops. -mz gets set.
  • Should User2 be able to request chathistory for the logs of when +mz was enabled?

  • coolserver.coolnework.net splits from hub.coolnetwork.net
  • User{1,2,3} chat on coolserver.* whilst split, User{4,5,6} chat on hub.*
  • coolserver.* rejoins hub.*
  • User1 wants to view the chat on hub.*'s chat, do they have to reconnect to hub to see it?

With that, an optional server parameter might be worth considering. Although with different specifications (like server-time) some sort of resync between messages for split servers could also exist in the future.


  • Services are down. My channel is SET RESTRICTED.
  • User2 joins my channel, and requests a CHATHISTORY.

With the above points, I really think that for a bouncer oriented specification, it'll be great. And while I'd like to keep the option for ircds to implement it, I have a hard time seeing how they would cover some of the problems. I imagine some sort of opt-in mechanic could work, but this comment is already getting quite long and it's a bit too much outside of the IRCv3 realm of influence.

@jwheare jwheare modified the milestone: Roadmap Jan 7, 2017

@jwheare jwheare added the ids label Jan 7, 2017

Show outdated Hide outdated extensions/chathistory-3.3.md
Show outdated Hide outdated extensions/chathistory-3.3.md
Show outdated Hide outdated extensions/chathistory-3.3.md
Show outdated Hide outdated extensions/chathistory-3.3.md
@prawnsalad

This comment has been minimized.

Show comment
Hide comment
@prawnsalad

prawnsalad Jan 10, 2017

The latest changes now forbids an empty BATCH response if there is no content to return, and instead forces an error to be returned. However if I request CHATHISTORY for a channel that has no more content then this isn't an error - there's just no more content. An empty BATCH is more descriptive and easier to handle here.

It now also states that draft/labeled-response is required when this could be easily be optional but recommended such as draft/msgid. Relying on any draft/* specs could hold back any clients wanting to implement CHATHISTORY for some time.

prawnsalad commented Jan 10, 2017

The latest changes now forbids an empty BATCH response if there is no content to return, and instead forces an error to be returned. However if I request CHATHISTORY for a channel that has no more content then this isn't an error - there's just no more content. An empty BATCH is more descriptive and easier to handle here.

It now also states that draft/labeled-response is required when this could be easily be optional but recommended such as draft/msgid. Relying on any draft/* specs could hold back any clients wanting to implement CHATHISTORY for some time.

@LFP6

This comment has been minimized.

Show comment
Hide comment
@LFP6

LFP6 Feb 13, 2017

Could this be extended so that you can request a certain number of messages to either side of a given timestamp/id? I'm imagining ircv3/ircv3-ideas#9 being implemented in a client so that you can jump back to that point in history (as Slack and Discord do).

LFP6 commented Feb 13, 2017

Could this be extended so that you can request a certain number of messages to either side of a given timestamp/id? I'm imagining ircv3/ircv3-ideas#9 being implemented in a client so that you can jump back to that point in history (as Slack and Discord do).

@Zarthus

This comment has been minimized.

Show comment
Hide comment
@Zarthus

Zarthus Feb 13, 2017

Extensions to the spec would probably make more sense as a separate specification, rather than to have one do-it-all spec full of "this is all an optional part of the spec".

Zarthus commented Feb 13, 2017

Extensions to the spec would probably make more sense as a separate specification, rather than to have one do-it-all spec full of "this is all an optional part of the spec".

@LFP6 LFP6 referenced this pull request Feb 13, 2017

Merged

Reaction client tag #289

@MuffinMedic

This comment has been minimized.

Show comment
Hide comment
@MuffinMedic

MuffinMedic Feb 13, 2017

Contributor

Adding "x lines before and after" would introduce an unneeded level of complexity as it requires an additional parameter for specify the "direction" of the history. I think a better solution is allowing the message_count to accept both positive and negative integers, where a positive value means "x lines after the timestamp" and a negative number represents "x lines before the timestamp".

A search-type spec could simply request content both before and after. This would give a simpler yet more versatile approach I feel, although I'd like to see what others say before implementing it into the spec.

Contributor

MuffinMedic commented Feb 13, 2017

Adding "x lines before and after" would introduce an unneeded level of complexity as it requires an additional parameter for specify the "direction" of the history. I think a better solution is allowing the message_count to accept both positive and negative integers, where a positive value means "x lines after the timestamp" and a negative number represents "x lines before the timestamp".

A search-type spec could simply request content both before and after. This would give a simpler yet more versatile approach I feel, although I'd like to see what others say before implementing it into the spec.

@LFP6

This comment has been minimized.

Show comment
Hide comment
@LFP6

LFP6 Feb 13, 2017

That seems good to me. While less concise, it also means that the server doesn't need to parse through the history nor does the client need to parse through the response to determine what comes before and what comes after.

LFP6 commented Feb 13, 2017

That seems good to me. While less concise, it also means that the server doesn't need to parse through the history nor does the client need to parse through the response to determine what comes before and what comes after.

@jwheare

This comment has been minimized.

Show comment
Hide comment
@jwheare

jwheare Mar 6, 2017

Member

Can we get an example of a full client/server exchange for this?

Member

jwheare commented Mar 6, 2017

Can we get an example of a full client/server exchange for this?

@DarthGandalf

This comment has been minimized.

Show comment
Hide comment
@DarthGandalf

DarthGandalf Mar 8, 2017

Member

Including the message with the specified id?

Member

DarthGandalf commented on extensions/chathistory.md in 0af7450 Mar 8, 2017

Including the message with the specified id?

This comment has been minimized.

Show comment
Hide comment
@jwheare

jwheare Mar 8, 2017

Member

s/specifie/specified

Member

jwheare replied Mar 8, 2017

s/specifie/specified

Show outdated Hide outdated extensions/chathistory-3.3.md
Show outdated Hide outdated extensions/chathistory.md
Show outdated Hide outdated extensions/chathistory.md
Show outdated Hide outdated extensions/chathistory.md
Show outdated Hide outdated extensions/chathistory.md
Show outdated Hide outdated extensions/chathistory.md
Show outdated Hide outdated extensions/chathistory.md
Show outdated Hide outdated extensions/chathistory.md
Show outdated Hide outdated extensions/chathistory.md
Show outdated Hide outdated extensions/chathistory.md
@kylef

This comment has been minimized.

Show comment
Hide comment
@kylef

kylef Apr 21, 2017

Contributor

I spotted #293 which states:

The chathistory batch is currently limited to PRIVMSG and NOTICE and has no way to some events because they may be processed by the client in a way that affects the present channel state.

I don't see this actually mentioned anywhere so I think it should be mentioned that only PRIVMSG and NOTICE messages can be apart of the chat history.

Contributor

kylef commented Apr 21, 2017

I spotted #293 which states:

The chathistory batch is currently limited to PRIVMSG and NOTICE and has no way to some events because they may be processed by the client in a way that affects the present channel state.

I don't see this actually mentioned anywhere so I think it should be mentioned that only PRIVMSG and NOTICE messages can be apart of the chat history.

@MuffinMedic

This comment has been minimized.

Show comment
Hide comment
@MuffinMedic

MuffinMedic Apr 21, 2017

Contributor

PRIVMSG and NOTICES aren't limitations with this specification - it derives from limitations with the batch type, which this extension requires be supported. This is a dependency limitation.

If #293 addresses these issues and batch/chathistory no longer has these limitations, then CHATHISTORY would automatically support the additional event types.

Contributor

MuffinMedic commented Apr 21, 2017

PRIVMSG and NOTICES aren't limitations with this specification - it derives from limitations with the batch type, which this extension requires be supported. This is a dependency limitation.

If #293 addresses these issues and batch/chathistory no longer has these limitations, then CHATHISTORY would automatically support the additional event types.

@medik medik referenced this pull request Aug 17, 2017

Open

ZNC *playback support #323

@DarthGandalf

This comment has been minimized.

Show comment
Hide comment
@DarthGandalf

DarthGandalf Aug 23, 2017

Member

Are you sure about this? With a self-contradicting spec, it's impossible to write a conforming implementation :)

Member

DarthGandalf commented on extensions/chathistory.md in d054b8b Aug 23, 2017

Are you sure about this? With a self-contradicting spec, it's impossible to write a conforming implementation :)

This comment has been minimized.

Show comment
Hide comment
@MuffinMedic

MuffinMedic Aug 23, 2017

Contributor

Nice catch, fixed in a210bfc

Contributor

MuffinMedic replied Aug 23, 2017

Nice catch, fixed in a210bfc

Show outdated Hide outdated extensions/chathistory.md
Show outdated Hide outdated extensions/chathistory.md
@jwheare

This comment has been minimized.

Show comment
Hide comment
@jwheare
Member

jwheare commented on extensions/chathistory.md in 4042cec Aug 25, 2017

z

@Zarthus

This comment has been minimized.

Show comment
Hide comment
@Zarthus

Zarthus commented on extensions/chathistory.md in 4042cec Aug 25, 2017

e

:irc.host BATCH -ID
@draft/label=ID :nick!ident@host CHATHISTORY ERR ERR_CODE

This comment has been minimized.

@Zarthus

Zarthus Aug 25, 2017

Without explicitly mentioning ERR as a verb (and WARN) implementors may think that this is not a constant string and get confused. A reference to some sort of document explaining subcommands may be useful

@Zarthus

Zarthus Aug 25, 2017

Without explicitly mentioning ERR as a verb (and WARN) implementors may think that this is not a constant string and get confused. A reference to some sort of document explaining subcommands may be useful

Content can also be requested up to a specified timestamp or `draft/msgid` in place of the `message_count`. The start and end parameter types do not have to match:
@draft/label=ID CHATHISTORY <target> timestamp=<timestamp> +draft/msgid=<message_id>

This comment has been minimized.

@Zarthus

Zarthus Aug 25, 2017

timestamp without an @ symbol?

@Zarthus

Zarthus Aug 25, 2017

timestamp without an @ symbol?

This comment has been minimized.

@MuffinMedic

MuffinMedic Aug 25, 2017

Contributor

timestamp in this case is a command parameter, not a client tag.

@MuffinMedic

MuffinMedic Aug 25, 2017

Contributor

timestamp in this case is a command parameter, not a client tag.

The `server-time` SHOULD be the time at which the message was received from the IRC server and the `batch id` a unique ID for the entire batch. `draft/label` SHOULD be included and MUST be a unique ID used to identify the `chathistory` request and any replies. A `draft/msgid` to identify each individual message MUST be the `draft/msgid` included when each message was first received from the IRC server.
### `CHATHISTORY` Command
`CHATHISTORY` content can be requested by the client to the server by sending the `CHATHISTORY` command to the server. A `batch` MUST be returned by the server. If no content exists to return, an empty batch SHOULD be returned to avoid the client waiting for a reply. Command support is sent to the client as the RPL_ISUPPORT 005 numeric `:irc.host 005 nick CHATHISTORY=max_message_count :are supported by this server`

This comment has been minimized.

@Zarthus

Zarthus Aug 25, 2017

Perhaps instead of "an empty batch ..." we could do "CHATHISTORY SUCCESS NOOP" / NO_DATA whatever can be returned

@Zarthus

Zarthus Aug 25, 2017

Perhaps instead of "an empty batch ..." we could do "CHATHISTORY SUCCESS NOOP" / NO_DATA whatever can be returned

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment