Add the CHATHISTORY extension specification.

[MuffinMedic]

Rendered view available here.

[jwheare]

Interesting spec. Can you elaborate why msgid is required?

[jwheare]

Interesting spec. Can you elaborate why msgid is required?

[prawnsalad]

As mentioned on IRC I've been getting this idea into the next version of Kiwi IRC. Something that stands out during implementation:

No acknowledgement by the server of the command is required other then returning the requested content.

It would be tidier if this was required to send an empty batch if there is no content to be returned. Just so the client doesn't sit there waiting for nothing.

[prawnsalad]

As mentioned on IRC I've been getting this idea into the next version of Kiwi IRC. Something that stands out during implementation:

No acknowledgement by the server of the command is required other then returning the requested content.

It would be tidier if this was required to send an empty batch if there is no content to be returned. Just so the client doesn't sit there waiting for nothing.

[MuffinMedic]

@jwheare The draft/msgid is to allow for usage of the draft/reply tag in servers where multiple users may have the same scrollback content and a user wishes to reply to an older message. The draft/msgid would be that which the server originally sent to clients. If a user replies to a scrollback message, other users with the same history can scrollback as well and see the original message replied to.

@prawnsalad Agreed & updated.

[MuffinMedic]

@jwheare The draft/msgid is to allow for usage of the draft/reply tag in servers where multiple users may have the same scrollback content and a user wishes to reply to an older message. The draft/msgid would be that which the server originally sent to clients. If a user replies to a scrollback message, other users with the same history can scrollback as well and see the original message replied to.

@prawnsalad Agreed & updated.

[dequis]

Thanks for taking the time to write a spec for this!

This should probably be together with the other extensions, not just a batch type. In fact, I don't see a strong reason for this to use a new batch type, if it's just chathistory plus event replay.

Please read the previous discussion on why event replay wasn't added as part of the chathistory batch: #156 - the short version is that the way this spec handles it isn't enough.

to allow for usage of the draft/reply tag

While I agree that sounds desirable and that all messages from history should have an ID (also helps with deduplication), there's no need to make that into a requirement.

[dequis]

Thanks for taking the time to write a spec for this!

This should probably be together with the other extensions, not just a batch type. In fact, I don't see a strong reason for this to use a new batch type, if it's just chathistory plus event replay.

Please read the previous discussion on why event replay wasn't added as part of the chathistory batch: #156 - the short version is that the way this spec handles it isn't enough.

to allow for usage of the draft/reply tag

While I agree that sounds desirable and that all messages from history should have an ID (also helps with deduplication), there's no need to make that into a requirement.

[MuffinMedic]

@dequis Do you suggest the extension be implemented as an RPL_ISUPPORT 005 command or as the scrollback capability? The original implementation I had was as a CAP but initial feedback suggested to simply send the 005 numeric instead.

While I agree that sounds desirable and that all messages from history should have an ID (also helps with deduplication), there's no need to make that into a requirement.

Agreed. Updated to make draft/msgid optional but recommended.

[MuffinMedic]

@dequis Do you suggest the extension be implemented as an RPL_ISUPPORT 005 command or as the scrollback capability? The original implementation I had was as a CAP but initial feedback suggested to simply send the 005 numeric instead.

While I agree that sounds desirable and that all messages from history should have an ID (also helps with deduplication), there's no need to make that into a requirement.

Agreed. Updated to make draft/msgid optional but recommended.

[dequis]

Yeah just an ISUPPORT token is fine for this.

[dequis]

Yeah just an ISUPPORT token is fine for this.

[prawnsalad]

The changes to use chathistory makes it more confusing now. Sending the SCROLLBACK command to get a chathistory batch type now uses 2 different terms to refer to the same thing - scrollback.

Would there be anything stopping us from using a CHATHISTORY command instead to keep things consistent?

[prawnsalad]

The changes to use chathistory makes it more confusing now. Sending the SCROLLBACK command to get a chathistory batch type now uses 2 different terms to refer to the same thing - scrollback.

Would there be anything stopping us from using a CHATHISTORY command instead to keep things consistent?

[SaberUK]

Typo.

[MuffinMedic]

Fixed in d261046

[SaberUK]

Wouldn't it be better to make it two parameters rather than using one separated with =?

[SaberUK]

Also:

• Can message_count be negative to go backwards?
• Should the max message_count be in the ISUPPORT token?
• What numerics are sent if message_count is invalid?

[MuffinMedic]

Wouldn't it be better to make it two parameters rather than using one separated with =?

No preference either way. Any reason two parameters are better?

Can message_count be negative to go backwards?

Backwards in what sense? Send messages in reverse order or else? What is the use case?

Should the max message_count be in the ISUPPORT token?

Added in c4883c0

What numerics are sent if message_count is invalid?

Oversight and not considered. Suggestions?

[SaberUK]

This should link to the website pages for those specs rather than the markdown documents.

[MuffinMedic]

Fixed in d261046

[MuffinMedic]

@prawnsalad Fixed in d261046 - Use chathistory as both extension and batch type for consistency.

[MuffinMedic]

@prawnsalad Fixed in d261046 - Use chathistory as both extension and batch type for consistency.

[SaberUK]

I'm wondering if for the purpose of this specification we might be better off introducing a new 'HISTORY' batch type which works similar to chathistory but for all messages instead of just PRIVMSG/NOTICE. The chathistory batch was only really intended for lightweight stuff like ZNC scrollback and InspIRCd's chathistory module.

[SaberUK]

I'm wondering if for the purpose of this specification we might be better off introducing a new 'HISTORY' batch type which works similar to chathistory but for all messages instead of just PRIVMSG/NOTICE. The chathistory batch was only really intended for lightweight stuff like ZNC scrollback and InspIRCd's chathistory module.

[dequis]

a new 'HISTORY' batch type which works similar to chathistory but for all messages instead of just PRIVMSG/NOTICE

A new batch type will have the same problems the chathistory batch had. Batch types by themselves can't introduce the sort of behavior changes we need to stop (for example) a QUIT from the past making someone quit in the present.

We haven't talked much about how to do the event replay thing, maybe it's worth doing it in a new issue. But the ideas i have in mind could reasonably be added retroactively to chathistory.

[dequis]

a new 'HISTORY' batch type which works similar to chathistory but for all messages instead of just PRIVMSG/NOTICE

A new batch type will have the same problems the chathistory batch had. Batch types by themselves can't introduce the sort of behavior changes we need to stop (for example) a QUIT from the past making someone quit in the present.

We haven't talked much about how to do the event replay thing, maybe it's worth doing it in a new issue. But the ideas i have in mind could reasonably be added retroactively to chathistory.

[Zarthus]

I have some serious concerns about how this is going to be properly handled security wise.

I think for bouncers this'll be great, but for an ircd I can think of many ways in which this can be abused, and i'm not sure to which point it would be a violation of privacy and where the line'll be drawn. Logs would be very unlikely to have the state of a channel knowledge.

Some of the below points don't necessarily need to be mentioned in the spec, but I'm curious on how some things would be handled nevertheless.

---

I would consider getting some numerics for errors (to which point it is relevant to IRCv3): What should happen if I do a /CHATHISTORY #channel-i'm-not-in or /CHATHISTORY #secret-channel? Personally, I think access should be rejected, but the specification gives no guidance on how to do this.

---

• User1 sends STATUSMSG(@) to #coolchannel
• User1 ops User2 in #coolchannel
• User2 does a /CHATHISTORY #coolchannel

Should they receive the opnotice they were not opped for?
I think it'll be hard to keep track of every single edge case.

---

• User1 accidentally pastes their password, email, or address. While they change it, do we want to permit a way to delete chathistory? Should the user be able to do it for their own messages?
  • This doesn't have to be the user themselves, plenty of doxxing attempts happen on IRC.

---

• Some channels have confidentiality settings (think +mz, or +qz on charybdis)
• opped User1 talks with deopped User{2,3,4}
• Spam stops. -mz gets set.
• Should User2 be able to request chathistory for the logs of when +mz was enabled?

---

• coolserver.coolnework.net splits from hub.coolnetwork.net
• User{1,2,3} chat on coolserver.* whilst split, User{4,5,6} chat on hub.*
• coolserver.* rejoins hub.*
• User1 wants to view the chat on hub.*'s chat, do they have to reconnect to hub to see it?

With that, an optional server parameter might be worth considering. Although with different specifications (like server-time) some sort of resync between messages for split servers could also exist in the future.

---

• Services are down. My channel is SET RESTRICTED.
• User2 joins my channel, and requests a CHATHISTORY.

---

With the above points, I really think that for a bouncer oriented specification, it'll be great. And while I'd like to keep the option for ircds to implement it, I have a hard time seeing how they would cover some of the problems. I imagine some sort of opt-in mechanic could work, but this comment is already getting quite long and it's a bit too much outside of the IRCv3 realm of influence.

[Zarthus]

I have some serious concerns about how this is going to be properly handled security wise.

I think for bouncers this'll be great, but for an ircd I can think of many ways in which this can be abused, and i'm not sure to which point it would be a violation of privacy and where the line'll be drawn. Logs would be very unlikely to have the state of a channel knowledge.

Some of the below points don't necessarily need to be mentioned in the spec, but I'm curious on how some things would be handled nevertheless.

---

I would consider getting some numerics for errors (to which point it is relevant to IRCv3): What should happen if I do a /CHATHISTORY #channel-i'm-not-in or /CHATHISTORY #secret-channel? Personally, I think access should be rejected, but the specification gives no guidance on how to do this.

---

• User1 sends STATUSMSG(@) to #coolchannel
• User1 ops User2 in #coolchannel
• User2 does a /CHATHISTORY #coolchannel

Should they receive the opnotice they were not opped for?
I think it'll be hard to keep track of every single edge case.

---

• User1 accidentally pastes their password, email, or address. While they change it, do we want to permit a way to delete chathistory? Should the user be able to do it for their own messages?
  • This doesn't have to be the user themselves, plenty of doxxing attempts happen on IRC.

---

• Some channels have confidentiality settings (think +mz, or +qz on charybdis)
• opped User1 talks with deopped User{2,3,4}
• Spam stops. -mz gets set.
• Should User2 be able to request chathistory for the logs of when +mz was enabled?

---

• coolserver.coolnework.net splits from hub.coolnetwork.net
• User{1,2,3} chat on coolserver.* whilst split, User{4,5,6} chat on hub.*
• coolserver.* rejoins hub.*
• User1 wants to view the chat on hub.*'s chat, do they have to reconnect to hub to see it?

With that, an optional server parameter might be worth considering. Although with different specifications (like server-time) some sort of resync between messages for split servers could also exist in the future.

---

• Services are down. My channel is SET RESTRICTED.
• User2 joins my channel, and requests a CHATHISTORY.

---

With the above points, I really think that for a bouncer oriented specification, it'll be great. And while I'd like to keep the option for ircds to implement it, I have a hard time seeing how they would cover some of the problems. I imagine some sort of opt-in mechanic could work, but this comment is already getting quite long and it's a bit too much outside of the IRCv3 realm of influence.

[DarthGandalf]

Move this below the CHATHISTORY Command header.
Before reading syntax of the command message_count and max_message_count don't make sense for the reader.

[MuffinMedic]

Fixed in 78b39f1

[DarthGandalf]

What example use case for this do you have in mind?

[MuffinMedic]

It was a leftover from an earlier version of the draft spec - removed in 78b39f1

[DarthGandalf]

What if client requested 40 lines, but server allows access only to 20 (which happened after client JOINed the channel), should it silently decrease size of batch, or report ACCESS_DENIED somewhere?

[MuffinMedic]

Addressed in 78b39f1 - the max_message_count should be returned an WARN MAX_MESSAGE_COUNT_EXCEEDED returned to let the client know the value was too high.

[DarthGandalf]

Is it all one example? Why does it have the same msgid for two messages?

[MuffinMedic]

One example with generic values - the IDs are a simple placeholder, not specific values.

[prawnsalad]

The latest changes now forbids an empty BATCH response if there is no content to return, and instead forces an error to be returned. However if I request CHATHISTORY for a channel that has no more content then this isn't an error - there's just no more content. An empty BATCH is more descriptive and easier to handle here.

It now also states that draft/labeled-response is required when this could be easily be optional but recommended such as draft/msgid. Relying on any draft/* specs could hold back any clients wanting to implement CHATHISTORY for some time.

[prawnsalad]

The latest changes now forbids an empty BATCH response if there is no content to return, and instead forces an error to be returned. However if I request CHATHISTORY for a channel that has no more content then this isn't an error - there's just no more content. An empty BATCH is more descriptive and easier to handle here.

It now also states that draft/labeled-response is required when this could be easily be optional but recommended such as draft/msgid. Relying on any draft/* specs could hold back any clients wanting to implement CHATHISTORY for some time.

[Zarthus]

in all caps? None other capabilities are capitalized are they?

[MuffinMedic]

The extension itself is lowercase. Only the command is capitalized for consistency (CHATHISTORY, PRIVMSG, NOTICE, etc...).

[LFP6]

Could this be extended so that you can request a certain number of messages to either side of a given timestamp/id? I'm imagining ircv3/ircv3-ideas#9 being implemented in a client so that you can jump back to that point in history (as Slack and Discord do).

[LFP6]

Could this be extended so that you can request a certain number of messages to either side of a given timestamp/id? I'm imagining ircv3/ircv3-ideas#9 being implemented in a client so that you can jump back to that point in history (as Slack and Discord do).

[Zarthus]

Extensions to the spec would probably make more sense as a separate specification, rather than to have one do-it-all spec full of "this is all an optional part of the spec".

[Zarthus]

Extensions to the spec would probably make more sense as a separate specification, rather than to have one do-it-all spec full of "this is all an optional part of the spec".

[MuffinMedic]

Adding "x lines before and after" would introduce an unneeded level of complexity as it requires an additional parameter for specify the "direction" of the history. I think a better solution is allowing the message_count to accept both positive and negative integers, where a positive value means "x lines after the timestamp" and a negative number represents "x lines before the timestamp".

A search-type spec could simply request content both before and after. This would give a simpler yet more versatile approach I feel, although I'd like to see what others say before implementing it into the spec.

[MuffinMedic]

Adding "x lines before and after" would introduce an unneeded level of complexity as it requires an additional parameter for specify the "direction" of the history. I think a better solution is allowing the message_count to accept both positive and negative integers, where a positive value means "x lines after the timestamp" and a negative number represents "x lines before the timestamp".

A search-type spec could simply request content both before and after. This would give a simpler yet more versatile approach I feel, although I'd like to see what others say before implementing it into the spec.

[LFP6]

That seems good to me. While less concise, it also means that the server doesn't need to parse through the history nor does the client need to parse through the response to determine what comes before and what comes after.

[LFP6]

That seems good to me. While less concise, it also means that the server doesn't need to parse through the history nor does the client need to parse through the response to determine what comes before and what comes after.

[jwheare]

Can we get an example of a full client/server exchange for this?

[jwheare]

Can we get an example of a full client/server exchange for this?

[Zarthus]

inclued

[DarthGandalf]

Including the message with the specified id?

[DarthGandalf]

Including the message with the specified id?

[jwheare]

s/specifie/specified

[kylef]

What if the server/bouncer doesn't have any limitations on the buffer size. Could 0 perhaps indicate there is no max limit?

[MuffinMedic]

Agreed - Will update accordingly to set 0 = no limit.

[kylef]

SHOULD should be in caps like MUST (RFC2119)

[kylef]

Is max_message_count required? Does omitting a value and having CHATHISTORY in ISUPPORT mean there are no limits enforced?

[MuffinMedic]

Yes, it's required. As per #292 (comment), it will be updating such that 0 can be used to indicate no limit exists.

[kylef]

What is CMD_INVALID I don't see mention of it here or in RFC1459/RFC2812.

Wouldn't ERR_NEEDMOREPARAMS be sent if the required parameters are missing and not "CMD_INVALID"? Or ERR_NOSUCHNICK/ERR_NOSUCHCHANNEL if the target is non-existent?

[kylef]

If target can be multiple targets, is it possible to receive the ERR_TOOMANYTARGETS numeric?

[kylef]

Can we make this format match that in other IRC specifications for consistency.

@draft/label=ID CHATHISTORY <target> draft/msgid=<message_id> <message_count>

[kylef]

Also not seeing much clarification on what target can be. Is it a direct target like #channel. Does it support comma separated values? #channel,#other-channel? Can we use * as wildcard?

[MuffinMedic]

I will specify the target as a single channel or query. The use of wildcards or multiple channels becomes problematic with draft/msgid, as each ID is unique only to a single message and the server would not know what to retrieve for each individual target.

What would the standard format be for the timestamp=YYYY-MM-DDThh:mm:ss.sssZ option? The use of server-time for either seems inappropriate.

[kylef]

I will specify the target as a single channel or query.

Okay if there is no wildcard, I am not sure how a client would know about the unread queries they have opened for individual nicks. Is there anyway a client would be able to load the available chat history targets?

znc-playback currently allows the wildcard so that clients can ask for all queries during launch so that they can load the nick queries.

[MuffinMedic]

CHATHISTORY is not intended to be a replacement for a playback buffer. Unless the spec is reworked, it currently has no concept of "read messages".

[kylef]

Link can be updated to http://ircv3.net/specs/extensions/message-ids.html

[kylef]

http://ircv3.net/specs/extensions/labeled-response.html

[kylef]

I think this might lead to some timing problems down the line, since the timestamp is not unique per message. A server may receive two messages at the very same time and only one of these make it to a connected client before it disconnects.

Then the client will reconnect and ask for messages since the last message it seen which would not include the missing message.

[MuffinMedic]

This is required to account for certain use cases in which draft/msgid tags are not available, such as in ZNC 1.6+ logs, in the format of YYYY-MM-DD.log [HH:mm:ss].

[kylef]

Secure identification of users and clients must exist in order to ensure that users cannot obtain history that does not belong to them

I'm not sure this is that clear. Some history do not necessarily "belong" to an individual user. Could I not request the chat history for a public channel on a server?

If I'm requesting public information I do not think any identification of the user or client needs to exist, only that the client is registered with the server.

[MuffinMedic]

Secure identification of users and clients must exist in order to ensure that users cannot obtain history that was not previously available to them at the time the messages were originally sent.

Using the reworded version:

Could I not request the chat history for a public channel on a server?

While you can certainly request chathistory from a public channel, you should not be able to request history if you weren't in the channel at the time the messages were originally sent. Allowing users to obtain history for channels they never used or are no longer joined to would essentially allow people to "spy" on channels.

[kylef]

While you can certainly request chathistory from a public channel, you should not be able to request history if you weren't in the channel at the time the messages were originally sent.

What would the point of requesting history if I have already seen the history because I was in the channel?

I think this should be down to networks/implementors, as a network might decide that channels without +p (private) and +s (secret) have publicly available logs. Or some other reasons behind it.

[MuffinMedic]

you should not be able to request history if you weren't in the channel at the time the messages were originally sent.

What would the point of requesting history if I have already seen the history because I was in the channel?

This limitation exists to prevent retrieval of content you weren't in the channel for. If you were in there at the time the messages were originally sent, then that qualifies as content "previously available to [you]"

[kylef]

This limitation exists to prevent retrieval of content you weren't in the channel for.

Shouldn't this be a consideration on an IRC server and not rules enforced by the specification? An implementation of CHATHISTORY might want to provide public history for some channels. For example some private IRC server at a company where all employees could join later and view existing history.

[MuffinMedic]

Good point. Will update to:

Secure identification of users and clients MUST exist in order to ensure that users cannot obtain history they are not authorized to view.

[kylef]

YYYY-MM-DDThh:mm:ss.sssZ implies a specific format and not any valid ISO8601 date. I think the spec should clearly specify what is supported (ISO 8601) instead of only mentioning this format. ISO8601 valid dates may be in other formats like 2017-04-21T12:45:50+00:00.

[MuffinMedic]

The format given consistent with the server-time extension.

[kylef]

I spotted #293 which states:

The chathistory batch is currently limited to PRIVMSG and NOTICE and has no way to some events because they may be processed by the client in a way that affects the present channel state.

I don't see this actually mentioned anywhere so I think it should be mentioned that only PRIVMSG and NOTICE messages can be apart of the chat history.

[kylef]

I spotted #293 which states:

The chathistory batch is currently limited to PRIVMSG and NOTICE and has no way to some events because they may be processed by the client in a way that affects the present channel state.

I don't see this actually mentioned anywhere so I think it should be mentioned that only PRIVMSG and NOTICE messages can be apart of the chat history.

[MuffinMedic]

PRIVMSG and NOTICES aren't limitations with this specification - it derives from limitations with the batch type, which this extension requires be supported. This is a dependency limitation.

If #293 addresses these issues and batch/chathistory no longer has these limitations, then CHATHISTORY would automatically support the additional event types.

[MuffinMedic]

PRIVMSG and NOTICES aren't limitations with this specification - it derives from limitations with the batch type, which this extension requires be supported. This is a dependency limitation.

If #293 addresses these issues and batch/chathistory no longer has these limitations, then CHATHISTORY would automatically support the additional event types.

[DarthGandalf]

Are you sure about this? With a self-contradicting spec, it's impossible to write a conforming implementation :)

[DarthGandalf]

Are you sure about this? With a self-contradicting spec, it's impossible to write a conforming implementation :)

[MuffinMedic]

Nice catch, fixed in a210bfc

[SaberUK]

draftCHATHISTORY?

[SaberUK]

Does each line need a title?

[SaberUK]

You should use a separate example for errors/warnings imo.

[Zarthus]

a`CHATHISTORY

[Zarthus]

Why are we mixing numerics and "return verbs" (L77) like WARN and ERROR. If one format is superior to the other we should strive to include it for all cases, not some of them, and some of them not.

[MuffinMedic]

Can you explain what you mean? I'm not clear on the distinction. Are you suggesting we stick with ERROR and WARN and provide plaintext reasons rather than the codes?

[jwheare]

z

[jwheare]

z

[Zarthus]

e

[Zarthus]

e

[Zarthus]

(18:53:52) <Zarthus> before i make an ass out of myself
(18:53:58) <Zarthus> return an `NO_SUCH_NICK` or `NO_SUCH_CHANNEL` error
(18:54:04) <Zarthus> typo "an" -> "a" yes/no
(18:54:14) <Sadie> y

[Zarthus]

MUST send an NO_TEXT_TO_SEND error

[Zarthus]

Without explicitly mentioning ERR as a verb (and WARN) implementors may think that this is not a constant string and get confused. A reference to some sort of document explaining subcommands may be useful

[Zarthus]

timestamp without an @ symbol?

[MuffinMedic]

timestamp in this case is a command parameter, not a client tag.

[Zarthus]

Perhaps instead of "an empty batch ..." we could do "CHATHISTORY SUCCESS NOOP" / NO_DATA whatever can be returned