New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[multiline] Multiline message batches #398
Conversation
|
||
### Message ids ([spec][message ids]) | ||
|
||
Servers MUST only include a message ID on the first message of a batch when sending a fallback to non-supporting clients. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why the first one? If the message ID is used eg for a reply or a reaction emoji, it visually makes more sense to have it attached to the last message.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If I understand correctly, this is saying that (some of) the fallback PRIVMSG
should be sent without any msgid
tag at all? This seems to upend some expectations (e.g., msgids being available for deduplication purposes).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For comparison, oragono.io/maxline-2 generates n+1 distinct message IDs for the original message and all n lines of the split message. This isn't perfect, but it maintains the expectation that PRIVMSG
should have an ID.
The msgid spec says: "If this tag is being used, it SHOULD be attached to all PRIVMSG and NOTICE events."
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
msgids can't appear more than once. Generating unique n+1 msgids and only sending them on the fallback creates undesirable situations where people might reply to a fallback message and the context would then be missing for people who fully support maxline (i.e. discard fallback message tags)
If you're concerned about missing msgids, the solution would be to properly support the maxline batch spec.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think that's true. It'd be easy to generate unique fallback msgids that the server can recognise as such and translate back or refuse replies for or something.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@jwheare, just to be clear, when you say "maxline" you're referring to the present multiline spec, not to oragono.io/maxline-2?
Some degradation of experience is inevitable under any proposal to support PRIVMSG over 512 bytes. I'm much less concerned about the case you're describing than I am about sending PRIVMSG without IDs, which seems to complicate the handling of chat history at a more fundamental level.
I'm not 100% sold on @edk0's suggestion because it violates an expectation I have that servers shouldn't need to care about client-only tags, but I could be convinced.
Hypothetically, we could define a new tag like draft/fmsgid
("fallback msgid") for transmitting the fallback msgids to clients that support the multiline spec.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm going to care about client-only tags anyway, so it's not a bother for me. You don't have to implement my suggestion; I'm just saying a solution is possible.
Are there any constraints on which messages in the client batch can carry client-only tags, and what those tags can be? |
The way I've implemented this in the client, the only tags within the batch that I use are the @draft/multiline-concat, every other tag must appear on the opening BATCH message to be recognised.
|
Actually that wording is from the fallback handling and refers to unbatched messages. Could use some clearer guidance on non-fallback BATCHed messages and where to include the tag, though I feel it is implied by the fact that the BATCH indicates a single logical message. |
Oh it is mentioned actually:
That means only |
extensions/multiline.md
Outdated
|
||
This section is non-normative. | ||
|
||
In the context of flood protection, keep in mind that clients might send an entire batched multiline message all at once. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I haven't thought this through fully, but there's an issue here with sendqs. If the sendq is small relative to the value of max-bytes, you can DoS clients by completing multiple multiline batches at once and filling up their sendqs.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Following up on a clarifying question from #ircv3: yeah, client A can try to disconnect client B from a server by filling up its sendq. Normally, flood control is a check on this. It's not clear what the best practices are for applying flood control to multiline messages --- the way to get parity with pre-multiline behavior w.r.t. the sendq would be to apply artificial delays to the outgoing batch.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If max-lines
is not set, the worst case for a DoS on the sendq is to send max-bytes
lines, with a single byte on each line, and multiline-concat
on every line after the first. Assuming the longest possible nickmask, the total size of the relayed message will be about 100 times max-bytes
. Even for a conservative value of max-bytes
, this should be large enough to overwhelm both the sendq and the TCP send buffer (I think it's typically around 85kB on modern Linux machines).
re. interactions between multiline and labeled-response: on which message does the client send the label tag, the |
BATCH + |
I have a (likely buggy) implementation of this up on testnet.oragono.io. It uses fallback msgids (each individual message in the multiline batch has an alternative msgid, which is sent to multiline-capable clients under the Please report any bugs at https://github.com/oragono/oragono :-) |
Do I understand correctly that the
This is almost explained in splitting long lines, but for the sake of verbosity, it probably warrants an explicit explanation. |
No, the |
Oh duh, I completely missed the tag there. The "leave the space character at the end of the line. This ensures that concatenated lines don't lose the space" paragraph mislead me. I do see why it's this way for fallback messages though. |
Any suggestions for clarifying that paragraph to make it less misleading? |
Probably mentioning the tag there would help clarify it. I can't think of an example where "This ensures that concatenated lines don't lose the space" is relevant besides buggy code that trims the messages? |
Any outstanding comments before we merge this as a draft? |
Two issues I raised haven't been addressed:
|
|
These messages would only be sent to fully compliant clients, which in any case need to take the batch tag into account when interpreting the message. So there should be no increase in client complexity over the baseline required to implement multiline. As for server complexity, it's a MAY, so servers don't have to implement it if they don't want it. The potential benefit is a 60% reduction in the size of relayed messages, which seems worthwhile to me. |
Compromise: Put only the nick as the sender. Could even put |
Implementations might want to use the target for routing messages (in addition to the batch tag), so I withdraw my proposal to send I still think we should allow servers to omit the source nickmask: this is syntactically unproblematic, and should be semantically unproblematic for clients as well (at least one implementation supports this "by default", since sources for messages within the relayed S2C batch are ignored). A lot of numbers have been bandied back and forth, so here's what I think is a realistic estimate: for a 100-line code snippet with an average line length of 60 characters, relayed to a channel with an 8-character name by a user with a 40-character nickmask, I compute a 30% reduction in wire size. |
Did we decide if this needs edits before merging as draft? |
|
I’ll take a look at 1 and 2. 3: I think it comes under the same “not worth the micro optimisation” justification as the other stuff on this theme. |
The other proposed optimizations had stronger arguments against them. I think this one is safer than the other proposals, and achieves most of their possible gains. |
I find the inclusion of entirely redundant information ( |
@slingamn those changes should cover your 1 and 2 suggestions. I still think we need some sort of guidelines for configuring limits, but I'm struggling to arrive at something that's sensible, future-proof, not overly restrictive etc. I tried plotting some numbers but it didn't really help. Any ideas? |
The proposed language in my earlier comment is aimed more at drawing attention to the issue, rather than suggesting concrete numbers (which I agree is a difficult problem). |
Yes I added a version of your proposal in ff7f73b but I still feel like something more is needed. |
Oh, sorry I missed that. I don't think I have much insight into this question, other than the suggestion of "envelope math", which you already tried. It sounds like there is an upwards constraint on these limits (supporting all realistic use cases for the feature) and a downwards constraint (blocking abuse), and probably the best way to get more information about both of these constraints is simply to move forward with the specifications process (i.e., make it an official draft, if that's the correct next step) and see what happens in the wild. This is because: a. Any guidance about limits we add later will be in a "non-normative" section of the spec p.s. In terms of moving forwards on the specifications process, I haven't given up on item 3, but I don't really have any further arguments for it beyond what I've already stated. |
OK, makes sense re: further limits guidance. I think in the absence of consensus over how to optimise these messages, again we should just proceed to merged draft and see if anything comes out of implementations. At the end of the day, the unbatched fallback messages will never be able to be optimised, so implementations being able to cope without it is effectively a requirement. |
That sounds fair. |
OK will merge if no further comments are forthcoming. |
@DarthGandalf all good? |
For draft, no further comments. |
A solution to multiline messages (#208) using client to server batches based on https://gist.github.com/jesopo/092866e52b40cda6f6205263d04dbcf9
Currently implemented and testable with the IRCCloud client and servers (prattle: teams, bnc, slack) and bitbot.
Client sending a mutliline batch
Server sending the same batch
Server sending messages to clients without multiline support
Final concatenated message