Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

SCTP stream mapping, addressing #708 #722

Closed
wants to merge 12 commits into from
Closed

SCTP stream mapping, addressing #708 #722

Show file tree
Hide file tree
Changes from 9 commits
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
7ee7082
whatever... just let me pull again
mwelzl Dec 23, 2020
8cb2d59
Merge branch 'master' of https://github.com/taps-api/drafts
mwelzl Dec 23, 2020
431fef9
More details on stream mapping. Closes #708.
mwelzl Dec 23, 2020
a3558a0
Fixes a compilation mistake? Hopefully?
mwelzl Dec 23, 2020
54a4993
Inserted a "todo note" that says that a text bit here will be updated…
mwelzl Dec 23, 2020
f26a783
using 2119 language, and making an IANA request for an SCTP Adaptatio…
mwelzl Jan 7, 2021
824f8b5
Update draft-ietf-taps-impl.md
mwelzl Jan 27, 2021
d0d85c9
Update draft-ietf-taps-impl.md
mwelzl Jan 27, 2021
476c36d
Update draft-ietf-taps-impl.md
mwelzl Jan 27, 2021
d7bb5e6
Removed the code point to negotiate the use of stream mapping and the…
mwelzl Feb 17, 2021
2d8309f
Delete draft-ietf-taps-interface.md
mwelzl Mar 1, 2021
317a330
removing -interface from PR
mwelzl Mar 1, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
118 changes: 87 additions & 31 deletions draft-ietf-taps-impl.md
View file
Open in desktop
Expand Up @@ -91,18 +91,20 @@ informative:
I-D.ietf-tcpm-2140bis:
NEAT-flow-mapping:
title: Transparent Flow Mapping for NEAT
seriesinfo: Workshop on Future of Internet Transport (FIT 2017)
authors:
seriesinfo: IFIP NETWORKING 2017 Workshop on Future of Internet Transport (FIT 2017)
author:
-
ins: F. Weinrank
name: Felix Weinrank
-
ins: M. Tuexen
name: Michael Tuexen
date: 2017
TCP-COUPLING:
title: "ctrlTCP: Reducing Latency through Coupled, Heterogeneous Multi-Flow TCP Congestion Control"
seriesinfo:
IEEE INFOCOM Global Internet Symposium (GI) workshop (GI 2018)
authors:
author:
-
ins: S. Islam
name: Safiqul Islam
Expand All @@ -122,7 +124,23 @@ informative:
ins: S. Gjessing
name: Stein Gjessing
date: 2018-04-15

NEATPy:
title: "Transport Services: A Modern API for an Adaptive Internet Transport Layer"
seriesinfo: IEEE Communications Magazine, April 2021
author:
-
ins: M. Welzl
name: Michael Welzl
-
ins: S. Islam
name: Safiqul Islam
-
ins: M. Gundersen
name: Michael Gundersen
-
ins: A. Fischer
name: Andreas Fischer
date: 2021
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this reference is no longer used, and should be removed.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK, will do this in the new & clean PR that replaces this one.


--- abstract

Expand Down Expand Up @@ -1030,39 +1048,78 @@ Data Unit: Message
API mappings for SCTP are as follows:

Connection Object:
: Connection objects represent a flow of SCTP messages between a client and a server, which may be an SCTP association or a stream in a SCTP association. How to map Connection objects to streams is described in {{NEAT-flow-mapping}}; in the following, a similar method is described.
To map Connection objects to SCTP streams without head-of-line blocking on the sender
side, both the sending and receiving SCTP implementation must support message interleaving {{!RFC8260}}.
Both SCTP implementations must also support stream reconfiguration. Finally, both communicating endpoints
must be aware of this intended multiplexing; {{NEAT-flow-mapping}} describes a
way for a Transport System to negotiate the stream mapping capability using SCTP's adaptation layer indication,
such that this functionality would only take effect if both ends sides are aware of it.
The first flow, for which the SCTP association has been created, will always use stream id zero.
All additional flows are assigned to unused stream ids in growing order. To avoid a conflict
when both endpoints map new flows simultaneously, the peer which initiated the transport connection
will use even stream numbers whereas the remote side will map its flows to odd stream numbers.
Both sides maintain a status map of the assigned stream numbers. Generally, new streams
must consume the lowest available (even or odd, depending on the side) stream number; this
rule is relevant when lower numbers become available because Connection objects associated
to the streams are closed.
: Connection objects can be mapped to an SCTP association or a stream in an SCTP association.
Mapping Connection objects to SCTP streams is called "stream mapping" and has additional requirements as follows.
The following explanation assumes a client-server communication model.

Stream mapping requires an association to already be in place between the client
and the server, and it requires the server to understand that a new incoming stream
theri marked this conversation as resolved.
Show resolved Hide resolved
should be represented as a new Connection Object by the Transport Services system.
To signal the use of stream mapping, the server SHOULD use the adaptation layer indication
parameter of LISTEN.SCTP and the client SHOULD use the adaptation layer indication
mwelzl marked this conversation as resolved.
Show resolved Hide resolved
parameter of CONNECT.SCTP, both with the "Stream Mapping" adaptation code point.
mwelzl marked this conversation as resolved.
Show resolved Hide resolved
Reception of an adaptation layer indication notification with the same adaptation code point value
then indicates that the peer is capable of stream mapping.

In the absence of an adaptation layer indication notification carrying a "Stream Mapping"
adaptation code point, stream mapping can still be
used if the application is aware that the peer application does not directly access SCTP streams.
mwelzl marked this conversation as resolved.
Show resolved Hide resolved
If, for example, a transport-protocol-agnostic client application tries to connect to a
native SCTP server application that does not implement stream mapping, and assumes
that a new Message sent on this Connection will look like the first Message of a new
Connection on the server side, communication will fail because the
mwelzl marked this conversation as resolved.
Show resolved Hide resolved
SCTP-based application on the server side does not obtain a new Connection.
As a counter-example, if both the client and server are
written over a Transport Services system, and the server listens on all available protocols, stream
mwelzl marked this conversation as resolved.
Show resolved Hide resolved
mapping works without the application having to be aware of it (a code example that works
without the adaptation layer notification, described in {{NEATPy}}, is available from the
NEATPy repository mentioned in {{appendix-implementations}} at the time of writing).

A new SCTP stream is created by sending an SCTP message with a new stream id. Thus, to
implement stream mapping,
the Transport Services system MUST provide a newly created Connection Object to the
application upon the reception of such a message. The necessary semantics to
implement a Transport Services system's `Close` and `Abort` primitives are provided
by the stream reconfiguration (reset) procedure described in {{?RFC6525}}. This also
allows to re-use a stream id after resetting ("closing") the stream.
To implement this functionality, SCTP stream reconfiguration {{?RFC6525}} MUST be
supported by both the client and the server side.

To avoid head-of-line blocking, stream mapping SHOULD only be implemented
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think I agree with this. A non-8260 SCTP still provides stream multiplexing with some HOLB avoidance properties, even if it's not as good. This is your question (2), which I think we agreed on?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confused. Question 2 was "close = stream reset", 3 was "I-DATA to avoid HOLB".
What is it you don't agree with? The "SHOULD only"?

I'm sceptical because I believe that small messages could get enqueued behind HUGE other messages, and there's also no per-stream flow control, at least not in the absence of I-DATA. A native SCTP application may know what it's doing, but a TAPS application would just use Connections and get a blockage surprise. So that doesn't seem good to me.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I-DATA actually was question 2 :) But i agree it seems a bit risky to use multi-streaming without it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ahhh yes right, I looked at a response by @martinduke and he had changed the numbering. Tsk :) First github turns the numbering into 1-8 for me, then this... sorry folks :)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So - what is it we have and don't have agreement about? The text now says MUST for the "stream reset" SCTP extension, as it allows to implement the equivalent of CLOSE (we could be less restrictive, but I fear that it makes things more complex... we have to invent timeouts for this; also, note that this hasn't been tested with an implementation - the only implementation that we have of doing this uses "stream reset"). For I-DATA, it says SHOULD.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm afraid we've QUIC-optimized our model; we have connection close and abort, and connection group abort. For SCTP, we'd want connection group close and abort, and an optional connection abort, IIUC

We could decide to add connection group close; or, we could just use the lack of a connection (stream) close in SCTP to simply signify it in the API by closing all the connections/streams.

Personally, I think we should have graceful close and abort for connection groups and write something in the QUIC mapping about how the former is handled.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You say "the lack of a connection (stream) close in SCTP": but we do have stream reset, and it's practically the same. This is why we require it with a MUST, and this capability is negotiated with the peer by SCTP itself - so if we require it on one side, we have it. A stream reset is quite like a FIN.

So, I don't see why you'd need a group close for SCTP? It may be a convenient thing to have, in general, sure, but I don't see what this has to do with SCTP in particular - being an API addition, I see this as separate from this PR, in fact.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, you're right; it's a clean close. It's the stream abort that we're missing.

So in theory a streaming protocol has four possible ending operations: connection close, stream close, connection abort, stream abort. QUIC has 3 of these; SCTP has 2, 3 with RFC 6525; HTTP/2 has all four. In some protocols these are half-closes or aborts, or full. TAPS currently has stream close, stream abort, and connection abort.

So I'm going to file a separate issue on this, because it's bigger than the SCTP mapping.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

About stream abort: not quite, I think. Using a "stream abort" would mean that the TCP-like half-closed behavior wouldn't work - you couldn't send a stream reset and then expect to receive something on a stream. However, that isn't possible anyway, because SCTP streams are only unidirectional. And, there's only a guarantee of a notification by a returning "stream reset" because we define it in this PR.

I agree that this is bigger and should be separate from this PR... but what's your plan? Do we really have a big problem here? In the end, it's a matter of having the right semantics at the API level. This began with just picking the strictest possible semantics: no half-closed connections, calling "close" means that what the app already gave to the transport system for transmission before the call will still be sent, but the app cannot expect to send or receive anything after this call. This way, SCTP (which doesn't allow half-closed associations) is covered, and the behavior is "correct" for TCP as well. These semantics also work for SCTP streams with stream reset as above.

So... altogether, I thought we'll be safe if we just go strict.

Now, quite recently, the wording was changed a bit, because of the need to support TCP half-closed connections (which is implemented via the "Final" Message Property). I felt a bit uneasy about this change, but in the end I couldn't see how things would go wrong with the wording that we now have in the API draft.

My high-level view is that we still have this rather strict thing, and then there is the option to use "Final", with no guarantee of it working (because we don't know if we get a protocol that supports half-closed connections). But there may be a devil in the details on how we worded it... definitely worth another careful look.

when both sides support message interleaving {{?RFC8260}}. This allows a sender to
schedule transmissions between multiple streams without risking that transmission
of a large message on one stream might block transmissions on other streams
for a long time.

To avoid conflicts between stream ids, the following procedure is recommended:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please help someone who hasn't read the SCTP specs in 15 years: is there really no deconfliction when there is simultaneous open of a new stream ID?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wouldn't really know myself. This is how Felix Weinrank and Michael Tuexen implemented it in NEAT - so I figured that this must be necessary. I can check with them if there's really no other way... but if there were, they would probably have used it. This said, this is also a few years ago now.

the first Connection, for which the SCTP association has been created, MUST always use stream id zero.
All additional Connections are assigned to unused stream ids in growing order. To avoid a conflict
when both endpoints map new Connections simultaneously, the peer which initiated association
MUST use even stream ids whereas the remote side MUST map its Connections to odd stream ids.
Both sides maintain a status map of the assigned stream ids. Generally, new streams
SHOULD consume the lowest available (even or odd, depending on the side) stream id; this
rule is relevant when lower ids become available because Connection objects associated
with the streams are closed.

SCTP stream mapping as described here has been implemented in a research prototype; a
desription of this implementation is given in {{NEAT-flow-mapping}}.

Initiate:
: If this is the only Connection object that is assigned to the SCTP association or stream mapping has
not been negotiated, CONNECT.SCTP is called. Else, unless the Selection Property `activeReadBeforeSend`
: If this is the only Connection object that is assigned to the SCTP association or stream mapping is
not used, CONNECT.SCTP is called. Else, unless the Selection Property `activeReadBeforeSend`
is Preferred or Required, a new stream is used: if there are enough streams
available, `Initiate` is just a local operation that assigns a new stream number to the Connection object.
available, `Initiate` is just a local operation that assigns a new stream id to the Connection object.
The number of streams is negotiated as a parameter of the prior CONNECT.SCTP call, and it represents a
trade-off between local resource usage and the number of Connection objects that can be mapped
without requiring a reconfiguration signal. When running out of streams, ADD_STREAM.SCTP must be called.

InitiateWithSend:
: If this is the only Connection object that is assigned to the SCTP association or stream mapping has
not been negotiated, CONNECT.SCTP is called with the "user message" parameter. Else, a new stream
: If this is the only Connection object that is assigned to the SCTP association or stream mapping is not used, CONNECT.SCTP is called with the "user message" parameter. Else, a new stream
is used (see `Initiate` for how to handle running out of streams), and this just sends the first message
on a new stream.

Ready:
: `Initiate` or `InitiateWithSend` returns without an error, i.e. SCTP's four-way handshake has completed. If an association with the peer already exists, and stream mapping has been negotiated and enough streams are available, a Connection Object instantly becomes Ready after calling `Initiate` or `InitiateWithSend`.
: `Initiate` or `InitiateWithSend` returns without an error, i.e. SCTP's four-way handshake has completed. If an association with the peer already exists, stream mapping is used and enough streams are available, a Connection Object instantly becomes Ready after calling `Initiate` or `InitiateWithSend`.

InitiateError:
: Failure of CONNECT.SCTP.
Expand All @@ -1071,13 +1128,13 @@ ConnectionError:
: TIMEOUT.SCTP or ABORT-EVENT.SCTP.

Listen:
: LISTEN.SCTP. If an association with the peer already exists and stream mapping has been negotiated, `Listen` just expects to receive a new message on a new stream id (chosen in accordance with the stream number assignment procedure described above).
: LISTEN.SCTP. If an association with the peer already exists and stream mapping is used, `Listen` just expects to receive a new message with a new stream id (chosen in accordance with the stream id assignment procedure described above).

ConnectionReceived:
: LISTEN.SCTP returns without an error (a result of successful CONNECT.SCTP from the peer), or, in case of stream mapping, the first message has arrived on a new stream (in this case, `Receive` is also invoked).

Clone:
: Calling `Clone` on an SCTP association creates a new Connection object and assigns it a new stream number in accordance with the stream number assignment procedure described above. If there are not enough streams available, ADD_STREAM.SCTP must be called.
: Calling `Clone` on an SCTP association creates a new Connection object and assigns it a new stream id in accordance with the stream id assignment procedure described above. If there are not enough streams available, ADD_STREAM.SCTP must be called.

Priority (Connection):
: When this value is changed, or a Message with Message Property `Priority` is sent, and there are multiple
Expand All @@ -1091,16 +1148,15 @@ Receive:
: RECEIVE.SCTP. The "partial flag" of RECEIVE.SCTP invokes a `ReceivedPartial` event.

Close:
If this is the only Connection object that is assigned to the SCTP association, CLOSE.SCTP is called. Else, the Connection object is one out of several Connection objects that are assigned to the same SCTP assocation, and RESET_STREAM.SCTP must be called, which informs the peer that the stream will no longer be used for mapping and can be used by future `Initiate`, `InitiateWithSend` or `Listen` calls. At the peer, the event RESET_STREAM-EVENT.SCTP will fire, which the peer must answer by issuing RESET_STREAM.SCTP too. The resulting local RESET_STREAM-EVENT.SCTP informs the transport system that the stream number can now be re-used by the next `Initiate`, `InitiateWithSend` or `Listen` calls.
If this is the only Connection object that is assigned to the SCTP association, CLOSE.SCTP is called. Else, the Connection object is one out of several Connection objects that are assigned to the same SCTP assocation, and RESET_STREAM.SCTP must be called, which informs the peer that the stream will no longer be used for mapping and can be used by future `Initiate`, `InitiateWithSend` or `Listen` calls. At the peer, the event RESET_STREAM-EVENT.SCTP will fire, which the peer must answer by issuing RESET_STREAM.SCTP too. The resulting local RESET_STREAM-EVENT.SCTP informs the Transport Services system that the stream id can now be re-used by the next `Initiate`, `InitiateWithSend` or `Listen` calls.

Abort:
If this is the only Connection object that is assigned to the SCTP association, ABORT.SCTP is called. Else, the Connection object is one out of several Connection objects that are assigned to the same SCTP assocation, and shutdown proceeds as described under `Close`.

# IANA Considerations

RFC-EDITOR: Please remove this section before publication.

This document has no actions for IANA.
This document requests the definition of a new SCTP Adaptation Code Point value
called "Stream Mapping". The SCTP Adaptation Coide Point is defined in RFC 5061.

# Security Considerations

Expand Down
8 changes: 7 additions & 1 deletion draft-ietf-taps-interface.md
View file
Open in desktop
Expand Up @@ -100,7 +100,7 @@ informative:
title: "ctrlTCP: Reducing Latency through Coupled, Heterogeneous Multi-Flow TCP Congestion Control"
seriesinfo:
IEEE INFOCOM Global Internet Symposium (GI) workshop (GI 2018)
authors:
author:
-
ins: S. Islam
name: Safiqul Islam
Expand Down Expand Up @@ -3237,3 +3237,9 @@ applications can obtain this information via the `Sent` Event ({{sent}}).

* Notification to a receiver that a partial message delivery has been aborted:
`ReceiveError` Event ({{receive-error}}).


## Unsupported Elements of the Minimal Set

* Notification of Excessive Retransmissions (early warning below abortion threshold):
This is not supported because it is TCP-specific and hardly
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is incomplete?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Argh! More importantly, it doesn't belong here - this is an accident :( will fix