TS 26.512 - Handling of 3XX HTTP redirects via interface at M2d

[davidjwbbc]

Description

During implementation and testing of the 5GMSd Application Function and 5GMSd Application Server it was noticed that pull ingest points for M2d that use 3XX HTTP status code redirections, to balance load, caused the 5GMS Aware App running on the UE to divert traffic away from requesting media content via the 5GMSd AS. This behaviour will have an impact on the 5G network's ability to provide QoS services for 5GMS Aware Apps running on the UE as the traffic may no longer be flowing over controlled networks.

There is no guidance in TS 26.512 relating to the behaviour of the 5GMSd Application Server when redirects are received via the interface at M2d.

Suggested solution

Clause 4.4 should contain some wording relating to the use of 3XX HTTP status codes and behaviour when 3XX HTTP status codes are received via the ingest over interface M2d.

Simple warning

Clause 4.4 could simply state the problem with pull ingest servers using 3XX status codes and that passing these through the AS to the 5GMS aware application will redirect traffic away from the 5GMSd Application Server. It is then up to the MNO and AP to come to agreement on the use of 3XX HTTP redirects.

Ban 3XX status codes

Clause 4.4 could prohibit 3XX redirects on M2d. This solves the problem, but as stated in the description above this may not be practical due to the ingest using a CDN which is using redirection to load balance.

Handling of 3XX redirects

Clause 4.4 could give details for handling 3XX redirects arriving for pull ingest via M2d.

The following discusses some possible behaviours.

AS follows redirects itself

The AS could follow any redirects received via M2d to obtain the HTTP object requested via the pull ingest. The obtained object can then be provided in response to requests via M4d.

Although this would receive the original requested object, it would be difficult to take the redirection into account when follow-up requests are made for other objects with URLs relative to the received object. For example, following redirects to obtain a DASH manifest would get you the manifest (which would be passed to the UE via M4d) but follow-up requests from the UE for the relative media objects would then be made relative to the original ingest URL (where the media objects may not exist). To overcome this the AS would have to match the media segment URLs from the manifest and then generate ingest URLs which follow the original manifest redirects to find the new relative media object ingest location. This would require the AS to understand different media manifest formats so that it could perform the correct redirected ingest URL transformations for any media objects.

AS follows redirects and modifies the manifests

Similar to the above, except that the obtained manifest is modified so that when the UE requests relative media objects they are easily identified as belonging to a particular manifest and can the AS can apply alternative ingest URL rewriting rules to find the ingest location of the media objects based on the original redirection URLs for the manifest.

AS rewrites redirection URLS

In this strategy the AS dynamically learns redirection locations and sets up new download path prefixes (either as a sub-path of the provisioning session or as a new M4d path) for these learned redirections on the interface at M4d. The redirection URL Location header is then rewritten to point to this new path prefix on the AS and the redirection passed through to the UE via M4d. The UE will then follow the redirection to the object under the new path prefix and the AS can then request the object from the newly learned redirected ingest URL. This then has the benefit that the UE will subsequently make the relative media object requests using the same AS learned redirection prefix path, allowing the same redirection base URL for the ingest to be used via M2d.

Example:

sequenceDiagram
  UE->>AS: Request object using M4d provisioning session URL prefix,<br/>e.g. http://as.mno.net/m4d/1234-5678-90AB/manifest.mpd
  AS->>Ingest1: Request object using M2d ingest prefix,<br/>e.g. http://ingest1.ap.net/media/asset1/manifest.mpd
  Ingest1-->>AS: Return a redirect to ingest2,<br/>e.g. Location: http://ingest2.ap.net/asset1/manifest.mpd
  AS->>AS: Learn the redirect and rewrite the Location header,<br/>e.g. remember /m4d/redirect-1/ will now map to http://ingest2.ap.net/asset1/<br/>and change the Location header to http://as.mno.net/m4d/redirect-1/manifest.mpd
  AS-->>UE: Return the rewritten redirect,<br/>e.g. http://as.mno.net/m4d/redirect-1/manifest.mpd
  UE->>AS: Request object using redirected URL,<br/>e.g. http://as.mno.net/m4d/redirect-1/manifest.mpd
  AS->>Ingest2: Request object using new mapping,<br/>e.g. http://ingest2.ap.net/asset1/manifest.mpd
  Ingest2-->>AS: Return object
  AS-->>UE: Return object
  UE->>AS: Request relative object,<br/>e.g. http://as.mno.net/m4d/redirect-1/video-1.m4s
  AS->>Ingest2: Request object,<br/>e.g. http://ingest2.ap.net/asset1/video-1.m4s
  Ingest2-->>AS: Return object
  AS-->>UE: Return object

Loading

Additional context

Original 5GMSd AS github issue: 5G-MAG/rt-5gms-application-server#70

[rjb1000]

Good idea to alert the reader to the high-level problem of M2 redirects in clause 4.4 ("stage 2.5").

But note that it only applies to pull-based content ingest at reference point M2d. More specific guidance on HTTP redirect handling rightly belongs in clause 8.2 ("stage 3") which should probably be referenced from clause 4.4 anyway.

The rewriting of HTTP redirects at M4d is a neat solution. This must be a problem for all CDNs, I imagine.

Let's see how this discussion goes in 3GPP SA4 with @haudiobe and @tlohmar.

If agreeable, I'd support describing it in general terms in clause 8.2. How about:

In the case where the 5GMSd Application Provider's origin server issues an HTTP 3xx redirect at reference point M2d pointing to another location, the 5GMSd AS shall issue an equivalent HTTP redirect to the Media Player whose location is a session-specific endpoint at reference point M4d. Requests to this location shall be rewritten by the 5GMSd AS to the target location of the M2d redirection.
NOTE: This explicit handling of HTTP redirects by the 5GMSd AS ensures that it is not bypassed.

I assume the session-specific redirect mapping stays in place until some inactivity timer expires. I think it's fine to leave garbage collection of these session-specific redirect mappings to implementation.

[davidjwbbc]

After a discussion we have come to the conclusion that there is a need to log the originating provisioning session id with media streaming access record. Therefore any dynamically learned redirection must be able to remember which provisioning session the request was for that originally caused the redirection. There should be a Provisioning Session Id added to Table 17.2-1 to hold this information as it is reported via R4.

[rjb1000]

There should be a Provisioning Session Id added to Table 17.2-1 to hold this information as it is reported via R4.

Having looked at table 17.2-1 for another reason today, I wondered whether the existing sessionIdentifier property might be sufficient to correlate requests to the dynamically learned redirect URL with requests to the original M4 URL, @davidjwbbc.

[davidjwbbc]

The sessionIdentifier states that it is for the "HTTP session", I'd therefore assumed (since this structure reflects the normal access logs of a web server or CDN node) that this field was the client session identifier, i.e. an identifier for the TCP connection between the AS and the UE, used to group multiple requests for manifests and media objects together for a single media playback session. There's also nothing stopping the Media Stream Handler from using this single HTTP session to hop back and forth between different media (or even possibly play two streams at the same time). This means that the single sessionIdentifier may end up being used for multiple provisioning sessions. Therefore I still think an extra provisioning session id field is necessary.

[rjb1000]

• TS 26.512 Rel-16 - Essential correction needs be backported.
• TS 26.512 Rel-17 - Essential correction needs be backported.
• TS 26.512 Rel-18 CR0037 in S4aI230105 - Work in progress.

[rjb1000]

Fixes broadly endorsed at post-SA4#124 ad hoc meeting:

• TS 26.512 CR0037 Rel-17 "[5GMS3, TEI17] Essential maintenance" in S4aI230105.

@ibouazizi indicated a preference for the 5GMS AS to follow the redirect itself, but @davidjwbbc cited the need for subsequent relative URLs to be resolved in the context of the redirected base path.

@ibouazizi offered to provide some additional information about Apache Traffic Server: https://docs.trafficserver.apache.org/en/latest/admin-guide/configuration/redirecting-http-requests.en.html

[davidjwbbc]

@ibouazizi offered to provide some additional information about Apache Traffic Server:
https://docs.trafficserver.apache.org/en/latest/admin-guide/configuration/redirecting-http-requests.en.html

The information about Apache Traffic Server indicates that it passes through redirects although it will rewrite the Location header to be relative to the ATS if the redirect from the Origin has the same protocol and authority (host & port) as the Origin HTTP request.

It provides a mechanism to add further mappings for other Origin servers that are statically known about and also provides a configuration flag which will convert the response into an error if the redirect is to an unmapped protocol and authority. However these configurations are not the default.

It does not follow redirects itself.

[rjb1000]

@ibouazizi offered to provide some additional information about Apache Traffic Server: https://docs.trafficserver.apache.org/en/latest/admin-guide/configuration/redirecting-http-requests.en.html

Just to cite a few of the relevant parts of the Apache Traffic Server documentation on redirect handling:

Handling Origin Server Redirect Responses

... When Traffic Server is configured as a reverse proxy, it must readdress redirects from origin servers so that browsers are redirected to Traffic Server and not to another origin server.

To readdress redirects, Traffic Server uses reverse-map rules. Unless you have proxy.config.url_remap.pristine_host_hdr enabled (the default) you should generally set up a reverse-map rule for each map rule. ...

and:

reverse-map rule

A reverse-map rule translates the URL in origin server redirect responses to point to Traffic Server so that clients are redirected to Traffic Server instead of accessing an origin server directly. ...

Traffic Server uses reverse_map rules to prevent clients (that receive redirects from origin servers) from bypassing Traffic Server and directly accessing the origin servers. ...

The proposed change to clause 8.2 of TS 26.512 adds a paragraph describing the above concept of a "reverse-map rule" in terms of the functional effect is has at reference point M4. In the case of the 5GMS AS, the "reverse-map rule" is created dynamically in response to the redirects it encounters at reference point M2, so it could better be described as a dynamic reverse mapping rule. A good implementation of the 5GMS AS would also implement a garbage collection strategy to remove dynamic reverse mapping rules that haven't been used for a long time, but we don't need to specify that since it's an implementation choice.

To help the reader, though, we could add another sentence to the NOTE below the new paragraph linking the specification text to the more familiar terminology commonly used by reverse HTTP proxies.

[rjb1000]

CR contributed to SA4#125 by @rjb1000:

• TS 26.512
  • Rel-16: CR0043 "Essential maintenance" in S4-231158.
  • Rel-17: CR0037r1 "Essential maintenance" in S4-231156.

[rjb1000]

CRs agreed at SA4#125:

• TS 26.512
  • Rel-16: CR0043r1 "Essential maintenance" in S4-231425.
  • Rel-17: CR0037r2 "Essential maintenance" in S4-231424.

[rjb1000]

CRs revised (YAML corrections) and approved at SA#101:

• TS 26.512
  • Rel-16: CR0043r2 "Essential maintenance" in SP-231082.
  • Rel-17: CR0037r3 "Essential maintenance" in SP-231053.

[rjb1000]

Approved changes published and available for download:

• TS 26.512 V16.11.0
• TS 26.512 V17.6.0