HLS + Fairplay streams buffers a lot in Mac Safari browser.

[lalit-tudip]

Description

I'm facing a lot of buffering issues in the Mac Safari browser for HLS + FAIRPLAY streams. below are the issues that I'm facing:

• The stream takes a lot of time to load the first frame of the videos whereas, on another platform like Apple TV, it loads in 2-3 seconds max which is using the same HLS + FAIRPLAY.
• After it loads it buffers every second or two whereas on another platform, it plays smoothly like it should.
• The DASH + Widevine combination for Chrome browser plays the stream as expected which is also using videojs.

Below is the code/config for the videojs that I'm using for handling the HLS + FAIRPLAY:

player.src({
            src: URL,
            type: Constants.HSL_STREAM_SRC_TYPE,
            keySystems: {
              "com.apple.fps.1_0": {
                initDataTypes: ["sinf"],
                videoCapabilities: [
                  {
                    contentType: "video/mp4",
                  },
                ],
                getCertificate: function (emeOptions, callback) {
                  videojs.xhr(
                    {
                      url: `${cert}`,
                      method: "GET",
                      responseType: "arraybuffer",
                      headers: {
                        "Access-Control-Allow-Origin": "*",
                      },
                    },
                    (err, response, responseBody) => {
                      if (err) {
                        callback(err);
                      }
                      let responseData = new Uint8Array(responseBody);
                      callback(null, responseData);
                    }
                  );
                },

                getContentId: function (emeOptions, initData) {
                  let uint16array = new Uint16Array(initData.buffer);
                  let contentId = String.fromCharCode.apply(null, uint16array);
                  contentId = contentId.substring(
                    contentId.indexOf("skd://") + 6
                  );
                  const url = new URL(contentId);
                  const urlParams = new URLSearchParams(url.search);
                  return urlParams.get("contentId");
                },

                getLicense: (emeOptions, contentId, keyMessage, callback) => {
                  let token = localStorage.getItem("auth");
                  let testToken = JSON.parse(token);
                  //JLO should replaced by base64EncodeUint8Array
                  let input = keyMessage;
                  let keyStr = Constants.FAIRPLAY_KEY_STR;
                  let output = "";
                  let chr1, chr2, chr3, enc1, enc2, enc3, enc4;
                  let i = 0;

                  while (i < input?.length) {
                    chr1 = input[i++];
                    chr2 = i < input?.length ? input[i++] : Number.NaN; // Not sure if the index
                    chr3 = i < input?.length ? input[i++] : Number.NaN; // checks are needed here

                    enc1 = chr1 >> 2;
                    enc2 = ((chr1 & 3) << 4) | (chr2 >> 4);
                    enc3 = ((chr2 & 15) << 2) | (chr3 >> 6);
                    enc4 = chr3 & 63;

                    if (isNaN(chr2)) {
                      enc3 = enc4 = 64;
                    } else if (isNaN(chr3)) {
                      enc4 = 64;
                    }
                    output +=
                      keyStr.charAt(enc1) +
                      keyStr.charAt(enc2) +
                      keyStr.charAt(enc3) +
                      keyStr.charAt(enc4);
                  }
                  //JLO should replaced by base64EncodeUint8Array
                  let spcMessage = output;
                  videojs.xhr(
                    {
                      url: `FAIRPLAY_LICENSE`,
                      method: "POST",
                      responseType: "text",
                      body: `{ "spc": "${spcMessage}", "assetId":"${contentId}"}`,
                      headers: {
                        "Content-Type": "application/json;charset=UTF-8",
                        Authorization: `JWT ${testToken.access_token}`,
                      },
                    },
                    (err, response, responseBody) => {
                      if (err) {
                        callback(err);
                        return;
                      }
                      let obj = JSON.parse(responseBody);

                      //JLO should replaced by base64DecodeUint8Array
                      let raw = window.atob(obj.ckc);
                      let rawLength = raw?.length;
                      let array = new Uint8Array(new ArrayBuffer(rawLength));

                      for (let i = 0; i < rawLength; i++)
                        array[i] = raw.charCodeAt(i);

                      //JLO should replaced by base64DecodeUint8Array

                      callback(null, array);
                    }
                  );
                },
              },
            },
          });

What could be the issue here, what changes would I need to make? or is it a streaming side issue for optimizing it for the browser?

Reduced test case

As this is a private project, so cannot share any URLs/Creds

Steps to reproduce

As this is a private project, so I cannot share any URLs/Creds.

Errors

No errors are logged in the Mac Safari's console.

What version of Video.js are you using?

7.21.4

Video.js plugins used.

"videojs-contrib-dash": "5.1.0", "videojs-contrib-eme": "3.9.0", "@videojs/http-streaming": "^2.8.0",

What browser(s) including version(s) does this occur with?

Mac Safari 17.4.1

What OS(es) and version(s) does this occur with?

All macOS

[gkatsev]

By default, Video.js uses native HLS on Safari. Also, I don't think we ever got MSE working with EME yet. Are you using native HLS or MSE? If it's native, then Safari is responsible for playback.
Additionally, have you ran your stream through Apple's mediastreamvalidator?

[lalit-tudip]

As I'm using the videojs-http-streaming library and not explicitly providing the overrideNative property, it should be false by default as stated in the library documentation. So it should be native.

I checked it on Apple's mediastreamvalidator and below is the report:

HTTP Content-Type: application/vnd.apple.mpegurl
--------------------------------------------------------------------------------
tracks-v2a1/mono.m3u8?token=3a3a66392e323036
--------------------------------------------------------------------------------
HTTP Content-Type: application/vnd.apple.mpegurl

Processed 49 out of 77 segments
Average segment duration: 4.004000
Total segment bitrates (all discontinuities): average: 3007.31 kb/s, max: 5282.78 kb/s
Playlist max bitrate: 4630.000000 kb/s
Audio Group ID: aac

Discontinuity: sequence: 0, parsed segment count: 49 of 77, duration: 308.308 sec, average: 3007.31 kb/s, max: 5282.78 kb/s
--------------------------------------------------------------------------------
tracks-a1/mono.m3u8?token=3a3a392e323036
--------------------------------------------------------------------------------
HTTP Content-Type: application/vnd.apple.mpegurl

Processed 76 out of 77 segments
Average segment duration: 4.004000
Total segment bitrates (all discontinuities): average: 160.79 kb/s, max: 168.28 kb/s
Rendition group ID: aac


Discontinuity: sequence: 0, parsed segment count: 76 of 77, duration: 308.308 sec, average: 160.79 kb/s, max: 168.28 kb/s

--------------------------------------------------------------------------------
tracks-v1a1/mono.m3u8?token=3a3a662e3233392e323036
--------------------------------------------------------------------------------
HTTP Content-Type: application/vnd.apple.mpegurl

Processed 33 out of 77 segments
Average segment duration: 4.004000
Total segment bitrates (all discontinuities): average: 3456.00 kb/s, max: 8882.77 kb/s
Playlist max bitrate: 9270.000000 kb/s
Audio Group ID: aac


Discontinuity: sequence: 0, parsed segment count: 33 of 77, duration: 308.308 sec, average: 3456.00 kb/s, max: 8882.77 kb/s

--------------------------------------------------------------------------------
tracks-a2/mono.m3u8?token=3a32e323036
--------------------------------------------------------------------------------
HTTP Content-Type: application/vnd.apple.mpegurl

Processed 76 out of 77 segments
Average segment duration: 4.004000
Total segment bitrates (all discontinuities): average: 162.58 kb/s, max: 168.28 kb/s
Rendition group ID: aac

Discontinuity: sequence: 0, parsed segment count: 76 of 77, duration: 308.308 sec, average: 162.58 kb/s, max: 168.28 kb/s

--------------------------------------------------------------------------------
MUST fix issues
--------------------------------------------------------------------------------

Error: The operation couldn’t be completed. (HTTPPumpErrorDomain error -12938 - HTTP 404: File Not Found)
Error: HTTP 404 - HTTP/2.0 404 Not Found
--------------------------------------------------------------------------------
SHOULD fix issues
--------------------------------------------------------------------------------
Warning: Non-canonical language subtag in language tag
--> Detail:  RFC5646 language: 'en', Language tag: 'eng'
--> Source:  URL
--> Compare: tracks-a1/mono.m3u8?token=3a3a666392e323036

Unable to open write stream at validation_data.json -- path.
Unable to write JSON at validation_data.json -- file:path.
--------------------------------------------------------------------------------
CAUTION
--------------------------------------------------------------------------------
MediaStreamValidator only checks for violations of the HLS specification. For a more
comprehensive check against the HLS Authoring Specification, please run hlsreport
on the JSON output.

@gkatsev Can you share any other areas I can check or make any updates? What do you think is causing this issue as of now?

[lalit-tudip]

@gkatsev any advice on it?

can you suggest any demo player where I can test the streams (HLS stream with Fairplay DRM).

I'm only able to find a demo player to test HLS streams with no encryption.