Add option to extract keyframe only during trickplay image generation

[gnattu]

This would be significantly faster than decoding every frame, but it does have compatibility issues. Not all decoders support this mode, notably the VP9 decoder, CUVID decoders, and QSV decoders.

Some videos with very long key-frame intervals may also perform poorly with this mode, as the image timing could become too inaccurate to reflect the actual frame.

Changes

• Add EnableKeyFrameOnlyExtraction option to TrickplayOptions

Issues

Fixes #11336

Web: jellyfin/jellyfin-web#5800

[github-actions]

Changes in OpenAPI specification found. Expand to see details.

What's Changed

---

GET /System/Configuration

Return Type:

Changed response : 200 OK

Application configuration returned.

• Changed content type : application/json

  • Changed property TrickplayOptions (object)

    Gets or sets the trickplay options.

    Updated TrickplayOptions :

    • Added property EnableKeyFrameOnlyExtraction (boolean)

      Gets or sets a value indicating whether to only extract key frames.
      Significantly faster, but is not compatible with all decoders and/or video files.

• Changed content type : application/json; profile="CamelCase"

  • Changed property TrickplayOptions (object)

    Gets or sets the trickplay options.

    Updated TrickplayOptions :

    • Added property EnableKeyFrameOnlyExtraction (boolean)

      Gets or sets a value indicating whether to only extract key frames.
      Significantly faster, but is not compatible with all decoders and/or video files.

• Changed content type : application/json; profile="PascalCase"

  • Changed property TrickplayOptions (object)

    Gets or sets the trickplay options.

    Updated TrickplayOptions :

    • Added property EnableKeyFrameOnlyExtraction (boolean)

      Gets or sets a value indicating whether to only extract key frames.
      Significantly faster, but is not compatible with all decoders and/or video files.

POST /System/Configuration

Request:

Changed content type : application/json

Updated ServerConfiguration :

• Changed property TrickplayOptions (object)

  Gets or sets the trickplay options.

  Updated TrickplayOptions :

  • Added property EnableKeyFrameOnlyExtraction (boolean)

    Gets or sets a value indicating whether to only extract key frames.
    Significantly faster, but is not compatible with all decoders and/or video files.

Changed content type : text/json

Updated ServerConfiguration :

• Changed property TrickplayOptions (object)

  Gets or sets the trickplay options.

  Updated TrickplayOptions :

  • Added property EnableKeyFrameOnlyExtraction (boolean)

    Gets or sets a value indicating whether to only extract key frames.
    Significantly faster, but is not compatible with all decoders and/or video files.

Changed content type : application/*+json

Updated ServerConfiguration :

• Changed property TrickplayOptions (object)

  Gets or sets the trickplay options.

  Updated TrickplayOptions :

  • Added property EnableKeyFrameOnlyExtraction (boolean)

    Gets or sets a value indicating whether to only extract key frames.
    Significantly faster, but is not compatible with all decoders and/or video files.

[jellyfin-bot]

This pull request has merge conflicts. Please resolve the conflicts so the PR can be successfully reviewed and merged.

[gnattu]

One more question for the handling: this mode is not compatible with a lot of hardware decoders. Should we add a whitelist to the server and force software decoding when this option is set and current hardware acceleration method is not compatible with this mode?

[crobibero]

Should we add a whitelist to the server and force software decoding when this option is set and current hardware acceleration method is not compatible with this mode?

I think if we know that there is a specific set of configurations that this new setting works with we should add a whitelist.

Also please be sure to PR the docs for next server release

[nyanmisaka]

Not all decoders support this mode, notably the VP9 decoder, CUVID decoders, and QSV decoders.

If FFmpeg's native software decoder supports AVDISCARD_NONKEY, then its corresponding hw accelerator should also support it. Hw decoder usually do not, unless they implement it separately.

So the following are exceptions:

• QSV w/ (Prefer OS native DXVA or VA-API hardware decoders)
• NVENC w/ (Enable enhanced NVDEC decoder)
• AMF, uses d3d11va on Windows.

[crobibero]

Don't forget to make the web change also (or at least create an issue to track it)

[gnattu]

Already here: jellyfin/jellyfin-web#5800

but will not build until the sdk is updated to include the api changes.