Skip to content

fix(amf): enable H.264 profile and coder selection#4927

Closed
k4idyn wants to merge 1 commit into
LizardByte:masterfrom
k4idyn:fix/amf-h264-profile-coder
Closed

fix(amf): enable H.264 profile and coder selection#4927
k4idyn wants to merge 1 commit into
LizardByte:masterfrom
k4idyn:fix/amf-h264-profile-coder

Conversation

@k4idyn

@k4idyn k4idyn commented Mar 30, 2026

Copy link
Copy Markdown
Contributor

Description

  • Currently, the AMD AMF encoder (h264_amf) in src/video.cpp does not have the profile and coder options wired up in its encoder_t definition. This prevents clients that strictly require H.264 Baseline/CAVLC (like the Sony PSP or other legacy hardware decoders) from successfully streaming, as the encoder defaults to High Profile and CABAC regardless of the client's request or the global configuration.
  • Why this is needed
    Older hardware decoders often have hardwired limitations. For example, the PSP-1000's Media Engine only supports:
  • H.264 Baseline Profile (not Main or High)
  • CAVLC entropy coding (not CABAC)

While Sunshine/Apollo allows selecting cavlc in the UI, this setting is ignored by the h264_amf encoder because it isn't passed to the FFmpeg avcodec_open2 dictionary.

  • Proposed Fix
    Update the amdvce definition in src/video.cpp to include the missing options:
        {"coder"s, &config::video.amd.amd_coder},
        {"profile"s, [](const config_t &cfg) {
           if (cfg.profile == 66) return "baseline"s;
           if (cfg.profile == 77) return "main"s;
           return "high"s;
         }},

This brings the AMD implementation into parity with NVENC and QuickSync.

Screenshot

image

Issues Fixed or Closed

Roadmap Issues

Type of Change

  • feat: New feature (non-breaking change which adds functionality)
  • fix: Bug fix (non-breaking change which fixes an issue)
  • docs: Documentation only changes
  • style: Changes that do not affect the meaning of the code (white-space, formatting, missing semicolons, etc.)
  • refactor: Code change that neither fixes a bug nor adds a feature
  • perf: Code change that improves performance
  • test: Adding missing tests or correcting existing tests
  • build: Changes that affect the build system or external dependencies
  • ci: Changes to CI configuration files and scripts
  • chore: Other changes that don't modify src or test files
  • revert: Reverts a previous commit
  • BREAKING CHANGE: Introduces a breaking change (can be combined with any type above)

Checklist

  • Code follows the style guidelines of this project
  • Code has been self-reviewed
  • Code has been commented, particularly in hard-to-understand areas
  • Code docstring/documentation-blocks for new or existing methods/components have been added or updated
  • Unit tests have been added or updated for any new or modified functionality

AI Usage

  • None: No AI tools were used in creating this PR
  • Light: AI provided minor assistance (formatting, simple suggestions)
  • Moderate: AI helped with code generation or debugging specific parts
  • Heavy: AI generated most or all of the code changes

@ReenigneArcher

Copy link
Copy Markdown
Member

Thank you for the PR submission, but I noticed you didn't use our PR template.

Please update the PR to use the correct template. You can find it at https://github.com/LizardByte/.github/blob/master/.github/pull_request_template.md?plain=1

@ReenigneArcher ReenigneArcher added the ai PR has signs of heavy ai usage (either indicated by user or assumed) label Mar 30, 2026
Comment thread .github/workflows/windows_build.yml Outdated
@k4idyn
k4idyn force-pushed the fix/amf-h264-profile-coder branch from 46ed0ac to 3b8a19f Compare March 30, 2026 19:04
@k4idyn
k4idyn requested a review from ReenigneArcher March 30, 2026 19:34
@k4idyn
k4idyn force-pushed the fix/amf-h264-profile-coder branch from 4fe233d to 3b8a19f Compare March 30, 2026 19:40
@k4idyn k4idyn closed this Mar 30, 2026
@k4idyn k4idyn reopened this Mar 30, 2026
@sonarqubecloud

Copy link
Copy Markdown

Comment thread src/video.cpp Outdated
@ReenigneArcher

Copy link
Copy Markdown
Member

@k4idyn
k4idyn force-pushed the fix/amf-h264-profile-coder branch from 3b8a19f to 7fd0fc5 Compare May 30, 2026 03:27
@k4idyn

k4idyn commented May 30, 2026

Copy link
Copy Markdown
Contributor Author

Updated this PR on top of current master and addressed the review feedback about \cfg.profile.

Changes in this update:

  • Added missing \coder\ wiring for \h264_amf.
  • Replaced the invalid \cfg.profile\ usage with AMF-valid profile mapping based on existing \amd_coder\ config:
    • \cavlc\ -> \constrained_baseline\
    • \cabac\ -> \high\
    • \auto\ -> \main\

Reference used: AMF encoder parameter listings for FFmpeg.

@sonarqubecloud

Copy link
Copy Markdown

Quality Gate Failed Quality Gate failed

Failed conditions
2 New issues
2 New Code Smells (required ≤ 0)

See analysis details on SonarQube Cloud

Catch issues before they fail your Quality Gate with our IDE extension SonarQube for IDE

@ReenigneArcher ReenigneArcher left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR claims to change the UI in the PR description, but no ui files are modified? Is any of this meant to be user configurable? Or only configured from the client itself?

For example, the PSP-1000's Media Engine only supports

Didn't realize there's a Moonlight client for PSP. You should add https://github.com/k4idyn/Moonlight-PSP to https://github.com/LizardByte/app-directory whenever you feel it's stable enough.

@k4idyn

k4idyn commented Jul 23, 2026

Copy link
Copy Markdown
Contributor Author

In the Configuration -> Encoder tabs, Sunshine/Apollo allows selecting cavlc/cabac in the UI, this setting is ignored by the encoder because it isn't passed to the FFmpeg dictionary; Forcefully streaming cabac regardless of what the user chooses.

@ReenigneArcher

Copy link
Copy Markdown
Member

Thanks for confirming. There are conflicts on this PR, can you resolve them?

@k4idyn
k4idyn force-pushed the fix/amf-h264-profile-coder branch from 7fd0fc5 to 96bec1c Compare July 23, 2026 04:25
@k4idyn

k4idyn commented Jul 23, 2026

Copy link
Copy Markdown
Contributor Author

Rebased on latest master and resolved merge conflicts.

@codecov

codecov Bot commented Jul 23, 2026

Copy link
Copy Markdown

Bundle Report

Bundle size has no change ✅

@codecov

codecov Bot commented Jul 23, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 9.09091% with 10 lines in your changes missing coverage. Please review.
✅ Project coverage is 26.68%. Comparing base (c6bef33) to head (0bb97be).
✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
src/video.cpp 9.09% 8 Missing and 2 partials ⚠️
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #4927      +/-   ##
==========================================
- Coverage   26.70%   26.68%   -0.02%     
==========================================
  Files         111      111              
  Lines       25406    25416      +10     
  Branches    11232    11236       +4     
==========================================
- Hits         6785     6783       -2     
- Misses      15624    16231     +607     
+ Partials     2997     2402     -595     
Flag Coverage Δ
Archlinux 0.00% <ø> (ø)
FreeBSD-aarch64 ?
FreeBSD-amd64 13.16% <ø> (-0.05%) ⬇️
Homebrew-macos-14 20.29% <ø> (ø)
Homebrew-macos-15 20.47% <ø> (+<0.01%) ⬆️
Homebrew-macos-26 20.58% <ø> (-0.01%) ⬇️
Homebrew-ubuntu-24.04 12.91% <ø> (ø)
Linux-AppImage 12.25% <ø> (ø)
Windows-AMD64 15.17% <0.00%> (-0.01%) ⬇️
Windows-ARM64 13.21% <12.50%> (-0.03%) ⬇️
macOS-arm64 17.69% <ø> (ø)
macOS-x86_64 18.36% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
src/video.cpp 51.94% <9.09%> (-0.35%) ⬇️

... and 44 files with indirect coverage changes


Continue to review full report in Codecov by Harness.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c6bef33...0bb97be. Read the comment docs.

@ReenigneArcher

Copy link
Copy Markdown
Member

Failures are probably all from docs. You can easily see the errors here: https://app.readthedocs.org/projects/sunshinestream/builds/33718953/ and search for error:

@k4idyn
k4idyn force-pushed the fix/amf-h264-profile-coder branch from 96bec1c to d497913 Compare July 23, 2026 14:30
@k4idyn

k4idyn commented Jul 23, 2026

Copy link
Copy Markdown
Contributor Author

Added Doxygen docstrings (/** @brief ... */) to
amespace amf and enum class coder_e in src/video.cpp to resolve the Doxygen documentation build.

@ReenigneArcher
ReenigneArcher force-pushed the fix/amf-h264-profile-coder branch from d497913 to 0bb97be Compare July 23, 2026 14:53
@sonarqubecloud

Copy link
Copy Markdown

Quality Gate Failed Quality Gate failed

Failed conditions
2 New issues
2 New Code Smells (required ≤ 0)

See analysis details on SonarQube Cloud

💡 Need a hand with PR review? Try Gitar by Sonar!

Comment thread src/video.cpp
Comment on lines +1042 to +1053
// SDR-specific options
{"profile"s, [](const config_t &) {
switch (config::video.amd.amd_coder) {
case (int) amf::coder_e::cavlc:
return "constrained_baseline"s;
case (int) amf::coder_e::cabac:
return "high"s;
default:
return "main"s;
}
}},
},

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

According to GPT 5.6,

Baseline profile is never applied. This new profile option is an AMF-private option, but Sunshine already sets AVCodecContext::profile to High for H.264 at line 1994. FFmpeg gives that public profile precedence and only consults the private option when no recognized public profile was supplied, as shown in FFmpeg’s AMF initializer. Consequently, CAVLC is enabled but the stream remains High Profile; the claimed PSP/legacy-decoder fix does not work. The AMF CAVLC path needs to set AVCodecContext::profile to constrained baseline instead.

So, I put up an alternate approach to this. Can you check if #5442 solves your issues? I also added some tests and resolved some existing sonar issues that were highlighted in this PR.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Physically tested PR #5442 on an AMD host PC streaming to real PSP hardware. Setting amd_coder = cavlc initialized in Constrained Baseline mode. PR #5442 works!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ai PR has signs of heavy ai usage (either indicated by user or assumed)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants