Properly Downmix 5.1 Audio

[Gr3q]

Disclaimer: could not find the ffmpeg downmix options in the source so I do not know the exact state of this, but I've seen the my logs and jellyfin mostly uses the -ac 2 flag.

Describe the bug
Default downmix option '-ac 2' in ffmpeg produces audio what is not true to the original 5.1 audio.

Logs
An example from my log:

 /usr/bin/ffmpeg -ss 00:15:42.000 -fflags +genpts -noaccurate_seek -i file:"/HDD/Sorozatok/Avatar - The Legend of Korra/Season 2/The.Legend.of.Korra.S02E06.The.Sting.720p.HDTV.x264.mkv" -map_metadata -1 -map_chapters -1 -threads 0 -map 0:0 -map 0:1 -map -0:s -codec:v:0 copy -bsf:v h264_mp4toannexb -copyts -vsync -1 -codec:a:0 aac -strict experimental -ac 2 -ab 384000 -af "volume=2" -f hls -max_delay 5000000 -avoid_negative_ts disabled -start_at_zero -hls_time 6 -individual_header_trailer 0 -hls_segment_type mpegts -start_number 157 -hls_segment_filename "/var/lib/jellyfin/transcoding-temp/8e1aac6772385f39f58dd79afd65f4a5%d.ts" -hls_playlist_type vod -hls_list_size 0 -y "/var/lib/jellyfin/transcoding-temp/8e1aac6772385f39f58dd79afd65f4a5.m3u8"

To Reproduce
Watch a movie with 5.1 what is downmixed to stereo

Additional context
There are a couple of alternative formulas tested for 5.1 downmix options here, - while I understand that this could be seen as personal preference - this could be explored further especially when others did the hard work already (and the default '-ac 2' option discards the subwoofer channel entirely).

[JustAMan]

For the reference: here's what I use in my pet project: -ac 2 -af 'pan=stereo|FL < 1.0*FL + 0.707*FC + 0.707*BL|FR < 1.0*FR + 0.707*FC + 0.707*BR'

[stale]

This issue has gone 120 days without comment. To avoid abandoned issues, it will be closed in 21 days if there are no new comments.
If you're the original submitter of this issue, please comment confirming if this issue still affects you in the latest release or nightlies, or close the issue if it has been fixed. If you're another user also affected by this bug, please comment confirming so. Either action will remove the stale label.
This bot exists to prevent issues from becoming stale and forgotten. Jellyfin is always moving forward, and bugs are often fixed as side effects of other changes. We therefore ask that bug report authors remain vigilant about their issues to ensure they are closed if fixed, or re-confirmed - perhaps with fresh logs or reproduction examples - regularly. If you have any questions you can reach us on Matrix or Social Media.

[Shadowghost]

Any reason not to change the current command to the one you used in your pet project if it's better?

[Gr3q]

I think no one is touching it because it can come down to personal preference, or what people are used to.
So I can only give you some "facts":

• The default -ac 2 flag discards the subwoofer and dialogue can be too quiet and the audio effects too loud.

• Then there is the one considered "true" to 5.1:
  -af "pan=stereo|FL=0.5*FC+0.707*FL+0.707*BL+0.5*LFE|FR=0.5*FC+0.707*FR+0.707*BR+0.5*LFE
  what uses the Subwoofer channel and has a better dialogue-effects-music balance.

• Then there is @JustAMan 's mixing, and there is a Nighttime mixing in this page along with the others as well.

If people don't want to commit to a new command it might be a good idea to offer it up as a config option.

[JustAMan]

@Gr3q reading your SO-linked answer it seems "my" formula is taken from there and already implemented in new enough ffmpeg if you say -ac 2.
So a "good enough" option to me seems to provide an editable drop-down box with a few pre-defined choices. Sounds right?

[Gr3q]

Yes, I agree

[stale]

This issue has gone 120 days without comment. To avoid abandoned issues, it will be closed in 21 days if there are no new comments.
If you're the original submitter of this issue, please comment confirming if this issue still affects you in the latest release or nightlies, or close the issue if it has been fixed. If you're another user also affected by this bug, please comment confirming so. Either action will remove the stale label.
This bot exists to prevent issues from becoming stale and forgotten. Jellyfin is always moving forward, and bugs are often fixed as side effects of other changes. We therefore ask that bug report authors remain vigilant about their issues to ensure they are closed if fixed, or re-confirmed - perhaps with fresh logs or reproduction examples - regularly. If you have any questions you can reach us on Matrix or Social Media.

[ejalal]

up

[stale]

This issue has gone 120 days without comment. To avoid abandoned issues, it will be closed in 21 days if there are no new comments.
If you're the original submitter of this issue, please comment confirming if this issue still affects you in the latest release or nightlies, or close the issue if it has been fixed. If you're another user also affected by this bug, please comment confirming so. Either action will remove the stale label.
This bot exists to prevent issues from becoming stale and forgotten. Jellyfin is always moving forward, and bugs are often fixed as side effects of other changes. We therefore ask that bug report authors remain vigilant about their issues to ensure they are closed if fixed, or re-confirmed - perhaps with fresh logs or reproduction examples - regularly. If you have any questions you can reach us on Matrix or Social Media.

[Gr3q]

up

[ejalal]

Hey guys,
Is there any way to manually replace the "-ac 2" parameter without building a custom jellyfin build ?

[dkanada]

You can replace your ffmpeg path in Jellyfin with a bash script that strips out the parameter and then calls the real ffmpeg with the resulting command.

[MateusAmin]

@Gr3q

I think this is the source you are looking for:

jellyfin/MediaBrowser.Controller/MediaEncoding/EncodingHelper.cs

Line 3284 in cb94b5a

args += " -ac " + channels.Value;

I would be interested in a better generic down-mixing solution. Although, to that end it may be better left to an upstream project.

As linked, it looks like ffmpeg is following the spec. so maybe the spec should be improved?

[stale]

This issue has gone 120 days without comment. To avoid abandoned issues, it will be closed in 21 days if there are no new comments.
If you're the original submitter of this issue, please comment confirming if this issue still affects you in the latest release or nightlies, or close the issue if it has been fixed. If you're another user also affected by this bug, please comment confirming so. Either action will remove the stale label.
This bot exists to prevent issues from becoming stale and forgotten. Jellyfin is always moving forward, and bugs are often fixed as side effects of other changes. We therefore ask that bug report authors remain vigilant about their issues to ensure they are closed if fixed, or re-confirmed - perhaps with fresh logs or reproduction examples - regularly. If you have any questions you can reach us on Matrix or Social Media.

[danielfomin96]

Hey all,
I created a script as a workaround, following @dkanada's idea, to be used as a wrapper for ffmpeg.
This allows jellyfin to downmix everything to stereo using a custom formula.
The used formula is a mix of Gregory's and Dave's answer from this link optimized for my stereo setup at home.

[ManuLinares]

TL;DR: if you have configured an alternative ffmpeg path in Jellyfin, make sure there's an ffprobe binary in the same location with the same file extension.

https://blog.jeroenhd.nl/article/jellyfin-rffmpeg-and-confusing-error-messages

[ManuLinares]

#!/bin/bash -
#title           :ffmpeg properly downmix to stereo
#description     :This script is a wrapper for jellyfin ffmpeg to downmix audio
#                :to stereo using a custom formula
#==============================================================================

ffmpeg="/usr/bin/ffmpeg"

function join { local IFS=$'\n'; echo "${*}"; }

##
args=$(join "${@}")
match='-ac\n[3-9]'
replace='-af\npan=stereo|FL < 0.5*FC + 0.3*FLC + 0.3*FL + 0.3*BL + 0.3*SL + 0.5*LFE | FR < 0.5*FC + 0.3*FRC + 0.3*FR + 0.3*BR + 0.3*SR + 0.5*LFE,volume=2'
readarray -t args < <(sed -z "s/$match/$replace/g" <<<"$args")
##

$ffmpeg "${args[@]}"
exit $?

This is the wrapper I've been using. Jellyfin should use this formulae instead of "-ac 2" for optimal downmix to stereo.

As a note: If replacing ffmpeg with this wrapper in jellyfin configuration, it should have ffprobe symlink in the same directory. You can do this with: ln -s $(which ffprobe) ./ffprobe

[justinlboyer]

In case anyone is trying to get this to work on Roku. It seems like the problem is the audio stream tries to direct play. I.e. -codec:a:0 copy is getting fed to the roku, when we really need -codec:a:0 <audio downmix encoding>.

My little hack to fix this is (riffing of the help from above... on this issue):

#!/bin/bash
ffmpeg="/usr/lib/jellyfin-ffmpeg/ffmpeg"

function join { local IFS=$'\n'; echo "${*}"; }

args=$(join "${@}")

match_roku='-codec:a:0\ncopy'
replace_roku='-codec:a:0\naac\n-ac\n2'
readarray -t args < <(sed -z "s/$match_roku/$replace_roku/g" <<<"$args")
#echo ${args[@]}

match='-ac\n[2-9]'
replace='-af\npan=stereo|FL < 0.5*FC + 0.3*FLC + 0.3*FL + 0.3*BL + 0.3*SL + 0.5*LFE | FR < 0.5*FC + 0.3*FRC + 0.3*FR + 0.3*BR + 0.3*SR + 0.5*LFE,volume=2'

args=$(join "${args[@]}")
readarray -t new_args < <(sed -z "s/$match/$replace/g" <<<"$args")
#echo ${new_args[@]}


$ffmpeg "${new_args[@]}"
exit $?

If someone has a cleaner way to get at this result, please share.

[ManuLinares]

If someone has a cleaner way to get at this result, please share.

Cleaner way.

#!/bin/bash -
#title           : ffmpeg properly downmix to stereo and downscale 4k to 1080p
#description     :  This script is a wrapper for jellyfin ffmpeg to downmix audio
#                :  to stereo using a custom formula, and downscale 4k to 1080p


ffmpeg="/usr/bin/ffmpeg"

#Add quotes for arguments with "spaces" and "parentheses")
for x in "${@}" ; do
    # try to figure out if quoting was required for the $x (Arguments)
    if [[ "$x" != "${x%[[:space:]]*}" ]] \
    || [[ "$x" != "${x%(*}" ]] ; then
        x="\""$x"\""
    fi
    _args=$_args" "$x
done

# add multiple replace, like:
#   _args=${_args/match/replace}
#   you can use bash pattern matching described in 
#   https://www.gnu.org/software/bash/manual/bash.html#Pattern-Matching

# Downmix audio to stereo
_args=${_args/-ac [3-9]/-af \"pan=stereo|FL < 0.5*FC + 0.3*FLC + 0.3*FL + 0.3*BL + 0.3*SL + 0.5*LFE | FR < 0.5*FC + 0.3*FRC + 0.3*FR + 0.3*BR + 0.3*SR + 0.5*LFE,volume=2\"}

# Downscale 4k to 1080p
_args=${_args/zscale=s=3840x????/zscale=s=1920x1080}

# Maybe change the codec in roku, I don't know, that is why I gave the examples I use above
_args=${_args/-codec:a:0 copy/-codec:a:0 aac}

# Execute wrapper
eval $ffmpeg $_args

exit $?

---

This is the best wrapper I've made so far:

• Easy to add multiple "replace".
• It works with wildly changing input
• Using standard https://www.gnu.org/software/bash/manual/bash.html#Pattern-Matching

[justinlboyer]

Maybe change the codec in roku, I don't know, that is why I gave the examples I use above
_args=${_args/-codec:a:0 copy/-codec:a:0 aac}

I was including the downmix as well, not sure if that is needed, not really sure what is going on. But my change does work.

This is the best wrapper I've made so far:

It looks great! Thank you!

[RTUnit]

The solution with wrapper worked great! Thanks!! Without wrapper I just can't hear dialogues.

These steps I execute after creating Jellyfin Docker container:
docker exec -it jellyfin mkdir /config/ffmpeg
docker exec -it jellyfin touch /config/ffmpeg/ffmpeg.wrap
docker exec -it -u 0 jellyfin ln -sf /config/ffmpeg/ffmpeg.wrap /usr/lib/jellyfin-ffmpeg/ffmpeg.wrap
docker exec -it -u 0 jellyfin ln -sf /usr/lib/jellyfin-ffmpeg/ffprobe /usr/lib/jellyfin-ffmpeg/ffprobe.wrap

I modify the /config/ffmpeg/ffmpeg.wrap with the wrapper code proposed above. For docker I needed just one change:

ffmpeg="/usr/lib/jellyfin-ffmpeg/ffmpeg"

then update the path to ffmpeg.wrap in Jellyfin server settings (under Playback):

[I-G-1-1]

I'm also facing the problem with low voice when playng movies with 5.1 tracks. So I'm trying to implement the workarround suggested in this thread for a better downmix to 2 stereo channels as I don't have any surround system / soundbar.

I'm using Jellyfin server 10.7.7 on Debian. This is my configuration:

sudo mkdir /usr/lib/jellyfin-ffmpeg-downmix
sudo ln -s /usr/lib/jellyfin-ffmpeg/ffprobe /usr/lib/jellyfin-ffmpeg-downmix/ffprobe.wrap
sudo touch /usr/lib/jellyfin-ffmpeg-downmix/ffmpeg.wrap
sudo chmod 755 -R /usr/lib/jellyfin-ffmpeg-downmix

with /usr/lib/jellyfin-ffmpeg-downmix/ffmpeg.wrap:

#!/bin/bash -
#title           : ffmpeg properly downmix to stereo and downscale 4k to 1080p
#description     :  This script is a wrapper for jellyfin ffmpeg to downmix audio
#                :  to stereo using a custom formula, and downscale 4k to 1080p

ffmpeg="/usr/lib/jellyfin-ffmpeg/ffmpeg"

#Add quotes for arguments with "spaces" and "parentheses")
for x in "${@}" ; do
    # try to figure out if quoting was required for the $x (Arguments)
    if [[ "$x" != "${x%[[:space:]]*}" ]] \
    || [[ "$x" != "${x%(*}" ]] ; then
        x="\""$x"\""
    fi
    _args=$_args" "$x
done

# add multiple replace, like:
#   _args=${_args/match/replace}
#   you can use bash pattern matching described in
#   https://www.gnu.org/software/bash/manual/bash.html#Pattern-Matching

# Downmix audio to stereo
_args=${_args/-ac [3-9]/-af \"pan=stereo|FL < 0.5*FC + 0.3*FLC + 0.3*FL + 0.3*BL + 0.3*SL + 0.5*LFE | FR < 0.5*FC + 0.3*FRC + 0.3*FR + 0.3*BR + 0.3*SR + 0.5*LFE,volume=2\"}

# Maybe change the codec in roku, I don't know, that is why I gave the examples I use above
_args=${_args/-codec:a:0 copy/-codec:a:0 aac}

# Execute wrapper
eval $ffmpeg $_args

exit $?

and Jellyfin playback/FFmpeg path set to use
/usr/lib/jellyfin-ffmpeg-downmix/ffmpeg.wrap

Now when I try to play a movie with a 5.1 track on it (eac3 or ac3) jellyfin doesn't force the conversion and direct stream to the client (I also can see using htop on the debian machine that there is no ffpeg conversion). As client I use the Android official app.
I also don't undestand what I have to choose in Settings/Playback/Audio Settings/Maximum Allowed Audio Channels: Auto, Stereo or 5.1 Channels?

I also have an AndroidTV Box (Mi Box S) with Jellyfin for AndroidTV official app and with this ffmpeg.wrap setup when I try to play the same movie with 5.1 ac3/eac3 all the players (libvlc, exoplayer, external mpv player, external Mx Player) cannot start to play the movie.

What I'm doing wrong? Any help would be really appreciated, thanks

[RTUnit]

@I-G-1-1 ,

Some ideas what to check...

I do not use this line, please try without it: _args=${_args/-codec:a:0 copy/-codec:a:0 aac}
For a test you can try with just empty wrapper - directly pass everything to ffmpeg without doing anything. See if it works.

I have ffprob.wrap in the same folder where ffmpeg:
I have no name!@QNAS:/usr/lib/jellyfin-ffmpeg$ ls -l total 40011 -rwxr-xr-x 1 root root 24232640 Jan 3 18:07 ffmpeg lrwxr-xr-x 1 root root 26 Mar 11 13:35 ffmpeg.wrap -> /config/ffmpeg/ffmpeg.wrap -rwxr-xr-x 1 root root 24089344 Jan 3 18:07 ffprobe lrwxr-xr-x 1 root root 32 Mar 11 20:04 ffprobe.wrap -> /usr/lib/jellyfin-ffmpeg/ffprobe drwxr-xr-x 4 root root 25 Jan 5 01:42 lib

Maybe compare file ownership for ffmpeg and ffmpeg.wrap to see if it match.

Is there anything in the logs when you test it on Mi Box S? Logs helped to resolve issues that I were facing.

For Jellyfin server 10.7.7 on Debian can you try to force subtitles? It should result in forced transcoding. Just I do not remember where was this setting.

[GNUGradyn]

This seems like a pretty hacky workaround for such a big issue. Dialogue is extremely quiet out of the box when downmixing and the official solution is to write a bash script to trick jellyfin into thinking ffmpeg is doing 1 thing when its actually doing something else?

Surely you do not expect the average joe running this in docker on their NAS or whatever to do this workaround? Or do we just expect everyone to have native stereo copies of all their content? Or are we just expecting everyone to have 5.1 surround and never want to watch something on their phone or their laptop?

I understand this is the default way ffmpeg is handling it when you do -ac 2 but I think having a functional experience out of the box should always be more important then rigidly adhering to standards. At a minimum we should add a toggle.

[Electry]

100% agreed.

It is easy to rebuild the docker image with the -ac 2 parameter changed to a different formula for proper downmixing, but this absolutely should not be something that an end-user would have to do.

[RTUnit]

If someone has a cleaner way to get at this result, please share.

Cleaner way.

#!/bin/bash -
#title           : ffmpeg properly downmix to stereo and downscale 4k to 1080p
#description     :  This script is a wrapper for jellyfin ffmpeg to downmix audio
#                :  to stereo using a custom formula, and downscale 4k to 1080p


ffmpeg="/usr/bin/ffmpeg"

#Add quotes for arguments with "spaces" and "parentheses")
for x in "${@}" ; do
    # try to figure out if quoting was required for the $x (Arguments)
    if [[ "$x" != "${x%[[:space:]]*}" ]] \
    || [[ "$x" != "${x%(*}" ]] ; then
        x="\""$x"\""
    fi
    _args=$_args" "$x
done

# add multiple replace, like:
#   _args=${_args/match/replace}
#   you can use bash pattern matching described in 
#   https://www.gnu.org/software/bash/manual/bash.html#Pattern-Matching

# Downmix audio to stereo
_args=${_args/-ac [3-9]/-af \"pan=stereo|FL < 0.5*FC + 0.3*FLC + 0.3*FL + 0.3*BL + 0.3*SL + 0.5*LFE | FR < 0.5*FC + 0.3*FRC + 0.3*FR + 0.3*BR + 0.3*SR + 0.5*LFE,volume=2\"}

# Downscale 4k to 1080p
_args=${_args/zscale=s=3840x????/zscale=s=1920x1080}

# Maybe change the codec in roku, I don't know, that is why I gave the examples I use above
_args=${_args/-codec:a:0 copy/-codec:a:0 aac}

# Execute wrapper
eval $ffmpeg $_args

exit $?

This is the best wrapper I've made so far:

• Easy to add multiple "replace".
• It works with wildly changing input
• Using standard https://www.gnu.org/software/bash/manual/bash.html#Pattern-Matching

Script breaks in situation when ffmpeg argument contains no spaces, but contains a semicolon. Script needs small modification to add || [[ "$x" != "${x%[;]*}" ]] \, here is the updated script:

#!/bin/bash -
#title           : ffmpeg properly downmix to stereo and downscale 4k to 1080p
#description     :  This script is a wrapper for jellyfin ffmpeg to downmix audio
#                :  to stereo using a custom formula, and downscale 4k to 1080p

ffmpeg="/usr/bin/ffmpeg"

#Add quotes for arguments with "spaces", "semicolon" and "parentheses")
for x in "${@}" ; do
    # try to figure out if quoting was required for the $x (Arguments)
    if [[ "$x" != "${x%[[:space:]]*}" ]] \
    || [[ "$x" != "${x%[;]*}" ]] \
    || [[ "$x" != "${x%(*}" ]] ; then
        x="\""$x"\""
    fi
    _args=$_args" "$x
done

# add multiple replace, like:
#   _args=${_args/match/replace}
#   you can use bash pattern matching described in 
#   https://www.gnu.org/software/bash/manual/bash.html#Pattern-Matching

# Downmix audio to stereo
_args=${_args/-ac [3-9]/-af \"pan=stereo|FL < 0.5*FC + 0.3*FLC + 0.3*FL + 0.3*BL + 0.3*SL + 0.5*LFE | FR < 0.5*FC + 0.3*FRC + 0.3*FR + 0.3*BR + 0.3*SR + 0.5*LFE,volume=2\"}

# Downscale 4k to 1080p
_args=${_args/zscale=s=3840x????/zscale=s=1920x1080}

# Maybe change the codec in roku, I don't know, that is why I gave the examples I use above
_args=${_args/-codec:a:0 copy/-codec:a:0 aac}

# Execute wrapper
eval $ffmpeg $_args

exit $?

The particular problem I faced with -filter_complex argument which had following argument: "[0:4]scale=s=720x576:flags=fast_bilinear,format=yuva420p,hwupload=derive_device=cuda[sub];[0:0]setparams=color_primaries=bt709:color_trc=bt709:colorspace=bt709,scale_cuda=format=yuv420p[main];[main][sub]overlay_cuda=eof_action=endall:shortest=1:repeatlast=0" and it caused the following error in Jellyfin log:

/usr/lib/jellyfin-ffmpeg/ffmpeg.wrap: line 38: [0:0]setparams=color_primaries=bt709:color_trc=bt709:colorspace=bt709,scale_cuda=format=yuv420p[main]: command not found
/usr/lib/jellyfin-ffmpeg/ffmpeg.wrap: line 38: [main][sub]overlay_cuda=eof_action=endall:shortest=1:repeatlast=0: command not found

Error does not occur with above updated script.

[Shadowghost]

This should all be unnecessary once #9001 is merged - at least if you just want to use another downmix algorithm.

[RTUnit]

This should all be unnecessary once #9001 is merged - at least if you just want to use another downmix algorithm.

Only for this feature, it is true my friend. However the script provided by ManuLinares is excellent way to customize so much more.

For example, I use the wrapper to clean up transcoding directory which is having limited space (RAM drive). It has finally made the playback without hiccups. More details on the solution is here: #3929. And even that feature is waiting for merge #8744.

Then again, there will be someone like me who is upgrading Jellyfin server once per year, so these workarounds let me benefit today and not wait for upgrade. Jellyfin upgrade is not always smooth - release comes with compatibility for minimum FFMPEG version, which has expected compatibility for GPU drivers (link is here), these compatibility requirements don't change often, but I have faced problem that QNAP didn't had released NVIDIA driver/kernel update for my NAS, hence transcoding was not working at all and I needed to rollback Jellyfin and continue to enjoy workarounds.

---

A further extension for this wonderful idea of FFMPEG WRAP to maintain free space in transcodes directory: Jellyfin-Transcodes-cleanup.