New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Kernel 4.14 behaves poorly in LibreELEC #2503

Open
smp79 opened this Issue Apr 10, 2018 · 29 comments

Comments

Projects
None yet
6 participants
@smp79

smp79 commented Apr 10, 2018

I have to use kernel 4.9 for my LibreELEC 9.0 builds because with the current default kernel 4.14 there are audio dropouts during heavy sdcard i/o activity.
Among other things, this makes impossible to watch and record the TV channel at the same time. The issue is present in current Milhouse builds. No such issue when building with kernel 4.9.

@popcornmix

This comment has been minimized.

Show comment
Hide comment
@popcornmix

popcornmix Apr 11, 2018

Collaborator

You'll need to provide some details. What model of Pi? What audio interface?
Any custom config.txt settings or hardware attached?
Mediainfo of a file that has problems?
Can you reproduce without PVR? (e.g. play a test file and copy a file on sdcard at same time)

Collaborator

popcornmix commented Apr 11, 2018

You'll need to provide some details. What model of Pi? What audio interface?
Any custom config.txt settings or hardware attached?
Mediainfo of a file that has problems?
Can you reproduce without PVR? (e.g. play a test file and copy a file on sdcard at same time)

@smp79

This comment has been minimized.

Show comment
Hide comment
@smp79

smp79 Apr 11, 2018

I reported about this issue right after the softirq/DVB fix was discovered. Kernel 4.9 and newer started to be usable for USB DVB owners and that's when I noticed another kernel issue (this time it seem to be Pi-specific).

The hardware is a Pi3, connected directly to a TV via HDMI. DVB tuner connected to the same Pi3.
I don't know how to reproduce without PVR but 1 user reported that my build with kernel 4.9 fixed a similar issue for him.

The easiest way for me to reproduce is to start recording when watching a TV channel. That's when the audio dropouts occur with tons of ActiveAE errors in kodi.log. Also any heavy network activity like copying a file over LAN will cause the short audio dropouts / ActiveAE errors.

I also tried to record to a USB drive instead of the sdcard - same issues.

The recorded stream is perfectly fine, only the playback is affected.

smp79 commented Apr 11, 2018

I reported about this issue right after the softirq/DVB fix was discovered. Kernel 4.9 and newer started to be usable for USB DVB owners and that's when I noticed another kernel issue (this time it seem to be Pi-specific).

The hardware is a Pi3, connected directly to a TV via HDMI. DVB tuner connected to the same Pi3.
I don't know how to reproduce without PVR but 1 user reported that my build with kernel 4.9 fixed a similar issue for him.

The easiest way for me to reproduce is to start recording when watching a TV channel. That's when the audio dropouts occur with tons of ActiveAE errors in kodi.log. Also any heavy network activity like copying a file over LAN will cause the short audio dropouts / ActiveAE errors.

I also tried to record to a USB drive instead of the sdcard - same issues.

The recorded stream is perfectly fine, only the playback is affected.

@mkreisl

This comment has been minimized.

Show comment
Hide comment
@mkreisl

mkreisl Apr 13, 2018

Similar issue reported from an XBian user, see this post

Reverting kernel 4.14.32+ to 4.9.91+ solves his issue

mkreisl commented Apr 13, 2018

Similar issue reported from an XBian user, see this post

Reverting kernel 4.14.32+ to 4.9.91+ solves his issue

@smp79

This comment has been minimized.

Show comment
Hide comment
@smp79

smp79 Apr 13, 2018

I will also add that I tested with different kernel versions - the issue was introduced in kernel 4.10.

smp79 commented Apr 13, 2018

I will also add that I tested with different kernel versions - the issue was introduced in kernel 4.10.

@popcornmix

This comment has been minimized.

Show comment
Hide comment
@popcornmix

popcornmix Apr 13, 2018

Collaborator

Instructions to reproduce would be useful. e.g. play this file and run this program at the same time causes audio dropouts.

I'm running Kodi on 4.14 kernel (Milhouse nightly build), including running PVR and haven't had this issue.

I assume enabling omxplayer avoids the issue?
I've seen some reports that disabling file/memory cache (e.g removing custom advancedsettings.xml helps).
Does changing the audio output format make a difference? (e.g. passthough on/off, ac3 transcoding on/off, outputting to 5.1 with/without stereo upmix).

Collaborator

popcornmix commented Apr 13, 2018

Instructions to reproduce would be useful. e.g. play this file and run this program at the same time causes audio dropouts.

I'm running Kodi on 4.14 kernel (Milhouse nightly build), including running PVR and haven't had this issue.

I assume enabling omxplayer avoids the issue?
I've seen some reports that disabling file/memory cache (e.g removing custom advancedsettings.xml helps).
Does changing the audio output format make a difference? (e.g. passthough on/off, ac3 transcoding on/off, outputting to 5.1 with/without stereo upmix).

@smp79

This comment has been minimized.

Show comment
Hide comment
@smp79

smp79 Apr 13, 2018

OMXPlayer does not avoid the issue, advancedsettings.xml is empty, audio output format does not make a difference.
I just tried to reproduce the issue without PVR and it is quite easy.

  1. Start the playback of a high bitrate 1080p video from a USB flash drive.
  2. During the playback start copying a large file to the Pi's sdcard over LAN.
    That's when the audio dropouts will start to occur. Check kodi.log for ActiveAE errors.

smp79 commented Apr 13, 2018

OMXPlayer does not avoid the issue, advancedsettings.xml is empty, audio output format does not make a difference.
I just tried to reproduce the issue without PVR and it is quite easy.

  1. Start the playback of a high bitrate 1080p video from a USB flash drive.
  2. During the playback start copying a large file to the Pi's sdcard over LAN.
    That's when the audio dropouts will start to occur. Check kodi.log for ActiveAE errors.
@smp79

This comment has been minimized.

Show comment
Hide comment
@smp79

smp79 Apr 20, 2018

Found this, possibly related issue.

smp79 commented Apr 20, 2018

Found this, possibly related issue.

@polojoe

This comment has been minimized.

Show comment
Hide comment
@polojoe

polojoe May 2, 2018

The described issue also exists if a download is started during livetv watching in le. After some time there's an audio dropout, after that a/v are out of sync.
Logs required?

polojoe commented May 2, 2018

The described issue also exists if a download is started during livetv watching in le. After some time there's an audio dropout, after that a/v are out of sync.
Logs required?

@Krawei

This comment has been minimized.

Show comment
Hide comment
@Krawei

Krawei Jun 15, 2018

Any updates here?

Krawei commented Jun 15, 2018

Any updates here?

@smp79

This comment has been minimized.

Show comment
Hide comment
@smp79

smp79 Jun 16, 2018

I tested with kernel 4.17 and there is no improvement.

smp79 commented Jun 16, 2018

I tested with kernel 4.17 and there is no improvement.

@SpeedyQ

This comment has been minimized.

Show comment
Hide comment
@SpeedyQ

SpeedyQ Jul 22, 2018

Any updates on this issue?

I recently tried updating to 4.14.54+ kernel (xbian on raspberry pi) which is current stable version, but the issue still exists where music playback seems to pause for millisecond(s) and continuing again where it paused.
I reverted back to 4.9.96+ kernel which is still ok regarding the stutter issue...

SpeedyQ commented Jul 22, 2018

Any updates on this issue?

I recently tried updating to 4.14.54+ kernel (xbian on raspberry pi) which is current stable version, but the issue still exists where music playback seems to pause for millisecond(s) and continuing again where it paused.
I reverted back to 4.9.96+ kernel which is still ok regarding the stutter issue...

@smp79

This comment has been minimized.

Show comment
Hide comment
@smp79

smp79 Aug 13, 2018

Similar issue reported here. I reverted commit a1b8913 but unfortunately it didn't improve things.

smp79 commented Aug 13, 2018

Similar issue reported here. I reverted commit a1b8913 but unfortunately it didn't improve things.

@smp79

This comment has been minimized.

Show comment
Hide comment
@smp79

smp79 Aug 14, 2018

I think I'm getting closer to solving this. I built 2 LibreELEC images: with kernel 4.10-rc1 and 4.10.0.
4.10-rc1: no audio dropouts at all when recording a DVB stream
4.10.0: lots of audio dropouts when recording a DVB stream

The offending commit is somewhere between 4.10-rc2 and 4.10-rc8. I will build more images and test.

smp79 commented Aug 14, 2018

I think I'm getting closer to solving this. I built 2 LibreELEC images: with kernel 4.10-rc1 and 4.10.0.
4.10-rc1: no audio dropouts at all when recording a DVB stream
4.10.0: lots of audio dropouts when recording a DVB stream

The offending commit is somewhere between 4.10-rc2 and 4.10-rc8. I will build more images and test.

@smp79

This comment has been minimized.

Show comment
Hide comment
@smp79

smp79 Aug 14, 2018

Ok, I narrowed it down to kernel 4.10-rc6 (no issues with rc5 and older).

smp79 commented Aug 14, 2018

Ok, I narrowed it down to kernel 4.10-rc6 (no issues with rc5 and older).

@popcornmix

This comment has been minimized.

Show comment
Hide comment
@popcornmix

popcornmix Aug 14, 2018

Collaborator

Interesting. Can you narrow it further?
Looking at the commit list (there's quite a few) the "mm" ones look like possible culprits.
Otherwise there's a number of net related commit which I guess could cause the issues here.

@smp79 Is network involved when you see the issue, or can you see it purely locally (dvb + local storage)?

Collaborator

popcornmix commented Aug 14, 2018

Interesting. Can you narrow it further?
Looking at the commit list (there's quite a few) the "mm" ones look like possible culprits.
Otherwise there's a number of net related commit which I guess could cause the issues here.

@smp79 Is network involved when you see the issue, or can you see it purely locally (dvb + local storage)?

@smp79

This comment has been minimized.

Show comment
Hide comment
@smp79

smp79 Aug 14, 2018

@popcornmix I see the issue when recording/writing to a local sdcard with no network involved. I will try to narrow it further.

smp79 commented Aug 14, 2018

@popcornmix I see the issue when recording/writing to a local sdcard with no network involved. I will try to narrow it further.

@mkreisl

This comment has been minimized.

Show comment
Hide comment
@mkreisl

mkreisl Aug 14, 2018

@SpeedyQ has reported here after changing hardware from Pi3B to Pi3B+ stuttering has gone with 4.14.58+ kernel
Maybe it's network related (as everybody knows, Pi3B+ is using different network driver)

mkreisl commented Aug 14, 2018

@SpeedyQ has reported here after changing hardware from Pi3B to Pi3B+ stuttering has gone with 4.14.58+ kernel
Maybe it's network related (as everybody knows, Pi3B+ is using different network driver)

@polojoe

This comment has been minimized.

Show comment
Hide comment
@polojoe

polojoe Aug 14, 2018

I'm using rpi3+ and also have the issue when watching tv during recording on local storage. Issue is also present during dvb and download.

polojoe commented Aug 14, 2018

I'm using rpi3+ and also have the issue when watching tv during recording on local storage. Issue is also present during dvb and download.

@smp79

This comment has been minimized.

Show comment
Hide comment
@smp79

smp79 Aug 14, 2018

Ok, I'm not a git expert and I need some help with this.
I narrowed it down to those 4 commits:
67ade05
1fdf419
57b59ed
99f300c

Commit a47b70e is good, no audio issues.
Commit 99f300c is bad, I get audio dropouts.

smp79 commented Aug 14, 2018

Ok, I'm not a git expert and I need some help with this.
I narrowed it down to those 4 commits:
67ade05
1fdf419
57b59ed
99f300c

Commit a47b70e is good, no audio issues.
Commit 99f300c is bad, I get audio dropouts.

@popcornmix

This comment has been minimized.

Show comment
Hide comment
@popcornmix

popcornmix Aug 14, 2018

Collaborator

How are you generating the commit hashes to test?
I wonder if you will be better off just using the merge commits
(e.g. git log --merges) as the points to test.

Collaborator

popcornmix commented Aug 14, 2018

How are you generating the commit hashes to test?
I wonder if you will be better off just using the merge commits
(e.g. git log --merges) as the points to test.

@smp79

This comment has been minimized.

Show comment
Hide comment
@smp79

smp79 Aug 14, 2018

I'm simply picking them one-by-one. I edit PKG_VERSION= in packages/linux/package.mk (in LibreELEC build system). The last good one is a47b70e.

I use vanilla kernel + this patch to make it build in LibreELEC build system.

smp79 commented Aug 14, 2018

I'm simply picking them one-by-one. I edit PKG_VERSION= in packages/linux/package.mk (in LibreELEC build system). The last good one is a47b70e.

I use vanilla kernel + this patch to make it build in LibreELEC build system.

@popcornmix

This comment has been minimized.

Show comment
Hide comment
@popcornmix

popcornmix Aug 14, 2018

Collaborator

I think the problem is that the git history is not linear due to the merge commits.
You may be better off using the merge commits from:

 git log --pretty=oneline --merges 99f300c...a47b70e

(but I wouldn't claim to be a git expert either).

Collaborator

popcornmix commented Aug 14, 2018

I think the problem is that the git history is not linear due to the merge commits.
You may be better off using the merge commits from:

 git log --pretty=oneline --merges 99f300c...a47b70e

(but I wouldn't claim to be a git expert either).

@smp79

This comment has been minimized.

Show comment
Hide comment
@smp79

smp79 Aug 15, 2018

I found the bad commit. It took more than 20 hours and 50+ LibreELEC/kernel builds :)

424f6c4

I also reverted that commit for kernel 4.14 - audio dropouts are now gone.

smp79 commented Aug 15, 2018

I found the bad commit. It took more than 20 hours and 50+ LibreELEC/kernel builds :)

424f6c4

I also reverted that commit for kernel 4.14 - audio dropouts are now gone.

popcornmix added a commit that referenced this issue Aug 15, 2018

Revert "mm: alloc_contig: re-allow CMA to compact FS pages"
The upstream commit caused poor video playback performance
on a busy system for many users.

See: #2503

This reverts commit 424f6c4.

Signed-off-by: popcornmix <popcornmix@gmail.com>
@popcornmix

This comment has been minimized.

Show comment
Hide comment
@popcornmix

popcornmix Aug 15, 2018

Collaborator

Thank you. That commit does look plausible for causing kernel stalls.
@mkreisl can you confirm the fix?

See #2647

Collaborator

popcornmix commented Aug 15, 2018

Thank you. That commit does look plausible for causing kernel stalls.
@mkreisl can you confirm the fix?

See #2647

@mkreisl

This comment has been minimized.

Show comment
Hide comment
@mkreisl

mkreisl Aug 15, 2018

@popcornmix
I was never able to reproduce this issue here, so it is impossible for me to confirm the fix

mkreisl commented Aug 15, 2018

@popcornmix
I was never able to reproduce this issue here, so it is impossible for me to confirm the fix

popcornmix added a commit that referenced this issue Aug 15, 2018

Revert "mm: alloc_contig: re-allow CMA to compact FS pages"
The upstream commit caused poor video playback performance
on a busy system for many users.

See: #2503

This reverts commit 424f6c4.

Signed-off-by: popcornmix <popcornmix@gmail.com>
@popcornmix

This comment has been minimized.

Show comment
Hide comment
@popcornmix

popcornmix Aug 16, 2018

Collaborator

@smp79 without the commit reverted, does increasing CMA avoid the issue?
E.g. add cma=64M to cmdline.txt ?

Collaborator

popcornmix commented Aug 16, 2018

@smp79 without the commit reverted, does increasing CMA avoid the issue?
E.g. add cma=64M to cmdline.txt ?

@smp79

This comment has been minimized.

Show comment
Hide comment
@smp79

smp79 Aug 16, 2018

No, cma=64M does nothing.

People with issues tend to be recording and playing live TV or running background jobs like torrents

I actually noticed the audio dropouts when only watching live TV with no extra background jobs. Those were rare but still annoying. Doing stuff like recording will just make the issue very obvious and easy to reproduce.

smp79 commented Aug 16, 2018

No, cma=64M does nothing.

People with issues tend to be recording and playing live TV or running background jobs like torrents

I actually noticed the audio dropouts when only watching live TV with no extra background jobs. Those were rare but still annoying. Doing stuff like recording will just make the issue very obvious and easy to reproduce.

@mkreisl

This comment has been minimized.

Show comment
Hide comment
@mkreisl

mkreisl Aug 16, 2018

Hmmm, that's weird. I'm running JDownloader2 in background while watching TV. Everything is on network (Videos, JDownloader saves data on network, even root fs is on iSCSI network disk) Never had audio dropouts even when playing 1080p HEVC vids

(XBian, Kodi Leia B1, 4.18.0+, Pi3B+)

mkreisl commented Aug 16, 2018

Hmmm, that's weird. I'm running JDownloader2 in background while watching TV. Everything is on network (Videos, JDownloader saves data on network, even root fs is on iSCSI network disk) Never had audio dropouts even when playing 1080p HEVC vids

(XBian, Kodi Leia B1, 4.18.0+, Pi3B+)

@smp79

This comment has been minimized.

Show comment
Hide comment
@smp79

smp79 Aug 16, 2018

The issue is more pronounced when the USB DVB tuner is used. It is harder to reproduce when just streaming video over LAN.
Also, those dropouts are sometimes very short - less than a second, so they can go unnoticed.

smp79 commented Aug 16, 2018

The issue is more pronounced when the USB DVB tuner is used. It is harder to reproduce when just streaming video over LAN.
Also, those dropouts are sometimes very short - less than a second, so they can go unnoticed.

popcornmix added a commit that referenced this issue Aug 17, 2018

Revert "mm: alloc_contig: re-allow CMA to compact FS pages"
The upstream commit caused poor video playback performance
on a busy system for many users.

See: #2503

This reverts commit 424f6c4.

Signed-off-by: popcornmix <popcornmix@gmail.com>

ahmedradaideh added a commit to ahmedradaideh/Pi-Kernel that referenced this issue Aug 18, 2018

Revert "mm: alloc_contig: re-allow CMA to compact FS pages"
The upstream commit caused poor video playback performance
on a busy system for many users.

See: raspberrypi/linux#2503

This reverts commit 424f6c4.

Signed-off-by: popcornmix <popcornmix@gmail.com>
Signed-off-by: ahmedradaideh <ahmed.radaideh@gmail.com>

ahmedradaideh added a commit to ahmedradaideh/Pi-Kernel that referenced this issue Aug 18, 2018

Revert "mm: alloc_contig: re-allow CMA to compact FS pages"
The upstream commit caused poor video playback performance
on a busy system for many users.

See: raspberrypi/linux#2503

This reverts commit 424f6c4.

Signed-off-by: popcornmix <popcornmix@gmail.com>
Signed-off-by: ahmedradaideh <ahmed.radaideh@gmail.com>

ahmedradaideh added a commit to ahmedradaideh/Pi-Kernel that referenced this issue Aug 18, 2018

Revert "mm: alloc_contig: re-allow CMA to compact FS pages"
The upstream commit caused poor video playback performance
on a busy system for many users.

See: raspberrypi/linux#2503

This reverts commit 424f6c4.

Signed-off-by: popcornmix <popcornmix@gmail.com>
Signed-off-by: ahmedradaideh <ahmed.radaideh@gmail.com>

popcornmix added a commit that referenced this issue Aug 22, 2018

Revert "mm: alloc_contig: re-allow CMA to compact FS pages"
The upstream commit caused poor video playback performance
on a busy system for many users.

See: #2503

This reverts commit 424f6c4.

Signed-off-by: popcornmix <popcornmix@gmail.com>

popcornmix added a commit that referenced this issue Aug 29, 2018

Revert "mm: alloc_contig: re-allow CMA to compact FS pages"
The upstream commit caused poor video playback performance
on a busy system for many users.

See: #2503

This reverts commit 424f6c4.

Signed-off-by: popcornmix <popcornmix@gmail.com>

popcornmix added a commit that referenced this issue Sep 5, 2018

Revert "mm: alloc_contig: re-allow CMA to compact FS pages"
The upstream commit caused poor video playback performance
on a busy system for many users.

See: #2503

This reverts commit 424f6c4.

Signed-off-by: popcornmix <popcornmix@gmail.com>

popcornmix added a commit that referenced this issue Sep 10, 2018

Revert "mm: alloc_contig: re-allow CMA to compact FS pages"
The upstream commit caused poor video playback performance
on a busy system for many users.

See: #2503

This reverts commit 424f6c4.

Signed-off-by: popcornmix <popcornmix@gmail.com>

popcornmix added a commit that referenced this issue Sep 10, 2018

Revert "mm: alloc_contig: re-allow CMA to compact FS pages"
The upstream commit caused poor video playback performance
on a busy system for many users.

See: #2503

This reverts commit 424f6c4.

Signed-off-by: popcornmix <popcornmix@gmail.com>

popcornmix added a commit that referenced this issue Sep 17, 2018

Revert "mm: alloc_contig: re-allow CMA to compact FS pages"
The upstream commit caused poor video playback performance
on a busy system for many users.

See: #2503

This reverts commit 424f6c4.

Signed-off-by: popcornmix <popcornmix@gmail.com>

popcornmix added a commit that referenced this issue Sep 17, 2018

Revert "mm: alloc_contig: re-allow CMA to compact FS pages"
The upstream commit caused poor video playback performance
on a busy system for many users.

See: #2503

This reverts commit 424f6c4.

Signed-off-by: popcornmix <popcornmix@gmail.com>

popcornmix added a commit that referenced this issue Sep 25, 2018

Revert "mm: alloc_contig: re-allow CMA to compact FS pages"
The upstream commit caused poor video playback performance
on a busy system for many users.

See: #2503

This reverts commit 424f6c4.

Signed-off-by: popcornmix <popcornmix@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment