Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

1080i deinterlace issue #70

Open
Gobam opened this issue Jun 5, 2014 · 111 comments
Open

1080i deinterlace issue #70

Gobam opened this issue Jun 5, 2014 · 111 comments

Comments

@Gobam
Copy link

@Gobam Gobam commented Jun 5, 2014

Interlaced HD videos (1080i) are not properly processed using the existing deinterlace options in XBMC GUI. All the existing options are causing high frame drops/skips. Same videos are working properly in other platofrms (Raspberry Pi, ION). Thanks!

@wolfgar

This comment has been minimized.

Copy link
Member

@wolfgar wolfgar commented Jun 5, 2014

Hi,
You are right,
Thanks for opening this ticket so that we will properly track this remaining issue...

@Ryo99

This comment has been minimized.

Copy link

@Ryo99 Ryo99 commented Jun 17, 2014

Do this commits to master-pr fix this issue?
3862644
e384ec1
5daa4b1
091b6e8

@smallint

This comment has been minimized.

Copy link
Member

@smallint smallint commented Jun 17, 2014

No. Deinterlacing of HD streams suffers from either limited memory bandwidth or bad implementation. I have now my Cubox with an up-to-date kernel up and running and will continue to work on this in future. If the latter is the case I have already some initial work in place towards an additional mixer thread that could help. In case of hardware limitations (VPU/IPU/GPU utilization) we have to test other options. Stephan already proposed some kernel tweaks but all this needs time and proper testing.

@susisstrolch

This comment has been minimized.

Copy link

@susisstrolch susisstrolch commented Jul 10, 2014

@smallint - can you state more precisely what you mean by "up-to-date kernel"?

@piotrasd

This comment has been minimized.

Copy link

@piotrasd piotrasd commented Aug 1, 2014

any progress with deinterlace issue ? for now i think this is the biggest problem :(

@jillesv

This comment has been minimized.

Copy link

@jillesv jillesv commented Aug 11, 2014

It is like there is no interest in solving this issue. For me it is also very important that 1080i Live TV playback is working. That is why I bought this device in the first place. My raspberry was not powerfull enough to play the 1080i stream. I thought this device was powerfull enough, but now there are other issues that prevent to watch Live-TV. This issue is also not on the known issue list for Openelec and Geexbox... I don't understand why... I think it is really important.

@koying

This comment has been minimized.

Copy link

@koying koying commented Aug 11, 2014

@jillesv Your priorities might not be the priorities of everybody...
I personally don't care for instance, and the devs who care may have more important issues at hand.

@rabeeh

This comment has been minimized.

Copy link

@rabeeh rabeeh commented Aug 11, 2014

Maybe someone can shed some light on where the bottlenecks are?
Previously smallint mentioned it either suffers from memory bandwidth or bad implementation; anyone can provide more details?
jillesv - for completeness; i suggest you provide a link for a test 1080i content to benchmark against.

@jillesv

This comment has been minimized.

Copy link

@jillesv jillesv commented Aug 11, 2014

@wolfgar

This comment has been minimized.

Copy link
Member

@wolfgar wolfgar commented Aug 11, 2014

Hi there,
@jillesv : first you are right, the issue is annoying for liveTV use case and is definitively real.
This thread is about it, so there is no attempt to hide it. But maybe distro should stress it and link to this place...
I understand that you might be frustrated and would like that this issue is already solved but you also have to understand that people who contribute to the development are not the vendors of imx6 devices and do it for free during their free time. So things are sometimes slow to happen
@rabeeh, in fact issue is that
decoding a frame + deinterlacing it (using IPU which has to split it in several blocks because of the vdic resolution limitation) + rendering is sometimes longer than a frame duration hence the jerky effect.
We are not very far from getting it to work smoothly so before changing significantly the current implementation we wanted to check whether tweaking AXI priority (indeed the deinterlacing requires additional memory bandwidth) can do the trick.
If it is not the case, then we would have to change the implementation : Today deinterlacing is performed in the rendering thread and delays the rendering of the frame.
We could try to improve it by having a scheme where, in nominal case, during a frame duration :
frame n+2 is decoded
frame n+1 is deinterlaced
frame n is rendered
Of course at the end we still have to do the 3actions during a frame duration but here we could deinterlace and render at the same time (to the extend of common hw resources of course) while in the current way of working, they are purely sequential operations. So conceptually it could work, but implementing requires a good synchronization and can be tricky...

@jillesv

This comment has been minimized.

Copy link

@jillesv jillesv commented Aug 11, 2014

Wolfgar,
Thank you for your explanation!

@piotrasd

This comment has been minimized.

Copy link

@piotrasd piotrasd commented Aug 16, 2014

so we waiting for fix :) is good to know there is hope :)

anyway i was testing last time - latest build and deinterlace for SD also i no perfect,
picture is not smooth like without deinterlace, when you watch you can feel in like little drop frames or jerky

@wolfgar

This comment has been minimized.

Copy link
Member

@wolfgar wolfgar commented Aug 16, 2014

yes there is hope for sure ! ;)
For SD, in fact there are 2 different algorithms : Try to select one instead of automatic, you may have better behavior..
Also what is your board ?

@piotrasd

This comment has been minimized.

Copy link

@piotrasd piotrasd commented Aug 16, 2014

MatrixTV v2.1 and i was used latest MatrixTV os v1.0.0.9 and also latest openelec.
(and compiled myself openelc with latets xbmc imx6 master branch)
About algorithms you mean half and full deinterlace from video settings in xbmc ? i was try both.
efect this same ....jerky picture is maybe to big word, but this is special visible when example some text
scrolling,example: some end subtitles of movies or some info bar down of picures on some news program. Without deinterlace eveything is smooth, i can record some samples for for test.

@wolfgar

This comment has been minimized.

Copy link
Member

@wolfgar wolfgar commented Aug 16, 2014

Yes I alluded to half an full deinterlace from video settings.
In fact these are xbmc naming but these 2 algos are wired to 2 different configuration of the vdic motion engine. That's why I suggested to try both : one should behave better for fast motion scenes...
OK we will try to improve this

@wolfgar

This comment has been minimized.

Copy link
Member

@wolfgar wolfgar commented Aug 18, 2014

@piotrasd : you should have received a mail from me

@smallint

This comment has been minimized.

Copy link
Member

@smallint smallint commented Aug 24, 2014

Sorry for my long absence. I am currently setting up a build environment on my cubox with Archlinux and xbmc compiled already. I am going to look into the deinterlacing issue and will try different methods to test and improve performance. This is scheduled for Monday.
@wolfgar: Should I fork master or is there a better branch to use in terms of the upcoming PR? I think we will need to include new enumerations for deinterlacing with better string representations. The more easy possible merges later the better.

@wolfgar

This comment has been minimized.

Copy link
Member

@wolfgar wolfgar commented Aug 24, 2014

@smallint : Nice to see you back !
As you have seen I have removed the deinterlacing support from initial PR in order to add proper imx dedicated values (in the enum) and I though it would be the opportunity to work on this remaining issue before pushing it upstream...
If you want to have a look, you are very welcome of course
I would advise to start from imx-pr branch as it is the candidate from the upstream PR...

@smallint

This comment has been minimized.

Copy link
Member

@smallint smallint commented Aug 27, 2014

I have tested deinterlacing with a dedicated mixer thread that does deinterlacing in parallel to decoding and rendering. The results are not very promising but I will investigate further. The synchronization itself is quite tricky and once this is stable I will do more tests on performance. What I can say is that the performance with my old kernel (3.0.xx) on the Wandboard and the Yocto build was much faster. Not sure if the kernel frequency setting did the trick before.
The hardest part is to do proper logging of all threads to analyze possible bottlenecks. Stay tuned ...

@wolfgar

This comment has been minimized.

Copy link
Member

@wolfgar wolfgar commented Aug 27, 2014

HI Jan,
Yes this way of working was expected to be tricky : Especially if we come to wait for deinterlacing to submit a new picture then the whole way of working is useless

Regarding your performance, I am unsure about the root cause :
Maybe the kernel config CONFIG_MX6_VPU_352M ?
or
maybe there are deeper changes in IPU driver for new kernels (I have not checked, I had a good understanding of the 3.0.35 driver because I had to look at it to remove the flickering lines but I don't know if major changes were introduced in the >3.10 kernels)

Take your time, as I know you are working on it there is no risk of duplicate work and do not hesitate to tell me if I can help in any way...

@smallint

This comment has been minimized.

Copy link
Member

@smallint smallint commented Aug 27, 2014

Currently I still need to figure out how exactly the sychronization performs. I am going to build a dummy deinterlacer (e.g. sleep(30ms)) and check if the playback is still smooth. If so I can assume that both pipelines are really parallel and do not block each other, the output should be in the interval of the maximum processing time (VPU or IPU). Once I introduce the real IPU and the performance drops dramatically it is likely that we are dealing with hardware limitations.

Take my word that I am continuously working on that during the next weeks and will let you know how things are going.

@cmichal2

This comment has been minimized.

Copy link

@cmichal2 cmichal2 commented Sep 2, 2014

I recompiled the Geexbox kernel with CONFIG_MX6_VPU_352M, and it makes a big improvement. It doesn't completely fix the issue, but obvious glitches in 1080i videos are much less frequent than without it.

@piotrasd

This comment has been minimized.

Copy link

@piotrasd piotrasd commented Sep 2, 2014

maybe kernel 3.14 resolve some problems

@wolfgar

This comment has been minimized.

Copy link
Member

@wolfgar wolfgar commented Sep 2, 2014

@cmichal2 : Thanks for your feedback : Which board and which kernel do you use for your test @352M ?

@cmichal2

This comment has been minimized.

Copy link

@cmichal2 cmichal2 commented Sep 2, 2014

I'm using a Cubox-i4. The kernel is 3.10.30 from geexbox-devel

@wolfgar

This comment has been minimized.

Copy link
Member

@wolfgar wolfgar commented Sep 2, 2014

Thanks a lot, I asked because contrary to the 3.0.35 version, the 3.10 freescale kernel seemed to ignore this configuration option...
Thanks for your answer, I will have a deeper look at the 3.10.30 kernel used by geexbox for cuboxi...

@cmichal2

This comment has been minimized.

Copy link

@cmichal2 cmichal2 commented Sep 2, 2014

You know, I went grepping through the source, and I couldn't find anywhere that that configuration option actually affected anything. But honestly, it does seem to make a big difference.

@wolfgar

This comment has been minimized.

Copy link
Member

@wolfgar wolfgar commented Sep 2, 2014

hehe the same from my previous look at the source (while in 3.0.35, this config was clearly taken into account through code)
For 3.10, maybe there is something hidden/correlated with dts when we select this option, I will check

Anyway, many thanks for confirming that it has the expected effect ! (even if not yet enough any improvement is good at this stage...)

@cmichal2

This comment has been minimized.

Copy link

@cmichal2 cmichal2 commented Sep 3, 2014

I'll be interested to hear if you can find any effect of that config option. I'm a little unsure now of what's going on. After xbmc (and tvheadend) have been running for a day the glitches seem to be more frequent again. I tried each kernel, and immediately after booting it does seem as though with the clock speed option it is better, but I'm not that confident. Could there be a memory management aspect to this?

@smallint

This comment has been minimized.

Copy link
Member

@smallint smallint commented Sep 3, 2014

Funny, while I am working currently on the deinterlacing issue I came to the conclusion that the IMX6 is not powerful enough with current kernel configuration (3.10.30, ArchLinux) to run VPU, IPU and GPU at the same time with 1080i. This is either related to limited memory bandwidth or that the IPU is stalling the VPU. Thats why I wanted to apply the CONFIG_MX6_VPU_352M option as well. But checking the sources I could not find any place where this option is used at all. So from my understanding this option cannot change anything unless there is some real magic done.

I am about to publish some numbers soon. What I would like to have is a small performance test tool built with the XBMC libraries to decode and process a stream using the IMX6 codec. Does there anything exist somewhere?

@rabeeh

This comment has been minimized.

Copy link

@rabeeh rabeeh commented Nov 30, 2014

@wolfgar / @smallint - will this be helpful -
http://pastebin.com/aPnfQ1Ed
?

Any one has content to test with?

@smallint

This comment has been minimized.

Copy link
Member

@smallint smallint commented Nov 30, 2014

@rabeeh This is @wolfgars patch for his Utilite kernel. Where did you get it from and why didn't you post a Github link but pastebin?

@Gobam

This comment has been minimized.

Copy link
Author

@Gobam Gobam commented Feb 13, 2015

Hi,
This is just to confirm if there is any advance in this issue. I installed latest OpenELEC version (5.0.1) but it still has problems with 1080i. Thanks!

@wolfgar

This comment has been minimized.

Copy link
Member

@wolfgar wolfgar commented Feb 13, 2015

The following PR fixes all "normal" 1080i contents : xbmc#6351

@wolfgar

This comment has been minimized.

Copy link
Member

@wolfgar wolfgar commented Feb 13, 2015

As a remembering, all development happens upstream now...

@fritsch

This comment has been minimized.

Copy link

@fritsch fritsch commented Feb 13, 2015

@Gobam 5.0.1 did not have this rework at all ... 5.0.2 has -> still this is not a user forum

@smallint

This comment has been minimized.

Copy link
Member

@smallint smallint commented Feb 16, 2015

We should close this issue, shouldn't we?

@Gobam

This comment has been minimized.

Copy link
Author

@Gobam Gobam commented Feb 16, 2015

Hi,

Not sure if I am doing something wrong, but in my case 1080i deinterlaced is still not working. I am testing in a Cubox-i4P with OpelenELEC 5.0.2, which should include the fixes. I am attaching two small samples of 1080i video I am using:

https://db.tt/hkdjYEjO : Captured from 1080i source using Hauppauge HD-PVR2
https://db.tt/Zqs3DnjG: Captured from DVB-T2 TV channel using Tvheadend

In both cases video is really choppy, it runs like in slow motion, and audio is running at normal speed, so it's not syncronized with the video. This is happening also with LiveTV, which is a critical usage scenario in my case. Both samples (and same LiveTV channels) are working flawless in both Raspberry PI B+ and Intel-ION systems, both using the same OpenELEC version (5.0.2)

Please let me know if there is another Kodi version I can test with. Thanks!

@piotrasd

This comment has been minimized.

Copy link

@piotrasd piotrasd commented Feb 16, 2015

im not sure if guys from OE apply this to build which is needed for good working deinterlace:

Recommendations for client setup:

enable VPU@352Mhz in kernel
setup 32 bit framebuffer in uEnv.txt (bpp=32)
disable console blanking in uEnv.txt (consoleblank=0)

@Gobam

This comment has been minimized.

Copy link
Author

@Gobam Gobam commented Feb 16, 2015

I can confirm that uEnv.txt parameters are correct:

mmcargs=setenv bootargs 'boot=/dev/mmcblk0p1 disk=/dev/mmcblk0p2 quiet video=mxcfb0:dev=hdmi,1920x1080M@60,if=RGB24,bpp=32 dmfc=3 consoleblank=0'

How can be confirmed that VPU@352Mhz is enabled in the kernel? Thanks.

@piotrasd

This comment has been minimized.

Copy link

@piotrasd piotrasd commented Feb 16, 2015

no i was checked
from OE - imx6 project config kernel:

MXC VPU(Video Processing Unit) support
CONFIG_MXC_VPU=y
CONFIG_MXC_VPU_DEBUG is not set
CONFIG_MX6_VPU_352M is not set

@susisstrolch

This comment has been minimized.

Copy link

@susisstrolch susisstrolch commented Feb 16, 2015

CONFIG_MX6_VPU_352M is a relict from older kernels.
There's no bit of code ifdef'ed by CONFIG_MX6_VPU_352M in imx kernel 3.14:
strolch@strolch:~/Development/OpenELEC/build.OpenELEC-imx6.arm-devel/linux-cuboxi-3.14-1f8e124>
grep -r "CONFIG_MX6_VPU_352M" .
./.config:# CONFIG_MX6_VPU_352M is not set
strolch@strolch:

On 02/16/2015 03:14 PM, piotrasd wrote:

no i was checked
from OE - imx6 project config kernel:

MXC VPU(Video Processing Unit) support

CONFIG_MXC_VPU=y

CONFIG_MXC_VPU_DEBUG is not set

CONFIG_MX6_VPU_352M is not set


Reply to this email directly or view it on GitHub
#70 (comment).

@smallint

This comment has been minimized.

Copy link
Member

@smallint smallint commented Feb 16, 2015

How can be confirmed that VPU@352Mhz is enabled in the kernel? Thanks.

dmesg | grep VPU
@smallint

This comment has been minimized.

Copy link
Member

@smallint smallint commented Feb 16, 2015

CONFIG_MX6_VPU_352M is a relict from older kernels.

There are patches to make that work again with >= kernel 3.10. I have my VPU running at 352 Mhz with kernel 3.14.

@susisstrolch

This comment has been minimized.

Copy link

@susisstrolch susisstrolch commented Feb 16, 2015

Do you have a patchset against current kernel?

On 02/16/2015 03:30 PM, smallint wrote:

CONFIG_MX6_VPU_352M is a relict from older kernels.

There are patches to make that work again with >= kernel 3.10. I have
my VPU running at 352 Mhz with kernel 3.14.


Reply to this email directly or view it on GitHub
#70 (comment).

@smallint

This comment has been minimized.

Copy link
Member

@smallint smallint commented Feb 16, 2015

Check xbmc#5805, there is a post from @cmichal2 from Nov 30, 2014 linking to all required patches.

@fritsch

This comment has been minimized.

Copy link

@fritsch fritsch commented Feb 16, 2015

Deinterlacing of 1080i50 content is fine for me, without that 352 overclock and overvoltage hack. I really don't like to force that frequency fixed and don't tell users, cause that might harm and if something gets broken, all start to whine.

@rabeeh did some rework with cpufreq - I think this dynamic higher clocking has more potential and is additionally more safe.

@rabeeh what do you suggest from a hw point of view? Clock to 352 Mhz fixed - what are the dangers when doing so? Which patches should distributions like OpenELEC ship to not put the hw at risk?

OpenELEC 5.0.2 ships the latest code done by @smallint but missing the RenderCapture code @wolfgar finished some days ago.

@piotrasd you are always the first whiner in such threads - why don't you just provide some builds for testing with the 352 Mhz patches in, the RenderCapture work and so on? It seems you have too much time.

@piotrasd

This comment has been minimized.

Copy link

@piotrasd piotrasd commented Feb 16, 2015

@fritsch sorry i just checked recomended stuff for client setup which was in PR and only show difrents from openelec confs
im not compiled to test, because i dont have time ;)

@smallint

This comment has been minimized.

Copy link
Member

@smallint smallint commented Feb 16, 2015

@fritsch, does OpenELEC 5.0.2 work for your streams as good as the test build with Kodi master you did?

@fritsch

This comment has been minimized.

Copy link

@fritsch fritsch commented Feb 16, 2015

@smallint: It works the same as kodi master. I only ported the patches that were needed to make it work correctly, see: https://github.com/fritsch/xbmc/commits/imx-rebase-pr

It was integrated into OpenELEC master already: https://github.com/OpenELEC/OpenELEC.tv/tree/master/projects/imx6/patches/kodi

5.0.x does not have this code.

@piotrasd

This comment has been minimized.

Copy link

@piotrasd piotrasd commented Feb 16, 2015

@fritsch big thanks and sorry i caused frustrations in you ;(

@wolfgar

This comment has been minimized.

Copy link
Member

@wolfgar wolfgar commented Feb 16, 2015

I have to say I have run several imx6Q based product (utilite, cuboxi, tbs2910) with VPU@352Mhz for several months without major troubles (it was my own default 3.0.35 kernel config and has become mine with the mentioned patches for 3.14)
Also official freescale support states "There should be no significant side-effect for this config, but it may bring a little higher power consumption." (https://community.freescale.com/message/338488#338488)
So I am not so reluctant to use a VPU @352Mhz as it opens the door to streams that cannot be played otherwise (carmen@60p sample for instance)
But it is a personal choice and I perfectly understand that a distro maintainer has concerns because he will be assaulted by users in case an issue occurs...

@fritsch

This comment has been minimized.

Copy link

@fritsch fritsch commented Feb 16, 2015

@wolfgar: I trust you there fully. But I still wait for @rabeeh comment on this - to not put our users at risk.

I expect more "official" support in all those regards. Can't be that you and @smallint invest day and night to get things going ... Cubox as is is on the crossroad currently ...

@wolfgar

This comment has been minimized.

Copy link
Member

@wolfgar wolfgar commented Feb 16, 2015

Of course, no problem,
As I said I perfectly understand this approach
Rabeeh's advice as cuboxi hw designer is clearly relevant...
I only wanted to tell about my own experience (and I might be lucky lol), and about the official freescale post that states there is no risk but to consume more power...

I leave this thread open so that @rabeeh can come and express his point of view regarding VPU frequency for cuboxi products but as soon as this exchange is finished I will close it and link to the current xbmc#6351 that fixes this issue...

@fritsch

This comment has been minimized.

Copy link

@fritsch fritsch commented Feb 16, 2015

Here is a build with the 352Mhz patch and the rendercapture code include and diverse fixups: https://dl.dropboxusercontent.com/u/55728161/OpenELEC-imx6.arm-devel-20150216220652-r20288-g20bad1e.tar

Everything is build from: https://github.com/fritsch/OpenELEC.tv/commits/imx-master

Edit: Handle with care - I don't want to take any guarantees with the 352 Mhz fix.
Edit2: Here is another version with only the 352 Mhz fix, but the RenderCapture (g2d) reverted: https://dl.dropboxusercontent.com/u/55728161/OpenELEC-imx6.arm-devel-20150216223931-r20290-gd87c335.tar

@cmichal2

This comment has been minimized.

Copy link

@cmichal2 cmichal2 commented Feb 17, 2015

I'll be interested too in hearing what rabeeh has to say about the VPU frequency, but to add a few bits more data: I've been running a cubox-i4 with the vpu frequency at 352MHz for ~ 6 months with no issues. I also did some measurements of power consumption and found only a tiny difference with and without the patch (details in xbmc#5805).

@rabeeh

This comment has been minimized.

Copy link

@rabeeh rabeeh commented Feb 17, 2015

As @wolfgar mentioned; vpu@352mhz can run without issues at all. The only bad impact it has is that it removes the lower processor frequency from the frequency scaling list; so when the processors are at idle they can't go to it's lower frequency state.
With that saying jnettlet had a patch to fix that too. I'll ping him to see what is the status of that patch too.

@fritsch

This comment has been minimized.

Copy link

@fritsch fritsch commented Feb 17, 2015

@rabeeh Thanks much - if you can link that patch here, that would be great and we would pick it for OpenELEC
@sraue ping

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
You can’t perform that action at this time.