Stereoscopic 3d rendering support

[elupus]

This series of commits add stereoscopic rendering support to xbmc. Note this need some squashing love before merge and some changes to GUI logic i think.

It would also be nice to have some comments on how the SBS and TAB logic is handled inside graphiccontext. It get's somewhat messy. If you have some nicer alternatives i'm all ears. Pushing it into windowing is best bet i think, with modification of our resolution structures. but that isn't very clean either.

Also, ffmpeg currently doesn't support bluray style 3d, so you need converted side by side or top bottom style video's.

In principal this could go in this merge window. It's working quite well.

[elupus]

Okey.. i rebased this and restored the dialog box with just some alterations in strings. Seems all the other comments ended up lost.

[da-anda]

hmm, after thinking about the mono/2D support some more, I think we have to remove it from stereo modes and make it a setting, otherwise we can't convert a stereoscopic video to 2D in case we couldn't detect the videos stereo format by filename or stream meta (like on internet streams/video on demand). What do you think? Downside - we couldn't cycle to this "stereo mode" anymore unless we add tons of additional code

[elupus]

I don't understand. If we can't detect the format of the video we are
screwed. We need to know source format or have a old setting for that.
Otherwise we will not select right part of the video.

[da-anda]

@elupus - in case we can't detect we have to trust/use selected GUI mode - and if mono is a GUI mode this obviously won't work.

[da-anda]

ok, forget it - using GUI mode is not an option in our case, because we don't only have OU and TAB gui modes but also others that require video conversion (anaglyph, interlaced) - so we would need a way to define the steremode of the video source. I could easily add a toggle to GUI in case we couldn't detect, something like "define stereo mode of video", or we show a popup/select dialog in case user switches to 3D mode and we don't know the source format. But this would require that we can somehow update the VideoRenderers configuration - doesn't it? We would also have to store that information along with the video so that user doesn't have to select it every time he plays the video. Sounds a little like a future feature then, because I don't think we find a clean solution for this today (would like to get this baby merged this merge window)

[elupus]

Well this window would be today. So in that case you need to get more eyes
on it. Like jmarshall since it touches gui code and also you need to fix
rbpi since I'm certain it breaks that.

[da-anda]

@jmarshallnz mind having a look? Would be great if this could make it
@popcornmix could you test this on the pi please?

[Memphiz]

@da-anda - file a merge window extension request for this PR in the internals for not hitting the merge button in hurry ...

[popcornmix]

The Pi had support for a 3D split GUI before this patch.
It used a different scheme of claiming the monitor was half-width or half-height, and then duplicating the GUI.

I'm assuming we want the Pi to follow this scheme and render both halves of the GUI (it is more overhead, but should be cleaner and allows 3D GUI effects, so I'm not against it).

After this PR, there's still a fair bit of code around dealing with half height/half width screen resolutions. Is that just leftovers from the Pi implementation that needs removing? E.g. in guilib/GraphicsContext.cpp

  case RES_HDTV_1080pSBS:
    CDisplaySettings::Get().GetResolutionInfo(res).iSubtitles = (int)(0.965 * 1080);
    CDisplaySettings::Get().GetResolutionInfo(res).iWidth = 960;
    CDisplaySettings::Get().GetResolutionInfo(res).iHeight = 1080;
    CDisplaySettings::Get().GetResolutionInfo(res).dwFlags = D3DPRESENTFLAG_PROGRESSIVE | D3DPRESENTFLAG_WIDESCREEN | D3DPRESENTFLAG_MODE3DSBS;
    CDisplaySettings::Get().GetResolutionInfo(res).fPixelRatio = 1.0f;
    CDisplaySettings::Get().GetResolutionInfo(res).strMode = "1080pSBS 16:9";
    break;

Is there a way of probing that SBS or TAB is supported from the display for a given resolution?
You can do this through HDMI, and it is possible that 1080p supports SBS only, and 720p supports TAB only. The Pi used to do this through half width/height resolutions, but full resolution with flags seems better.

[popcornmix]

I'd prefer OMXPlayer was fixed up like:
popcornmix@0079b91
(basically do exactly the same as dvdplayer).
OMXPlayerVideo.cpp can then use m_hints.stereo_mode to decide what to do.
I haven't done that yet, as I'm unsure if I need the current doubling of width/height.

[elupus]

Well, the pi has real 3d resolutions. With its current code, those have
modified width/height and the hw does some 2d +3d composing i think. Ie gui
only need to be rendered in half height once. So it's not clear how to
mimic that behavior.

It's fully possible we need separate modes for that.

[popcornmix]

@elupus
As long as performance is okay, I think Pi should render both halves of the GUI, as other platform do. This allows GUI to have depth, as well as making the code more common.

Because the Pi has good HDMI control, it can query what 3D modes are supported, and can switch into 3D mode automatically, so it would be good if that is supported.

The RESOLUTION_INFO structure already has flags in for indicating 3D support (e.g. D3DPRESENTFLAG_MODE3DSBS and D3DPRESENTFLAG_MODE3DTB) which we populate. We currently also halve the width or height for these modes, but I think it would be better not to do that.

SetNativeResolution will auto switch 3DTAB or 3DSBS when those flags are present.

[popcornmix]

I can see one problem on the Pi. SetVideoRect is being called to switch video window to left then right side every frame. This makes no sense on a bypass renderer and results in a flickery mess.

[popcornmix]

On Pi, system Info/summary shows 70fps when running normally and 42fps when in side by side mode.
"over/under" doesn't display an image at all.

https://github.com/popcornmix/xbmc/tree/steroscopic
contains changes to remove Pi specific 3d code, and use this PR.
Currently automatic switching TV into 3D mode is not supported, and over/under is broken.

[elupus]

@popcornmix do you know why over/under is broken?

Also that extra stuff should be RBPI only yes. No other system have supported this before.

[popcornmix]

No. GUI goes completely black in over/under mode when selected from settings.

When over/under is switched to on playing a video, the video plays correctly, but overlay is invisible.
There is a small dimmed (alpha) area at bottom of UI (full width, ~1/16 of height) when I bring up info (i key) playing a video. That dimmed area is not replicated in top half as you might expect.

I get the same in 1280x720 resolution. (I wondered if the 1280x720 GUI with 1920x1080 display was casuing the problem, but it seems not).
I also get the same in VGA resolution. (I wondered if it was somehow memory or CPU related).

[afedchin]

@elupus, at 8b69909 RenderFlags.cpp and RenderFlags.h are not included into the VC project, so build failed.

[baijuxavior]

Build failed for me also. 6 errors like:

Error 5 error LNK2019: unresolved external symbol "unsigned int __cdecl RenderManager::GetFlagsChromaPosition(unsigned int)" (?GetFlagsChromaPosition@RenderManager@@yaii@Z) referenced in function "protected: int __thiscall CDVDPlayerVideo::OutputPicture(struct DVDVideoPicture const *,double)" (?OutputPicture@CDVDPlayerVideo@@IAEHPBUDVDVideoPicture@@n@Z)

[baijuxavior]

OK. Added the renderflags.cpp and renderflags.h to xbmc.vcxproj and xbmc.vcxproj.filters files and build succeeded.

[baijuxavior]

Any idea why a black bar comes in the middle of movies (irrespective of o/u, sbs or 2d movies) when switched to o/u mode? See da-anda's imag above. In the original ou movie there is no black bar. This makes part of the upper subtitle hidden when I'm trying for subtitle depth.

[elupus]

You could be hitting frame packed mode. But that should affect gui too.
Otherwise we have some error in rendermanager in where things end up.

[elupus]

@baijuxavior could you get me a patch for the win changes?
@da-anda yea that selection dialog on play looks quite nice.

[da-anda]

@elupus I also noticed that when I watch a tab movie and switch to tab stereo mode, XBMC is adding an extra black bar and scaling down the video even more

[elupus]

It could have an invalid aspect set i suppose. The full hd tab movie on the
forum had that for example.

[da-anda]

Movie resolution is 1920x1072, but this also happens on 1902x1080 tab movies (tested two others) - there is always a black bar/space added in between

[baijuxavior]

I tested a movie in tab mode in my FHD TV and there was no bar. The black bar is seen in laptop screen (1366x768).

[elupus]

@jmarshallnz could you have a look at the last commit here: elupus@24dbac2

Think i got it correct, but i'm not fully sure about the multi line scrolling text. It now only updates scroll info on the first line. It used to call it on the other lines with zero pixel speed, but i don't think that matters.

[da-anda]

@elupus - what do you think is still missing here, besides of a rebase and cleanup? Personally I was not able to test interlaced mode on Linux (didn't work for me in Ubuntu 13.04 with ATI, no interlacing visible, maybe I messed up the compile) and a forum user reported that at least on his TV the eyes where flipped in interlaced mode while all other modes where just fine. Shall we drop interlaced for now and add it when it's better tested? Everything else seems fine so far, besides of the comment I wrote on 11aa3e4 (option "use movies stereo mode" should be hidden if preferred stereo mode is set to it).

The JSON stuff I'm working on (the PR in your branch/repo) can IMO be left out for the initial commit and be added in a second step.

[giaur]

@da-anda - please don't suggest to remove interlaced mode. Currently it works very well - debian/nvidia tested. Eyes was flipped, but it seems to be fixed (for over-under at least).

But even if eyes are flipped, it's not any big problem, because there is option to swap eyes, so we can get correct picture in any case. I don't know about amd cards (maybe some driver issue - amd support for linux is very poor), but on the forum some users reported that interlaced also works on intel cards.

Personally I found interlaced mode as very stable (I use it all the time) - I see no problems and no reasons to remove it.

[elupus]

well it still breaks the old code for the rbpi. I've started working on
that, but it's somewhat of a re-design of over-under. Suppose that
technically can go in later thou.

On Wed, Jul 3, 2013 at 9:43 PM, giaur notifications@github.com wrote:

@da-anda https://github.com/da-anda - please don't suggest to remove
interlaced mode. Currently it works wery well - debian/nvidia tested. Eyes
was flipped, but it seems to be fixed (for over-under at least).

But even if eyes are flipped, it's not any big problem, because there is
option to swap eyes, so we can get correct picture in any case. I don't
know about amd cards, but on the forum some users reported that interlaced
also works on intel cards.

Personally I found interlaced mode as very stable (I use it all the time)

• I see no problems and no reasons to remove it.

—
Reply to this email directly or view it on GitHubhttps://github.com//pull/2848#issuecomment-20440327
.

[popcornmix]

I think that's fine. As the comment says:
/* inform TV of any 3D settings. Note this property just applies to next hdmi mode change, so no need to call for 2D modes */

I think the "DESKTOP" bug is in CWinSystemEGL::UpdateResolutions. It only looks for matching width/height/refresh, but ignores 3D flags.

This means:
1920x1080
1920x1080 SBS
1920x1080 TAB

all match, and it just picks the last one (hence the default to TAB).

[elupus]

okey. so spotted another such place in there as well.

[popcornmix]

Better. DESKTOP mode now has correct (none) 3d flags.
I can choose from resolution list the TAB, SBS and NONE variants of each resolution, and the TV switches correctly.

If I choose non-3D resolution (e.g. 1280x720) and then choose "stereoscopic mode" SBS, the gui goes to SBS, but the TV mode is not chosen.

Also shouldn't the resolution list have the SBS/TAB variants filtered, and the "stereoscopic mode" be used for selecting 3D mode?

[elupus]

Good then it works as expected. Well some resolutions are 3d only. Like 1920x22xx frame packed resolutions (ie double resolution displasy). Those would not be possible to get to with that approach.

We could probably automatically switch to a 3d mode when going to a stereoscopic sbs/tab mode if there exist an identical 3d mode (ie same rez and rate and everything).

We can work on better selection strategy later.

[giaur]

After yesterday changes on screen osd stopped to work. When I move mouse while full screen playback, incomplete osd appears (without any buttons and icons) and disappears after 0.5 sec. So I can't even test 3d. I mean raspberry pi, not pc.

Anyone experienced the same issue?

[popcornmix]

@elupus
I've found the reason the video + OSD doesn't render properly on Pi when in 3d modes.
In CLinuxRendererGLES::RenderUpdate with RENDER_BYPASS, the glScissor calls use m_destRect which is
always 960x1080 for SBS and 1920x540 for TAB.

This means that GUI can never render into other half of screen (and in fact leaves it opaque so video is not visible either).

[elupus]

That should be corrected for using g_graphicContext.StereoCorrection()

[elupus]

@popcornmix should be fixed i think.

[popcornmix]

@elupus
Thanks. Much better.