3D strength and convergence suggester tool on iw3 desktop

[Isobel999]

Hello I'm quite new to iw3 as my friends recently taught me how to use it, however I have technical difficulties in figuring out the best 3D strength and/or convergence and I was wondering if it's possible to add a tool on iw3 desktop that can analyze and suggest the best strength and convergence that the eyes can fuse the most easily while not being flat.

I hope I explained it well and apologize if this has been asked before.

Really appreciate this software. Thank you very much. :)

[nagadomi]

The 3D appearance varies depending on the viewer settings, such as screen distance and IPD.
In iw3 desktop, you can adjust parameters like strength and convergence in real time by hovering the mouse cursor over the setting and scrolling.

[Isobel999]

Thank you for the reply, I do understand that but what I meant to say is if there was a feature that can automatically detect the most suitable strength and convergence combination based on the film or picture.

For example because of ghosting the left image is uncomfortable for the eyes to fuse well when the image on the right is the kind of image you would get when you use the correct strength and convergence combination.

It takes me quite a while of trial and error and even in some cases I couldn't find the right settings on my own for some films. What I notice is that films that use the same cinematography or cinematographer almost always end up with the same strength and/or convergence.

I'm asking if there could be a feature that can be used to suggest the best 3D strength and/or convergence that could make viewing on a 3D projector that most comfortable and the eyes can fuse easily.

I understand if it's not technically possible but it's a feature that could help me and quite a few people I know at least. :)

[Roo515151]

It's probably not the same situation you have but I just wanted to point out I had left eye ghosting on my 3dtv when converting videos with IW3 which after a lot of trial and error ultimately turned out to be simply the angle of the screen.

when I tilted it slightly upwards instead of slightly downwards the ghosting was gone. (I wasted a lot of time on setting changes before I realised)

[Isobel999]

With the projector screen unlike the tv or VR it's not possible to tilt anything or adjust the 3D strength once it has already been converted. There is usually the right strength and convergence combination which when you use your left and right eye it fuses well and doesn't require much muscle power to combine. When testing convergence or strength you need human eyes to trial and error and see where the best fusion is. I was wondering if there was a feature that can help reduce this and where I can choose the best setting after getting the suggested divergence and convergence.

[nagadomi]

If that is an issue specific to the projector, I don't have one, so I'm not familiar with the details.
For a projector, the parameters maybe the screen size and your interpupillary distance (IPD).
The physical horizontal displacement on the screen should be smaller than your IPD.
Since the average human IPD is about 6.5 cm, exceeding that would require the eyes to diverge outward, which is generally difficult and causes fatigue.
To keep it within 6.5 cm, reduce the screen size or lower the 3D strength.

[Isobel999]

Thank you for the reply, it's actually something that's an issue with not just the projector, but also with VR and 3D TVs, having a feature that can help suggest the best parallax (divergence/convergence) would be helpful to anyone using the software. Although I'm not sure it can be done, it would be a huge benefit to everyone I'm sure.

[nagadomi]

Then it's the same as #555.
The metric depth is the result of that analysis, but the current iw3 discards that information.

[diverswan9]

Whenever I'm planning on doing a whole film I run a quick test or two first. You could make a test clip from the source looking for three types of scenes... a dark scene, a bright scene, and an action scene... like one minute per type. I use either Avidemux or LosslessCut to make a 3 minute test clip. Then when I've found settings that work for that film I run the whole film with those.

[Isobel999]

Thank you for the reply, I do run tests but my question is if it's possible to get a feature which would kind of bypass that, and suggest the best settings for a specific film, it would be extremely helpful for users like me, however I am not aware if it's technically possible. :)

[JackSkellington90]

I also mentioned this not too long ago coincidentally, I find it difficult to find strength and convergence. But i can recommend using subtitles to see the best convergence, once the images look fused it means you found the most comfortable convergence, then you can try to find the best 3D strength. Also I do this with constant convergence and once I find the right 3D strength and convergence I add +1.00 strength and turn on auto convergence, I'm not sure how or why it works but it always does. Lol

But yes I can understand how difficult it is to find the right strength in some films, I'm focusing more on converting tv series now, since usually the 3D strength and convergence would work for all the episodes.

Hope this helps

[nagadomi]

The appearance of stereo images in VR is more complex than many people think, because human perception plays a major role.

Subtitle behavior is a good example.
When subtitles are displayed in 3D space separately from the stereo image, it becomes difficult to focus on both the screen and the subtitles at the same time, and during zoom operations, the image and subtitles may appear to move with different sensitivities. This is because the 3D subtitles appear at the exact position and size where they are rendered, whereas the stereo image does not. For example, a screen rendered at a distance of 2 meters may actually appear to be 4 meters away. Because it involves not only the screen position but also the disparity.　

Many of the problems attributed to 3D Strength or convergence actually result from the interaction between those parameters and the video player settings.
I believe that many of these issues can be solved solely through the implementation or settings of the video player.
I plan to add a fully controllable stereo video and image player to this repository soon, so please try that first.

[JackSkellington90]

This is an interesting observation I can share.

So these are all 3 stereoscopic images (I used potplayer anaglyph Dubois, because it shows the disparity more clearly) basically if there's a lot of red fringing it's a bad sign. What I notice is that for this specific film at 0.8 convergence there is the least amount of red hues and usually that means this is the most comfortable of the 3, notice there's a certain degree of redness in all 3 but 0.8C has the least. I also notice this differs based on the film. (Note: All films here are the same strength)

This judging from the still has the most comfortable convergence: (0.8c)

I was thinking perhaps the iw3 viewer could be a great idea if we could change the hue to display stereoscopic anaglyph Dubois (like in the stills I posted) so we could judge disparity and convergence before running tests.

It's probably the most helpful in that case or it could even have like a "heat map" that can hint on whether you are closer or further away from the most comfortable convergence/strength. Perhaps this might be a good solution.

[nagadomi]

There are two types of subtitles: hard subtitles, where the subtitles are included in the video as image data, and soft subtitles, where they are stored as subtitle stream.
That example is a hard subtitle case, where the disparity simply becomes zero when the subtitle depth and convergence align.

The iw3-player (tentative name) can use soft subtitle stream data to display subtitles at positions separate from the video screen, allowing full control over subtitle distance, size, and position.
However, if the subtitles are hard-coded into the video, they cannot be controlled.

[nagadomi]

It might be off-topic, but I think a tool that converts hard subtitles to soft subtitles would be very useful — though it might already exist. Such a tool would cut out the subtitle area from the video (possibly along with the black bars), run OCR on it to extract the text, and then convert it into soft subtitle data.

[JackSkellington90]

May I ask is it possible to judge disparity based on the hard subtitles of a film ? In my case I find it useful to pinpoint the best and most comfortable convergence in a film.

[nagadomi]

However, if the subtitles are hard-coded into the video, they cannot be controlled.

This was incorrect. Eye separation (IPD) can be adjusted in the player. However, the correct value still varies by scene.

May I ask is it possible to judge disparity based on the hard subtitles of a film ? In my case I find it useful to pinpoint the best and most comfortable convergence in a film.

What you want is AutoConvergence feature for subtitles.
The current AutoConvergence focuses on salient objects such as people, but it can simply use a model that targets subtitles (text characters) instead. However, I am somewhat doubtful about how useful that would actually be, since it means prioritizing subtitles over the visuals. Using soft subtitles is definitely the correct direction, though I understand that the conversion can be a practical issue.

[nagadomi]

A simple manual way to achieve it using ffmpeg and iw3:

1. Remove the black bars and subtitles from the video.
2. Convert only the video part to SBS using iw3.
3. Take the cropped video that includes the subtitles and black bars, place two copies side by side, and encode it as a video.
4. Combine the results from steps 2 and 3 vertically.

This way, you can create an SBS video where the subtitles have zero parallax.

[JackSkellington90]

The subtitles isn't the issue, it's more of a way for me to judge if I'm using the right convergence plane before testing it if that makes sense.

[nagadomi]

If subtitles are not the real issue, then judging convergence based on subtitles might be a mistake or a misunderstanding.

[JackSkellington90]

Thank you for clarifying, perhaps if we had an anaglyph viewer it might help testing.

[nagadomi]

In fact, assuming that subtitles are not the issue is probably the real misunderstanding. People who read subtitles must constantly keep their focus on them, so if the depth of the subtitles shifts from scene to scene, it would cause significant eye strain. The idea of fixing the subtitles at the zero parallax point makes perfect sense.

[JackSkellington90]

I thought so 😅 basically it would make sense because some films I end up with 0.6C others I end up with 0.8c, whatever I find alignment with the subtitles usually works and then from there I fix the strength.

[Roo515151]

Just in case anyone come across this just for the subtitles mentions.

The program "subtitle edit" can export subtitles from an srt file to vobsub/idx with 3d settings (and you can control the convergence as part of it ie so the words aren't behind a foreground object). I use this regularly so can vouch for it. I simply open the video created with IW3 in vlc player, right click and add the subtitles file I exported from SE.

[Isobel999]

Thank you for the replies, I think it does make sense to use subtitles as an anchor to see the best convergence as mentioned, I would also be interested to know if there's any other hacks to figure out when the two images have fused without relying on the tests. If there was maybe a way to view the SBS in a fused image and study it before conversion it could be useful. Maybe at some point we could have a feature that kind of suggests to increase/ reduce the strength or convergence to get the most comfortable viewing experience. This feature could help make iw3 a lot more accessible to people who don't have the technical skills or knowledge but still have the enthusiasm for 3D conversion. :)

[JackSkellington90]

I find this book to be very helpful, it also mentioned that 3D disparity can be calculated and that film makers usually have a way to calculate it. Perhaps this could be used for a 3D strength analyzer software.

[Isobel999]

This is a bit technical but at the same time actually easy to follow. I have tried using the method you mentioned to find the best convergence and I actually kind of get it, the subtitles align when you find the right convergence and 3D strength. 😄

It's still a long process at least for me, if you don't mind could you please share the title of the book Maybe it can help me understand better.

[nagadomi]

I do not think there is anything I can do about this, so I will move it to a discussion.