Ensure pictures taken with CALF VR180 are supported

[atkulp]

My Wordpress site is stripping the metadata and it results in an error after parsing the EXIF and other headers:

Uncaught (in promise) TypeError: Cannot read properties of undefined (reading 'GPano')
    at parseVR (stereo-img.js:23:19)
    at async HTMLElement.parse (stereo-img.js:218:25)
    at async HTMLElement.parseImageAndInitialize3DScene (stereo-img.js:275:5)
    at async HTMLElement.init (stereo-img.js:292:5)

A simple null-coalesce should solve the problem, or just short-circuiting and going with defaults if exif is undefined.

[atkulp]

I've continued working on this a bit. I made sure the original photo with metadata was being returned and I'm still getting the error. Your demo page loads it from my local machine, but I can't get it to work on my remote image URL. Here it is in case it helps. Here's my embed if it helps:

Note that the image is 10MB. Not sure if it matters. It appears to load fine and parse ok, just not the EXIF metadata.

[steren]

Is the image served from the same origin (domain) as the page?

It is a requirement that it does. See #14

[atkulp]

Yes. It's the same domain, same site.

…

On Sun, Oct 22, 2023, 11:47 AM Steren ***@***.***> wrote: Is the image served from the same origin (domain) as the page? It is a requirement that it does. See #14 <#14> — Reply to this email directly, view it on GitHub <#20 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAYMEEV5376PRKZRC4YK2F3YAVEZJAVCNFSM6AAAAAA6KTJ3ASVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONZUGE2DCMZXGA> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[atkulp]

I put a test page up:

https://www.vrvoyaging.com/stereo-image-test/
password: stereo

I got it to render the image correctly once when I had an unclosed quote somewhere. I didn't take note of it closely enough so I'm not sure what it is.

[steren]

I see you are using an absolute URL with the domain, try using a relative URL. In my example, the URL is examples/vr180-lenovo-mirage.vr.jpg

[atkulp]

I changed it to a relative url (/wp-content/uploads/2023/10/V1000064.jpg). Same result

[steren]

Can you try with an example at https://github.com/steren/stereo-img/tree/main/examples to be sure that the issue doesn't come from your image?

[steren]

In other words: please try to troubleshoot the root cause with more testing

[atkulp]

It must be the images. These are coming straight from the new Calf VR180 camera. It might be good to see what's different about them as it's a popular camera (in VR180 circles) right now. I wonder if it's because it's expecting a single image in the main image section and the other eye in the metadata like the Lenovo was. This is L-R but also fisheye. If I switch to left-right, it shows the image, but it treats it as equirectangular instead of fisheye. Does it require new mode maybe?

I wrote a review on the camera and link to a zip file with some sample photos (it produces fisheye L-R 3D and flat SBS images in addition to video).

Article: https://www.vrvoyaging.com/calf-3d-camera-powerful-tech-at-a-mid-range-price/
Samples: https://drive.google.com/file/d/1FSjZVGfc-0Ax_yJiT2-pnuRj-nvPcRq0/view?usp=share_link

[steren]

Nice, I didn't know about the Calf VR180 camera, I might get one! (It's definitely less than going with a Canon EOS R5 + Dual Fisheye Lens)

Thanks for the samples. Are you OK if I add one of them to the examples folder?

[steren]

On the demo website, I am able to load the samples if I use the file picker.

On a local version of the demo website, I am able to load the image from the URL if I add it to examples and use <stereo-img src="examples/V1000064.jpg" type="left-right", angle="180"></stereo-img>

So the problem seems to come from how you embed these images on your website. Can you try on your website to display this image (we're sure that it works)

[steren]

Also, I am curious can you share more details how each file has been created?
For example, V1000060.jpg and V1000064.jpg do not seem to be using the same technique / format, as they don't have the same aspect ratio and one seems to be more fisheye than the other.

What mode has been used for each image? Your blog mentions VR180 3D, side-by-side (flat) 3D, single panoramic. I don't think there was a VR180 example in the zip file, correct? (VR180 pictures are supposed to show only one image when viewed in a regular viewer, since the other eye is stored in the exif)

I'd gladly add support for this new camera to this library if I have samples.

[atkulp]

Here are some additional samples right off the camera. These aren't ok to distribute, but you're welcome to use what's in the original sample zip as samples in this repo. I can provide more for that purpose if you need.

I must have been tripping. I could have sworn there was a panorama or some other picture mode, but I'm only seeing the VR180 and flattened options now.

Still images
https://drive.google.com/file/d/1B0Aeoa55B4lylHsG7Ug-4cGrS0_BUCfk/view?usp=drive_link

Videos
https://drive.google.com/file/d/1E1PFi8KbouNMI7HZOnSw1dLiBRmZrxaN/view?usp=drive_link

[steren]

Thanks.

That's interesting that VR180 images from CALF are left-right images. VR180 images produced by Google VR180 cameras or apps (Cardboard Camera, or Lenovo Mirage Camera) are storing the right eye pixels in the exif of the image, so that on a regular viewer, only the left eye is showed. This is what stereo-viewer is doing: it extracts right eye from the GImage exif.
In the case of CALF, it looks like it just stores left and right, which still works with stereo-img.

[steren]

Back to your problem: it looks like https://www.vrvoyaging.com/stereo-image-test/ doesn't serve stereo-img anymore. Is it intentional?

[atkulp]

It's still using it. Just on the first image, not on the second. If I use type="left-right" and angle="180" is that actually going to be the correct projection? There's the question of equirectangular vs fisheye.

[steren]

using <stereo-img src="examples/V1000064.jpg" type="left-right", angle="180"></stereo-img>, were you able to have the image work on your website?

I am not able to see any content on https://www.vrvoyaging.com/stereo-image-test/

[atkulp]

Sorry, I changed the visibility of it and thought you'd still see it. You can try again. Yes, left-right and 180 works, but I'm not positive the projection is right.

[steren]

Great to know. Have you figured out what the problem was?

I'll keep this issue open to track any bug with your example files

[atkulp]

Looking at the spec, I'm not actually convinced that VR180 requires using metadata for the right image. it looks like left-right SBS could be considered conforming. it seems to leave lots of leeway really, other than fisheye (no reprojecting), motion stream (for correcting/compensating later), and lots of metadata fields to ensure correct decoding.

in Windows, it's weird to me that Explorer shows mono fisheye thumbnail preview for the Calf MP4 files, but Calf images are all fisheye SBS. My old Lenovo Mirage video files are mono fisheye thumbnail, while the images are mono equirect projection (though the image files were indeed right-image-as-metadata in the full file).

So either Calf is being generous calling it "vr180" or it's just a variant to support. Regardless, hopefully its metadata gives enough clues to be able to handle it without needing to set very specific type and angle properties.

[steren]

The spec you pointed to seems to say it applies to "Videos"
As far as I know, spec for images is here, notably, see the GImage exif.

Like you, from my experience with VR180 generated via Lenovo Mirage Camera are:

• side to side for videos
• right eye in exif for images

I need to inspect the exif of the files produced by Calf, it's likely they contain the many metadata that I need to extract (e.g. pitch and roll, which I only extract in the case of type="vr").

I'll update this issue with my findings.

Thanks a lot for the samples.

[atkulp]

anything else I can help with? there seems to be a total lack of exif data which makes it pretty tough... maybe it would be helpful to engage with the devs on Discord

[steren]

I haven't found the time yet to dive into this issue, I might around winter holidays.

[steren]

Wow, it seems that the sample images you shared have absolutely no exif data (try for yourself at https://mutiny.cz/exifr/)

This is going to be a problem. It means that this component will never be able to auto-detect these images. Usually, by looking at the make, model, lens model or exif, I can infer the type of stereo image, and also use the focal length to create the appropriate field of view

Here, there's absolutely nothing. Yes, engaging with the developers would help. Do you want to do this?

[steren]

Update: I bought a Canon EOS R5 with the dual fisheye lens. I will be adding support for it first (see #21), I will try to add support for CALF shortly after, as the work seems very similar

[steren]

Both pictures are working fine:

For the left-right: <stereo-img src="examples/calf-left-right.jpg"></stereo-img>

For the "VR180": <stereo-img src="examples/calf-vr180.jpg" angle="180"></stereo-img>

Note that you should not use type="vr" since the picture does not conform to Google's VR Photo format