-
Notifications
You must be signed in to change notification settings - Fork 1.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
iOS camera api all formats are unknown #9930
Comments
It's probably just an oversight in CoreMediaFormatToSDL(), please feel free to extend it and submit a PR. |
I did some light research on this and the issue isn't so much with The SDL camera code for coremedia uses I will implement the method for getting formats that is suggested in that SO post and see if the formats will then match. If they don't however I am not quite sure how to proceed, would it mean that SDL needs to add support for more formats? |
It's possible, yes, but it's likely that SDL already supports the common formats used by the camera. |
It seems like that SO post about CoreMediaIO was a bad lead, as CoreMediaIO is for macos only. I instead tried to print all the formats the cameras on my iPhone 13 support, it seems like they are part of another enum than what SDL uses. SDL is looking for formats in the In particular x420, 420v and 420f seem very common. These format codes translate to the following formats: kCVPixelFormatType_420YpCbCr8BiPlanarVideoRange = '420v'
kCVPixelFormatType_420YpCbCr8BiPlanarFullRange = '420f'
kCVPixelFormatType_420YpCbCr10BiPlanarVideoRange = 'x420' I think I could modify the SDL code to use this header and values instead, but from there on I am sadly not quite knowledgeable. Does SDL support these formats? Is there a way for SDL to support this formats? Not sure what the implications are here and what needs to be done so any help would be appreciated, I would really love to use the Camera API for my mobile game. |
Yes, SDL does support these formats. The difference between kCVPixelFormatType_420YpCbCr8BiPlanarVideoRange and kCVPixelFormatType_420YpCbCr8BiPlanarFullRange is not represented in the SDL pixel format, but instead in other metadata for the surface. I'll have to look at this later, but for now you can just return SDL_PIXELFORMAT_NV12 for both of them and it should work. |
Awesome, I will modify the code on my side for now and see if I can get the camera stuff to work. Thanks a lot. |
You're welcome! |
This is related to #9610 I was investigating this earlier today and found that adding While debugging I found that the built-in camera on my M1 Macbook Air reports the pixel format In
In
I had to add |
@slouken I tried your suggestion of using I then tried @hkva's suggestion of mapping I then tried updating SDL, since I last got it from the main branch 2 weeks ago or so, and now I just get Any help regarding what I could try next to debug or fix this would be appreciated, I really want to use the Camera API so I would like to contribute in any way I can, tho sadly I am not very well educated in this particular area. Some logs that stand out to me and I'll investigate now are Here is a picture of the |
@SasLuca I was able to get my integrated webcam working with this change, maybe you'll have some luck? https://github.com/hkva-fork/SDL/commit/b8ef9ffa41a591a8c6943e348d840d415241973a 0001-camera-Support-CoreMedia-devices-with-NV12-pixel-for.patch I am seeing the bad access as well. I can repro by running |
Ill try your fork and let you know if it works, thanks a lot. |
@hkva tried the wrong branch initially, but I did try 9610-coremedia-camera-nv12 now and still getting bad accesses in that place sadly. |
Fixed, thanks! |
Awesome, will be updating and trying it again soon! |
I've been trying the new Camera api on a real iPhone 13 device and while I got SDL to get a list of camera devices, it seems like all the available formats of all the available camera are unsupported.
Specifically
CoreMediaFormatToSDL
always returnsSDL_PIXELFORMAT_UNKNOWN
.Is there any undocumented setting or something that I might be missing on causing this issue or is it just a limitation of the API at the moment?
The text was updated successfully, but these errors were encountered: