-
Notifications
You must be signed in to change notification settings - Fork 26
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
GLTFRealityKitLoader bad material rendering on some GLB Files #51
Comments
Thanks for the report. This should be enough to start looking into the issue. |
This was due to a misunderstanding of how RealityKit maps the materials list in a If you're able to test this fix without me needing to spin a new release, I'd appreciate any feedback. Otherwise this fix will be incorporated into an upcoming release. Thanks again for the report. |
Thanks for taking a look Warren! We'll test it out and revert back. |
Thanks for the rapid turnaround Still seeing issues with 95d6aaf ![]() ![]() |
Can you share a .glb that reproduces this issue? I suspect it's a separate bug. |
I can but GLB doesn't compress well and these files are bigger than the GitHub max attachment size Google Drive link work? |
The immediate issue here is that we're trying to extract a color channel from a monochrome image as if it's an RGBA image. That's fairly easy to fix, but in the process of fixing it, I've encountered a lot of flakiness in the runtime behavior, and the perennial flakiness of Swift+lldb is making it hard to debug. I'll continue looking at this. |
…ityKit converter (#51) Added code to "pre-decode" potentially problematic images when running on visionOS
The situation should be much improved with c5501dd. In addition to never attempting to extract a channel from a single-channel image, I switched to a more efficient method from the Accelerate framework to do the extraction when necessary. I also added some defensive image decoding code that only runs on visionOS to address some problems I encountered there. You're likely to hit validation errors when running with the Metal API validation layer due to the use of 8K images for all material properties, but you can address that when and how you see fit. |
Tested with c5501dd Things do seem improved, there's intermittency on some of the failures (not sure if that's this library or RealityKit/visionOS Simulator) However some of the models (the jet engine) remain broken with that "zebra" white/gray material failure mode. Here's is another model that consistently zebras: https://drive.google.com/file/d/1cQDxuaT44CjQHENjNUlJSmaspaRd8Srp/view?usp=share_link Are you able to reproduce that issue with that model?
(edit:) my testing environment:
Nothing too crazy here, I can provide the project if that helps with reproduction ![]() ![]() |
Sample project, very simple modifications to the default project setup from 15.2 Large size due to embedded gib files Hope I'm just holding it wrong :) https://drive.google.com/file/d/1gPBJwy8YBSyL7oc6mHnpwCjqufudWp9L/view?usp=share_link |
I think if you're consuming the framework as a Swift package (which I don't recommend to anyone, but grudgingly support), it uses the contents of the Package.swift in the checkout to determine how to build. In this case, because the package file points to the latest GitHub release, you're not actually building and linking against the code at top of tree; you're linking against version 0.5.7, which doesn't include the changes made to address this issue. |
Here's an XCFramework built from top of tree. It should work as a drop-in replacement for the SPM dependency. https://drive.google.com/file/d/1SmnapYontNls7Wuwv2lyXmhs648derKV/view?usp=sharing SHA256: |
Oh wow okay that "makes sense" but damn So my project's package.swift points to GLTFKit master, and so the git history of the local GLTFKit copy shows your recent commits (which is why I thought I was using them) But because GLTFKit's package.swift on master points to a specific release, it "overrides" my project's package.swift file, and what actually gets installed is the older release copy... HUH. Good to know. Okay that's great... I'll be able to test this later today. Thanks Warren! Much appreciated |
Yep, you got it. Sorry for the confusion. I only distribute binaries because SPM still doesn't support mixed-language targets after all these years. Maybe we get that some time this decade. |
Thanks so much for the support here, @warrenm! If you're interested, we'd be happy to give you free access to Polycam Pro. If you create a Polycam account, send me the email address you used to create it, and I'll upgrade it. My email is my first name "elliott" at polycam.ai |
Perfection, thank you so much @warrenm |
Sweet! Thanks for sticking with it. |
@warrenm Awesome! Just upgraded your Polycam account - you should have access to Pro indefinitely. |
Loading certain GLB files via GLTFRealityKitLoader into a RealityView scene on visionOS is resulting in bad rendering
Example 1:
![Screenshot 2024-01-16 at 10 12 01 PM](https://private-user-images.githubusercontent.com/150742504/297268448-e1b98c20-f32f-468e-9367-90b0855f4101.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjEwNTY1MDYsIm5iZiI6MTcyMTA1NjIwNiwicGF0aCI6Ii8xNTA3NDI1MDQvMjk3MjY4NDQ4LWUxYjk4YzIwLWYzMmYtNDY4ZS05MzY3LTkwYjA4NTVmNDEwMS5wbmc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjQwNzE1JTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI0MDcxNVQxNTEwMDZaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT0yMjc0YWYzZDMyMTFjYTY3NmZhMWI0NmVkNWI4YTdjNGNmMTA3ZThkMzQ4OTk0MjZkZDVkNDIxZTAyZWUzMWNlJlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCZhY3Rvcl9pZD0wJmtleV9pZD0wJnJlcG9faWQ9MCJ9.K9Wpe7_mSE2FlkHVib0aWNPj8h3p8f08b8GElClliQE)
Expected:
Result:
![Screenshot 2024-01-16 at 10 14 40 PM](https://private-user-images.githubusercontent.com/150742504/297268471-f2f1fee9-7774-480f-929b-41181557b945.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjEwNTY1MDYsIm5iZiI6MTcyMTA1NjIwNiwicGF0aCI6Ii8xNTA3NDI1MDQvMjk3MjY4NDcxLWYyZjFmZWU5LTc3NzQtNDgwZi05MjliLTQxMTgxNTU3Yjk0NS5wbmc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjQwNzE1JTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI0MDcxNVQxNTEwMDZaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT1iZjEwZTU0OGI5OTZkMThlZDA5ZjIyMmMwNWRlNWYwMDljNGI0YTkxMDQ0NzYwOGI0NGE5MjY4Yjc1OWI1NWUwJlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCZhY3Rvcl9pZD0wJmtleV9pZD0wJnJlcG9faWQ9MCJ9.T_twsDcXHQp2YzsBtFVbeK6q-nVDEqGs9xPnS454JdE)
Example 2:
![Screenshot 2024-01-16 at 10 24 39 PM](https://private-user-images.githubusercontent.com/150742504/297269166-18ab3967-8d2d-427b-b537-442c3b124cec.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjEwNTY1MDYsIm5iZiI6MTcyMTA1NjIwNiwicGF0aCI6Ii8xNTA3NDI1MDQvMjk3MjY5MTY2LTE4YWIzOTY3LThkMmQtNDI3Yi1iNTM3LTQ0MmMzYjEyNGNlYy5wbmc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjQwNzE1JTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI0MDcxNVQxNTEwMDZaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT04ODk2OGFkMGVlYmFjMzlmYzEzMjlkZjE4MjY3NTU1YjFiZGU5NmQzM2YxMTI0MWVmNWY0MjYwMjg4Y2MyN2I4JlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCZhY3Rvcl9pZD0wJmtleV9pZD0wJnJlcG9faWQ9MCJ9.YEY1QWcovlQ8EJx7KLfwbwpQlRpGzLFc0i8O_5c5fGw)
Expected:
Result:
![Screenshot 2024-01-16 at 9 50 05 PM](https://private-user-images.githubusercontent.com/150742504/297269050-fd6a3b46-9b1c-458a-ba73-43f9d85cf84d.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjEwNTY1MDYsIm5iZiI6MTcyMTA1NjIwNiwicGF0aCI6Ii8xNTA3NDI1MDQvMjk3MjY5MDUwLWZkNmEzYjQ2LTliMWMtNDU4YS1iYTczLTQzZjlkODVjZjg0ZC5wbmc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjQwNzE1JTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI0MDcxNVQxNTEwMDZaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT0zNmYwNzUxOTljZmZkMGQ4MWExNjhkZmJmZjViMWEwNzRjODJjN2EzYTM1OWM3MDgyNTZiMDI1MzgwYTdlNTFlJlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCZhY3Rvcl9pZD0wJmtleV9pZD0wJnJlcG9faWQ9MCJ9.RPKz8im7TxoprgOoCfzjFsbt_8PuhlhU1ntUkvGTmMo)
GLB Samples:
tank.glb.zip
More samples providable upon request, GitHub attachments max at 25mb
The text was updated successfully, but these errors were encountered: