Unable to load glb model in Android with sceneform using filament

[udayk07]

Hi team,

Unable to load below attached models in android with filament v1.9.24.

Model 1:
I tried to load using https://google.github.io/filament/viewer/ and it is loading fine ( But, console showing warning ). Also, materials are missing in viewer.

expected output:

** In Android studio console found no specific filament logs for this case.

castle.zip

Model2:
While loading below model, it is not displaying in both filament-viewer and Android.

car.zip

[romainguy]

I cannot repro the issue with the first model in the Web viewer. There are no warnings in the console. The second model seems invalid and fails to load in both our Web and native viewers.

[udayk07]

Tried once again the first model in Android device ( SAMSUNG S9 device ) ( https://github.com/google/filament/files/6502198/castle.zip ) and another model ( size 60MB ) (https://www.dropbox.com/s/z5q4twlfjwqf96o/kirala.zip?dl=0). Both models failed to load.

In both cases found no specific filament related logs in Android Studio console with filament. Attached error logs from console.

com.example.augmentedreality E/native: motion_tracking_context.cc:1331 Unable to find measurements corresponding to VIO status at timestamp 387896745592848. com.example.augmentedreality E/native: hit_test.cc:427 generic::internal: No point hit. com.example.augmentedreality E/native: hit_test.cc:427 generic::internal: No point hit. com.example.augmentedreality E/native: hit_test.cc:427 generic::internal: No point hit. com.example.augmentedreality E/native: hit_test.cc:427 generic::internal: No point hit. com.example.augmentedreality E/native: hit_test.cc:427 generic::internal: No point hit. com.example.augmentedreality E/native: hit_test.cc:427 generic::internal: No point hit. com.example.augmentedreality E/native: hit_test.cc:427 generic::internal: No point hit. com.example.augmentedreality E/native: hit_test.cc:427 generic::internal: No point hit. com.example.augmentedreality I/ViewRootImpl@389914[ARActivity]: ViewPostIme pointer 0 com.example.augmentedreality I/ViewRootImpl@389914[ARActivity]: ViewPostIme pointer 1 com.example.augmentedreality E/native: hit_test.cc:427 generic::internal: No point hit. com.example.augmentedreality D/Toast: show: focusDisplayId = 0, isFocusInDesktop = false mCustomDisplayId=-1 com.example.augmentedreality V/Toast: SHOW: android.widget.Toast$TN@88adfa4 com.example.augmentedreality E/native: camera_image_stream.cc:269 FAILED_PRECONDITION: Failed to extract the native metadata, status=DEADLINE_EXCEEDED: Timed out waiting for metadata. com.example.augmentedreality W/native: data_manager.cc:225 The used RAM memory size by current process is 1066348 kb which is greater than the threshold, so stop online mapping. com.example.augmentedreality V/Toast: HANDLE SHOW: android.widget.Toast$TN@88adfa4 mView=null mNextView=android.widget.LinearLayout{84b6d3 V.E...... ......I. 0,0-0,0} com.example.augmentedreality V/Toast: HANDLE HIDE: android.widget.Toast$TN@88adfa4 mView=null com.example.augmentedreality I/ugmentedrealit: Starting a blocking GC NativeAlloc com.example.augmentedreality I/ViewRootImpl@235f0e[Toast]: setView = android.widget.LinearLayout@84b6d3 TM=true MM=false com.example.augmentedreality V/Toast: Text: Pn T in android.widget.Toast$TN@88adfa4 com.example.augmentedreality I/Choreographer: Skipped 124 frames! The application may be doing too much work on its main thread. com.example.augmentedreality E/native: hit_test.cc:427 generic::internal: No point hit. com.example.augmentedreality V/Toast: HIDE: android.widget.Toast$TN@88adfa4 com.example.augmentedreality I/ViewRootImpl@235f0e[Toast]: Relayout returned: old=(0,65,1080,2089) new=(335,1795,745,1915) req=(410,120)0 dur=24 res=0x7 s={true 483589222400} ch=true com.example.augmentedreality I/mali_winsys: new_window_surface() [410x120] return: 0x3000 com.example.augmentedreality E/native: camera_image_stream.cc:269 FAILED_PRECONDITION: Failed to extract the native metadata, status=DEADLINE_EXCEEDED: Timed out waiting for metadata. com.example.augmentedreality E/native: camera_image_stream.cc:269 FAILED_PRECONDITION: Failed to extract the native metadata, status=DEADLINE_EXCEEDED: Timed out waiting for metadata. com.example.augmentedreality V/Toast: HANDLE HIDE: android.widget.Toast$TN@88adfa4 mView=android.widget.LinearLayout{84b6d3 V.E...... ........ 0,0-410,120} com.example.augmentedreality V/Toast: REMOVE! android.widget.LinearLayout{84b6d3 V.E...... ........ 0,0-410,120} in android.widget.Toast$TN@88adfa4 com.example.augmentedreality I/mali_egl: eglDestroySurface() in com.example.augmentedreality I/mali_winsys: delete_surface() [410x120] return com.example.augmentedreality I/mali_egl: eglDestroySurface() out com.example.augmentedreality W/libEGL: EGLNativeWindowType 0x716a949ed0 disconnect failed com.example.augmentedreality I/ViewRootImpl@235f0e[Toast]: dispatchDetachedFromWindow com.example.augmentedreality D/InputTransport: Input channel destroyed: 'c28aea0', fd=153 com.example.augmentedreality I/Choreographer: Skipped 86 frames! The application may be doing too much work on its main thread. com.example.augmentedreality E/native: hit_test.cc:427 generic::internal: No point hit. com.example.augmentedreality E/native: hit_test.cc:427 generic::internal: No point hit. com.example.augmentedreality E/native: hit_test.cc:427 generic::internal: No point hit. com.example.augmentedreality D/InputMethodManager: prepareNavigationBarInfo() DecorView@9d23288[ARActivity] com.example.augmentedreality D/InputMethodManager: getNavigationBarColor() -855310 com.example.augmentedreality V/InputMethodManager: Starting input: tba=com.example.augmentedreality ic=null mNaviBarColor -855310 mIsGetNaviBarColorSuccess true , NavVisible : false , NavTrans : false com.example.augmentedreality D/InputMethodManager: startInputInner - Id : 0 com.example.augmentedreality I/InputMethodManager: startInputInner - mService.startInputOrWindowGainedFocus com.example.augmentedreality D/InputTransport: Input channel destroyed: 'ClientS', fd=110 com.example.augmentedreality E/native: hit_test.cc:427 generic::internal: No point hit. com.example.augmentedreality D/InputMethodManager: prepareNavigationBarInfo() DecorView@9d23288[ARActivity] com.example.augmentedreality D/InputMethodManager: getNavigationBarColor() -855310 com.example.augmentedreality V/InputMethodManager: Starting input: tba=com.example.augmentedreality ic=null mNaviBarColor -855310 mIsGetNaviBarColorSuccess true , NavVisible : false , NavTrans : false com.example.augmentedreality D/InputMethodManager: startInputInner - Id : 0 com.example.augmentedreality I/InputMethodManager: startInputInner - mService.startInputOrWindowGainedFocus

com.example.augmentedreality/com.example.augmentedreality.MainActivity (server)' ~ Channel is unrecoverably broken and will be disposed!

Console_log.txt

[prideout]

In a debug build on Desktop, the car.zip model triggers an assert in OpenGLDriver::allocateHandle. (@pixelflinger) I'd be surprised if the handle arena is overflowing -- this model has a lot of renderables (7151 meshes) but not a totally crazy number.

[udayk07]

@prideout Do we have any workaround to fix this rendering issue for below car.zip ( dracco compression also removed) model as it seems to be in valid format while checking with gltf-validator ?

car.zip

[prideout]

@udayk07 which tool did you use to generate this glTF?

It might be interesting to see what would happen if you were to combine the meshes to reduce the number of renderables.

[udayk07]

@prideout The model generated by three.js exporter ( to load as .glb file in Android ).

Will look into combine meshes in this model. Do we have any maximum mesh count in filament ?

[prideout]

Our internal handle arena is limited to 2 MB which should be plenty. To be honest my mesh reduction idea is just an experiment to see what will happen, since I don't understand this bug yet. (note however that your model will be more optimal if you reduce the mesh count)

[udayk07]

@prideout Tried to combine meshes for car.zip model. Found below results,

1. car_partial_meshjoin.glb.zip -> combined only few meshes in this model (total mesh count 4965) which is not loading in Android device and filament viewer also ( original mesh count 7151 )

2. car_high_meshjoin.glb.zip -> Combined almost all meshes in model which is loading fine in Android device ( total mesh count 258 )

Even though above one with 258 meshes loads fine, the model https://github.com/google/filament/files/6502198/castle.zip with only 5 meshes not loading in Android ( found no filament logs in console also )

[prideout]

Thanks, that is interesting. I haven't had time to investigate yet, so I'm not sure if this is an actual bug or if a resource limit has been exceeded, either way we should probably try to emit better error messages.

@pixelflinger these models might be good stress tests, not sure how they compare to bistro and sponza

[pixelflinger]

So car.zip seems to be invalid. At least, our ResourceLoader asserts with a problem where there are no triangles when processing tangents.

The model also uses more memory than we set by default, thankfully, we have compile time configuration flags you can set. It seems that you need to:

• increase FILAMENT_OPENGL_HANDLE_ARENA_SIZE_IN_MB -- but I'm not sure to what value yet. Maybe try 8 MB.
• increase FILAMENT_MIN_COMMAND_BUFFERS_SIZE_IN_MB to 4MB

Finally there is another problem where we're running out of jobs, I have a PR that fixes that.

Because of the 1st problem, I wasn't able to load this model.

[udayk07]

@pixelflinger

I am using filament in Android app through maven dependency ( implementation 'com.google.android.filament:filament-android:1.10.0' ).

Is it possible to set above mentioned flags in Android build ?

[pixelflinger]

Ah no not yet. It's on my to-do list to make them settable at runtime.

I'm also working on some changes that will make this car work better.

So that said, such a model won't run well, due to the overhead of referring very small geometry with a lot of calls. On my desktop computer, I can't even get good performance.

It's important to group meshes together that are close from each other and of similar size and of same material.

[udayk07]

Will there be any update on memory size change option for Android to resolve rendering issue ?

[prideout]

Yes, thanks for the reminder: I think @pixelflinger and I have discussed increasing this constant for the WebGL build only, because the WebGL build is more likely to be used as a generic viewer. For Android builds, we probably want to avoid increasing this limit.

[prideout]

The handle limit has been increased via #4201.

[udayk07]

Thanks for the help.

It seems the changes available in version 1.10.5. But, the android version 1.10.5 not available ( Could not find com.google.android.filament:filament-android:1.10.5 )