Live object detection example not working as should

[DavidOrakpo]

I cloned the repo this morning, ran the shell script to download the tflite and label files, and ran on my device.

However, it is not detecting as the demo gifs show it should.
Find a screen recording of how it looks below.
https://imgur.com/a/UoIrljg

[B4eva]

I actually have the same issue here. I successfully cloned and ran the script but it does not identify and label items as intended.

WhatsApp.Video.2023-08-06.at.17.27.48.mp4

[PaulTR]

That's really weird. Are either of you able to provide logs that maybe show that the model didn't load properly? I was able to run from main without seeing this issue.

Thanks!

[DavidOrakpo]

Thanks for the speedy reply! Please find below the logs from the debug console when the project is running.

❯ flutter run
Connected devices:
SM G781B (mobile)           • RFCR11C3JYL   • android-arm64 • Android 13 (API 33)
sdk gphone64 arm64 (mobile) • emulator-5554 • android-arm64 • Android 13 (API 33) (emulator)

No wireless devices were found.

[1]: SM G781B (RFCR11C3JYL)
[2]: sdk gphone64 arm64 (emulator-5554)
Please choose one (or "q" to quit): 1
Launching lib/main.dart on SM G781B in debug mode...
Running Gradle task 'assembleDebug'...                              3.7s
✓  Built build/app/outputs/flutter-apk/app-debug.apk.
Syncing files to device SM G781B...                                 80ms

Flutter run key commands.
r Hot reload. 🔥🔥🔥
R Hot restart.
h List all available interactive commands.
d Detach (terminate "flutter run" but leave application running).
c Clear the screen
q Quit (terminate the application on the device).

A Dart VM Service on SM G781B is available at: http://127.0.0.1:59597/b554bFG4mwM=/
I/CameraManagerGlobal(12045): Connecting to camera service
D/VendorTagDescriptor(12045): addVendorDescriptor: vendor tag id 14172875900359437128 added
I/CameraManagerGlobal(12045): Camera 0 facing CAMERA_FACING_BACK state now CAMERA_STATE_CLOSED for client com.example.live_object_detection_ssd_mobilenet API Level 2User Id 0
I/CameraManagerGlobal(12045): Camera 1 facing CAMERA_FACING_FRONT state now CAMERA_STATE_CLOSED for client com.samsung.android.smartface API Level 2User Id 0
I/CameraManagerGlobal(12045): Camera 2 facing CAMERA_FACING_BACK state now CAMERA_STATE_CLOSED for client android.system API Level 2User Id 0
I/CameraManagerGlobal(12045): Camera 20 facing CAMERA_FACING_BACK state now CAMERA_STATE_CLOSED for client com.sec.android.app.camera API Level 2User Id 0
I/CameraManagerGlobal(12045): Camera 21 facing CAMERA_FACING_BACK state now CAMERA_STATE_CLOSED for client android.system API Level 2User Id 0
I/CameraManagerGlobal(12045): Camera 23 facing CAMERA_FACING_BACK state now CAMERA_STATE_CLOSED for client android.system API Level 2User Id 0
I/CameraManagerGlobal(12045): Camera 3 facing CAMERA_FACING_FRONT state now CAMERA_STATE_CLOSED for client com.samsung.adaptivebrightnessgo API Level 2User Id 0
I/CameraManagerGlobal(12045): Camera 4 facing CAMERA_FACING_FRONT state now CAMERA_STATE_CLOSED for client client.pid<1370> API Level 2User Id 0
I/CameraManagerGlobal(12045): Camera 40 facing CAMERA_FACING_BACK state now CAMERA_STATE_CLOSED for client android.system API Level 2User Id 0
I/CameraManagerGlobal(12045): Camera 41 facing CAMERA_FACING_FRONT state now CAMERA_STATE_CLOSED for client android.system API Level 2User Id 0
I/CameraManagerGlobal(12045): Camera 52 facing CAMERA_FACING_BACK state now CAMERA_STATE_CLOSED for client android.system API Level 2User Id 0
I/n_ssd_mobilenet(12045): Compiler allocated 6133KB to compile void android.view.ViewRootImpl.performTraversals()
W/libc    (12045): Access denied finding property "ro.mediatek.platform"
I/tflite  (12045): Created TensorFlow Lite XNNPACK delegate for CPU.
D/CompatibilityChangeReporter(12045): Compat change id reported: 206033068; UID 10917; state: ENABLED
I/tflite  (12045): Initialized TensorFlow Lite runtime.
W/Camera  (12045): The selected imageFormatGroup is not supported by Android. Defaulting to yuv420
The Flutter DevTools debugger and profiler on SM G781B is available at: http://127.0.0.1:9101?uri=http://127.0.0.1:59597/b554bFG4mwM=/
I/CameraManagerGlobal(12045): Camera 0 facing CAMERA_FACING_BACK state now CAMERA_STATE_OPEN for client com.example.live_object_detection_ssd_mobilenet API Level 2User Id 0
I/Camera  (12045): startPreview
I/BLASTBufferQueue(12045): [SurfaceView[com.example.live_object_detection_ssd_mobilenet/com.example.live_object_detection_ssd_mobilenet.MainActivity]@0#1](f:0,a:0) onFrameAvailable the first frame is available
D/SurfaceView@894d91f(12045):  setAlpha: mUseAlpha = false alpha=1.0
D/SurfaceView@894d91f(12045):  updateSurfaceAlpha: setUseAlpha() is not called, ignored.
I/ViewRootImpl@dff68e9[MainActivity](12045): Setup new sync id=0
I/ViewRootImpl@dff68e9[MainActivity](12045): Setting syncFrameCallback
I/ViewRootImpl@dff68e9[MainActivity](12045): registerCallbacksForSync syncBuffer=false
D/SurfaceView@894d91f(12045): updateSurfacePosition RenderWorker, frameNr = 1, position = [0, 0, 1080, 2361] surfaceSize = 1080x2361
I/SurfaceView@894d91f(12045): uSP: rtp = Rect(0, 0 - 1080, 2361) rtsw = 1080 rtsh = 2361
I/SurfaceView@894d91f(12045): onSSPAndSRT: pl = 0 pt = 0 sx = 1.0 sy = 1.0
I/SurfaceView@894d91f(12045): aOrMT: ViewRootImpl@dff68e9[MainActivity] t = android.view.SurfaceControl$Transaction@78ed018 fN = 1 android.view.SurfaceView.-$$Nest$mapplyOrMergeTransaction:0 android.view.SurfaceView$SurfaceViewPositionUpdateListener.positionChanged:1537 android.graphics.RenderNode$CompositePositionUpdateListener.positionChanged:373 
I/ViewRootImpl@dff68e9[MainActivity](12045): mWNT: t=0xb40000787d306700 mBlastBufferQueue=0xb4000078f7178400 fn= 1 caller= android.view.SurfaceView.applyOrMergeTransaction:1455 android.view.SurfaceView.-$$Nest$mapplyOrMergeTransaction:0 android.view.SurfaceView$SurfaceViewPositionUpdateListener.positionChanged:1537 
I/ViewRootImpl@dff68e9[MainActivity](12045): mWNT: t=0xb40000787d306500 mBlastBufferQueue=0xb4000078f7178400 fn= 1 caller= android.view.ViewRootImpl.lambda$applyTransactionOnDraw$11$android-view-ViewRootImpl:12865 android.view.ViewRootImpl$$ExternalSyntheticLambda1.onFrameDraw:4 android.graphics.HardwareRenderer$FrameDrawingCallback.onFrameDraw:924 
I/ViewRootImpl@dff68e9[MainActivity](12045): Received frameDrawingCallback syncResult=0 frameNum=1.
I/ViewRootImpl@dff68e9[MainActivity](12045): Setting up sync and frameCommitCallback
I/BLASTBufferQueue(12045): [ViewRootImpl@dff68e9[MainActivity]#0](f:0,a:0) onFrameAvailable the first frame is available
I/ViewRootImpl@dff68e9[MainActivity](12045): Received frameCommittedCallback lastAttemptedDrawFrameNum=1 didProduceBuffer=true
D/OpenGLRenderer(12045): CFMS:: SetUp Pid : 12045    Tid : 12076
W/Parcel  (12045): Expecting binder but got null!
I/ViewRootImpl@dff68e9[MainActivity](12045): onSyncComplete
I/ViewRootImpl@dff68e9[MainActivity](12045): setupSync seqId=0 mSyncId=0 fn=1 caller=android.view.ViewRootImpl$$ExternalSyntheticLambda11.accept:6 android.window.SurfaceSyncer.lambda$setupSync$1$android-window-SurfaceSyncer:128 android.window.SurfaceSyncer$$ExternalSyntheticLambda1.accept:8 android.window.SurfaceSyncer$SyncSet.checkIfSyncIsComplete:382 android.window.SurfaceSyncer$SyncSet.markSyncReady:359 android.window.SurfaceSyncer.markSyncReady:151 android.view.ViewRootImpl.performTraversals:4503 
I/ViewRootImpl@dff68e9[MainActivity](12045): reportDrawFinished seqId=0 mSyncId=-1 fn=1 mSurfaceChangedTransaction=0xb40000787d4fdc00
I/ViewRootImpl@dff68e9[MainActivity](12045): MSG_WINDOW_FOCUS_CHANGED 1 0
I/ViewRootImpl@dff68e9[MainActivity](12045): mThreadedRenderer.initializeIfNeeded()#2 mSurface={isValid=true 0xb40000785ecad000}
D/InputMethodManager(12045): startInputInner - Id : 0
I/InputMethodManager(12045): startInputInner - mService.startInputOrWindowGainedFocus
D/InputMethodManager(12045): startInputInner - Id : 0
I/Camera  (12045): CameraCaptureSession onConfigured
I/Camera  (12045): Updating builder settings
D/Camera  (12045): Updating builder with feature: ExposureLockFeature
D/Camera  (12045): Updating builder with feature: ExposurePointFeature
D/Camera  (12045): Updating builder with feature: ZoomLevelFeature
D/Camera  (12045): Updating builder with feature: AutoFocusFeature
D/Camera  (12045): Updating builder with feature: NoiseReductionFeature
I/Camera  (12045): updateNoiseReduction | currentSetting: fast
D/Camera  (12045): Updating builder with feature: FocusPointFeature
D/Camera  (12045): Updating builder with feature: ResolutionFeature
D/Camera  (12045): Updating builder with feature: SensorOrientationFeature
D/Camera  (12045): Updating builder with feature: FlashFeature
D/Camera  (12045): Updating builder with feature: ExposureOffsetFeature
D/Camera  (12045): Updating builder with feature: FpsRangeFeature
I/Camera  (12045): refreshPreviewCaptureSession
I/CameraManagerGlobal(12045): Camera 0 facing CAMERA_FACING_BACK state now CAMERA_STATE_ACTIVE for client com.example.live_object_detection_ssd_mobilenet API Level 2User Id 0
I/CameraManagerGlobal(12045): Camera 0 facing CAMERA_FACING_BACK state now CAMERA_STATE_IDLE for client com.example.live_object_detection_ssd_mobilenet API Level 2User Id 0
I/Camera  (12045): startPreviewWithImageStream
I/Camera  (12045): CameraCaptureSession onConfigured
I/Camera  (12045): Updating builder settings
D/Camera  (12045): Updating builder with feature: ExposureLockFeature
D/Camera  (12045): Updating builder with feature: ExposurePointFeature
D/Camera  (12045): Updating builder with feature: ZoomLevelFeature
D/Camera  (12045): Updating builder with feature: AutoFocusFeature
D/Camera  (12045): Updating builder with feature: NoiseReductionFeature
I/Camera  (12045): updateNoiseReduction | currentSetting: fast
D/Camera  (12045): Updating builder with feature: FocusPointFeature
D/Camera  (12045): Updating builder with feature: ResolutionFeature
D/Camera  (12045): Updating builder with feature: SensorOrientationFeature
D/Camera  (12045): Updating builder with feature: FlashFeature
D/Camera  (12045): Updating builder with feature: ExposureOffsetFeature
D/Camera  (12045): Updating builder with feature: FpsRangeFeature
I/Camera  (12045): refreshPreviewCaptureSession
W/n_ssd_mobilenet(12045): Long monitor contention with owner main (12045) at void android.hardware.camera2.impl.CameraDeviceImpl.waitUntilIdle()(CameraDeviceImpl.java:1402) waiters=0 in void android.hardware.camera2.impl.CameraDeviceImpl$4.run() for 479ms
W/n_ssd_mobilenet(12045): Long monitor contention with owner main (12045) at void android.hardware.camera2.impl.CameraDeviceImpl.waitUntilIdle()(CameraDeviceImpl.java:1402) waiters=1 in void android.hardware.camera2.impl.CameraDeviceImpl$CameraDeviceCallbacks.onResultReceived(android.hardware.camera2.impl.CameraMetadataNative, android.hardware.camera2.impl.CaptureResultExtras, android.hardware.camera2.impl.PhysicalCaptureResultInfo[]) for 294ms
I/CameraManagerGlobal(12045): Camera 0 facing CAMERA_FACING_BACK state now CAMERA_STATE_ACTIVE for client com.example.live_object_detection_ssd_mobilenet API Level 2User Id 0
I/Camera  (12045): CameraCaptureSession onClosed
W/System  (12045): A resource failed to call Surface.release. 
D/ProfileInstaller(12045): Installing profile for com.example.live_object_detection_ssd_mobilenet
I/ViewRootImpl@dff68e9[MainActivity](12045): ViewPostIme pointer 0
I/ViewRootImpl@dff68e9[MainActivity](12045): ViewPostIme pointer 1
I/ViewRootImpl@dff68e9[MainActivity](12045): ViewPostIme pointer 0
I/ViewRootImpl@dff68e9[MainActivity](12045): ViewPostIme pointer 1
I/ViewRootImpl@dff68e9[MainActivity](12045): ViewPostIme pointer 0
I/ViewRootImpl@dff68e9[MainActivity](12045): ViewPostIme pointer 1
I/n_ssd_mobilenet(12045): Background young concurrent copying GC freed 334(94KB) AllocSpace objects, 14(7952KB) LOS objects, 69% free, 2728KB/8838KB, paused 9.185ms,18us total 17.308ms
I/n_ssd_mobilenet(12045): Background young concurrent copying GC freed 285(93KB) AllocSpace objects, 10(6508KB) LOS objects, 52% free, 4206KB/8839KB, paused 7.099ms,15us total 15.502ms
I/ViewRootImpl@dff68e9[MainActivity](12045): ViewPostIme pointer 0
I/ViewRootImpl@dff68e9[MainActivity](12045): ViewPostIme pointer 1
I/ViewRootImpl@dff68e9[MainActivity](12045): ViewPostIme pointer 0
I/ViewRootImpl@dff68e9[MainActivity](12045): ViewPostIme pointer 1
I/ViewRootImpl@dff68e9[MainActivity](12045): ViewPostIme pointer 0
I/ViewRootImpl@dff68e9[MainActivity](12045): ViewPostIme pointer 1

Application finished.

 

Hope this helps!

That's really weird. Are either of you able to provide logs that maybe show that the model didn't load properly? I was able to run from main without seeing this issue.

Thanks!

[DavidOrakpo]

Update, I tested the code in my emulator and it works fine. I'm not sure what's causing the mis match on a real device

[B4eva]

After a few tries and double-checking, found out I did not load the model correctly. After doing that again it works fine now. Thanks for quick assist @PaulTR

[DavidOrakpo]

Update. I changed the Resolution enum to MAX and not MEDIUM, and it worked.