Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Facedetection Static Bitmap image - Internal error occurred while executing MLKit tasks - Android #345

Closed
archidvignesh opened this issue Aug 23, 2021 · 13 comments
Assignees

Comments

@archidvignesh
Copy link

archidvignesh commented Aug 23, 2021

So I am following Jetpack architecture and MVVM pattern. In the fragment, I use the image picker to fetch an image . If the user has successfully picked an image, then I call an isFaceDetected function from the viewModel. Here is the function:

//View Model
private fun detectFace(image: Bitmap) {
        val inputImage = InputImage.fromBitmap(image, 0)
        val detector = FaceDetection.getClient()
        val result = detector.process(inputImage).addOnFailureListener {
            Log.d("Authentication", it.toString())
        }.addOnSuccessListener {
            Log.d("Authentication", "Successful face detection")
        }
    }

Here is how I call this function from the fragment

@RequiresApi(Build.VERSION_CODES.P)
    private val getContentA = registerForActivityResult(ActivityResultContracts.StartActivityForResult()) { it ->
        val resultCode = it.resultCode
        val data = it.data

        if(resultCode == Activity.RESULT_OK) {
            val fileUri = data?.data!!
            val bitmap = when {
                Build.VERSION.SDK_INT < 28 -> MediaStore.Images.Media.getBitmap(
                    requireActivity().contentResolver,
                    fileUri
                )
                else -> {
                    val source = ImageDecoder.createSource(
                        requireActivity().contentResolver,
                        fileUri
                    )
                    ImageDecoder.decodeBitmap(source)
                }
            }

            viewModel.detectFace(bitmap)
        }
    }

I have followed all steps in documentation. I am using bundled model. I am getting error Internal error occurred while executing MLKit tasks. Here is the trace from logs

2021-08-23 17:21:24.116 464-1061/com.example.chemisstry I/DynamiteModule: Considering local module com.google.mlkit.dynamite.face:10000 and remote module com.google.mlkit.dynamite.face:0
2021-08-23 17:21:24.117 464-1061/com.example.chemisstry I/DynamiteModule: Selected local version of com.google.mlkit.dynamite.face
2021-08-23 17:21:24.117 464-767/com.example.chemisstry W/GooglePlayServicesUtil: Google Play services out of date for com.example.chemisstry.  Requires 203390000 but found 203019047
2021-08-23 17:21:24.118 464-767/com.example.chemisstry W/GoogleApiManager: The service for com.google.android.gms.common.internal.service.zap is not available: ConnectionResult{statusCode=SERVICE_VERSION_UPDATE_REQUIRED, resolution=null, message=null}
2021-08-23 17:21:24.156 464-1061/com.example.chemisstry W/LibraryVersion: Failed to get app version for libraryName: optional-module-face
2021-08-23 17:21:24.169 464-504/com.example.chemisstry I/mple.chemisstr: NativeAlloc concurrent copying GC freed 55198(2566KB) AllocSpace objects, 10(328KB) LOS objects, 62% free, 7514KB/19MB, paused 149us total 112.133ms
2021-08-23 17:21:24.193 464-768/com.example.chemisstry D/TransportRuntime.SQLiteEventStore: Storing event with priority=VERY_LOW, name=FIREBASE_ML_SDK for destination cct
2021-08-23 17:21:24.210 464-768/com.example.chemisstry D/TransportRuntime.JobInfoScheduler: Upload for context TransportContext(cct, VERY_LOW, MSRodHRwczovL2ZpcmViYXNlbG9nZ2luZy5nb29nbGVhcGlzLmNvbS92MGNjL2xvZy9iYXRjaD9mb3JtYXQ9anNvbl9wcm90bzNc) is already scheduled. Returning...
2021-08-23 17:21:24.228 464-768/com.example.chemisstry D/TransportRuntime.SQLiteEventStore: Storing event with priority=DEFAULT, name=FIREBASE_ML_SDK for destination cct
2021-08-23 17:21:24.230 464-767/com.example.chemisstry W/GooglePlayServicesUtil: Google Play services out of date for com.example.chemisstry.  Requires 203390000 but found 203019047
2021-08-23 17:21:24.230 464-767/com.example.chemisstry W/GoogleApiManager: The service for com.google.android.gms.common.internal.service.zap is not available: ConnectionResult{statusCode=SERVICE_VERSION_UPDATE_REQUIRED, resolution=null, message=null}
2021-08-23 17:21:24.233 464-1061/com.example.chemisstry V/FaceDetectorV2Jni: initialize.start()
2021-08-23 17:21:24.239 464-768/com.example.chemisstry D/TransportRuntime.JobInfoScheduler: Upload for context TransportContext(cct, DEFAULT, MSRodHRwczovL2ZpcmViYXNlbG9nZ2luZy5nb29nbGVhcGlzLmNvbS92MGNjL2xvZy9iYXRjaD9mb3JtYXQ9anNvbl9wcm90bzNc) is already scheduled. Returning...
2021-08-23 17:21:24.239 464-1061/com.example.chemisstry I/native: I0823 17:21:24.239522    1061 face_detector_v2_jni.cc:33] Loading models/fssd_25_8bit_v1.tflite
2021-08-23 17:21:24.240 464-1061/com.example.chemisstry I/native: I0823 17:21:24.240793    1061 face_detector_v2_jni.cc:33] Loading models/fssd_25_8bit_gray_v1.tflite
2021-08-23 17:21:24.242 464-1061/com.example.chemisstry I/native: I0823 17:21:24.242186    1061 face_detector_v2_jni.cc:33] Loading models/LMprec_600.emd
2021-08-23 17:21:24.257 464-768/com.example.chemisstry D/TransportRuntime.SQLiteEventStore: Storing event with priority=VERY_LOW, name=FIREBASE_ML_SDK for destination cct
2021-08-23 17:21:24.257 464-464/com.example.chemisstry I/InputTransport: Create ARC handle: 0x7476e027c0
2021-08-23 17:21:24.257 464-1061/com.example.chemisstry I/native: I0823 17:21:24.257549    1061 face_detector_v2_jni.cc:33] Loading models/BCLlefteyeclosed_200.emd
2021-08-23 17:21:24.257 464-1061/com.example.chemisstry I/native: I0823 17:21:24.257946    1061 face_detector_v2_jni.cc:33] Loading models/BCLrighteyeclosed_200.emd
2021-08-23 17:21:24.258 464-1061/com.example.chemisstry I/native: I0823 17:21:24.258188    1061 face_detector_v2_jni.cc:33] Loading models/BCLjoy_200.emd
2021-08-23 17:21:24.258 464-1061/com.example.chemisstry I/native: I0823 17:21:24.258554    1061 face_detector_v2_jni.cc:33] Loading models/MFT_fssd_fastgray.pb
2021-08-23 17:21:24.258 464-1061/com.example.chemisstry I/native: I0823 17:21:24.258692    1061 face_detector_v2_jni.cc:33] Loading models/contours.tfl
2021-08-23 17:21:24.265 464-787/com.example.chemisstry V/FA: Activity resumed, time: 2643730424
2021-08-23 17:21:24.269 464-464/com.example.chemisstry I/Choreographer: Skipped 30 frames!  The application may be doing too much work on its main thread.
2021-08-23 17:21:24.272 464-768/com.example.chemisstry D/TransportRuntime.JobInfoScheduler: Upload for context TransportContext(cct, VERY_LOW, MSRodHRwczovL2ZpcmViYXNlbG9nZ2luZy5nb29nbGVhcGlzLmNvbS92MGNjL2xvZy9iYXRjaD9mb3JtYXQ9anNvbl9wcm90bzNc) is already scheduled. Returning...
2021-08-23 17:21:24.273 464-1061/com.example.chemisstry I/native: I0823 17:21:24.273821    1061 face_detector_v2_jni.cc:33] Loading models/blazeface.tfl
2021-08-23 17:21:24.285 464-1061/com.example.chemisstry I/tflite: Initialized TensorFlow Lite runtime.
2021-08-23 17:21:24.295 464-768/com.example.chemisstry D/TransportRuntime.SQLiteEventStore: Storing event with priority=VERY_LOW, name=FIREBASE_ML_SDK for destination cct
2021-08-23 17:21:24.296 464-1061/com.example.chemisstry V/FaceDetectorV2Jni: initialize.end()
2021-08-23 17:21:24.308 464-768/com.example.chemisstry D/TransportRuntime.JobInfoScheduler: Upload for context TransportContext(cct, VERY_LOW, MSRodHRwczovL2ZpcmViYXNlbG9nZ2luZy5nb29nbGVhcGlzLmNvbS92MGNjL2xvZy9iYXRjaD9mb3JtYXQ9anNvbl9wcm90bzNc) is already scheduled. Returning...
2021-08-23 17:21:24.416 464-464/com.example.chemisstry D/ViewRootImpl[AuthenticationActivity]: msg2 update new insets InsetsState: {mDisplayFrame=Rect(0, 0 - 720, 1544), mSources= { InsetsSource: {mType=ITYPE_STATUS_BAR, mFrame=[0,0][720,56], mVisible=true}, InsetsSource: {mType=ITYPE_NAVIGATION_BAR, mFrame=[0,1460][720,1544], mVisible=true}, InsetsSource: {mType=ITYPE_TOP_GESTURES, mFrame=[0,0][720,56], mVisible=true}, InsetsSource: {mType=ITYPE_BOTTOM_GESTURES, mFrame=[0,1460][720,1544], mVisible=true}, InsetsSource: {mType=ITYPE_LEFT_GESTURES, mFrame=[0,0][0,1544], mVisible=true}, InsetsSource: {mType=ITYPE_RIGHT_GESTURES, mFrame=[720,0][720,1544], mVisible=true}, InsetsSource: {mType=ITYPE_TOP_TAPPABLE_ELEMENT, mFrame=[0,0][720,56], mVisible=true}, InsetsSource: {mType=ITYPE_BOTTOM_TAPPABLE_ELEMENT, mFrame=[0,1460][720,1544], mVisible=true}, InsetsSource: {mType=ITYPE_LEFT_DISPLAY_CUTOUT, mFrame=[0,0][-2147483648,1544], mVisible=true}, InsetsSource: {mType=ITYPE_TOP_DISPLAY_CUTOUT, mFrame=[0,0][720,54], mVisible=true}, InsetsSource: {mType=ITYPE_RIGHT_DISPLAY_CUTOUT, mFrame=[2147483647,0][720,1544], mVisible=true}, InsetsSource: {mType=ITYPE_BOTTOM_DISPLAY_CUTOUT, mFrame=[0,2147483647][720,1544], mVisible=true}, InsetsSource: {mType=ITYPE_IME, mFrame=[0,0][0,0], mVisible=false} }
2021-08-23 17:21:24.424 464-464/com.example.chemisstry D/Authentication: com.google.mlkit.common.MlKitException: Internal error has occurred when executing ML Kit tasks
2021-08-23 17:21:29.291 464-787/com.example.chemisstry V/FA: Inactivity, disconnecting from the service
2021-08-23 17:22:13.661 464-768/com.example.chemisstry D/TransportRuntime.CctTransportBackend: Making request to: ************************************************************************
2021-08-23 17:22:13.789 464-768/com.example.chemisstry D/Linux: [Posix_connect Debug]Process com.example.chemisstry :443 
2021-08-23 17:22:14.037 464-768/com.example.chemisstry I/TransportRuntime.CctTransportBackend: Status Code: 200
2021-08-23 17:22:14.038 464-768/com.example.chemisstry I/TransportRuntime.CctTransportBackend: Content-Type: application/json; charset=UTF-8
2021-08-23 17:22:14.038 464-768/com.example.chemisstry I/TransportRuntime.CctTransportBackend: Content-Encoding: gzip
@archidvignesh
Copy link
Author

Here is another useful information:

If I try to do this with URI, it works fine. But bitmap is causing internal error.

This might be a useful tip as well. I was following the documentation step by step but I accidentally included both google play services module and the bundle module and built the gradle. Then I deleted the google plays services module as I need the model to be bundled to the app.

@zhouyiself
Copy link
Collaborator

From the log, the face engine is initialized successfully, so I'm wondering if there's something wrong with the input bitmap. Can you check if the bitmap is valid before feeding to ML Kit?

@archidvignesh
Copy link
Author

Bitmap is ok. I upload to firebase storage as bitmap data and I’m able to get the url and set an image so I guess it’s fine.

@SilverDestiny
Copy link
Collaborator

Hi,

2021-08-23 17:21:24.424 464-464/com.example.chemisstry D/Authentication: com.google.mlkit.common.MlKitException: Internal error has occurred when executing ML Kit tasks

Could you print out the cause of this MlKitException as well? It'd be easier to check what failed if we have the full details. Thanks!

@archidvignesh
Copy link
Author

@SilverDestiny Can you give me instructions on how I can get that for you? Sorry kind of new to android

@SilverDestiny
Copy link
Collaborator

MlKitException was constructed by the error message and cause Throwable by https://developer.android.com/reference/java/lang/Exception#Exception(java.lang.String,%20java.lang.Throwable)

From your code:

val result = detector.process(inputImage).addOnFailureListener {
            Log.d("Authentication", it.toString())
        }

I think the toString() here only prints the top level error message without underlying cause details.
Probably you can log with Throwable https://developer.android.com/reference/android/util/Log#d(java.lang.String,%20java.lang.String,%20java.lang.Throwable) or just printStackTrace() https://developer.android.com/reference/java/lang/Throwable#printStackTrace()

@jackqdyulei
Copy link
Collaborator

Hey have you fixed the issue? Can we close this?

@archidvignesh
Copy link
Author

archidvignesh commented Sep 29, 2021 via email

@jackqdyulei
Copy link
Collaborator

Hey could you at least give us following information?

  1. SDK version
    com.google.mlkit:face-detection:

  2. The Device API level where it breaks. (Since you added @RequiresAPI and have special code on API level < 28 )

@archidvignesh
Copy link
Author

archidvignesh commented Sep 29, 2021 via email

@jackqdyulei
Copy link
Collaborator

Thanks for your report. The root cause is ImageDecoder.decodeBitmap(source) by default returns HARDWARE bitmap, which is not support by ML Kit.

We will consider whether to support it, or explicitly mention it in javadoc.

@archidvignesh
Copy link
Author

archidvignesh commented Oct 6, 2021 via email

@jackqdyulei jackqdyulei self-assigned this Oct 6, 2021
@jackqdyulei
Copy link
Collaborator

Issue been fixed internally and will be available in future release.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants