Is there any way I can get the preview size selected by camera view. I dont need the list of previews supported but the preview size which is eventually chosen after all the logic done. I have an overlay over camera view which needs preview width and height

[amit2273]

How do I?

Describe your problem here. Please, read the docs first.
Questions not strictly related to CameraView should be asked elsewhere.

Version used

CameraView exact version.

[natario1]

Just call getWidth() or getHeight() as with any view in Android

[amit2273]

Zest of problem is that I am using camera view over which i am using MLKit for face detection using Frame Processor. However, My Bounding box is getting shifted to the left in many devices.

So, I have debugged a lot in your code and found that
in onMeasure () you have code something like below :

    final int widthValue = MeasureSpec.getSize(widthMeasureSpec);
    final int heightValue = MeasureSpec.getSize(heightMeasureSpec);
    final float previewWidth = mLastPreviewStreamSize.getWidth();
    final float previewHeight = mLastPreviewStreamSize.getHeight();

What I'm looking for is previewWidth and previewHeight value, so that I can pass it to my overlay for setting camera info:

fun setCameraInfo(previewWidth: Int, previewHeight: Int, facing: Facing) {
synchronized(lock) {
this.previewWidth = previewWidth
this.previewHeight = previewHeight
this.facing = facing
}
postInvalidate()
}
for eg: for a phone on which it is getting shifted to left has following values :
widthValue =1080
heightValue = 1911
previewWidth= 1440
previewHeight= 1920

IF i'm passing widthValue and HeightValue, my bounding box is getting shifted to the left(as you can see above difference in widthValue and previewWidth value), however when I debugged and hardcoded the values of previewWidth and previewHeight(because its not provided as much as I know in this lib), its working fine in all the device.

Can you suggest me how can I take these values??

[amit2273]

there should be some way for anyone using an overlay above cameraView to get previewWidth and previewHeight.
We get picture size but not preview . I think you should expose Preview Size choosen by camera view as well.

[Sachin1503]

Hi
I have been struggling with the same issue since long. After trying what You @amit2273 have mentioned, it worked like a charm.
did you find how can we use previewWidth and previewHeight in our overLay.
or is there any alternative measurement which are exposed and we can use for previewWidth and previewHeight @natario1

[natario1]

You are processing a frame of size frame.getSize() and drawing it onto a view of size cameraView.getWidth() (or height), which can be bigger or smaller - of course you have to scale. So I don't see how the preview stream size would get into the equation.

[amit2273]

Hi @natario1 ,

I hope this can give you a better picture:

Our Moto :
we are creating an experience where we will trigger "takePictureSnapshot()" when a user is inside the oval which is drawn over camera view. Please find screenshot attached for better reference.

xml Design :
this is how we have our xml designed for drawing an overlay above camera view. We are using Graphic overLay.

<com.otaliastudios.cameraview.CameraView
android:id="@+id/camera_view"
android:layout_width="0dp"
android:layout_height="0dp"
android:adjustViewBounds="true"
android:keepScreenOn="true"
app:cameraAudio="off"
app:cameraFacing="front"
app:cameraFlash="auto"
app:cameraMode="picture"
app:layout_constraintBottom_toBottomOf="@id/bottom_view"
app:layout_constraintLeft_toLeftOf="parent"
app:layout_constraintRight_toRightOf="parent"
app:layout_constraintTop_toTopOf="@id/bottom_view">

        <com.incred.customer.cameracapture.selfie.customview.GraphicOverlay
            android:id="@+id/overlay"
            android:layout_width="wrap_content"
            android:layout_height="wrap_content" />
    </com.otaliastudios.cameraview.CameraView>

What is Graphic overlay :

A view which renders a series of custom graphics to be overlayed on top of an associated preview

• (i.e., the camera preview). The creator can add graphics objects, update the objects, and remove
• them, triggering the appropriate drawing and invalidation within the view.

We are using it to create our Bounding Box of face provided by ML face recognition Kit.

Link : https://github.com/googlesamples/android-vision/blob/master/visionSamples/FaceTracker/app/src/main/java/com/google/android/gms/samples/vision/face/facetracker/ui/camera/GraphicOverlay.java

ML Kit Bounding Box :

We are using firebase Ml kit face recognition to create a bounding box of the face detected. We are also scaling x and y coordinate from the preview's coordinate system to the view coordinate system.

    val face = firebaseVisionFace ?: return
    // An offset is used on the Y axis in order to draw the circle, face id and happiness level in the top area
    // of the face's bounding box
    val x = translateX(face.boundingBox.centerX().toFloat())
    val y = translateY(face.boundingBox.centerY().toFloat())
    // Draws a bounding box around the face.
    val xOffset = scaleX(face.boundingBox.width() / 2.0f)
    val yOffset = scaleY(face.boundingBox.height() / 2.0f)
    val left = x - xOffset
    val top = y - yOffset
    val right = x + xOffset
    val bottom = y + yOffset
    canvas.drawRect(left, top, right, bottom, boxPaint)

Not to mention but we are passing the frame size as well to the ML Kit

addFrameProcessor {
val frameMetadata =
FrameMetadata(it.size.width, it.size.height, it.rotation / 90, Facing.FRONT)
}

Issue :
Issue is our bounding box is getting shifted to the left in many devices . please find screenshot attached.

We only call "takePictureSnapshot()" when our bounding box is inside the oval.

Probable cause :
If you check out GraphicOverlay.java thoroughly (1 min read)(used widely by developers for drawing overlay over camera view) for which I have mentioned a link above, uses scaling and mirroring of the graphics relative the camera's preview properties and not view properties. Because of which the bounding box is drawn according to the width and Height of the view and not the preview.
If we are providing preview Size which is expected, it works fine.

[natario1]

I understand that it works with preview stream size, but it's just by coincidence. You should probably call setCameraInfo(frame.size.width, frame.size.height), or if this doesn't work, setCameraInfo(0, 0) and do the scaling yourself.

I also imagine that this graphic overlay should be match_parent to work correctly.

[reactivedroid]

We recently used CameraView for MLKit to detect faces live. When we launched the app in production, we started receiving multiple issues where the bounding box was not overlaying properly. This affected a lot of issues on production and we had to revert the release.

@amit2273 Going by your finding, it looks like exposing previewWidth and previewHeight to CameraSource will solve the issue.
@natario1 It would be great if you can give some of your time to this so that whole Android community can be benefitted.

[amit2273]

Hi @natario1 ,

I have tried setCameraInfo(frame.size.width, frame.size.height) and it doesn't work.
also, my bad in pasting wrong code snipplet as we are using match_parent for graphic overlay.

Also, I have used firebase sample tutorial for implementing the same
Link : https://github.com/firebase/quickstart-android/tree/master/mlkit

where they are using their own camera source
https://github.com/firebase/quickstart-android/blob/master/mlkit/app/src/main/java/com/google/firebase/samples/apps/mlkit/common/CameraSource.java

When I went through the internal working of code, they are calculating the preview Width and preview Height , which you are also doing inside the your library and setting it to graphic overlay in CameraSourcePreview.java 's startIfReady() method :
https://github.com/firebase/quickstart-android/blob/master/mlkit/app/src/main/java/com/google/firebase/samples/apps/mlkit/common/CameraSourcePreview.java

@SuppressLint("MissingPermission")
private void startIfReady() throws IOException {
if (startRequested && surfaceAvailable) {
if (PreferenceUtils.isCameraLiveViewportEnabled(context)) {
cameraSource.start(surfaceView.getHolder());
} else {
cameraSource.start();
}
requestLayout();

  if (overlay != null) {
    Size size = cameraSource.getPreviewSize();
    int min = Math.min(size.getWidth(), size.getHeight());
    int max = Math.max(size.getWidth(), size.getHeight());
    if (isPortraitMode()) {
      // Swap width and height sizes when in portrait, since it will be rotated by
      // 90 degrees
      overlay.setCameraInfo(min, max, cameraSource.getCameraFacing());
    } else {
      overlay.setCameraInfo(max, min, cameraSource.getCameraFacing());
    }
    overlay.clear();
  }
  startRequested = false;
}

}

I'm just trying to replicate the same code .
What I think exposing preview width and height does the work for any one following firebase's sample project for ML Kit (which people widely follow) using your Lib and wouldn't have to write their own overlay and scaling Logic.
Would be better for android family.

[natario1]

Yeah well you have to rotate them. It's much cleaner in CameraView than it is in their app:

int width = frame.getRotationToView() % 180 == 0 ? frame.getWidth() : frame.getHeight();
int height = frame.getRotationToView() % 180 == 0 ? frame.getHeight() : frame.getWidth();
overlay.setCameraInfo(width, height, frame.getFacing())

And then you will have to change something else in your code to make it work correctly. For example, ML results are rotated based on the frame rotation that you pass to the ML detector, and you might have to rotate the detector results as well. GraphicOverlay does not care about rotation and this is wrong.
It's not a well written class.

I don't have much time to help you, feel free to open a PR with getPreviewStreamSize() and use it, but I would not use it in production as it makes no sense. Frame size and view size should be used. The preview stream size does never make it into the equation.

[amit2273]

Thanks @natario1 for your time. Really appreciative.
with some minor changes and using frame width and height , I was able to achieve what I was looking for.
thanks once again for your insights

[hpkaushik121]

Thanks @natario1 for your time. Really appreciative.
with some minor changes and using frame width and height , I was able to achieve what I was looking for.
thanks once again for your insights

Hi @amit2273 can you please tell what exactly you have done i am stuck with the same

[jeffreyfjohnson]

@hpkaushik121 I was struggling with the same thing. What I found may help you.

When you set the CameraView to wrap_content, the frames the frame processor spits out are exactly the size of the CameraView in the layout, and vice-versa. This is key (and also not how it works in some other popular Camera libraries).

To illustrate: Let's say you have a screen that is 300px wide by 400px high. Let's also say that you have your CameraView centered in that container, but the output from the camera doesn't have the exact same ratio. So the CameraView is sized to 300px wide by 350px high. Since it is centered in the container, there are 25px of black bars on the top and bottom of the camera preview.

Let's also say you have an overlay view on top of the CameraView that displays rectangles around objects detected in frame processing. If you can make your overlay view the exact same size as the CameraView (300x350 in this example), then a bounding box detected in the frame should overlay perfectly on the CameraView. But if you have to make your overlay view the size of the container (300x400), you must account for the 25 px of offset from the top of the container to the top of the CameraView

[ghost]

Hi @amit2273 could you share a sample or tell what exactly you have done?
Please help me out.

[amit2273]

Hi @abhishektiwari-kiwi, @hpkaushik121
with frame rotation of 180, frame width and height also needed to be adjusted and rotated for overlay to perfectly sit over the camera preview.

fun setCameraInfo(previewWidth: Int, previewHeight: Int, facing: Facing) {
synchronized(lock) {
this.previewWidth = previewWidth
this.previewHeight = previewHeight
this.facing = facing
}
postInvalidate()
}

So, Please interchange the values of frame.size.width and frame.size.height when passing it to camera overlay.
cameraView.apply {
addFrameProcessor {frame->

            cameraOverlay.setCameraInfo(
                frame.size.height,          // frame height passed to previewWidth of Camera View
                frame.size.width,            // frame width passed to previewHeight of Camera View
                Facing.FRONT
            )
        }
    }