Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Get raw depth data in Android #10310

Closed
oddityyyy opened this issue Mar 12, 2022 · 3 comments
Closed

Get raw depth data in Android #10310

oddityyyy opened this issue Mar 12, 2022 · 3 comments
Labels

Comments

@oddityyyy
Copy link

Issue Description

Hello,
I want to know how to get raw depth data like this in android:
image
I have tried some ways like this:
`try (FrameReleaser fr = new FrameReleaser()) {
try (FrameSet frames = mPipeline.waitForFrames().releaseWith(fr)) {

            Align align = new Align(StreamType.COLOR);
            
            String folderPath = "/storage/emulated/0/graduateDesign/";
            String profile = "lhh";
            SimpleDateFormat sdf = new SimpleDateFormat("yyyyMMdd_HHmmss");
            String currentDateAndTime = sdf.format(new Date());
          
            try (FrameSet processed = frames.applyFilter(align).releaseWith(fr).applyFilter(mColorizer).releaseWith(fr)) {

                Frame depthColored = processed.first(StreamType.DEPTH,StreamFormat.RGB8).releaseWith(fr);
                DepthFrame depthColorFrame = depthColored.as(Extension.DEPTH_FRAME);
                depthColorFrame.releaseWith(fr);

                Frame color = processed.first(StreamType.COLOR).releaseWith(fr);
                VideoFrame colorFrame = color.as(Extension.VIDEO_FRAME);
                colorFrame.releaseWith(fr);

                Frame depth = processed.first(StreamType.DEPTH,StreamFormat.Z16).releaseWith(fr);
                DepthFrame depthFrame = depth.as(Extension.DEPTH_FRAME);
                depthFrame.releaseWith(fr);

                // capturing color image
                Mat mRGB = new Mat(colorFrame.getHeight(), colorFrame.getWidth(), CvType.CV_8UC3);
                byte[] return_buff = new byte[colorFrame.getDataSize()];
                colorFrame.getData(return_buff);
                mRGB.put(0, 0, return_buff);

                Bitmap bmp = Bitmap.createBitmap(mRGB.cols(), mRGB.rows(), Bitmap.Config.ARGB_8888);
                org.opencv.android.Utils.matToBitmap(mRGB, bmp);

                File imgFile = new File(folderPath, profile + "colorFrame-" + currentDateAndTime + ".jpg");
                try {
                    FileOutputStream out2 = new FileOutputStream(imgFile);
                    bmp.compress(Bitmap.CompressFormat.JPEG, 100, out2);
                    out2.flush();
                    out2.close();
                } catch (Exception e) {
                    e.printStackTrace();
                }

                // capturing colorized depth image 
                Mat mDepthColorized = new Mat(depthColorFrame.getHeight(), depthColorFrame.getWidth(), CvType.CV_8UC3);
                byte[] return_buff_DepthColor = new byte[depthColorFrame.getDataSize()];
                depthColorFrame.getData(return_buff_DepthColor);
                mDepthColorized.put(0, 0, return_buff_DepthColor);

                Bitmap bmpDepthColor = Bitmap.createBitmap(mDepthColorized.cols(), mDepthColorized.rows(), Bitmap.Config.ARGB_8888);
                org.opencv.android.Utils.matToBitmap(mDepthColorized, bmpDepthColor);

                File imgFileDepthColor = new File(folderPath, profile + "depthFrame-" + currentDateAndTime + ".jpg");
                try {
                    FileOutputStream out3 = new FileOutputStream(imgFileDepthColor);
                    bmpDepthColor.compress(Bitmap.CompressFormat.JPEG, 100, out3);
                    out3.flush();
                    out3.close();
                } catch (Exception e) {
                    e.printStackTrace();
                }

                
                // capturing raw data
                Mat mDepth = new Mat(depthFrame.getHeight(), depthFrame.getWidth(), CV_8UC1);//8UC1 //8UC3  //16UC1
                byte[] return_buff_Depth = new byte[depthFrame.getDataSize()];
                depthFrame.getData(return_buff_Depth);
                mDepth.put(0, 0, return_buff_Depth);

                Bitmap bmpDepth = Bitmap.createBitmap(mDepth.cols(), mDepth.rows(), Bitmap.Config.ARGB_8888);
                org.opencv.android.Utils.matToBitmap(mDepth, bmpDepth);

                File imgFileDepth = new File(folderPath, profile + "grayFrame-" + currentDateAndTime + ".png");
                try {
                    FileOutputStream out4 = new FileOutputStream(imgFileDepth);
                    bmpDepth.compress(Bitmap.CompressFormat.PNG, 100, out4);
                    out4.flush();
                    out4.close();
                } catch (Exception e) {
                    e.printStackTrace();
                }`

But the resuit is:
image
image
image
It looks like raw grey depth image going wrong.
Could you give me some advice, thanks!

@MartyG-RealSense
Copy link
Collaborator

Hi @oddityyyy There are very few references about achieving on Android the type of depth map that you are aiming for, unfortunately. Most of the references are linked to #5446 that you have already likely visited because it has the depth map image that you shared above.

A RealSense programming expert shared Java advice with a RealSense user in #5571 (comment) that led to that RealSense user achieving in #5571 (comment) a depth map image below that resembles what you want.

image

@oddityyyy
Copy link
Author

@MartyG-RealSense Thanks, #5571 is useful to me, the problem has been solved.

@MartyG-RealSense
Copy link
Collaborator

Thanks very much @oddityyyy for the update!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants