Color order of retrieved image #72
Description
Some information first:
- Zed SDK version: 2.8.0
- Cuda version: 10.0
- Language: C++
We recently noticed some confusion of the color order of retrieved images by Camera::retrieveImage function. It seems to us the retrieved image is in RGBA format, based on:
// Create an RGBA sl::Mat object
sl::Mat image_zed(zed.getResolution(), MAT_TYPE_8U_C4);
// Create an OpenCV Mat that shares sl::Mat data
cv::Mat image_ocv = slMat2cvMat(image_zed);
- in the
sl_zed/defines.hpp, inenum VIEW, theVIEW_LEFTwhich is passed as a parameter ofCamera::retrieveImage, also has the comment:
VIEW_LEFT, /**< Left RGBA image. Each pixel contains 4 usigned char (R,G,B,A). sl::MAT_TYPE_8U_C4. */
- and your example code:
void saveSbSImage(Camera& zed, std::string filename) {
...
cv::cvtColor(sbs_image, sbs_image, cv::COLOR_RGBA2RGB);
...
}
- I also had a look at the headers, and even found in
sl_core/utils/types.hpp:
/**
\class Vector4
\ingroup Core_group
\brief Represents a four dimensions vector for both CPU and GPU.
*/
template <typename T>
class /*@cond SHOWHIDDEN*/SL_CORE_EXPORT/*@endcond*/ Vector4 {
static const int nbElem = 4;
public:
/// @cond
union {
struct {
T x, y, z, w;
};
struct {
T r, g, b, a;
};
struct {
T ox, oy, oz, ow;
};
T v[nbElem];
};
/// @endcond
...
};
where struct { T r, g, b, a; }; indicates the pixel value order is RGBA. (This struct is used in the docstring of ERROR_CODE retrieveImage(Mat& mat, VIEW view = VIEW_LEFT, MEM type = MEM_CPU, int width = 0, int height = 0); in sl_zed/Camera.hpp, which is the returned type of sl::Mat::getValue:
Mat leftImage, depthView; //create sl::Mat objects to store the images
while (true) {
// Grab an image
if (zed.grab() == SUCCESS) { // A new image is available if grab() returns SUCCESS
zed.retrieveImage(leftImage, VIEW_LEFT); // Get the rectified left image
zed.retrieveImage(depthView, VIEW_DEPTH); // Get a grayscale preview of the depth map
//Display the center pixel colors
sl::uchar4 leftCenter;
leftImage.getValue<sl::uchar4>(leftImage.getWidth() / 2, leftImage.getHeight() / 2, &leftCenter);
std::cout << "leftImage center pixel R:" << (int)leftCenter.r << " G:" << (int)leftCenter.g << " B:" << (int)leftCenter.b << std::endl;
sl::uchar4 depthCenter;
depthView.getValue<sl::uchar4>(depthView.getWidth() / 2, depthView.getHeight() / 2, &depthCenter);
std::cout << "depthView center pixel R:" << (int)depthCenter.r << " G:" << (int)depthCenter.g << " B:" << (int)depthCenter.b << std::endl;
}
}
After testing a bit, we finally confirmed the pixel values are actually in BGRA order instead of RGBA order. However, searching in the documentations, code, issues, etc., we didn't find any information about this fact. I'd be nice if you could confirm the values are truly in BGRA order but not RGBA order and update the document to prevent similar confusion in the future. Thank you in advance for the reading and look forward to hearing from you.