Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

What is the process of MNN :: CV :: ImageProcess module processing images? #854

Closed
TCBocean opened this issue Jun 1, 2020 · 4 comments
Closed

Comments

@TCBocean
Copy link

TCBocean commented Jun 1, 2020

My expected processing flow is to first reduce the picture to the input size of the network, then divide by 255.0, subtract mean, and divide by std.
My code is:

    auto input = interpreter->getSessionInput(session, nullptr);
    int width = 1436;
    int height = 1920;
    MNN::CV::ImageProcess::Config config_data;
    config_data.filterType = MNN::CV::BILINEAR;
    const float mean_vals[3] = {140.97361f, 129.48639f, 125.60532f};
    const float norm_vals[3] = {68.57847f, 66.457595f, 65.706699f};
    ::memcpy(config_data.mean, mean_vals, sizeof(mean_vals));
    ::memcpy(config_data.normal, norm_vals, sizeof(norm_vals));
    config_data.sourceFormat = MNN::CV::RGB;
    config_data.destFormat = MNN::CV::ImageFormat::RGB;
    std::unique_ptr<MNN::CV::ImageProcess> process(MNN::CV::ImageProcess::create(config_data));
    MNN::CV::Matrix matrix;
    matrix.setIdentity();
    matrix.postScale(width/352.0, height/352.0);
    process->setMatrix(matrix);

    jbyte *bufferData = env->GetByteArrayElements(jbufferData, NULL);
    if (bufferData == NULL) {
        MNN_ERROR("Error Buffer Null!\n");
        return JNI_FALSE;
    }
    process->convert((const unsigned char *)bufferData, width, height, 0, input); 
    env->ReleaseByteArrayElements(jbufferData, bufferData, 0);

The mean and std here are the result of multiplying by 255, because my mean and std are obtained in the range of 0 ~ 1.
I am not sure if the processing flow of MNN :: CV :: ImageProcess module is divided by 255.

    auto inputtensor = new MNN::Tensor(input, MNN::Tensor::TENSORFLOW);
    float * temp = inputtensor->host<float>();
    jbyte *bufferData = env->GetByteArrayElements(jbufferData, NULL);
    if (bufferData == NULL) {
        MNN_ERROR("Error Buffer Null!\n");
        return JNI_FALSE;
    }
    detector_component->normalization_source_image((uint8_t const*)bufferData, temp,3, 0, 1);
    input->copyFromHostTensor(inputtensor);

I used the interface I wrote and processed it according to the process I mentioned before, and I can get the correct results.
So I guess that my code can't be executed according to my expectations. Can someone teach me how to configure MNN :: CV :: ImageProcess module before resize, then reduce maen, and then divide std?
Thank you very much~

@jxt1234
Copy link
Collaborator

jxt1234 commented Jun 5, 2020

You should div normal by 255
68.57847f, 66.457595f, 65.706699f -> (68.57847f, 66.457595f, 65.706699f) / 255.0f

@TCBocean
Copy link
Author

TCBocean commented Jun 5, 2020

@jxt1234
Do you mean the processing flow of MNN::CV::ImageProcess module is as follows?
((img - mean) / 255) / norm

@jxt1234
Copy link
Collaborator

jxt1234 commented Jun 5, 2020

no. It's dst = (img - mean) * normal

@TCBocean
Copy link
Author

TCBocean commented Jun 5, 2020

I understand

@TCBocean TCBocean closed this as completed Jun 5, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants