Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

"Batched output tensor has 0 dimensions" problem when post only one image with '--enable_batching=true' #116

Closed
troycheng opened this issue Nov 8, 2018 · 3 comments

Comments

@troycheng
Copy link

@troycheng troycheng commented Nov 8, 2018

At first Thanks for sharing this project.

Follow the instruction I successfully set up the tensorflow_model_server and tried to post one or more images in the same request, it worked well as follows

REQ1: request only one image at a time

b=\`base64 -w 0 ./test.jpg\`; j='{"inputs":{"input":[{"b64":"'"$b"'"}]}}'; curl -X POST -d "$j" http://localhost:8600/v1/models/aocr:predict
{
    "outputs": {
        "probability": 21.5299,
        "output": "420H2N20"
    }
}

REQ2: request more than one image at a time

b=`base64 -w 0 ./test.jpg`; j='{"inputs":{"input":[{"b64":"'"$b"'"},{"b64":"'"$b"'"},{"b64":"'"$b"'"}]}}'; curl -X POST -d "$j" http://192.168.16.207:8600/v1/models/aocr:predict
{
    "outputs": {
        "probability": [
            21.5299,
            21.5299,
            21.5299
        ],
        "output": [
            "420H2N20",
            "420H2N20",
            "420H2N20"
        ]
    }
} 

However, when I trying the same requests with '--enable_batching=true', REQ1 returns an Batched output tensor has 0 dimensions error and REQ2 works well

REQ1:

b=`base64 -w 0 ./test.jpg`; j='{"inputs":{"input":[{"b64":"'"$b"'"}]}}'; curl -X POST -d "$j" http://192.168.16.207:8601/v1/models/aocr:predict
{ "error": "Batched output tensor has 0 dimensions" }

REQ2:

b=`base64 -w 0 ./test.jpg`; j='{"inputs":{"input":[{"b64":"'"$b"'"},{"b64":"'"$b"'"},{"b64":"'"$b"'"}]}}'; curl -X POST -d "$j" http://192.168.16.207:8601/v1/models/aocr:predict
{
    "outputs": {
        "probability": [
            21.5299,
            21.5299,
            21.5299
        ],
        "output": [
            "420H2N20",
            "420H2N20",
            "420H2N20"
        ]
    }
}

I tried to solve it but failed. Although with batching turned off it can work, I still want to know why...
It can't make sense that using batch means it must be request more than one image at a time...

I'm not sure it's an issue about this project or tf-serving, and sorry if it is placed wrong. I would be grateful if someone can help.

@MariusMez

This comment has been minimized.

Copy link
Contributor

@MariusMez MariusMez commented Nov 8, 2018

Hi,

It seem's more related to TF Serving than attention-ocr.

You can read my last comment here: #94 (comment)

--> I don't understand well the differences between the two options explained in the tensorflow serving documentation about how specifying input tensors in row or column format. Maybe your tests can help to understand this tensorflow serving functionnality (you can test with row format check if the error still appear)

@troycheng

This comment has been minimized.

Copy link
Author

@troycheng troycheng commented Nov 9, 2018

Thanks, @MariusMez. I tried row format and the result is: REQ2 can work with or without batching, while REQ1 failed in both modes.

REQ1:

b=`base64 -w 0 ./test.jpg`; j='{"instances":[{"b64":"'"$b"'"}]}'; curl -X POST -d "$j" http://192.168.16.207:8601/v1/models/aocr:predict
{"error": "Tensor name: probability has no shape information"} // normal mode
{"error": "Batched output tensor has 0 dimensions"} // batching mode

REQ2:

b=`base64 -w 0 ./test.jpg`; j='{"instances":[{"b64":"'"$b"'"},{"b64":"'"$b"'"},{"b64":"'"$b"'"}]}'; curl -X POST -d "$j" http://192.168.16.207:8601/v1/models/aocr:predict
{
    "predictions": [
        {
            "probability": 21.5299,
            "output": "420H2N20"
        },
        {
            "probability": 21.5299,
            "output": "420H2N20"
        },
        {
            "probability": 21.5299,
            "output": "420H2N20"
        }
    ]
} // in both modes

It seems the row or column format is just the way to organize the input data and can work in both modes when posting multiple images. Posting one image will cause some dimension problem. It should be tf-serving's problem

@emedvedev emedvedev closed this Nov 24, 2018
@emedvedev

This comment has been minimized.

Copy link
Owner

@emedvedev emedvedev commented Nov 24, 2018

Closing the issue, as it really is the problem of TF Serving. Nothing I can do here, sadly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.