Pose estimator - invalid output

[GiTackers]

Hello everyone.

We have a P1467-LE camera. We're trying example 'pose-estimator-with-flask', but when we execute

docker-compose --tlsverify --host tcp://$DEVICE_IP:$DOCKER_PORT --env-file ./config/env.$ARCH.$CHIP up

it returns

Error response from daemon: pull access denied for acap4-pose-estimator-python, repository does not exist or may require 'docker login': denied: requested access to the resource is denied

and, in the camera log, we find:

2023-06-19T17:36:33.498+02:00 axis-b8a44f82b9a9 [ INFO ] dockerdwrapper[12194]: time="2023-06-19T17:36:33.496465080+02:00" level=error msg="Not continuing with pull after error: errors:\ndenied: requested access to the resource is denied\nunauthorized: authentication required\n"
2023-06-19T17:36:33.498+02:00 axis-b8a44f82b9a9 [ INFO ] dockerdwrapper[12194]: time="2023-06-19T17:36:33.496587200+02:00" level=info msg="Ignoring extra error returned from registry: unauthorized: authentication required"
2023-06-19T17:36:33.607+02:00 axis-b8a44f82b9a9 [ INFO ] dockerdwrapper[12194]: time="2023-06-19T17:36:33.606334960+02:00" level=warning msg="error aborting content ingest" digest="sha256:3fda7495b281a86f22bccae769d752db33a9e89af264570a1853b107390599a6" error="context canceled" remote="docker.io/axisecp/acap-runtime:1.2.0-aarch64-containerized"
2023-06-19T17:36:33.607+02:00 axis-b8a44f82b9a9 [ INFO ] dockerdwrapper[12194]: time="2023-06-19T17:36:33.606507960+02:00" level=warning msg="Error persisting manifest" digest="sha256:3fda7495b281a86f22bccae769d752db33a9e89af264570a1853b107390599a6" error="error writing manifest to content store: failed to send write: EOF: unknown"

Any idea about it?

Thank you very much in advance.

[Corallo]

Hello @GiTackers
Quite unexpected issue.

First it looks l that docker in the camera doesn't have the application image.
Please check with
docker --tlsverify --host tcp://$DEVICE_IP:$DOCKER_PORT images
If the image acap4-pose-estimator-python was correctly uploaded in the camera

[GiTackers]

OK, the problem was that there was no more space left on the device: solved with a larger sd card.

But, now, another question.

1. When we connect to http://$DEVICE_IP:5000, log in the terminal shows always key points array and their confidences, even if there is no one in the scene.
2. However, even when there is a person in the scene, no keypoints are drawn in the video stream.

Any idea about it?

Thank you very much in advance.

[Corallo]

1. That is because of how the model is. It will always print a vector as output with location and confidence of each keypoint. If there are no people in the frame, the confidence will be just close to zero.
2. Strange. What is the confidence of the points in the output? Is your firmware update to the latest version?

[GiTackers]

Good morning and thank you for your response!

Yes, the firmware is the latest one (11.4.63), and keypoints/confidences are the same without or with person in the scene.

Without person it prints:

Key points: [[0.29087663 0.4014917 ]
pose-estimator-with-flask-pose-estimator-1 | [0.5407847 0.4629445 ]
pose-estimator-with-flask-pose-estimator-1 | [0.5407847 0.4629445 ]
pose-estimator-with-flask-pose-estimator-1 | [0.3113609 0.36052316]
pose-estimator-with-flask-pose-estimator-1 | [0.52030045 0.31545776]
pose-estimator-with-flask-pose-estimator-1 | [0.52030045 0.48342878]
pose-estimator-with-flask-pose-estimator-1 | [0.52030045 0.48342878]
pose-estimator-with-flask-pose-estimator-1 | [0.4793319 0.5858501 ]
pose-estimator-with-flask-pose-estimator-1 | [0.45884764 0.4629445 ]
pose-estimator-with-flask-pose-estimator-1 | [0.4793319 0.5858501 ]
pose-estimator-with-flask-pose-estimator-1 | [0.45884764 0.4629445 ]
pose-estimator-with-flask-pose-estimator-1 | [0.45884764 0.5858501 ]
pose-estimator-with-flask-pose-estimator-1 | [0.52030045 0.4629445 ]
pose-estimator-with-flask-pose-estimator-1 | [0.64320606 0.5243973 ]
pose-estimator-with-flask-pose-estimator-1 | [0.6677872 0.50391304]
pose-estimator-with-flask-pose-estimator-1 | [0.93817955 0.61043125]
pose-estimator-with-flask-pose-estimator-1 | [0.89311415 0.50391304]]
pose-estimator-with-flask-pose-estimator-1 | Confidences: [0.07374337 0.01638742 0.01638742 0.07374337 0.04096854 0.02048427
pose-estimator-with-flask-pose-estimator-1 | 0.01638742 0.01638742 0.01229056 0.02048427 0.02048427 0.09422764
pose-estimator-with-flask-pose-estimator-1 | 0.07374337 0.05735596 0.05735596 0.07374337 0.12290562]

With person it prints:

Key points: [[0.29087663 0.4014917 ]
pose-estimator-with-flask-pose-estimator-1 | [0.5407847 0.4629445 ]
pose-estimator-with-flask-pose-estimator-1 | [0.5407847 0.4629445 ]
pose-estimator-with-flask-pose-estimator-1 | [0.3113609 0.36052316]
pose-estimator-with-flask-pose-estimator-1 | [0.52030045 0.31545776]
pose-estimator-with-flask-pose-estimator-1 | [0.52030045 0.48342878]
pose-estimator-with-flask-pose-estimator-1 | [0.52030045 0.48342878]
pose-estimator-with-flask-pose-estimator-1 | [0.4793319 0.5858501 ]
pose-estimator-with-flask-pose-estimator-1 | [0.45884764 0.4629445 ]
pose-estimator-with-flask-pose-estimator-1 | [0.4793319 0.5858501 ]
pose-estimator-with-flask-pose-estimator-1 | [0.45884764 0.4629445 ]
pose-estimator-with-flask-pose-estimator-1 | [0.45884764 0.5858501 ]
pose-estimator-with-flask-pose-estimator-1 | [0.52030045 0.4629445 ]
pose-estimator-with-flask-pose-estimator-1 | [0.64320606 0.5243973 ]
pose-estimator-with-flask-pose-estimator-1 | [0.6677872 0.50391304]
pose-estimator-with-flask-pose-estimator-1 | [0.93817955 0.61043125]
pose-estimator-with-flask-pose-estimator-1 | [0.89311415 0.50391304]]
pose-estimator-with-flask-pose-estimator-1 | Confidences: [0.07374337 0.01638742 0.01638742 0.07374337 0.04096854 0.02048427
pose-estimator-with-flask-pose-estimator-1 | 0.01638742 0.01638742 0.01229056 0.02048427 0.02048427 0.09422764
pose-estimator-with-flask-pose-estimator-1 | 0.07374337 0.05735596 0.05735596 0.07374337 0.12290562]

So, the same output.

Any idea about it?

Thank you very much in advance.

[GiTackers]

Hello everyone.

We even reset the camera and reinstall all over again but the same problem happens.

Any idea about it?

Thank you very much in advance.

[Corallo]

Hi @GiTackers

We are going to look into this.
Are you using the example as it is, or did you make changes?
When you connect to the flask video stream, do you see the video working properly?

[GiTackers]

Hello Corallo and thank you for your response.

Yes, we are using example as it is, without changes: we followed this https://github.com/AxisCommunications/acap-computer-vision-sdk-examples/tree/main/pose-estimator-with-flask.

Only one thing. Even if in the live stream resolution (width x height) is 2592x1944, in the flask video stream it appears as part of that full image, it shows just the central region: that is, full height but width reduced (equally to both right and left). We don't know if this behaviour is intentional.

Thank you very much in advance.

[Corallo]

@GiTackers
We reproduced the issue, it looks like it is a firmware problem, that occours in 11.3 or after.
Unfrotunatly it will take some time from our side to investigate and solve the issue.

In the mean time, as a work around you can try with an older firmware that doesn't have this issue:
http://ftp.axis.com/pub_soft/MPQT/P1467-LE/11_0_89/
with 11.0 you should not have this problem.

If latency is not an issue and you need to use the latest firmware, another workaround is to switch to CPU for running the inference (change configuration file on the docker compose up command).

Regarding the resolution, the model takes in in put a squared aspect ratio. To avoid distortion in the image we are cutting only the central part of it. Another option would be to pad the height and obtain the same result without loosing the view area.

[GiTackers]

Hello Corallo and thank you for your response.

We installed that firmware (11.0.89), but it's incompatible with Docker ACAP.

Following the guide, at the start, giving the command

ssh root@$DEVICE_IP 'command -v containerd >/dev/null 2>&1 && echo Compatible with Docker ACAP || echo Not compatible with Docker ACAP'

returns "Not compatible with Docker ACAP".

In fact, in the "App" section of the camera, flagging the "Docker Daemon" returns "Unable to start".

Thank you very much in advance.

[Corallo]

This is unfortunate...

I am afraid there's only 11.2 that has both docker daemon and doesn't have this issue with the model.
11.2 doesn't seem to be public anymore, you might want to try asking for it on a ticket and see if they can provide it to you.

In the meantime, we are investigating the issue to see if we can solve it for a future firmware release.

[GiTackers]

Hello Corallo,

just as information: after ticket, they send us P1468-LE_11.2.68 firmware. Using this bin, it works.

Thank you very much.

[Corallo]

Glad that worked, we'll keep this issue open until we solve it for future firmware

[Corallo]

This issue should be solved in the next firmware release 11.6

[Corallo]

The issue is fixed in 11.6