Animal Pose estimation using KAPAO

[abhigoku10]

@Kanav123 thanks for the open source code base I had a few queries

1. can we train kapo for animal pose estimation on the animal pose dataset? if so what are the changes to be made
2. can we have hand pose, facial landmarks, and body pose from the same architecture? can we modify kapo architecture for these things if so what is ur suggestions

thanks in advance

[wmcnally]

Steps for training on new dataset:

1. Create a new dataset configuration file similar to data/coco-kp.yaml and data/crowdpose.yaml
2. Modify utils/labels.py and val.py to suit your needs.

[abhigoku10]

@wmcnally thanks for response , did as you have mentioned but i am getting error in val.py file in line number 285, i have verified the values of coco and results which are getting passed into "eval = COCOeval(coco, result, iouType='keypoints')" but i am getting this following error
Traceback (most recent call last):
File "train.py", line 601, in
main(opt)
File "train.py", line 499, in main
train(opt.hyp, opt, device)
File "train.py", line 360, in train
flips=val_flips)
File "/usr/local/lib/python3.7/dist-packages/torch/autograd/grad_mode.py", line 28, in decorate_context
return func(*args, kwargs)
File "/content/kapao/val.py", line 291, in run
eval.evaluate()
File "/usr/local/lib/python3.7/dist-packages/pycocotools/cocoeval.py", line 149, in evaluate
for imgId in p.imgIds
File "/usr/local/lib/python3.7/dist-packages/pycocotools/cocoeval.py", line 150, in
for catId in catIds}
File "/usr/local/lib/python3.7/dist-packages/pycocotools/cocoeval.py", line 231, in computeOks
e = (dx2 + dy**2) / vars / (gt['area']+np.spacing(1)) / 2
ValueError: operands could not be broadcast together with shapes (20,) (17,)
0% 0/376 [01:08<?, ?it/s]
ERROR:torch.distributed.elastic.multiprocessing.api:failed (exitcode: 1) local_rank: 0 (pid: 2707) of binary: /usr/bin/python3
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/usr/local/lib/python3.7/dist-packages/torch/distributed/launch.py", line 193, in
main()
File "/usr/local/lib/python3.7/dist-packages/torch/distributed/launch.py", line 189, in main
launch(args)
File "/usr/local/lib/python3.7/dist-packages/torch/distributed/launch.py", line 174, in launch
run(args)
File "/usr/local/lib/python3.7/dist-packages/torch/distributed/run.py", line 692, in run
)(*cmd_args)
File "/usr/local/lib/python3.7/dist-packages/torch/distributed/launcher/api.py", line 116, in call
return launch_agent(self._config, self._entrypoint, list(args))
File "/usr/local/lib/python3.7/dist-packages/torch/distributed/launcher/api.py", line 246, in launch_agent
failures=result.failures,
torch.distributed.elastic.multiprocessing.errors.ChildFailedError:

---

        train.py FAILED            

=======================================
Root Cause:
[0]:
time: 2021-12-28_19:59:36
rank: 0 (local_rank: 0)
exitcode: 1 (pid: 2707)
error_file: <N/A>
msg: "Process failed with exitcode 1"

Other Failures:
<NO_OTHER_FAILURES>

Please share your thougths

[wmcnally]

If you are using a custom dataset, you cannot use the COCO evaluation toolbox. You should create a new val.py that manually computes the accuracy or loss and return it.

[abhigoku10]

@wmcnally i am using a standard animal pose dataset ' https://github.com/AlexTheBad/AP-10K " for that also should i manually compute the accuracy and loss ?

[wmcnally]

I see. It looks like the annotations are in COCO format so you should be able to use the COCO evaluation tools. The error is a shape mismatch, 20 vs 17. This should provide some clues as to where the bug is.

[abhigoku10]

@wmcnally thanks for the response , i was able to figure out where the issue was , it was in cocoeval.py in line number 523 "self.kpt_oks_sigmas = np.array([.26, .25, .25, .35, .35, .79, .79, .72, .72, .62,.62, 1.07, 1.07, .87, .87, .89, .89])/10.0" there are only 17 values due to which the error in mis match was happening
Please share your thougths

[wmcnally]

Aren't there only 17 keypoints in the animal pose dataset? I'm confused as to why you have 20.

[abhigoku10]

@wmcnally oh in animal pose dataset there are 20 key points , should i modify it to have 17 key points only

[abhigoku10]

@wmcnally based on your suggestion i trained the model with 17 key points and below is the output which i a getting , from the observations key point are getting detected well , but the bounding box are not proper , when debugged it felt that the mosaic augmentation is creating the issue . Would you share ur thougths
Output results:

Moasic augmentation output

Please share your thougths

[wmcnally]

The bounding boxes don't look correct in your training images. Mosaic augmentation is likely not the issue. It is possible that your bounding boxes are in a different format. You should check and fix the format if necessary.