Demo example is lagging and hand detection for single hand is not working well

[delebash]

Issue Description
Demo example is lagging and hand detection for single hand is not working well
Steps to Reproduce
Run Live Demo https://vladmandic.github.io/human/demo/index.html
Turn off all modules except HAND POSE
Change MAX OBJECTS to 1
Expected Behavior
Hand detector detects hand intermittently at a distance but should not have any problem.
**Environment
Windows 10
Microsoft Edge 86.0.622.63

Additional
May be related to issue #16

[vladmandic]

yes, it's a duplicate of #16
i'm still tweaking the model and not every daily update is a success.
i'll post here when i think results are satisfactory.

[delebash]

ok thanks

[vladmandic]

new version published, feedback is welcome

[djthegr8]

Can we use only the hand detection model?
I do not require the handpose one, and need just the inference from hand detection one.
Thanks

[vladmandic]

hand detection model has very poor reliability. trick to make this work is to figure out which box to grab and then cache it as long as possible as detection will frequently loose focus even if hand is perfect.

which is not that difficult if you only ever search for one hand, but caching and cache expiration gets tricky with multiple unreliable items in random order. that's why original handpose model works "more reliably".

so no, we can't just use hand detection - it doesn't solve anything.

[djthegr8]

Apologies, didn't make myself clear. Not relevant to this conversation, I wanted to use just the hand detector model, and was wondering if there was a way to do so...

[vladmandic]

you want to use original model? whole point of human is to improve on what's available, not to just repackage.
i'll get it right - remember this is still marked preproduction and my work on handmodel is less than 2 days total so far.

also, if you just repackage existing code, then there is no shared processing between them - each does it's own image pre-processing, etc. - and that takes far longer.

[djthegr8]

I wanted to use the handdetect.json and handdetect.bin files given in /models/ as my target application doesn't require landmarking

[vladmandic]

so just the hand box and no finger detection?
yes, i could expose that as a configurable value.

[djthegr8]

Could you please do so? Thanks

[djthegr8]

And, also, what is the output shape of the detection model???

[vladmandic]

i just pushed new code, quite a lot of changes in hand model processing

new config value config.hand.landmarks: <true|false>, if set to false you only get bounding box and no landmarks.
like i said before, bounding box detection on its own is really unreliable, but since you wanted it...

if you're interested in detector output shape, take a look at 'src/hand/handdetector.js estimateHandBounds() and getBoxes()

with landmarks enabled, original bounding box is just used as starting value and is actually discarded with new one calculated based on maximum coordinates of detected landmarks and newly calculated value is cached
that happens in src/hand/handpipeline.js estimateHands()

[djthegr8]

Great... What are the default values for models in config (disabled?)

[vladmandic]

default values are the same regardless of landmarks being disabled or enabled.

but...confidence level of landmark detection is much more reasonable while confidence value of just box detection is off-the-chart (sometimes above 90% for false-positive), so if you want to use landmarks = off, best to put minConfidence = 0.96

i already modified defaults for iouThreshold = 0.1 and scoreThreshold = 0.8 so nonMaxSuppression is as pessimistic as it can be without dropping things completely.

[djthegr8]

Great, thanks

[djthegr8]

A suggestion, could you add an exhaustive list of all attributes of the configs? That would be really great for reference..,

[vladmandic]

more detailed than https://github.com/vladmandic/human/wiki/Configuration ?

i haven't updated wiki with new defaults, but will soon.

you can always take a look at config.js in project root, that file contains all the current defaults, wiki is just a copy&paste of that file.

[djthegr8]

Didn't see that, apologies

[djthegr8]

Another question ~ does keeping all models (except the one required) disable help performance??

[vladmandic]

absolutely. you can disable each model in demo app ui and see the impact on performance.
also, if in ui you disable async ops, you'll see in the page footer per-model performance output in miliseconds.

additional benefit if you disable models before startup, they don't even get loaded - library loads models only when it needs them, but for demo everything gets pre-loaded at startup for better user experience (see last few lines of demo/browser.js)

[djthegr8]

How do I resolve this?

[vladmandic]

are you using library from browser or node? only node supports loading from file.
if you're using browser, you must host models on a web server.

for convenience i've included a micro web-server
https://github.com/vladmandic/human/wiki/Demos

[vladmandic]

i'm closing this issue as there are several unrelated items discussed in the thread.
please open a new issue for any outstanding items.