Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

False post-process value witch "extended Resolution" false #2465

Closed
haverland opened this issue Jun 11, 2023 · 0 comments · Fixed by Slider0007/AI-on-the-edge-device#34
Closed
Labels
bug Something isn't working

Comments

@haverland
Copy link
Collaborator

The Problem

The first digit will on value >9.5 to 0, if extended resolution is set false.

In second part of the condition should be (result_after_decimal_point >= (1-Digital_Uncertainty) * 10)

if ((result_after_decimal_point <= Digital_Uncertainty * 10) || (result_after_decimal_point >= Digital_Uncertainty * 10)) // Band around the digit --> Rounding, as digit reaches inaccuracy in the frame

But I am wondering why the first digit must be handled in transition. Where's no need for it.

Version

15.2.4

Logfile

[0d08h38m12s] 2023-06-07T06:45:01     <DBG>   [CNN] PointerEvalHybridNew - No predecessor - Result = 0 number: 9.500000 number_of_predecessors = -1.000000 eval_predecessors = -1 Digital_Uncertainty = 0.200000
[0d08h38m12s] 2023-06-07T06:45:01     <DBG>   [CNN] getReadout(dig100)  prev=0
[0d08h38m12s] 2023-06-07T06:45:01     <DBG>   [CNN] PointerEvalHybridNew - NO analogue predecessor, zero crossing has taken placen = 2 number: 2.500000 number_of_predecessors = 9.500000 eval_predecessors = 0 Digital_Uncertainty = 0.200000
[0d08h38m12s] 2023-06-07T06:45:01     <DBG>   [CNN] getReadout#PointerEvalHybridNew()= 2
[0d08h38m12s] 2023-06-07T06:45:01     <DBG>   [CNN] getReadout#result= 20
[0d08h38m12s] 2023-06-07T06:45:01     <DBG>   [CNN] PointerEvalHybridNew - NO analogue predecessor, no change of digits, as pre-decimal point far enough away = 0 number: 0.000000 number_of_predecessors = 2.500000 eval_predecessors = 2 Digital_Uncertainty = 0.200000
[0d08h38m12s] 2023-06-07T06:45:01     <DBG>   [CNN] getReadout#PointerEvalHybridNew()= 0
[0d08h38m12s] 2023-06-07T06:45:01     <DBG>   [CNN] getReadout#result= 020

Expected Behavior

Result of the first digit should be the int-value of the recognition.

Screenshots

No response

Additional Context

No response

@haverland haverland added the bug Something isn't working label Jun 11, 2023
jomjol pushed a commit that referenced this issue Jun 12, 2023
* added more debug for #2447

* new model on new images dig-class100-0165_s2

* #2465 fix first digit with extended_Resolution=false
Slider0007 pushed a commit to Slider0007/AI-on-the-edge-device that referenced this issue Jun 20, 2023
* added more debug for jomjol#2447

* new model on new images dig-class100-0165_s2

* jomjol#2465 fix first digit with extended_Resolution=false
Slider0007 added a commit to Slider0007/AI-on-the-edge-device that referenced this issue Jun 20, 2023
* jomjol#2465 fix first digit with extended_Resolution=false

Co-authored-by: Frank Haverland <fspapaping@googlemail.com>
caco3 pushed a commit that referenced this issue Jul 1, 2023
* added more debug for #2447

* new model on new images dig-class100-0165_s2

* #2465 fix first digit with extended_Resolution=false

* fix #2491
jomjol pushed a commit that referenced this issue Jul 30, 2023
* added more debug for #2447

* new model on new images dig-class100-0165_s2

* #2465 fix first digit with extended_Resolution=false

* fix #2491

* #2540 fix trunc of first with imprecise precision
LordGuilly pushed a commit to LordGuilly/AI-on-the-edge-device that referenced this issue Oct 12, 2023
* added more debug for jomjol#2447

* new model on new images dig-class100-0165_s2

* jomjol#2465 fix first digit with extended_Resolution=false

* fix jomjol#2491

* jomjol#2540 fix trunc of first with imprecise precision
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant