Wrong segmentation

[ZhengdiYu]

Hi, outputs/segms/train/Capture9/0129_indextip/cam410062/image32412.png. This segmentation has a value of '33', which is not expected. Do you have any ideas?

[ZhengdiYu]

[zc-alexfan]

Hi, I have not noticed this before. Maybe you can see if there is 33 on the original texture map? Alex

…

On Tue, May 10, 2022 at 8:28 PM ZhengdiYu ***@***.***> wrote: Reopened #5 <#5>. — Reply to this email directly, view it on GitHub <#5 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AJH2YBKNCBKVDOLRR6DYKSDVJKTFDANCNFSM5VSMBC3A> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.*** com>

[ZhengdiYu]

Hi, I have not noticed this before. Maybe you can see if there is 33 on the original texture map? Alex
…
On Tue, May 10, 2022 at 8:28 PM ZhengdiYu @.> wrote: Reopened #5 <#5>. — Reply to this email directly, view it on GitHub <#5 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AJH2YBKNCBKVDOLRR6DYKSDVJKTFDANCNFSM5VSMBC3A . You are receiving this because you are subscribed to this thread.Message ID: @. com>

Weird. The training process always went normally before.

But it suddenly began to throw out an error yesterday when computing loss, like:

block: [0,0,0], thread: [16,0,0] Assertion t >= 0 && t < n_classes failed.
RuntimeError: cuda runtime error (59) : device-side assert triggered

Sometimes the error would also be the assertion from the swapr_lr_label() function, while sometimes it would be the RuntimeError above.

Do you still have the segmentation I mentioned above? It would be great if you could have a check with this frame.

[ZhengdiYu]

Also, one tiny question, what is the tol value you used in experiments? (in def segm_iou(pred, target, n_classes, tol, background_cls=0):)

[zc-alexfan]

hmm. sorry, I don't notice this problem before. I would suggest to save the rendered image with 33 and visualize what's in 33.

I use tol= 20. This is just to avoid numerical problem when computing the IoU.