red flags for annotation

[rbavery]

Jona has listed out some questions to consider for our labels, transferring to a ticket to track:

• For the fainter (less distinctive) slicks, the magic wand is often not a great tool, and it will be tighter to the slick in clearly defined areas, and grab unnecessary pixels accidentally when clicking in fainter areas–which leads to an over abundance of less interesting pixels.

• If you switch to the polygon lasso then the polylasso ends up grabbing more pixels than would otherwise have been selected, and generally increases the average area of those less stark examples.

• For slicks that cross tile boundaries, it will be VERY difficult to determine coincident vs recent, and that may cause a large amount of challenge when trying to learn the difference. Hard example: https://www.photopea.com/#i4lghhSj7

• Standards shift slightly in areas with a lot of pollution, in a bimodal fashion:

• Sometimes the annotation gets drawn smaller and smaller and details get annotated that might be labeled ignorable in other images

• At other times, the presence of obvious slicks makes annotation ignore finer details by comparative insignificance, even though they might have been annotated if they were on their own in a different image.

• Infrastructure slicks leading into large amorphous slicks are difficult to choose how to annotate. Currently truncating them when the branch hits an area of indeterminate source.

• Sometimes the infrastructure is tightly packed and all of it producing slicks, creating a matted appearance which is impossible to annotate separately, but which deserves to be labeled “infra”, leading to potential confusion in the ML.

• Sometimes seemingly arbitrary inclusions or exclusions can drastically change the size and shape of a bounding box, with the addition of very few pixels. Whether or not the annotator captures them seems likely to be based more on chance than intention.

• This applies also to old slicks that may be classified as two distant pieces of a single slick or independent old slicks…

• https://www.photopea.com/#iIP1teSjy

• Natural slicks can be in such high quantity that it becomes challenging to identify what should be annotated and what shouldn’t

• https://www.photopea.com/#il74_IGyt