Labels not placed directly on obstacles

[misaugstad]

I suspect that this is just coming from user behavior (maybe they want to avoid covering up the thing that they are evaluating), but we haven't ruled out that it is a bug of ours yet. I think I want to collect a bunch of examples of this and then check to see if there are any patters in the browser, OS, or user(s) who are contributing data that look like this. That should inform how we deal with this issue. Here are a few examples from a quick look through Gallery. But I've seen much more egregious examples recently.

Label IDs: 9067, 29691, 49109, 3367 in SPGG.

[jonfroehlich]

Thanks Mikey. Yah, wonder if this is just one individual's user labeling idiosyncrasy or a tech issue (e.g., Safari doing something weird again).

…

On Tue, Mar 2, 2021 at 11:56 AM Mikey Saugstad ***@***.***> wrote: I suspect that this is just coming from user behavior (maybe they want to avoid covering up the thing that they are evaluating), but we haven't ruled out that it is a bug of ours yet. I think I want to collect a bunch of examples of this and then check to see if there are any patters in the browser, OS, or user(s) who are contributing data that look like this. That should inform how we deal with this issue. Here are a few examples from a quick look through Gallery. But I've seen much more egregious examples recently. [image: Screenshot from 2021-03-02 11-49-24] <https://user-images.githubusercontent.com/6518824/109706273-97897b80-7b4d-11eb-8015-cf62e7343b65.png> [image: Screenshot from 2021-03-02 11-48-59] <https://user-images.githubusercontent.com/6518824/109706278-98221200-7b4d-11eb-9c3b-26b7cd81c540.png> [image: Screenshot from 2021-03-02 11-46-46] <https://user-images.githubusercontent.com/6518824/109706279-98baa880-7b4d-11eb-87fa-9374e657f654.png> [image: Screenshot from 2021-03-02 11-45-27] <https://user-images.githubusercontent.com/6518824/109706280-98baa880-7b4d-11eb-8e2b-67a68ba3f210.png> Label IDs: 9067, 29691, 49109, 3367 in SPGG. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#2478>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAML55PZHFSLJXNUZIXIW7DTBU7GDANCNFSM4YPUMTGA> .

-- Jon E. Froehlich <https://jonfroehlich.github.io/> (he/him <https://www.mypronouns.org/he-him>) | @jonfroehlich <https://twitter.com/jonfroehlich> Associate Professor, Allen School of Computer Science & Engineering Core Faculty, Urban Design and Planning, College of Built Environments Director, Makeability Lab <https://makeabilitylab.cs.washington.edu/> | Associate Director, CREATE <http://create.uw.edu/> University of Washington Help make sidewalks more accessible: http://projectsidewalk.io

[uditpatwal]

Seattle Label ID's

• Good Label ID's
  • 30323, 81303, 96152, 12856, 105832, 72448, 125815, 113147, 72764, 78289
• Bad Label ID's
  • 124566, 132673, 93066, 128793, 39303, 139701, 27132, 41822, 76845, 94954, 64407, 87612, 128793, 79360, 96821

SPGG Label ID's

• Good Label ID's
  • 33381, 31335, 17018, 98169, 83766, 72293, 47946, 97685, 23495, 62437
• Bad Label ID's
  • 41252, 28109, 59656, 65860, 79982, 2291, 95235, 64232, 36354, 5814, 97685

[robertyoung2]

Hi @misaugstad,

I ran the cropping tool on about ~2,500 dropped curb labels. I did a manual review of the cropped images and noted down a sample of label_ids where I felt the labels were poorly positioned (floating too high, sometimes too low) as in your above examples.

The list comprises 100 labels, hopefully this is a decent enough sample size for you to be able dig a little deeper into the issue.

labels_not_on_object.csv

Sample - 20136

If you need more information, or want a larger sample size, let me know and that's something I can work on.

[misaugstad]

This is incredibly helpful, thank you! I expect this to be a large enough sample size. I'll get back to you when I have more info!

[misaugstad]

I'm probably not going to be doing this analysis today, but note to self for the future. Probably using a query like this:

SELECT label_id, sidewalk_user.user_id, username,
       browser, browser_version, browser_width, browser_height, avail_width, avail_height,
       screen_width, screen_height, operating_system, language
FROM label
INNER JOIN audit_task ON label.audit_task_id = audit_task.audit_task_id
INNER JOIN audit_task_environment ON audit_task.audit_task_id = audit_task_environment.audit_task_id
INNER JOIN sidewalk_user ON audit_task.user_id = sidewalk_user.user_id
WHERE label_id IN ()
GROUP BY label_id, sidewalk_user.user_id, username,
          browser, browser_version, browser_width, browser_height, avail_width, avail_height,
          screen_width, screen_height, operating_system, language;

[jonfroehlich]

Here's a video of the problem @uditpatwal and @misaugstad

LabelMovementOnValidationScreen.mp4

[jonfroehlich]

More examples:

[jonfroehlich]

Another recent example from St. Louis

[misaugstad]

Found another extreme example of this. In Seattle, user 38f27bb3-eb84-44ea-8284-780f1675d621. Looking at their labels placed on May 6th, 2024. What's interesting about them is that they seem to be off in the horizontal direction as well as the vertical, when I most commonly see the labels off in the vertical direction only.

I tried to label in the same places on the same panos and it worked fine on my computer, Linux, Chrome.

A quick look at the audit_task_environment table shows me

• browser: Chrome
• OS: Windows
• language: en-US
• browser width: 1812
• available width: 1812
• screen width: 1812
• browser height: 854
• available height: 974
• screen height: 1019

I also tried doing this, shortening my browser like they did and still got good results from my labels... But again this was on Linux.

Another weird piece is that it looks like they were auditing outside of where they needed to be (pic below)... It's definitely possible that this was just user error, they are a new HS volunteer. But it's also possible that it could be related?

[misaugstad]

This happened to the data that I added to Walla Walla when first setting up the city. I recorded a few pano IDs where I found that my labels were placed in the wrong spot:
ZaS_D67N2iAEgEEcMcFnzw
2Nn0aR2CfUhZ9oTYORAADg