-
Notifications
You must be signed in to change notification settings - Fork 0
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Sidewalk CV Code Image Resolution Bugs #2
Comments
I don't think the solution should be to use constants. Instead, why can't we read the resolution of the input image? |
Gotcha.... well.... Damnit! I should have asked about depth data because this is a totally reasonable type of error to expect. Regardless, @kaviMD I don't think your conclusion sounds correct – it sounds to me like the |
Yeah, I think you're right. For the higher resolution images, the X,Y coordinate needs to be effectively "unscaled" by multiplying by |
Good catch! Just want to add that even with correct depth data, it might be appropriate to reevaluate the formula used to compute the crop size from distance. The way I originally came up with that formula, I just manually made a bunch of crops, plotted a chart of distance vs. the crop size I used, and fitted a formula using Excel. I'm not sure whether it might be necessary to redo this process to come up with a new formula for the Seattle resolution, or if simply scaling up the predicted crop size by a little bit would be sufficient. |
Thanks @tongning. Yes, the results of some of @tongning's initial crop experiments are here: ProjectSidewalk/SidewalkWebpage#633 (comment). Is there another location with a fuller examination? This is definitely something that @kaviMD or @sarda-devesh could look into improving in the future. We should create a new Issue for it. Update: created Issue: #3 |
@galenweld and I just met and we came up with a universal solution that eliminates all future issues with image sizes. It involves three changes to the code. First: Second: Third: These three changes should not only automatically handle the current different image resolutions in every part of the CV pipeline, but they should also make the code forwards and backward compatible in the event that google changes image resolution again. |
And you read the resolution of the current pano image, right? Rather than
using constants?
…On Thu, Jul 11, 2019 at 4:09 PM Kavi Dey ***@***.***> wrote:
@galenweld <https://github.com/galenweld> and I just met and we came up
with a universal solution that eliminates all future issues with image
sizes. It involves three changes to the code.
First:
The global variables GSV_IMAGE_WIDTH and GSV_IMAGE_HEIGHT will always be
equal to 13312 and 6656, the resolution of the lower-res Newberg/DC panos.
This has the effect that any conversions between SV and XY coordinate along
with other math like generating crop locations, will be calculated as if
they were for the lower-res panos.
Second:
When computing the *center* location of a crop, *IF* the image is a
higher resolution Seattle image, the x and y coordinate of the crop will be
scaled up respectively. (This can easily automatically happen in code).
Third:
When computing the *size* of a crop, *IF* the image is a higher
resolution Seattle image, the size of the crop will be scaled up. (Again
this is easy to do automatically in code).
These three changes should not only automatically handle the current
different image resolutions in every part of the CV pipeline, but they
should also make the code forwards and backward compatible in the event
that google changes image resolution again.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#2?email_source=notifications&email_token=AAML55LGDWDA6XB5XQM2ZHTP664TFA5CNFSM4IBH7GQKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODZYHGGY#issuecomment-510685979>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AAML55MGZZ72RAPUX2MYWODP664TFANCNFSM4IBH7GQA>
.
--
Jon Froehlich
Associate Professor
Paul G. Allen School of Computer Science & Engineering
University of Washington
http://makeabilitylab.io
@jonfroehlich <https://twitter.com/jonfroehlich> - Twitter
Help make sidewalks more accessible: http://projectsidewalk.io
|
To elaborate on one point - the crop size needs to be scaled up when cropping from a panorama of a higher resolution because (since the resolution is higher) the corresponding crop needs to be proportionally larger.... a curb ramp that takes up 1 degree of the field of view will be more pixels wide in a higher resolution image. |
And yes, Jon, your understanding is correct. The coordinate system that the |
I just found an issue with how we were generating crops for panos in Seattle.
The issue is that the depth data in both Seattle and DC/Newberg is the same resolution. BUT the pano sizes are different resolutions. The function that calculates the crop size (
predict_crop_size
) usesGSV_IMAGE_WIDTH
andGSV_IMAGE_HEIGHT
to decide where in the depth data file to sample BUT when it tries to sample depth data for the points on the bottom or right of the panorama, it can’t find any data because the resolution of the depth data didn’t increase with the resolution of the panorama. This can be fixed by instead of inputting the panorama’s actual width and height to the function, always inputting13312
and6656
(the resolution of the DC/Newberg images). Right now when the function fails, it falls back onto a less accurate function that calculates crop size based on position in the image.I don’t think this is a big deal for the ASSETS paper because of how few labels are on the very bottom and right of the panos, but it is definitely something that we should fix and it makes me wonder how many other pieces of code are being affected in weird ways by the difference in resolution. I think that this thread can be a place to gather and resolve any additional bugs we find related to image resolution.
The text was updated successfully, but these errors were encountered: