add ability to measure distances with AR #3709
Conversation
There was a problem hiding this comment.
I just glanced over the code.
Unfortunately, it doesn't run on the emulator (logcat error EGL_BAD_ATTRIBUTE) or on my OnePlus ONE running LineageOS 17.1 (Android 10) without Google Play Services, but with microG 0.2.19.211515 (logcat error: ARCore-InstallService: Play Store install service could not be bound.). When clicking "Show AR Measure", nothing happens in the UI.
app/src/main/java/de/westnordost/streetcomplete/measure/MeasureActivity.kt
Outdated
Show resolved
Hide resolved
app/src/main/java/de/westnordost/streetcomplete/measure/MeasureActivity.kt
Show resolved
Hide resolved
app/src/main/java/de/westnordost/streetcomplete/measure/MeasureActivity.kt
Show resolved
Hide resolved
app/src/main/java/de/westnordost/streetcomplete/measure/MeasureActivity.kt
Show resolved
Hide resolved
app/src/main/java/de/westnordost/streetcomplete/measure/MeasureActivity.kt
Outdated
Show resolved
Hide resolved
…eActivity.kt Co-authored-by: Flo Edelmann <florian-edelmann@online.de>
…etComplete into ar-core-measure
|
Thanks for the review! I implemented the activity to be self-contained, i.e. it should not crash if the prerequisites for AR are not given. It just exits itself. I did that so that it could theoretically be used from outside this app (e.g. by Vespucci) without any necessary compatibility check provisions in that app itself. |
|
The measure AR view only works if it is in the list of supported devices: Additionally, it only works
|
|
Then sadly I will be unable to test it. |
|
|
||
| // measuring distance with AR | ||
| implementation("com.google.ar:core:1.29.0") | ||
| implementation("com.google.ar.sceneform:core:1.17.1") |
There was a problem hiding this comment.
Will StreetComplete still be eligible for F-Droid when this is included? Or will it need another build flavor?
There was a problem hiding this comment.
For some apps F-Droid maintains patches applied before build.
There was a problem hiding this comment.
I don't know, we will see what happens. Maybe it will just get a flag "contains non-free software" or something(?)
In any case, shouldn't be hard (for them) to remove this functionality with patches.
There was a problem hiding this comment.
GRR GITHUB
I had written a whole comment here but not yet posted, then clicked "unresolve conversation", and GitHub decided to delete my comment. But anyway.
My understanding is that F-Droid does not knowingly distribute non-free apps, period. So, once someone noticed, they would probably remove F-Droid, unless someone cares enough to write a patch.
I care enough to write a patch. Actually, I've previously considered volunteering to be the "SC F-Droid maintainer" if the need should arise. It seems that time may be here. However, I must first raise an objection to adding a dependency on Google Play Services, and especially to including a non-free library in the app. Yes, we're already using Android and thus, trusting Google, but I think proprietary dependencies and shipping proprietary code are both meaningful lines that I'd rather not cross. Of course, I know it isn't my decision to make.
If you decide to continue down this route, I'd rather keep the patches in this repo than in the F-Droid repo. That way it's less complicated to build the foss version; you only need to get the necessary code from one location. I'm not sure if it would be best to maintain a patchset, use a foss/fdroid git branch, or create a new build flavor. A branch is probably easiest to maintain code-wise (git helps with merge), but is the most extra effort for releases.
There was a problem hiding this comment.
Well, mainline (master) will be the release on github and on google play, signed with my private key.
Maybe it is possible to exclude dependencies via a gradle build config constant. E.g. the AddCyclewayWidth quest could exist without AR, just the button to start the AR activity would be missing.
There was a problem hiding this comment.
Actually, all of Google Filament (rendering engine), Google Sceneform (connecting ARCore/Filament easily) and the AR Core SDK are open source.
What is not open source is the ARCore (aka "Google Play Services for AR") app. That app does all the heavy lifiting, image processing etc.
The AR Core SDK just communicates with this app and makes sure that that app is installed and up-to-date before doing anything useful. But communicating with a closed source app doesn't sound like it would violate against F-Droid guidelines.
If the user cannot install the app because the Google Play Services are not installed at all (such in the case of @FloEdelmann ) or the user doesn't want to install the app, it will simply do nothing. 🤷
There was a problem hiding this comment.
Ah, cool! I didn't realize— usually it's Android components which are Open Source and everything Google-branded is proprietary, so I didn't look further.
Yes, that should be fine for F-Droid. There might be an antifeature tag (similar to NonFreeNet for apps which use non-free network services), but I'm not sure. A 'wait and see' approach should be fine.
|
I have two devices (mine and my wife's) which are on the list. How can I test this? |
|
In the settings screen (only in debug builds), there is a new entry at the very bottom. Other than if it works at all, I am very interested to see how precise it is on different devices. For a fair test setup and comparison between devices, let's presume: daytime, outside but cloudy/rainy, measurement of sidewalk width (sidewalk pattern will be similar everywhere) |
This comment has been minimized.
This comment has been minimized.
app/src/main/java/de/westnordost/streetcomplete/measure/MeasureActivity.kt
Show resolved
Hide resolved
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
|
Thank you @westnordost for the APK. I just installed that version, but the app is crashing when calling the AR feature. I've sent you the error reports by mail. If I can further assist, please give me a note :) |
|
Uhm, should have tested it |
|
Here's another one: https://github.com/streetcomplete/StreetComplete/actions/runs/1786937861 |
Well this one works for me (after I updated the AR tools which it prompted for). On Sony Xperia XZ1 Compact with Android 9. Measuring two carpet tiles, which appear to actually be 1m, it came up with 1.03m, which could be my use of the arrow as much as anything else. I'll try and do some real world tests soon. If this gets merged, it could do with some guidance/UI around it, it was a bit random out of the box! |
|
Tested on a Samsung Galaxy S20+, it works well. Very much looking forward to some width quests. |
|
You have to get used to it. There is explanation needed when integrating the feature.
|
|
Yeah as said, please test it outside on the sidewalk. I don't plan to provide an explanation other than what is already there, users will just need to try it out. They can't break anything, as the result is not applied immediately but the user will be led back to the then-filled-out width form.
It would be great if you could do this and state your device, so we have a few random samples on how well/precise different smartphones are able to perform in AR. |
|
I am also thinking about removing this visualization of the detected plane (those dots), as the arrow as a cursor should already be enough. Also, the plane render doesn't render vertical planes. But vertical planes should be detected too. Try to measure the height of a house or something. It will likely not work on surfaces that are just one color (like room walls). |
I was thinking about adding sanity checks to the input but kind of decided against it. Though what I could add is a check whether it has more than 2 decimal places (less than one 1cm) and simply don't allow this input. |
https://www.engadget.com/flight-simulator-open-street-map-building-205545509.html |
|
I checked the times. It's not the ARCore check that takes long. It looks like it is the inflation of the layout 🤷 |
|
I did not check the details here, but as far as I see ARCore is not fully open-source and ships with proprietary binaries under some Google ToS. How do you envision the F-Droid build version to work here? Just strip that feature? Or do we have an alternative lib? Also, did any anyone already link the related issues #879 and e.g. the one possibly use-case #698? Well, apparently no, so I just did so… (I know, bad rhetorical questions here… 🙃 ) |
|
The F-Droid version will work just like the normal version IF the user has the google play services installed and chooses to install ARCore (if it is not already installed). If not, the button to start the AR view will simply not display. |
|
Ah okay, hmm, maybe if one installs that proprietary app it could also be made to work with microG or so at least. In any case for all those affacted, I have opened microg/GmsCore#1672 already. |
|
@westnordost |
|
not yet, I want to wait till Google Play reviewed the v40.1 update before I post another beta. There is only the debug APK: https://github.com/streetcomplete/StreetComplete/actions/runs/1825403355 |
It say it is expired and cannot be downloaded. |
|
With the new quest I am able to tag |
|
I tried the last debug APK a bit and could see quite good results up to about 2 meters. |
It's not about it working, if it's not FOSS it will not be build on F-Droid. /LE: will take a look, as I see that this might have been already built. |
|
@licaon-kter See #3709 (comment) The newest version is already built and released on F-Droid: https://f-droid.org/de/packages/de.westnordost.streetcomplete/ |
|
Does the app show this https://github.com/google-ar/arcore-android-sdk#user-privacy-requirements when AR is available and used? There's a Yes/Cancel thing? |
|
ARCore is a separate app. If it is not installed, a dialog will open which prompts the user to download it from the play store. It is requirement for all apps on the play store that the privacy statement is linked from there. |
Trying the feature out – I didn't come to a conclusion how it is supposed to work. I think, it needs explanation.
Sometimes an icon showing a hand on a phone circles cryptically across the live camera image shown. Then eventually sometimes an arrow pointing down appears at seemingly random locations (if it does not appear: good luck guessing why and what to do about it – is it, that no pane has been detected? or no line detected to be interpreted as border of thing to be measured?), drifting erratically when tilting or turning the phone. Upon a touch of the screen, tilting/turning does indeed display a measuring line, the arrow now at the other end of that line; turning the phone the arrow moves against the cam image but ever so often stops before "reaching" the other side of the road (in the instance of trying to measure the road width); a bubble note gives the mysterious advice "to stay near the arrow (and mind the traffic)" – but trying to walk with the arrow makes you move the entire measurement line or "overtaking" the arrow.
A brief explanation should tell the user:
|
|
Shouldn't description on how ARCore app works be described somewhere on the internet? If so, perhaps a link on how to use ARcore might be provided in StreetComplete FAQ (by someone who has it working and can verify that it the instructions are correct; i.e. not me). Or shortly described directly there... |
|
Short one:
|
|
Support for Huawei's AR Engine would be really great too :) https://developer.huawei.com/consumer/en/hms/huawei-arengine/ some Huawei phones (for example my p40 pro) have a time-of-flight sensor which are used for AR, and would probably give very accurate measurements in streetcomplete 😄 |
|
Regarding Huawei phones and AR not working there: #3917 |
|
BTW, for anyone having a Google-free or only microG device which is otherwise compatible and who watches this PR, I have a big news for you: More information (to keep it in one place): #2093 (comment) |
|
@rugk although adding Google app on Google-free device kind of defeats the point I would think 😆 |
|
I don't think it necessarily does. If your aim is to have a 100% FLOSS system, sure. However, if your aim is just to have a privacy-respecting, secure system installing a sandbkxed app and preventing internet access seems fine. |
Just this so far, this is not used in any quest yet. I think it is maybe a good idea to have this merged first (to master, after v40 is released) so that other people usually working on/using master can test this feature early as I expect the AR experience will be very different depending on the device.
Particularly, I am interested in comparisons how precise is the measurement on your device (compare with measure tape).
Fixes #698.