Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding location to SMS/Email datasources. #3534

Open
rowasc opened this issue May 13, 2019 · 14 comments

Comments

Projects
None yet
4 participants
@rowasc
Copy link
Contributor

commented May 13, 2019

Which user group(s) are the primary audience for this feature/enhancement.

  • Reporters (as the ones adding the location)
  • Admins (as the ones configuring the integration in their deployment)

Include deployment names if possible
A user in Ushahidi.io and OSS contributor ( @lexoyo ) has this deployment https://monidom.ushahidi.io/views/data that they are setting up..
While setting up their datasources (Email, SMS) they asked where the location field would go and how the user would add the location data. This makes perfect sense really, since our main view is a map, I think I'd expect location to be a field in most surveys if I was setting it up without guidance.

Specify key workflows (e.g. crisis, elections)
I can see a strong use case in crisis monitoring.

Specify key use cases (program/data workers, incident management, advocacy/storytelling)

Incident management IMO, but would love people to chime in here and let us know if there's a need for this.

Is your feature request related to a problem? Please describe.
I can't set up location in email or sms .

Describe the solution you'd like
I'm not totally sure how to go about this.. or if it's actually something we want or not. .I think at least for email it could make a lot of sense to let users add something like
map=[link-to-map-pin]
ie map=maps.google.com/bblabblalabla&lat={lat}&lon={lon}
or whatever the format for the map service is, and being able to parse that location to add to a field specified by the deployment administrator.

I have some concerns on how usable this would be, but I think it's worth exploring/spec'ing out at least?
Describe alternatives you've considered
Not having location.. Or I suppose asking the user to add their google map link but not map it to any field

Additional context

@rowasc

This comment has been minimized.

Copy link
Contributor Author

commented May 13, 2019

@Erioldoesdesign @tuxpiper @Angamanga @StaicyG hey folks, sorry to spam you all, but was wondering if anyone has thoughts on this?
cc @lexoyo following up here on our chat , feel free to jump in and let us know what you think any time

@rowasc

This comment has been minimized.

Copy link
Contributor Author

commented May 13, 2019

Tagging @Shadrock in case he's interested in giving me all the reasons this feature idea could be super problematic :)
I know it would be a thing if we just "took" that information (ie getting the location from a twitter post's geolocation data) , but would it also be a bad idea as an explicit thing we can tell users to do ?

Is this even a reasonable thing to ask of a non (necessarily) technical user? @Erioldoesdesign specifically might have opinions or data here. Not sure how other platforms do things like this, if at all.

@Erioldoesdesign

This comment has been minimized.

Copy link

commented May 14, 2019

YES! I have huge thought's on this.

I was actually with this deployment owner last week: https://bristolstreetharassmentproject.ushahidi.io/views/map

They just got email submissions working on their deployment but were concerned they'd loose the location data that is hugely important to them. The only way to currently do it is to edit a post before you publish it and add in the location yourself.
That also requires that the reporter included location info in the body of the email/text.

There's a few ways I can think of UX wise that could make this work but it wouldn't be 'pure' email.

Essentially what you could try to do is sending an email to a deployment triggers a follow up emailf rom that deployment that includes all the survey question info in the email body and then the responder fills out info in the replied email body. Sends that but then the deployment would need to be smart enough to 'scan' the email for those specific responses.

The other idea I had was that a user sends a report by email. The user receives an email back with a link to their post via webpage/app that is a direct edit version of that post. So here we're asking the user to add more detail to their post themselves so not the deployment owner.
This solution doesn't account for users in countries where Ushahidi web is blocked though :(

@Erioldoesdesign

This comment has been minimized.

Copy link

commented May 14, 2019

Regarding your idea here:

I'm not totally sure how to go about this.. or if it's actually something we want or not. .I think at least for email it could make a lot of sense to let users add something like
map=[link-to-map-pin]
ie map=maps.google.com/bblabblalabla&lat={lat}&lon={lon}
or whatever the format for the map service is, and being able to parse that location to add to a field specified by the deployment administrator.

I'm not sure again, how technically savvy reporter/users will be when trying this. (or even non-techy deployment owners for that matter!)

The only thing I know is that user understand human-worded questions like 'Where did this incident take place? (give as much detail as you can)" and they know to type text into text fields. Asking for a google map exact location could work from some but you probably aren't gonna grab the less technically savvy and also the folks not on smart phones (for sms) they would have to describe their location and machines would need to do the rest.

@tuxpiper

This comment has been minimized.

Copy link
Member

commented May 14, 2019

I think that sadly there's just no standard that fits all countries or user groups. It doesn't mean that nothing can be done, just that any solution would just be a first stab at the problem.

There are standards for geotagging sms , dating from a few years ago. See https://en.wikipedia.org/wiki/GeoSMS and https://www.opengeospatial.org/standards/opengeosms , which has been used with Platform in the past: http://www.opengeospatial.org/blog/1520 .

Detecting and parsing these geo URLs (or even google maps, OSM URLs) in SMS or e-mail messages would be a relatively affordable improvement to this version of platform. However, as pointed out, it probably remains a technically savvy thing to use, because widespread support in the client side is just not there (as in, there is no "Add Geo-coordinates" button to most SMS android apps or feature phones).

Still, I guess it could work for some cases, with proper messaging and training.

As an aspirational blue-sky kind of goal, maybe we would be looking at natural language processing combined with context-aware navigation engine, i.e. something that can parse text such as "Two blocks east of Bishop Magua Center along Ngong Road" . This is the best kind of geographical reference that can be produced in some countries, where people navigate based on landmarks, rather than coordinates or street addresses.

This would be pretty hard to make work reliably, too. Not just technically, which it is, but just because the world is ... very dynamic. If you've got 20-25 mins to spare, this is a talk I really enjoyed: https://www.youtube.com/watch?v=_t5DxV7cXgQ , about the challenges of finding directions in Aşgabat, Turkmenistan.

So even with the biggest efforts, you can only be so sure of how accurate your geo data is. Geo data is not necessarily accurate. ( Which is a reminder of issue #600 , there should be a better job done at capturing that complexity ).

As a first stab at the problem, I really like the idea of follow up questions/instructions via email, SMS (or even chatbot in other platforms), because:

  • it facilitates the technical problem of identifying the relevant data in a free form block of text
  • it makes it clear that the platform is asking for a location, instead of what could feel to the end user as surreptitiously gaining access to that data.
  • it makes it easier for the deployers to educate end-users on valid ways to provide geographical data. Technically-savvy deployers may even code their own way to parse geographical data in a way that works for their context. Platform code just shouldn't make that any harder than it needs to be.
  • it introduces a flow that can be used to solve so many other practical problems in structuring posts
@rowasc

This comment has been minimized.

Copy link
Contributor Author

commented May 14, 2019

As a first stab at the problem, I really like the idea of follow up questions/instructions via email, SMS (or even chatbot in other platforms)

Agree. Chatbot / questionnaire sent in steps to email or sms could work. The problem with the chatbot and targeted surveys is that we don't have a good way to support them appropriately for all our deployments at the moment (ie not just the enterprise/grants) , so to implement something similar we should probably think of how to tackle those two issues in a way that doesn't get too expensive to run and maintain , I think?

@Erioldoesdesign

This comment has been minimized.

Copy link

commented May 14, 2019

hmm indeed - sounds like this feature needs to be rolled into a grant application as well as being well described for our OSS community! #on it

@rowasc

This comment has been minimized.

Copy link
Contributor Author

commented May 14, 2019

+1 , this is a relatively large project with maintenance costs for .io , if we want to make it as a more complete/multi use thing.
As a simple feature it could be "send this email when you get a response to the form if this checkbox is ticked" which is less resource heavy, but I think it requires some more thinking and I would prefer not to do a super constrained solution so that we could actually do something like the targeted survey flows work in a generic way and not be 1 way only to start It, if that makes sense .

@tuxpiper

This comment has been minimized.

Copy link
Member

commented May 14, 2019

+1 , this is a relatively large project with maintenance costs for .io , if we want to make it as a more complete/multi use thing.

I think this is true, I am just somewhat conflicted with that perspective, from the point of view of roadmap building.

<!DOCTYPE possible, completely personal, rant>

Just a personal thought forming here: as Ushahidi as an org opens up and encourages development on platform, there may be things that fall outside of the appropriate cost/benefit spot for rolling as a SaaS solution in ushahidi.io . In other words, I'm not sure I'm anymore at the point of not wanting to see a feature in the platform roadmap, just because such feature is not SaaS friendly.

ushahidi.io , being a SaaS solution, could present itself as the most vanilla flavor of platform. The baseline that is quickly available to anyone. And that can be OK, hugely valuable.

It's of course good to try to raise the baseline as much as possible, but it may not be beneficial to expect each and every initiative around this project to want to stay close to that baseline.

@rowasc

This comment has been minimized.

Copy link
Contributor Author

commented May 14, 2019

Oh, absolutely.
This could easily be feature flagged in a way that we can

  • document how it'd be used if you self host it
  • document how to self host it
  • not have to add it to .io unless/until we are ready for it
    It wouldn't even be the first time we feature flag things to not be a part of general .io so there's already a system for it (targeted surveys and HDX being the more interesting examples)

regarding io as a vanilla version of platform:
I think there's something there, in .io as a vanilla, no frills solution to get you started very quickly and host your deployment if you don't have very advanced requirements, yes. I think as we shift our perspective on this, we will need to put more and more focus on documenting the "advanced" implementations and deployments people would be able to do if they wanted to self-host. I imagine the dev hub will have a lot of benefits for this exact thing.

I would argue that a basic implementation for this should be part of vanilla/io, but it's important to realize that it doesn't have to go to .io from the beginning, so thank you for that nudge, I appreciate it.

@lexoyo

This comment has been minimized.

Copy link

commented May 14, 2019

Hello all, thank you for this very interesting convo @rowasc as well as your help in the chat

My 2 cents concerning the UX, as I understand it there are several features in this feature request:

1- Feedback: I like the idea of sending back an email to let the user know what we understood of his submission
1.1 - could even be done for SMS too?

2- Better ways to collect info by email, closer to web, more structured: the feedback email of 1- could ask for the missing info, and probably let the user a chance to change the submitted info
2.1 - could even be done for SMS too?
2.2 - this could be several messages, more like a chat bot, step by step questions

3- The ideal feature for me would be to be able to automatically geolocalize the user and this would be available as an "Email field" and "SMS field".
3.1- Don't the SMS platform give us the approx coordinates of the user?
3.2- In an email sent back as a confirmation email (same as 1- maybe) we could include an image downloaded by the user which would give us a coordinate?

@tuxpiper

This comment has been minimized.

Copy link
Member

commented May 15, 2019

Thank you @lexoyo for the points!

I think 1 and 2 are much on spot. As I see it, the main objective here is to reduce the workload for an organization that is running a Ushahidi deployment.

Presently, running a deployment can mean having significant amounts of people dedicated just in order to give structure to posts: converting (copy-pasting) plain short messages to the structure of the survey, and following up with the reporter to ask for any missing information.

Automated E-mail, SMS and chatbot interactions could lighten this load a lot.

I feel there will probably always be a manual step, for confirming that the reporting end understood and fulfilled the interaction properly, but that is still a lot less work.

About item 3:

3.1- Don't the SMS platform give us the approx coordinates of the user?

No, it really doesn't, in all of the cases we have seen. It's true that cell tower triangulation is a thing ( although not sure how much it's ingrained in the comm protocols involved), but we have never seen such data trickling down to the consuming end of a SMS gateway.

3.2- In an email sent back as a confirmation email (same as 1- maybe) we could include an image downloaded by the user which would give us a coordinate?

This is interesting. If I understand it correctly, you mean that the platform would translate the origin IP address requesting the image into a location? Unfortunately, there are many factors that hugely diminish the accuracy of this: outdated IP-to-geo databases (varies by country), transparent proxies rolled out by telcos, VPNs... map markers can easily end up in completely different countries because of any of these.

There are cases where the geographical data is there, as a feature that the comms platform provides. Prime example is twitter. Many users send their geolocation with their tweets. However, twitter terms of service disallow storing and aggregating that data, iirc.

And finally, some people may just not be comfortable with sending their precise location, specially without them knowing they are doing so. I think that, when it comes to location, explicitly asking is best.

@Erioldoesdesign

This comment has been minimized.

Copy link

commented May 15, 2019

+100 for this statement:

And finally, some people may just not be comfortable with sending their precise location, specially without them knowing they are doing so. I think that, when it comes to location, explicitly asking is best.

The safest way to collect that kind of data is to explicitly ask for it and ask the machines to try and make sense of anything more 'fuzzy' like 'I'm at the corner of rue de garibaldi' and then the deployment owner verify, confirm or change those details.

@lexoyo

This comment has been minimized.

Copy link

commented May 15, 2019

It totally makes sense, I understand and respect that privacy and accuracy are more important than automation

So I guess the discussion now is about "Automated E-mail, SMS and chatbot interactions" and the "manual steps"

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.