Terminology and User Stories
- Instance
- A website which is running an Ushahidi map. It usually has a specific cause (e.g., "Zombie attacks in Washington DC") on which it seeks to gather data. Generally, allows anyone to send a message via public interfaces (web form, email, sms) and may also gather data from web streams (twitter, RSS).
- Crowdmap
- A platform which allows anyone to host a free Ushahidi instance.
- Message
- May be received by email, SMS, or Twitter (voice also possible)
- Report (=Incident)
- An annotated message, including additional fields such as
- Category
- A "tag" which describes a report (e.g., "Buildings on Fire" or "). In Ushahidi 2.x, sometimes categories are also used to describe workflow information for _annotators_ (e.g., "Needs to be translated"); we believe this is planned to change in Ushahidi 3.x.
- "Parent"/"child" category
- A "parent" category is the top-level of the categorization heirarchy. For example, the "parent" category might be crime while the "child" categories might include assault or theft. In Ushahidi 2.x, a "parent" category cannot be directly selected.
- Approve
- Allow a _report_ to appear as a dot the map, visible to public.
- Verify
- Mark a _report_ as reliable/true information, because you are confident in it (from viewing multiple sources, seeing a news article, viewing in person, etc)
- Trusted
- Can mark a reporter/user as trusted, which results in their _reports_ automatically being _approved_.
- Citizen
- A person unaffiliated with the Ushahidi _instance_, who submits information to the system via a _message_.
- Annotator
- A volunteer or employee who takes incoming _messages_ and creates a _report_. They provide additional information (such as one or more _categories_, a location name, and latitude/longitude).
- Admin
- Runs the _instance_. Can modify the list of available _categories_. Has the ability to approve or verify a message. May also be an annotator.
See [Crowdsourced Crisis Reporting](Crowdsourced Crisis Reporting) for more.
How to Submit a Report
Message Queue
Story #0: A Citizen be assisted in choosing categories.
An citizen files a report from the public user interface. She types the message text, the some categories are automatically highlighted in bold, with a confidence percentage written in parentheses. Sometimes, a "parent" category is highlighted, while other times a "child" category is highlighted.
Story #1: An Annotator should be assisted in choosing categories.
An annotator opens the message queue in the admin interface. She selects a message to annotate. Upon opening the message, several of the categories are already highlighted in bold, with a confidence percentage written in parentheses. The annotator is quickly able to check those boxes to confirm what the system has suggested.
Story #2: An Annotator should not have to waste time processing near-duplicate messages.
An annotator opens the message queue in the admin interface. Several messages have a tag next to them ("suspect duplicate"). Upon clicking this tag, the two messages appear side-by-side and the annotator is presented with a choice ("Is this message a duplicate? Yes/No"). The annotator sees that this new message is a re-tweet of a previous message. The Annotator chooses "Yes", and the new message is then linked to the previous message without any additional user input.
Alternatives:
-
An annotator selects a new message from the queue. They see a listing of all existing reports that are near duplicates in their content. They annotator then knows whether they've already recorded this information on the map, and it's useless to re-annotate.
-
Several annotators are facing 100s of new messages in the queue. 20 of these new, non-annotated messages are grouped together before the annotator reads them, because they are near duplicates. The annotator can then respond to the near-duplicates (e.g. Twitter re-tweets) as a unit, and saves time rather than annotating each of them individually.
Story #3: An admin should be able to be assisted regardless of the scale of their instance.
An admin creates a new instance and begins receiving messages. When the admin (also playing role of annotator) views the message queue and file reports, she notices that categories are already being suggested for the default category set. She is impressed that there's no warm-up period before the machine learning kicks in!
(We may be able to pre-train our classifier to work on specific non-vanilla Ushahidi instances, e.g., if there are 10 baseline categories to use for an Ushahdi instance with an environmental focus, we could guess those categories based on previous training external to this particular instance.)
Story #4 "Global Review": An admin should be able to quickly review and correct mis-labeled data.
A group of expert admins prepares to review all reports, updating their categories to strictly adhere to the UN OCHA guidelines. Because these guidelines are complex and specific, sometimes citizens and volunteer annotators miscategorize reports initially, despite their best intentions. The machine suggests a list of reports to the experts, which are likely to be labeled incorrectly or have urgent content. These messages are then addressed first by the experts, and the highest risk (incorrect, urgent) items are fixed promptly.
Story #5 "PII": An admin should be able to quickly remove personally identifying information (PII), so that it never appears on the public map.