Meet Tia

TIA -- Texting Internet Assistant -- is a nimble, NLP-powered AI assistant which performs internet tasks for users without internet access. Tia communicates entirely over SMS, ideal for off-the-grid situations or those concerned about cell-provider data limits.

Fig.1 - TIA's Welcome Messages.

TIA utilizes machine learning NLP via Facebook's open source Wit.AI, and personalizes each user experience based on their previous search history (in MongoDB).

Fig.1.5 - TIA's Wit.AI Interface

For full demo, text 'Hi' to 📲 347-352-6247 📲 (currently down for maintenance)

What Can Tia Do?

1. Driving | Walking | Public Transit Directions
2. Current Weather | 5-Day-Forecast
3. Information Summaries (Wolfram-Alpha and Wikipedia)
4. Top Headlines from 75+ News Sources
5. Jeopardy Trivia Questions
6. Yelp Reviews
7. Language Translation
8. Late-Night Jokes
9. Scheduled Reminders (One-Time and Recurring)
10. Email Alerts*

The app is divided into the following parallel servers communicating over the following RabbitMQ task queues:

1. Worker -- worker_gmail.py -- sends new user messages to gmail queue from GMAIL API

2. Listener -- listener_gmail.py -- receives gmail queue items, logs into (Mongo) database

3. Worker -- worker_preprocess.py -- sends new database messages to preprocessing queue

4. Listener -- listener_preprocess.py -- receives messages from preprocessing queue, performs processing, and updates message data on database

5. Worker -- worker_voice.py -- sends processed messages from database to google voice queue

6. Listener -- listener_voice.py -- receives messages from google voice queue, performs voice processing, and sends reply to user.

7. Worker -- worker_timer.py -- sends future messages, i.e., recurring and one-off reminders, from database to timer queue.

8. Listener -- listener_timer.py -- receives messages from timer queue and also doubles as the scheduling server, which uses a Background instance of the APScheduler library.

   

Fig.2 - Selenium Webdriver Controlling Google Voice Server

How it works

Tia is made up of several python servers communicating via RabbitMQ task queues as well as dozens of RESTful APIs, beginning with:

GMAIL API

TIA has a dedicated google voice number - (347) 352-6247 (currently unstable) - which auto-forwards all texts to its own GMAIL account.

1. Our first Rabbit queue worker -- worker_gmail.py -- checks if anything new has been added to the list of messages currently marked UNREAD. If there's nothing new in the 'unread' list, TIA skips ahead to #5.

2. If there is a new unread item, TIA makes another GET request for that most recent 'UNREAD' message, and parses the body/subject of the email, checking whether it's a forwarded text from the GV number or a regular email.

3. If the text is a GV text, TIA parses the email, extracting the body of the text as well as the sender's number, then marks the e-mail as 'read'.

4. The new info is added to TIA's MongoDB, which features three collections, message_records, user_records, and timed_records (for send later messages). If there is a previous user_record with the sender's phone number, saved personal info -- name, home address, how many times they've interacted with TIA -- is transferred to the new message_record. Finally, the message is listed on the MongoDB as 'unsent'.

5. TIA checks for any 'unsent' message_records in the MongoDB. If there are, TIA begins to process that message_record.

6. TIA queries Wit.AI, a natural language processing API, which parses the message for various intents and keywords. Once the response is returned from the WIT API, the processing begins:

TIA COMMANDS

Check Weather / 5-Day-Forecast

Example: "How's the weather in Dubuque?" Example: "What's the forecast near me?"

First, the Geocoder API by Here is queried to find the exact location entered -- for example, if the enters "malibu" without a zip code or state, Geocoder fetches more accurate information (i.e., zip, latitude/longitude) necessary for the next API call.

This second call is sending the zip code information to Open Weather Map API. Once the request has been returned as JSON, weather information is extracted, formatted, and decorated with condition-specific weather emojis.

Next, a create request is made in the GMAIL API, to craft a new email, populate it with the formatted (and emoji-fied) results of the weather call. Next, another request is made to the Gmail API to send this email to the cell-email address. This process is the same for each of Tia's non-gmail commands (except Jeopardy, which actually sends two emails).

For both weather-based commands, if left blank, the location defaults to home.

Yelp

Example: "Find sushi in Brooklyn."

The Yelp API is queried, using a specific Yelp category, i.e., 'Pizza' or 'movie theaters'and a location, 'in brooklyn', 'near me', etc. Once the general query is made, another query is made with more specific business info data (open/closing times, reviews) for the top three results and formatted.

<a href="https://www.turboimagehost.com/p/39220395/ezgif-5-b0d0f30272.gif.html" target="_blank"><img src="https://s7d1.turboimg.net/t1/39220395_ezgif-5-b0d0f30272.gif" border="0" alt="Free Image Hosting" /></a>

Fig.3 - Driving Directions Request

Turn-By-Turn-Directions

There are three types of direction commands:

1. "I want to drive from home to Santa Clarita, CA"
2. "Let's walk from the brooklyn bridge to williamsburg"
3. "How would I take public transit from faneuil hall marketplace boston to 1575 cambridge street cambridge massachusetts?"

Similar to the weather commands, each of the two waypoint locations are used in a GET request to the Geocoder API by Here. Once Latitude/Longitude information is returned, a call is made to the Route API, also by Here. The GET request URL is altered depending on the transit mode requested (pedestrian, driving, or public transit), but each uses the same route-based API.

Again, "home" synonyms can be used in either direction to default to the creator's home location.

<a href="https://www.turboimagehost.com/p/39220385/ezgif-5-82efca872a.gif.html" target="_blank"><img src="https://s7d1.turboimg.net/t1/39220385_ezgif-5-82efca872a.gif" border="0" alt="Free Image Hosting" /></a>

Fig.4 - News Headlines Request

News Sources

There are four commands related to news services:

Example: "What's going on at the New York Times?" Example: "Tell me the latest from hacker news" Examples: "ABC headlines please"

The News Api features a plethora of updated news summaries from providers across the globe, 75 of which were hand-picked for use on Tia.

For this command a GET request is made to the News API for the latest ABC news headlines, which then return up to 10 of the most recent Article Titles along with brief summary, which are numbered, formatted, and emoji-fied for sms-viewing.

While NY Times and Hacker News headlines are also available through the News API, these two have their own respective APIs, which TIA defaults to when requested, since there is more content available by querying the NY Times API and Hacker News API.

Language Translation

Example: "How would an Italian say, 'How much for that book?'" Example: "Translate お元気ですか into English"

The translation command utlizes another NLP resource, TextBlob, an open-source NLP library which in turn uses Google Translate to translate text to and from over 100 different languages.

<a href="https://www.turboimagehost.com/p/39218805/ezgif-5-11737366e6.gif.html" target="_blank"><img src="https://s7d1.turboimg.net/t1/39218805_ezgif-5-11737366e6.gif" border="0" alt="Free Image Hosting" /></a> 

Fig.5 - Setting One-Time Reminders Request

Scheduled Reminders

Example: "Remind me to pick up my sister in 5 minutes'" Example: "don't forget to take out the trash tomorrow"

The Scheduled Reminders command utlizes the <a href="https://timezonedb.com>TimeZoneDB API, which helps convert and get us local time for specific timezones. Currently, the Scheduled Reminders are currently available only for the users' home time zones.

<a href="https://www.turboimagehost.com/p/39218873/ezgif-5-fdde4b64c6.gif.html" target="_blank"><img src="https://s7d1.turboimg.net/t1/39218873_ezgif-5-fdde4b64c6.gif" border="0" alt="Free Image Hosting" /></a>

Fig.6 - Jeopardy Category Questions and Answers

Knowledge

Example: "Bio of Betty White" Example: "Let's play some Jeopardy" Example: "How many baseballs could fit inside a boeing 747?"

The wiki search makes a GET request using the simple Wikipedia API, where the response is a substantial summary of the requested wikipedia page.

More advanced, non-biographical questions use the Wolfram-Alpha algorithmic answer API, which has a wide variety of topics and sources, including ➗ Mathematics, 🔬 Science & Technology, 🎭 Society & Culture and 🍴 Everyday Life 🏀

The jeopardy call uses a Jeopardy Trivia API to first get a random question. Using the category ID from that number, a second GET request is made for all the clues ever aired in that category. Tia randomly picks a matched set of five clues for that category, and sends the category questions along with the spoiler answers below it.

Late Night Jokes

Example: "Make me laugh" Example: "I want some late night jokes from January 1st, 2010" Example: "Give me some random jokes"

This author put together a google sheets-based API using topical Late Night Monologue jokes from Newsmax's vast collection from 2009-Present. A call to the API will yield all hosts' jokes (Fallon, Kimmel, Colbert, etc.) combined for a specific night. This request defaults to the most recent airdate, but users can select a specific date or request jokes from a random date as well.

Help

There are two helper commands which give more information to the user:

Example: "Please help" Example: "Which news sources can I choose from again?"

These commands do not utilize any APIs, and are hard-coded text messages to remind the user of TIA's commands and which news sources are available.

^* New gmail e-mails are automatically forwarded from TIA to the user, and can be deleted/marked-as-read on request. However, this features is not currently available via the primary TIA number, and must be self-hosted -- see gmailfeatures.py file.

tia-text-internet-assistant