Question-answering over Twilio voice recordings with LangChain.js

Prerequisites

• A Twilio account - sign up for a free Twilio account here
• A Twilio phone number with Voice capabilities - learn how to buy a Twilio Phone Number here
• Node.js (version 18 or above) installed - download Node.js here
• OpenAI account and API key – make an OpenAI account here and get an OpenAI API Key here
• ngrok, a handy utility to connect the development version of our Python application running on your machine to a public URL that Twilio can access.

Your .env file should contain one line:

OPENAI_API_KEY= {REPLACE-WITH-YOUR-OPENAI-API-KEY}

record.js

Records an inbound phone call to a Twilio phone number and transcribe the call

Run the file with node record.js and in another terminal tab, run ngrok http 3000. Grab that forwarding URL so you can configure your purchased Twilio phone number to send a request to your web application. Configure your Twilio phone # with the ngrok url so after calling the Twilio phone number, your application runs:

In your ngrok tab, you can see the /record and /handle_transcription endpoints were hit.

Now go to your local web server in the browser and append /recent_transcription (http://localhost:3000/recent_transcription). You can see the audio recording there in the browser!

handle_transcription.js

Code imports OpenAI so we can use their models, LangChain's loadQAStuffChain to make a chain with the LLM, and Document so we can create a Document the model can read from the audio recording transcription

Now, running the file (containing the speech from the movie Miracle) with node handle_transcription.js should yield the following output: