Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
131 changes: 131 additions & 0 deletions fern/calls/voice-mail-detection.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,131 @@
Voicemail is basically a digital answering machine. When you can’t pick up, callers can leave a message so you don’t miss anything important. It’s especially handy if you’re in a meeting, driving, or just can’t get to the phone in time.

### **The Main Problem**

If a lot of your calls are landing in voicemail, you could be spending too much time and money on calls that never connect to a real person. This leads to wasted resources, and sometimes missed business opportunities.

### **The Solution: Early Voicemail Detection**

By detecting voicemail right away, your VAPI Assistant can either hang up (if leaving a message isn’t necessary) or smoothly play a recorded message. This cuts down on useless call time and makes your entire call flow more efficient.

## **Two Ways to Detect Voicemail**

### **1. Using Twilio’s Voicemail Detection**

Twilio has built-in features to detect when a machine picks up. You configure these settings in your VAPI Assistant so it knows when a voicemail system has answered instead of a live person.

```jsx
voicemailDetection: {
provider: "twilio",
voicemailDetectionTypes: [
"machine_start",
"machine_end_beep",
"machine_end_silence",
"unknown",
"machine_end_other"
],
enabled: true,
machineDetectionTimeout: 15,
machineDetectionSpeechThreshold: 2500,
machineDetectionSpeechEndThreshold: 2050,
machineDetectionSilenceTimeout: 2000
}

```

- **provider**: Tells VAPI to use Twilio’s system.
- **voicemailDetectionTypes**: Defines the events that mean “voicemail.”
- **machineDetectionTimeout**: How many seconds to wait to confirm a machine.
- The other settings let you fine-tune how quickly or accurately Twilio identifies a machine based on speech or silence.

#### Quick Reference

| Setting | Type | Valid Range | Default |
| ---------------------------------- | ------ | ------------- | -------- |
| machineDetectionTimeout | number | 3 – 59 (sec) | 30 (sec) |
| machineDetectionSpeechThreshold | number | 1000–6000 ms | 2400 ms |
| machineDetectionSpeechEndThreshold | number | 500–5000 ms | 1200 ms |
| machineDetectionSilenceTimeout | number | 2000–10000 ms | 5000 ms |

### **2. Using VAPI’s Built-In Voicemail Tool**

VAPI also has an LLM-powered tool that listens for typical voicemail greetings or prompts in the call’s audio transcription. If you prefer an approach that relies more on phrasing and context clues, this is a great option.

```jsx
{
...yourExistingSettings,
"model": {
"tools": [{ type: "voicemail" }]
}
}

```

Here, `tools: [{ type: "voicemail" }]` signals that your VAPI Assistant should look for keywords or patterns indicating a voicemail greeting.

## **Combining Both Approaches**

For the best of both worlds, you can enable Twilio’s detection **and** the built-in voicemail tool at the same time:

```jsx
{
...yourExistingSettings,
voicemailDetection: {
provider: "twilio",
voicemailDetectionTypes: [
"machine_start",
"machine_end_beep",
"unknown"
],
enabled: true,
machineDetectionTimeout: 15
},
model: {
tools: [{ type: "voicemail" }]
}
}

```

When one method doesn’t catch it, the other might—boosting your overall detection accuracy.

## **Tips for Better Voicemail Handling**

1. **Adjust Detection Timing**

Lower `machineDetectionTimeout` (e.g., to 5 seconds) if you want the system to decide faster. But remember, shorter timeouts can lead to occasional false positives.

2. **Fine-Tune Speech and Silence Thresholds**

For example:

```jsx
{
"provider": "twilio",
"enabled": true,
"machineDetectionTimeout": 5,
"machineDetectionSpeechThreshold": 2400,
"machineDetectionSpeechEndThreshold": 1000,
"machineDetectionSilenceTimeout": 3000
}

```

These values tweak how quickly Twilio “listens” for human speech or background silence.

3. **Think Through Your Call Flow**
- **Give It Time**: If you’re leaving a message, you might want to increase `startSpeakingPlan.waitSeconds` so the detection has enough time before the tone.
- **firstMessageMode**: Setting it to `assistant-waits-for-user` can also give you smoother call handling—your assistant won’t barge in if someone unexpectedly picks up late

## **What Happens When a Call Ends?**

- **Detected Voicemail + No Message**: The call will end, and you’ll see a reason like `customer-did-not-answer`.
- **Detected Voicemail + Have a Message**: Your assistant leaves the recorded message, and the call ends with a reason like `voicemail`.

## **Testing and Next Steps**

1. **Make a Test Call**: Dial a known voicemail number and watch how quickly (and accurately) your VAPI Assistant identifies the machine.
2. **Tweak Settings**: Adjust your timeout and threshold values based on real-world performance.
3. **Repeat**: Keep testing until you’re confident your configuration is catching voicemail reliably without cutting off real people.

By following these steps, you’ll save time, improve call-handling efficiency, and ensure your system feels more professional. If you need to fine-tune or add new features later, you can always revisit these settings and make quick adjustments.
4 changes: 4 additions & 0 deletions fern/docs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -238,6 +238,8 @@ navigation:
path: squads.mdx
- page: Example
path: squads-example.mdx
- page: Silent Transfers
path: squads/silent-transfers.mdx
- section: Advanced Concepts
contents:
- section: Calls
Expand All @@ -250,6 +252,8 @@ navigation:
path: calls/call-ended-reason.mdx
- page: Live Call Control
path: calls/call-features.mdx
- page: Voice Mail Detection
path: calls/voice-mail-detection.mdx
- section: SIP
contents:
- page: SIP Introduction
Expand Down
237 changes: 237 additions & 0 deletions fern/squads/silent-transfers.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,237 @@
- **The Problem**: In traditional AI call flows, when transferring from one agent to another, announcing the transfer verbally can confuse or annoy callers and disrupt the conversation's flow.
- **The Solution**: Silent transfers keep the call experience _uninterrupted_, so the user doesn’t know multiple assistants are involved. The conversation flows more naturally, boosting customer satisfaction.

If you want to allow your call flow to move seamlessly from one assistant to another _without_ the caller hearing `Please hold while we transfer you` here’s what to do:

1. **Update the Destination Assistant’s First Message**
- Set the assistant’s `firstMessage` to an _empty string_.
- Make sure the `firstMessageMode` is set to `assistant-speaks-first-with-model-generated-message`.
2. **Trigger the Transfer from the Source Assistant**

- In that assistant’s prompt, include a line instructing it to transfer to the desired assistant:

```json
trigger the transferCall tool with 'assistantName' Assistant.
```

- Replace `'assistantName'` with the exact name of the next assistant.

3. **Direct the Destination Assistant’s Behavior**
- In that assistant’s prompt, include a line instructing it to _`Proceed directly to the Task section without any greetings or small talk.`_
- This ensures there’s no awkward greeting or “Hello!” when the next assistant begins speaking.

### **Example Usage Scenario**

- **HPMA (Main Assistant)** is talking to the customer. They confirm the order details and then quietly passes the conversation to **HPPA (Payment Assistant)**.
- **HPPA** collects payment details without the customer ever hearing, `We’re now transferring you to the Payment Assistant.` It feels like one continuous conversation.
- Once payment is done, **HPPA** transfers the call again—this time to **HPMA-SA (Main Sub Assistant)**—which takes over final shipping arrangements.

Everything happens smoothly behind the scenes!

## **Squad and Assistant Configurations**

Below are the key JSON examples you’ll need. These show how to structure your assistants and squads so they work together for silent transfers.

### **HP Payment Squad With SubAgent**

```json
{
"members": [
{
"assistantId": "2d8e0d13-1b3c-4358-aa72-cf6204d6244e",
"assistantDestinations": [
{
"message": " ",
"description": "Transfer call to the payment agent",
"type": "assistant",
"assistantName": "HPPA"
}
]
},
{
"assistantId": "ad1c5347-bc32-4b31-8bb7-6ff5fcb131f4",
"assistantDestinations": [
{
"message": " ",
"description": "Transfer call to the main sub agent",
"type": "assistant",
"assistantName": "HPMA-SA"
}
]
},
{
"assistantId": "f1c258bc-4c8b-4c51-9b44-883ab5e40b2f",
"assistantDestinations": []
}
],
"name": "HP Payment Squad With SubAgent"
}
```

### **HPMA Assistant (Main Assistant)**

```json
{
"name": "HPMA",
"voice": {
"voiceId": "248be419-c632-4f23-adf1-5324ed7dbf1d",
"provider": "cartesia",
"fillerInjectionEnabled": false
},
"createdAt": "2024-11-04T17:15:08.980Z",
"updatedAt": "2024-11-30T13:04:58.401Z",
"model": {
"model": "gpt-4o",
"messages": [
{
"role": "system",
"content": "[Identity]\nYou are the Main Assistant..."
}
],
"provider": "openai",
"maxTokens": 50,
"temperature": 0.3
},
"firstMessage": "",
"transcriber": {
"model": "nova-2",
"language": "en",
"provider": "deepgram"
},
"backchannelingEnabled": false,
"backgroundDenoisingEnabled": false,
"isServerUrlSecretSet": false
}
```

(Similar JSON information for the HPPA and HPMA-SA assistants can follow, just like in the original text.)

## **Assistant Prompts (In Plain Text)**

Each assistant has its own system prompt outlining identity, context, style, and tasks. These prompts ensure the conversation is smooth, customer-centric, and aligned with your call flow needs. Here’s a streamlined version for reference:

### **HPMA (Main Assistant Prompt)**

```
[Identity]
You are the Main Assistant, a friendly and helpful agent assisting customers
in purchasing widgets over the phone.

[Context]
You're engaged with the customer to book an appointment.
Stay focused on this context and provide relevant information.
Once connected to a customer, proceed to the Task section.
Do not invent information not drawn from the context.
Answer only questions related to the context.

[Style]
- Be polite and professional.
- Use a conversational and engaging tone.
- Keep responses concise and clear.

[Response Guidelines]
- Ask one question at a time and wait for the customer's response before
proceeding.
- Confirm the customer's responses when appropriate.
- Use simple language that is easy to understand.
- Never say the word 'function' nor 'tools' nor the name of the
Available functions.
- Never say ending the call.
- Never say transferring.

[Task]
1.Greet the customer and ask if they are interested in purchasing widgets.
- Wait for the customer's response.
2. If the customer is interested, ask for their name.
- Wait for the customer's response.
3.Ask how many widgets the customer would like to purchase.
- Wait for the customer's response.
4.Confirm the order details with the customer.
- trigger the transferCall tool with Payment `HPPA` Assistant.
```

### **HPPA (Payment Assistant Prompt)**

```
[Identity]
You are the Payment Assistant, operating in secure mode to collect payment information from customers safely and confidentially.

[Context]
You're engaged with the customer to collect payment details. Stay focused
on this context and provide relevant information.
Do not invent information not drawn from the context.
Answer only questions related to the context.
Once connected to a customer, proceed to the Task section without
any greetings or small talk.

[Style]
- Be professional and reassuring.
- Maintain confidentiality at all times.
- Speak clearly and calmly.

[Response Guidelines]
- Collect the customer's credit card number, expiration date, and CVV.
- Confirm each piece of information after it is provided.
- Ensure the customer feels secure during the transaction.
- Do not record or log any information.
- Never say the word 'function' nor 'tools' nor the name of the
Available functions.
- Never say ending the call.
- Never say transferring.

[Task]
1. Ask for the credit card number.
- Wait for the customer's response.
2. Ask for the expiration date of the card.
- Wait for the customer's response.
3. Ask for the CVV number.
- Wait for the customer's response.
4. Confirm that the payment has been processed successfully.
- trigger the transferCall tool with Payment `HPMA-SA` Assistant.
```

### **HPMA-SA (Main Sub Assistant Prompt)**

```
[Identity]
You are the Main Assistant, a friendly and helpful agent assisting customers
in purchasing widgets over the phone.

[Context]
You're engaged with the customer to book an appointment.
Stay focused on this context and provide relevant information.
Do not invent information not drawn from the context.
Answer only questions related to the context.
Once connected to a customer, proceed to the Task section without any greetings
or small talk.

[Style]
- Be professional and reassuring.
- Maintain confidentiality at all times.
- Speak clearly and calmly.

[Response Guidelines]
- Collect the customer's credit card number, expiration date, and CVV.
- Confirm each piece of information after it is provided.
- Ensure the customer feels secure during the transaction.
- Do not record or log any information.
- Never say the word 'function' nor 'tools' nor the name of the
Available functions.
- Never say ending the call.
- Never say transferring.

[Task]
1.Ask for the customer's shipping address to deliver the widgets.
- Wait for the customer's response.
2.Confirm the shipping address and provide an estimated delivery date.
3.Ask if the customer has any additional questions or needs further assistance.
- Wait for the customer's response.
4.Provide any additional information or assistance as needed.
5.Thank the customer for their purchase and end the call politely.
```

## **Conclusion**

By following these steps and examples, you can configure your call system to conduct **silent transfers** ensuring that callers experience a single, uninterrupted conversation. Each assistant does its job smoothly, whether it’s capturing payment, finalizing a shipping address, or collecting basic info.

Enjoy setting up your silent transfers!
Loading