# Twilio AI Provider Switch Guide

## Overview

Your Twilio voice integration supports **two AI providers** for conversational AI:

1. **OpenAI Realtime API** - Maximum control, custom function calling (~$0.30/min)
2. **ElevenLabs Conversational AI** - Cost-effective, native agent platform (~$0.10/min)

You can switch between them with a single environment variable!

## Quick Switch Guide

### Step 1: Set the Provider

In your `.env` file:

```bash
# Use OpenAI (default)
TWILIO_AI_PROVIDER=OPENAI

# Or use ElevenLabs
TWILIO_AI_PROVIDER=ELEVENLABS
```

### Step 2: Configure Provider Credentials

#### For OpenAI:
```bash
OPENAI_API_KEY=sk-proj-xxxxxxxxxxxxx
```

#### For ElevenLabs:
```bash
ELEVENLABS_API_KEY=sk_xxxxxxxxxxxxx
ELEVENLABS_AGENT_ID=your-agent-id
```

### Step 3: Restart Your Application

```bash
mvn spring-boot:run
```

That's it! All Twilio calls will now use your selected AI provider.

---

## Detailed Configuration

### OpenAI Realtime API Setup

**Cost**: ~$0.06/min input + $0.24/min output = ~$0.30/min total

**Features**:
- Maximum control over conversation flow
- Custom function calling for structured data extraction
- Real-time bidirectional audio streaming
- Full access to GPT-4o capabilities

**Setup**:
1. Get your API key from https://platform.openai.com/api-keys
2. Add to `.env`:
   ```bash
   OPENAI_API_KEY=sk-proj-xxxxxxxxxxxxx
   TWILIO_AI_PROVIDER=OPENAI
   ```

**Advantages**:
✅ Complete customization of system prompts  
✅ Built-in function calling for patient data extraction  
✅ Proven stability and reliability  
✅ Full conversation transcription  

**Disadvantages**:
❌ Higher cost (~$0.30/min)  
❌ More complex integration  

---

### ElevenLabs Conversational AI Setup

**Cost**: ~$0.10/min (67% cheaper than OpenAI)

**Features**:
- Native conversational AI agent
- Pre-configured voice personalities
- Built-in tool/function support
- Lower latency in many regions

**Setup**:
1. Sign up at https://elevenlabs.io
2. Create a Conversational AI agent in the dashboard
3. Get your API key and Agent ID
4. Add to `.env`:
   ```bash
   ELEVENLABS_API_KEY=sk_xxxxxxxxxxxxx
   ELEVENLABS_AGENT_ID=your-agent-id
   TWILIO_AI_PROVIDER=ELEVENLABS
   ```

**Advantages**:
✅ 67% cost savings vs OpenAI  
✅ Simpler setup with dashboard configuration  
✅ Native voice optimization  
✅ Built-in tool support  

**Disadvantages**:
❌ Less control over prompt engineering  
❌ Dashboard-based configuration (vs code)  
❌ Newer platform (less proven at scale)  

---

## ElevenLabs Agent Configuration

When using ElevenLabs, you need to configure your agent in the ElevenLabs dashboard:

### 1. Create Agent
- Go to https://elevenlabs.io/app/conversational-ai
- Click "Create New Agent"
- Name it "Medical Assistant" (or your preferred name)

### 2. Configure System Prompt & First Message

**Important**: When using ElevenLabs, the agent's "first message" replaces the TwiML welcome message. The code will NOT play the `<Say>` greeting when `TWILIO_AI_PROVIDER=ELEVENLABS`.

#### System Prompt
Copy the medical assistant prompt from `src/main/java/com/saas/voip/service/OpenAIRealtimeService.java` (lines 43-133) and paste it into the ElevenLabs agent configuration.

#### First Message (REQUIRED!)
Configure the agent's **first message** to include:
```
Bonjour, vous êtes en contact avec la Clinique La Rive Bleue. Cet appel est enregistré afin d'améliorer la qualité de nos services. Comment puis-je vous aider aujourd'hui ?
```

⚠️ **Without a "first message" configured, the agent will be silent after connection!**

### 3. Configure Tools/Functions

Add these tools to extract patient data:

**Tool: `enregistrer_patient`**

```json
{
  "name": "enregistrer_patient",
  "description": "Enregistre les informations du patient et son rendez-vous une fois toutes les données collectées et confirmées",
  "parameters": {
    "type": "object",
    "properties": {
      "nom": {
        "type": "string",
        "description": "Nom complet du patient"
      },
      "date_naissance": {
        "type": "string",
        "description": "Date de naissance du patient au format JJ/MM/AAAA"
      },
      "telephone": {
        "type": "string",
        "description": "Numéro de téléphone du patient"
      },
      "maladie": {
        "type": "string",
        "description": "Motif de consultation ou maladie du patient"
      },
      "motif_visite": {
        "type": "string",
        "description": "Motif détaillé de la visite ou consultation"
      },
      "appointment_date_time": {
        "type": "string",
        "description": "Date et heure du rendez-vous au format ISO 8601 (YYYY-MM-DDTHH:MM:SS)"
      },
      "doctor_name": {
        "type": "string",
        "description": "Nom complet du médecin pour le rendez-vous"
      },
      "appointment_confirmed": {
        "type": "boolean",
        "description": "true si le rendez-vous est confirmé par le patient, false sinon"
      }
    },
    "required": ["nom", "date_naissance", "telephone", "maladie"]
  }
}
```

**Webhook URL**: `https://your-domain.com/api/voip/elevenlabs/call-summary`

This webhook receives the extracted patient data when the agent calls the function.

### 4. Voice Selection

Choose a French voice that sounds professional and empathetic. Recommended:
- **Charlotte** - Warm, professional female voice
- **Antoine** - Calm, reassuring male voice

### 5. Save Agent ID

After creating the agent, copy the Agent ID and add it to your `.env` file.

---

## Architecture Details

### How the Switch Works

The system uses a **Factory Pattern** to instantiate the correct AI handler:

```
TwilioMediaStreamHandler
  ↓
VoiceAiSessionFactory (checks TWILIO_AI_PROVIDER)
  ↓
├── OpenAiSessionHandler → OpenAIRealtimeService
└── ElevenLabsSessionHandler → ElevenLabs WebSocket
```

### Welcome Message Handling

**OpenAI Mode** (`TWILIO_AI_PROVIDER=OPENAI`):
- TwiML plays `<Say>` welcome message **before** WebSocket connects
- OpenAI takes over after the welcome message

**ElevenLabs Mode** (`TWILIO_AI_PROVIDER=ELEVENLABS`):
- TwiML **skips** the `<Say>` element (no code greeting)
- WebSocket connects immediately
- Agent's "first message" from dashboard plays as the greeting
- ⚠️ **Critical**: You MUST configure a "first message" in the ElevenLabs dashboard, or the call will be silent!

### Code Flow

1. **Twilio call comes in** → TwilioVoiceController generates TwiML with WebSocket URL
2. **WebSocket connects** → TwilioMediaStreamHandler creates AI session handler via factory
3. **Audio streaming** → Handler processes bidirectional audio (Twilio ↔ AI Provider)
4. **Data extraction** → 
   - **OpenAI**: Function calling → Direct save to database
   - **ElevenLabs**: Tool callback webhook → ElevenLabsCallbackController → Save to database
5. **SMS confirmation** → Sent if appointment is confirmed
6. **Call ends** → Session closed, cleanup

### Multi-Tenant Routing

Both providers support the same multi-tenant routing:
- **Call data** → Saved to tenant-specific schema
- **Phone number lookup** → Identifies tenant
- **TenantContext** → Automatically switches to correct database

---

## Cost Comparison

| Provider | Input Cost | Output Cost | Total/Min | Savings |
|----------|-----------|-------------|-----------|---------|
| **OpenAI** | $0.06/min | $0.24/min | **$0.30/min** | Baseline |
| **ElevenLabs** | - | - | **$0.10/min** | **67% cheaper** |

### Cost Example (1000 minutes/month)

- **OpenAI**: $300/month
- **ElevenLabs**: $100/month
- **Savings**: $200/month

---

## Testing Your Configuration

### 1. Check Logs on Startup

When your app starts, you should see:

```
INFO  c.s.voip.config.VoiceAiConfig - AI Provider configured: OPENAI
```

or

```
INFO  c.s.voip.config.VoiceAiConfig - AI Provider configured: ELEVENLABS
```

### 2. Test with Twilio

Make a call to your Twilio number. In the logs, you should see:

**For OpenAI**:
```
INFO  c.s.voip.handler.TwilioMediaStreamHandler - Created AI session handler: OpenAiSessionHandler
INFO  c.s.voip.handler.OpenAiSessionHandler - OpenAI handler - Client connected: callSid=CAxxxxx
```

**For ElevenLabs**:
```
INFO  c.s.voip.handler.TwilioMediaStreamHandler - Created AI session handler: ElevenLabsSessionHandler
INFO  c.s.voip.handler.ElevenLabsSessionHandler - ElevenLabs handler - Client connected: callSid=CAxxxxx
```

### 3. Verify Data Extraction

After a successful call with patient information:

**Check database**:
```sql
SELECT * FROM tenant_xxx.inbound_call_request ORDER BY created_at DESC LIMIT 1;
```

You should see the patient data (nom, telephone, appointment details, etc.)

---

## Troubleshooting

### Issue: "No AI provider configured"

**Solution**: Set `TWILIO_AI_PROVIDER` in `.env`

```bash
TWILIO_AI_PROVIDER=OPENAI
```

### Issue: "ElevenLabs connection failed"

**Checklist**:
- ✅ Valid `ELEVENLABS_API_KEY`
- ✅ Valid `ELEVENLABS_AGENT_ID`
- ✅ Agent is active in ElevenLabs dashboard
- ✅ Webhook URL is publicly accessible (ngrok/Replit)

### Issue: "ElevenLabs agent is silent / no response"

**Most Common Cause**: No "first message" configured in the dashboard!

**Solution**:
1. Go to your ElevenLabs agent settings
2. Find the "First Message" field
3. Add your welcome message (in French for this use case)
4. Save the agent

Without this, the agent connects but doesn't speak.

**Also Check**:
- ✅ Agent is "Active" in the dashboard
- ✅ API key and Agent ID are correct in `.env`
- ✅ Check logs for "ElevenLabs handler - Client connected" message

### Issue: "Patient data not saved with ElevenLabs"

**Checklist**:
- ✅ Tool `enregistrer_patient` configured in agent
- ✅ Webhook URL points to `/api/voip/elevenlabs/call-summary`
- ✅ Webhook is publicly accessible
- ✅ Check logs for callback errors

### Issue: "OpenAI works but ElevenLabs doesn't"

This is normal! The implementations are different:
- **OpenAI**: Direct WebSocket with function calling
- **ElevenLabs**: Agent tools + webhook callback

Make sure the ElevenLabs agent is properly configured with tools and webhook.

---

## Best Practices

### Development
- Use **OpenAI** for development (more control, easier debugging)
- Test function calling and prompt engineering

### Production
- Use **ElevenLabs** for production (cost savings)
- Monitor call quality and data extraction accuracy
- Have OpenAI as fallback option

### Switching Strategy
1. Test thoroughly with both providers
2. Compare data extraction accuracy
3. Consider cost vs control tradeoffs
4. Switch via environment variable (no code changes needed!)

---

## Environment Variable Summary

```bash
# Required for all
TWILIO_ACCOUNT_SID=ACxxxxx
TWILIO_AUTH_TOKEN=xxxxx
TWILIO_PHONE_NUMBER=+1234567890
SERVER_BASE_URL=https://your-domain.com

# Choose Provider
TWILIO_AI_PROVIDER=OPENAI  # or ELEVENLABS

# OpenAI Configuration (if using OpenAI)
OPENAI_API_KEY=sk-proj-xxxxx

# ElevenLabs Configuration (if using ElevenLabs)
ELEVENLABS_API_KEY=sk_xxxxx
ELEVENLABS_AGENT_ID=your-agent-id
```

---

## Need Help?

- **OpenAI Issues**: Check https://platform.openai.com/docs/guides/realtime
- **ElevenLabs Issues**: Check https://elevenlabs.io/docs/conversational-ai
- **Twilio Issues**: Check https://www.twilio.com/docs/voice/media-streams

---

**Last Updated**: October 2025  
**Version**: 1.0
