Telephony agents built on Voice AI platforms like Retell AI make it easy for businesses to run live phone conversations through an AI agent instead of an IVR or human call center staff. Getting one of these agents into production means solving two things: integrating it with existing telephony infrastructure, and adding a compliance layer that preserves the organization’s security and governance standards.
On the integration side, the agent must connect to existing PBXs, SIP trunks, contact center workflows, business applications, and operational processes. On the compliance side, healthcare gives a concrete example: any AI agent handling patient information must keep HIPAA compliance intact.
In this article, we demonstrate a reference architecture for Retell AI integration into an existing telephony infrastructure, including one layer of a HIPAA-compliant system: an interception point that reviews LLM output using a SIP trunk. A custom Node.js and TypeScript service sits between Retell AI and an LLM, giving us an interception point before a generated response is spoken. This lets us minimize unnecessary PHI repetition, enforce required call-opening language, integrate appointment-management functions, and log what the model generated against what the caller ultimately heard. The same interception-point architecture applies anywhere a voice AI platform needs to respect rules a general-purpose LLM doesn’t know about on its own.
The SIP Trunk and Custom LLM Architecture
Retell AI runs a streaming STT → LLM → TTS pipeline with proprietary turn-taking models. You can use their built-in agents (Single Prompt, Conversation Flow), or bring your own LLM server via WebSocket. We’re taking the “Custom LLM” path (and choosing OpenAI for this example) because it gives us an interception point between the LLM and the caller: sentence-level PHI redaction, structured audit logging, and full control over what reaches the caller.
The two key elements:
- Elastic SIP Trunking. Your PBX creates a SIP trunk pointing at sip.retellai.com, and Retell picks up calls on that trunk. Your PBX doesn’t know or care what’s on the other end. It just sees another trunk. Don’t have a PBX handy? We provide a Terraform template that provisions a disposable FreePBX instance on EC2 for testing below.
- Custom LLM server (Node.js + TypeScript). Sits between Retell and OpenAI, streaming responses sentence-by-sentence with per-sentence compliance validation and audit logging. Runs locally with ngrok during development.
Here’s a demo with the full call with HIPAA disclosure, identity verification with PHI redaction, and appointment booking:
A note on response times: You’ll notice a considerable delay in the demo between the patient finishing and the agent responding. Most of that time comes from the fact that the server app waits for the full response from the model (for simplicity) and it runs locally, exposed through ngrok. This adds latency on every hop. In production, two things would bring this down significantly:
- Streaming directly from the LLM and redacting at sentence boundaries as tokens arrive
- Deploying the server as closest as possible (or next to if you host it) to your LLM provider instead of routing through a tunnel
The Compliance Layer for This Telephony Agent in Action
Here’s the turn-by-turn breakdown of what you just watched. The compliance layer handles each turn: modifying some responses, passing others clean, and logging everything.
Turn 1. Call Connects: HIPAA Disclosure Delivered
The WebSocket opens and the server sends the mandatory disclosure as a hardcoded begin message with no LLM involvement. Every call gets this, guaranteed.
| What the patient hears |
|---|
| “Thank you for calling Valley Health Clinic. Before we begin, I want to let you know that you are speaking with an AI assistant. This call may be recorded for quality and training purposes. If you have any medical concerns or need clinical advice, I will connect you directly with our nursing staff. How can I help you today?” |
Turn 2. Patient Identifies Themselves: Passes Clean
Patient: “Hey. I would like to reschedule my appointment. My name is James Wilson.”
| Field | Value |
|---|---|
| Raw LLM response | “I can help with that, James. To verify your identity, please provide your date of birth.” |
| Compliance action | pass |
| What the patient hears | (same as raw response — no modification needed) |
Turn 3. Identity Verification: PHI Redaction Fires
Patient: “Yeah. It’s November two nineteen seventy two.”
The LLM confirmed identity by repeating the patient’s full name and date of birth back. The compliance layer catches this, repeating PHI over the phone is unnecessary exposure.
| Field | Value |
|---|---|
| Raw LLM response | “I’ve confirmed your identity, James Wilson, date of birth November 2, 1972. Your next appointment is currently scheduled for July 12, 2026 at 3:30 PM. Would you like to see available times to reschedule?” |
| Compliance action | modify |
| Reason | Unnecessary PHI repetition redacted |
| What the patient hears | “I’ve verified your identity. Your next appointment is currently scheduled for July 12, 2026 at 3:30 PM. Would you like to see available times to reschedule?” |
Turns 4–5. Appointment Booking: Function Calling Works
Patient: “Yes, please.”
The agent calls get_available_slots, and offers options. The patient picks July 14 at 9 AM. The agent calls book_appointment. Both turns pass compliance clean.
| What the patient hears |
|---|
| “Here are some available appointment times: July 14th at 9:00 AM or 11:00 AM, July 15th at 2:00 PM, and July 16th at 10:00 AM or 3:30 PM. Which slot would you prefer?” |
| “I have booked your appointment for July 14th at 9:00 AM. If you need any further assistance, feel free to ask. Thank you!” |
The Audit Log
The demo also featured the server’s stdout which provides visibility on what the LLM tried to do and what ended up doing for each turn. For instance, here’s the log for turn 3 (Formatted for display width. Line breaks inside string values are not valid JSON, do not copy this snippet directly for parsing):
{
"timestamp": "2026-07-12T16:53:21.040Z",
"callId": "call_728462acbc9dcbe3df69e0bdf70",
"turnNumber": 3,
"transcriptIn": "Yeah. It's November two nineteen seventy two.",
"rawLlmResponse": "I've confirmed your identity, James Wilson, date of
birth November 2, 1972. Your next appointment is currently scheduled
for July 12, 2026 at 3:30 PM. Would you like to see available times
to reschedule?",
"complianceAction": "modify",
"complianceReason": "Unnecessary PHI repetition redacted",
"finalResponseSpoken": "I've verified your identity. Your next
appointment is currently scheduled for July 12, 2026 at 3:30 PM.
Would you like to see available times to reschedule?"
}
Every turn is recorded:
- what the LLM generated,
- whether the compliance layer modified it,
- and what was actually spoken to the caller.
If a response ever slips through that shouldn’t have, this log demonstrates you had controls in place and identifies the exact scope of the incident.
In production you’d pair this with your monitoring setup: alerting when compliance action rates change unexpectedly, periodic review of “pass” decisions to catch false negatives, and a process for updating rules when new PHI patterns emerge.
Building the Compliance Layer: The Middleware
Now let’s see what’s happening behind the scenes. If you want to follow along, the code from the demo above is available at WebRTCventures/retell-hipaa-demo on GitHub. You’ll need a Retell AI account with a Custom LLM agent configured, an OpenAI API key, Node.js 18+, and ngrok.
For the telephony side, you can configure a SIP trunk from your PBX pointing to sip.retellai.com as shown on (Retell custom telephony docs), or use the repo’s Terraform module to spin up a test FreePBX instance on EC2 and follow the steps provided.
The Custom LLM server sits between Retell and OpenAI. It generates responses via function calling, streams them sentence-by-sentence with per-sentence PHI redaction, and logs a complete audit entry after each turn.
The WebSocket Handler
When a call starts, Retell opens a WebSocket connection to your server at /llm-websocket/{call_id}. One connection per call. The most important event is response_required, which is Retell signaling that it’s the agent’s turn to speak.
The first thing the server does on connection is deliver the HIPAA disclosure. Retell’s protocol expects your server to send the opening message proactively:
// websocket-handler.ts — The begin message delivers HIPAA disclosure on connect
wss.on("connection", (ws: WebSocket, request: IncomingMessage) => {
const callId = extractCallId(request.url);
const session = new CallSession(callId);
const beginMessage = {
response_type: "response",
response_id: 0,
content: HIPAA_DISCLOSURE + "How can I help you today?",
content_complete: true,
end_call: false,
};
ws.send(JSON.stringify(beginMessage));
session.disclosureDelivered = true;
session.incrementTurn();
ws.on("message", (data) => handleMessage(ws, session, data));
ws.on("close", () => { session.end(); logCallSummary(session); });
});
Here’s the response_required handler. It generates the full response (including function calling), then streams it sentence-by-sentence with PHI redaction on each sentence:
// websocket-handler.ts — Stream response with per-sentence PHI redaction
async function handleResponseRequired(
ws: WebSocket,
session: CallSession,
event: ResponseRequiredEvent
): Promise<void> {
session.updateTranscript(event.transcript);
const llmResult = await generateResponse(session);
// Split into sentences, redact PHI per-sentence, stream to Retell
const sentences = llmResult.content.split(/(?<=[.!?])\s+/).filter(Boolean);
const spokenSentences: string[] = [];
for (let i = 0; i < sentences.length; i++) {
const { output } = redactSentence(sentences[i], session);
spokenSentences.push(output);
ws.send(JSON.stringify({
response_type: "response",
response_id: event.response_id,
content: output,
content_complete: i === sentences.length - 1,
end_call: false,
}));
}
session.incrementTurn();
// Log the complete turn after streaming finishes
const finalResponseSpoken = spokenSentences.join(" ");
logTurn({
timestamp: new Date().toISOString(),
callId: session.callId,
turnNumber: session.turnCount,
transcriptIn: event.transcript.filter((e) => e.role === "user").pop()?.content ?? "",
rawLlmResponse: llmResult.content,
complianceAction: finalResponseSpoken !== llmResult.content ? "modify" : "pass",
complianceReason: finalResponseSpoken !== llmResult.content
? "Unnecessary PHI repetition redacted"
: "Response passed compliance checks",
finalResponseSpoken,
});
}A few notes on this implementation:
- Sentence-level splitting enables surgical redaction. Rather than passing or blocking the entire response, the code splits it into sentences so it can replace only the one containing PHI. The check (does this sentence contain both the patient’s full name and date of birth?) is self-contained per sentence with no cross-sentence context needed.
- Production should stream from the LLM directly. For simplicity, the demo waits for the full LLM response before splitting (necessary when function calls are in-flight). In production, you’d stream tokens from OpenAI, buffer until you hit a sentence boundary, redact, and send to Retell immediately, cutting time-to-first-byte from seconds to sub-second for longer responses.
- Audit logging happens after the turn completes. The log entry records both the raw LLM output and what was actually spoken, providing the full before/after picture for compliance auditing. Pair this with alerting on compliance action rates: if “modify” actions spike or drop to zero, something has changed in LLM behavior that warrants investigation.
The Compliance Validator: PHI Redaction Logic
Here’s the PHI redaction logic in its entirety. This is the core of what makes this a compliance layer rather than just a chatbot:
// compliance-validator.ts — Per-sentence PHI redaction
export function redactSentence(sentence: string, session: CallSession): {
redacted: boolean;
output: string;
} {
if (!session.patientContext) {
return { redacted: false, output: sentence };
}
const { fullName, dob } = session.patientContext;
const lower = sentence.toLowerCase();
const dobFormats = getDobFormats(dob);
const hasDob = dobFormats.some((f) => lower.includes(f.toLowerCase()));
const hasName = lower.includes(fullName.toLowerCase());
if (hasName && hasDob) {
return { redacted: true, output: "I've verified your identity." };
}
return { redacted: false, output: sentence };
}If the LLM echoes back both the patient’s full name and their date of birth in the same sentence, replace it with a generic confirmation. The condition is AND: name alone (“Thanks, James Wilson”) is normal conversational flow, and DOB alone after the patient just stated it isn’t an additional disclosure. For this scenario, it’s the combination that creates an unnecessary linkage of identifiers over the phone.
A production implementation would expand the rule set: redact SSN patterns regardless of context, flag medication names repeated outside clinical necessity, apply different thresholds per data type, and likely use a policy engine (OPA, Cedar) rather than hardcoded conditions.
The main takeaway is having an interception point where code provides a verifiable enforcement layer on top of your prompt engineering.
Taking It to Production
The demo gives you the architecture pattern: SIP trunk integration, real-time compliance interception, function calling through to an EHR, and structured audit logging. Here’s what you layer on top for a production deployment.
Legal & Contracts
| Area | What to add |
|---|---|
| BAA chain | Signed BAAs with Retell, OpenAI (Enterprise/Team), TTS subprocessor, and AWS |
| LLM hosting | OpenAI with BAA, or self-hosted LLM (Llama-based) to keep PHI off third-party infra |
| Audit retention | Six-year minimum per HIPAA, Athena or similar for querying |
| Access controls | Role-based access to logs/recordings, documented justification required |
The BAA chain is the most common gap. HIPAA compliance is not a single vendor decision. It’s a contract chain across every service that touches PHI. Retell offers self-serve BAAs on all plans. OpenAI requires Enterprise or Team tier for a BAA; without one, patient transcripts cannot be sent through their API. A single missing link means PHI is being processed without contractual safeguards, which is a violation regardless of technical controls.
Security & Encryption
| Area | What to add |
|---|---|
| SIP transport | mTLS — mutual authentication between PBX and Retell |
| Media encryption | SRTP enforced — unencrypted RTP rejected |
| Data at rest | S3 with SSE-KMS encryption, Object Lock (compliance mode), key rotation |
| PHI in LLM context | Minimum necessary: function calling retrieves only what’s needed per turn |
| Session data | Cleared on call end, not persisted in server state |
Minimum necessary is the principle most often overlooked in LLM-based systems. A Production implementation should use function calling to retrieve patient data only when needed, clear session data when the call ends, and restrict audit log access by role. Operators see compliance actions, not raw transcripts.
Intelligence & Safety
| Area | What to add |
|---|---|
| Policy rules | Policy engine (OPA or Cedar) — rules vary by specialty and department |
| EHR | Epic FHIR / Cerner via OAuth, fuzzy matching, MPI lookups, write-back safety |
| Agent orchestration | Agent framework (LangGraph, Strands) or MCP for standardized tool interfaces |
| Streaming Tokens | Streaming tokens as they are generated to reduce delay between responses |
Operations & Scale
| Area | What to add |
|---|---|
| Deployment | Containerized behind a load balancer with WebSocket support, multi-AZ |
| Failover | Retell transfers to human if Custom LLM server unreachable |
| Scale | Horizontal scaling with sticky sessions, request queuing, rate limit management |
| SIP redundancy | Secondary trunk path (redundant PBX or cloud SIP provider as fallback) |
| Observability | Latency dashboards, compliance action alerting, function call error rate monitoring |
| Model Colocation | Deploy compliance layer server colocated (or as close as possible) with inference endpoint |
Extending the Compliance Layer to Production
This example shows one layer of a HIPAA-compliant system: the interception point that reviews LLM output before it reaches the caller. A production deployment adds the legal, security, and operational work covered above: signed BAAs across every vendor, encryption at rest and in transit, audit retention, and a policy engine that scales past hardcoded rules.
The same pattern applies beyond healthcare. Any telephony agent operating under rules a general-purpose LLM doesn’t know about on its own can use this interception-point architecture: financial services under PCI or SOC 2, legal intake under privilege rules, or any regulated call center workflow.
WebRTC.ventures builds telephony infrastructure for regulated industries. Our work includes SIP and PSTN bridging, real-time communication platforms, telehealth systems, and compliance-sensitive deployments. We have built multi-function teletherapy applications, call center platforms with remote interpreting, and telehealth systems that operate at scale.
Whether you’re running Asterisk, FreeSwitch, FreePBX, Avaya, or Cisco, we can help you integrate a telephony agent without replacing your existing infrastructure, keeping your compliance intact. Reach out today.
