Tel-Agent connects phone systems to self-hosted AI agents

What this is about

Tel-Agent is an open-source gateway between an existing phone system and an AI voice agent. Incoming SIP calls are routed to a person, blocked, or handed to the agent according to user-defined rules. The tool runs on privately controlled hardware and stores conversations, transcripts, and configuration in that installation.

Its practical appeal extends beyond phones. According to the project, the same agent can serve web chat, SMS, email, WhatsApp, Telegram, Messenger, Instagram, Discord, and Slack. It targets small teams, practices, and businesses that want to test a digital receptionist without handing all conversation data to a closed platform.

What Tel-Agent actually does

Tel-Agent connects to an existing system such as Asterisk, FreePBX, or 3CX as a SIP endpoint. It does not replace the PBX. A call passes through caller and destination rules. The agent can speak, handle interruptions, take a message, transfer a call, check a calendar, or call an approved HTTP endpoint.

The stack combines LiveKit Agents, FastAPI, Next.js, SQLite or PostgreSQL, Redis, and Caddy. The repository lists packages for Windows, macOS, and Linux as well as Docker Compose. The current default configuration requires outside services for speech recognition, the language model, and speech synthesis. Local models through Ollama, Whisper, and Piper are listed for a later stage.

Calls are recorded, transcribed, and made searchable. Credentials are encrypted at rest and are not returned in full to the browser. Published ports listen only on the local machine by default. Remote access requires a deliberate private-network, VPN, or TLS reverse-proxy setup.

Why it matters

Many voice agents are available only as hosted services. Tel-Agent separates communication channels from model providers: the team operates the gateway and supplies its own credentials. Routing, archives, contacts, and recordings therefore become more controllable.

That is particularly useful when telephony already uses SIP. An existing PBX can treat the agent like another extension. A general HTTP tool can also reach systems such as n8n or Home Assistant without turning Tel-Agent into a full automation platform.

The code uses AGPL-3.0. Anyone operating a modified version as a network service must publish those modifications under the license terms or arrange a commercial license. This directly affects how agencies and vendors can distribute their extensions.

In plain language

Tel-Agent is like a receptionist with a rule binder. For each call, the receptionist checks whether to connect it, reject it, or answer directly. Unlike an outsourced call center, the filing cabinet stays in your office, although voice and model services may still come from outside providers.

A practical example

A repair shop receives 80 calls per day, about 25 concerning opening hours or order status. Its FreePBX system sends those calls to a Tel-Agent extension. The agent answers standard questions from a reviewed knowledge base and may query order status only through a narrowly scoped, read-only HTTP endpoint.

Complaints or unclear names trigger a transfer to a person. The shop announces recording at the start, deletes test data on a defined schedule, and reviews 50 trial calls for incorrect answers. Only then does it use the agent outside business hours.

Scope and limits

First, the project calls itself alpha. Unsigned installers, early releases, and unfinished integrations mean production use needs additional testing and maintenance.

Second, self-hosted does not automatically mean fully local. The current default pipeline uses outside providers for recognition, the language model, and speech output. Data flows, processing agreements, storage locations, and deletion periods must be reviewed.

Third, call recording is legally sensitive. A technical notice does not replace checking applicable consent, information, and privacy duties. An automated agent also needs a reachable human fallback when recognition, routing, or an answer fails.

A sensible trial starts with one SIP extension, synthetic calls, and a read-only tool. Real customer calls should follow only after measuring misroutes, response latency, and handoffs.

SEO & GEO keywords

Tel-Agent, AI phone agent, SIP, self-hosted AI, voice agent, LiveKit Agents, phone automation, call routing, transcription, privacy, AGPL