# LiveKit Agents plugin

URL: https://www.atmee.ai/docs/v2/livekit-plugin/livekit-agents-plugin

> Bring your own LiveKit voice agent; Atmee renders a talking-head avatar from a single portrait into your room and bills per minute.

## What it is

You already run a voice agent on [LiveKit Agents](https://docs/v2.livekit.io/agents/) with your own
speech recognition, model and voice. The Atmee plugin adds a face: a talking-head avatar,
rendered from one portrait, joins **your** LiveKit room as its own participant and speaks your
agent's audio with lips in sync. Atmee never sees your LiveKit secret and never hosts your
conversation; it only renders, and bills by the minute while it does. If you would rather have
Atmee run the whole conversation and only join a room from your page, use the Session API
instead; [Choose your integration](/docs/v2/api/choose-integration) compares the two.

```python
from livekit.plugins import atmee

avatar_id = await atmee.AtmeeAPI().create_avatar(name="Val", image="portrait.jpg")

avatar = atmee.AvatarSession(avatar_id=avatar_id)   # ATMEE_API_KEY in the environment
await avatar.start(session, room=ctx.room)
await session.start(agent=..., room=ctx.room)       # the agent's TTS drives the avatar's video
```

## Installation

```bash
pip install livekit-plugins-atmee
```

Python 3.10 or newer, `livekit-agents` 1.6.8 or newer.

## Prerequisites

* An **Atmee API key** (`sk_atmee_…`) with the `sessions` scope, in `ATMEE_API_KEY`. Keep it on
  your agent's side; it never goes to a browser. See [API keys](/docs/v2/api/api-keys).
* Your **own LiveKit project** (LiveKit Cloud or self-hosted): `LIVEKIT_URL`, `LIVEKIT_API_KEY`,
  `LIVEKIT_API_SECRET`, the same variables your agent already uses. The plugin mints the avatar's
  room token locally with your secret and hands only that token to Atmee.

## Usage

### 1. Create an avatar from a portrait

Once, from a script or your backend:

```python
async with atmee.AtmeeAPI() as api:
    avatar_id = await api.create_avatar(name="Val", image="portrait.jpg")   # path, bytes, or https URL
```

A portrait-only avatar is **render-only** (`kind: render_only`): it has no voice or knowledge of
its own and is ready immediately, because there is nothing to build. It can only be used through
avatar sessions, not on atmee.ai or through `POST /v1/sessions`. Any avatar of your account that has
a portrait works too, including ones created in the Studio or with a voice through
[`POST /v1/avatars`](/docs/v2/api/create-avatar).

### 2. Render it in your agent

```python
async def entrypoint(ctx: agents.JobContext):
    await ctx.connect()
    session = AgentSession(stt=..., llm=..., tts=..., vad=...)

    avatar = atmee.AvatarSession(avatar_id=os.environ["ATMEE_AVATAR_ID"])
    await avatar.start(session, room=ctx.room)        # before session.start

    await session.start(agent=MyAgent(), room=ctx.room)
```

`start()` returns as soon as Atmee's rendering worker acknowledged the start, about a second. The
avatar's video and audio tracks appear in the room a few seconds later, published by the
participant `atmee-avatar-agent`. Your agent publishes no audio of its own; the avatar speaks
for it.

**Atmee V2 avatars can take minutes to start.** They render on a separate pool of GPUs that scales
down when it's quiet, so the first V2 session after a quiet period can take a few minutes: Atmee
waits up to 4 minutes for the render to start and up to 6 for the avatar to join. Use
`wait_for="avatar_joined"` with a client timeout of at least 7 minutes, or retry on
`503 avatar_warming` after its `Retry-After` seconds.

Options on `AvatarSession`:

| Argument                      | Default                          | Meaning                                                                                    |
| ----------------------------- | -------------------------------- | ------------------------------------------------------------------------------------------ |
| `avatar_id`                   | required                         | The Atmee avatar to render.                                                                |
| `api_key`, `api_url`          | `ATMEE_API_KEY`, `ATMEE_API_URL` | Credentials and API base.                                                                  |
| `avatar_participant_identity` | `atmee-avatar-agent`             | Identity the avatar joins with; set it when one room hosts several avatars.                |
| `max_duration_seconds`        | `3600`                           | Hard ceiling of the session; also the most it can bill if your agent dies without a trace. |
| `wait_for`                    | `"initializing"`                 | `"avatar_joined"` makes `start()` block until the avatar is in the room.                   |
| `metadata`                    | `None`                           | Free-form JSON stored with the session on the Atmee side.                                  |

`avatar.session_id` and `avatar.session_info` hold the Atmee session after `start()`;
`avatar.on("avatar_disconnected", ...)` fires if the avatar participant leaves while your agent is
still running.

## Lifecycle and billing

Billing starts when the avatar sees your agent in the room and runs by the minute at the render
rate, from your account's credits: **0.5 credits per minute** for an Atmee V1 avatar and 1 credit
per minute for an Atmee V2 avatar. That is half a hosted session's rate, since Atmee only renders
(see [Costs](/docs/v2/api/overview#costs)). The plugin needs API access, which starts on the Starter
plan; Atmee V2 avatars need Growth or above. It stops when

* your agent leaves the room or the room closes (the rendering worker notices and reports the end
  itself, so a crashed agent stops being billed within seconds),
* `aclose()` runs, registered automatically as a job shutdown callback: it ends the session and
  removes the avatar participant from your room,
* the rendering worker itself dies or loses contact: it proves it is alive once a minute, and a
  session silent for a few minutes is closed and billed only up to the last proof,
* or the session reaches `max_duration_seconds`.

Each session is a normal API-key session for billing purposes: your key's spend cap, concurrency
limit and `billingMode` apply (see the [API reference](/docs/v2/api/reference/avatar-sessions/startAvatarSession)).

## Errors

All API failures raise `atmee.AtmeeException` with `status_code`, `code` and `message`. Two
subclasses are worth handling: `AtmeeNoCapacityError` (every rendering worker is busy; check
`retry_after` and try again later) and `AtmeeAvatarNotReadyError` (the avatar has no portrait
yet). The codes:

| Status | `code`                       | Meaning                                                                                                           |
| ------ | ---------------------------- | ----------------------------------------------------------------------------------------------------------------- |
| `400`  | `invalid_livekit_url`        | `livekitUrl` is not a `ws(s)://` server URL.                                                                      |
| `400`  | `invalid_livekit_token`      | Not a LiveKit access token, no room grant, or expired.                                                            |
| `400`  | `missing_publish_on_behalf`  | The token names no agent to render for (`lk.publish_on_behalf`).                                                  |
| `400`  | `agent_identity_mismatch`    | `agentIdentity` differs from the token's `lk.publish_on_behalf`.                                                  |
| `402`  | `insufficient_credits` …     | Out of credits, spend cap reached, or too many sessions for this key.                                             |
| `403`  | `avatar_not_allowed`         | The avatar is not on this key's allow-list.                                                                       |
| `409`  | `avatar_not_renderable`      | The avatar has no reference image to animate.                                                                     |
| `502`  | `avatar_start_failed`        | The worker refused the render before joining (for example the portrait has no usable face). Nothing is billed.    |
| `503`  | `no_capacity`                | Every rendering worker is busy; retry after `Retry-After` seconds.                                                |
| `503`  | `avatar_sessions_disabled`   | This deployment has no rendering backend.                                                                         |
| `503`  | `avatar_warming`             | An Atmee V2 avatar is being prepared on a rendering node; retry after `Retry-After` seconds. Nothing is billed.   |
| `503`  | `avatar_start_interrupted`   | The worker accepted the start but stopped before the avatar joined; retry after `Retry-After`. Nothing is billed. |
| `503`  | `avatar_version_unavailable` | The avatar is an Atmee V2 avatar and this deployment renders Atmee V1 only.                                       |
| `503`  | `hosted_room_unavailable`    | You left out the room details and this deployment can't host a room.                                              |

A token your LiveKit server rejects never joins; the session then fails as `AVATAR_DID_NOT_JOIN`
after the join timeout and is not billed.

## Endpoints

The plugin is a thin client over three endpoints. Use them directly from any language; all are
authenticated with `X-Api-Key` like the rest of the [API](/docs/v2/api/api-keys).

### `POST /v1/avatar_sessions`

Starts an avatar session. Responds `202` once the worker acknowledged the start; add
`?waitFor=avatar_joined` to block until the avatar participant is in your room.

| Field                | Type                    | Notes                                                                                                                                                                                                                                                                  |
| -------------------- | ----------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `avatarId`           | string (uuid), required | The avatar to render. Must be on the key's allow-list.                                                                                                                                                                                                                 |
| `livekitUrl`         | string                  | Your LiveKit server URL (`wss://<project>.livekit.cloud` or a self-hosted `ws(s)://` address). Send it with `livekitToken`; omit both for an Atmee-hosted room (see the [API reference](/docs/v2/api/reference/avatar-sessions/startAvatarSession)).                   |
| `livekitToken`       | string                  | Access token for the avatar participant, minted with **your** LiveKit key and secret: `identity` = the avatar's participant identity, `kind: agent`, a `roomJoin` grant for your room, attribute `lk.publish_on_behalf` = your agent's identity. Read, never verified. |
| `agentIdentity`      | string                  | Optional cross-check; must equal the token's `lk.publish_on_behalf`. In an Atmee-hosted room, your agent's identity (default `agent`).                                                                                                                                 |
| `maxDurationSeconds` | integer 60–14400        | Hard ceiling (default 3600). Capped by your key's limits.                                                                                                                                                                                                              |
| `metadata`           | object, ≤50 keys        | Stored with the session.                                                                                                                                                                                                                                               |

Response `202`: `sessionId`, `status` (`initializing` or `avatar_joined`),
`avatarParticipantIdentity`, `agentIdentity`, `roomName`, `maxDurationSeconds`, `billingMode`; in
an Atmee-hosted room also `livekitUrl` and `agentToken`, for your agent to join with.

### `GET /v1/avatar_sessions/{id}`

The session's durable state: `status` (`pending`, `active`, `completed`, `failed`),
`avatarJoinedAt`, `agentJoinedAt` (billing starts here), `endedAt`. `403 forbidden` for another
account's session, `404 not_found`.

### `POST /v1/avatar_sessions/{id}/end`

Stops billing now. Idempotent: ending a session that already ended answers `200` with
`alreadyEnded: true`, so you can call it from every teardown path. The worker's own report when
your agent leaves is the primary end signal; this call is the fallback, and the way to stop paying
for an avatar you are done with while your room stays open. It does not remove the avatar from
your room; remove the participant yourself (the plugin does so on close).

## How it works

1. `AvatarSession.start()` mints a LiveKit access token for the avatar participant with your
   LiveKit credentials: identity `atmee-avatar-agent`, `kind: agent`, a `roomJoin` grant for your
   room, and the attribute `lk.publish_on_behalf` set to your agent's identity. Atmee receives only
   that token, never your secret.
2. It calls `POST /v1/avatar_sessions` with the avatar's id, your LiveKit URL and the token.
   Atmee reserves a session, starts a rendering worker, and answers as soon as the worker
   acknowledged the start.
3. The worker joins your room with the token, waits for your agent, receives its audio over the
   LiveKit data stream (`lk.audio_stream`, 16 kHz PCM, the standard `DataStreamAudioOutput`), and
   publishes lip-synced video and audio on behalf of your agent.
4. When your agent leaves or the room closes, the worker notices from inside the room and reports
   the end to Atmee, which stops billing. `aclose()` removes the avatar participant and ends the
   session explicitly as well.

## Examples

The plugin repository ships two examples: `examples/agent.py`, a Deepgram + OpenAI + Silero agent
with an Atmee avatar (run with `python examples/agent.py dev` and open the LiveKit Agents
Playground), and `examples/create_avatar.py`, which creates an avatar from a portrait and prints
its id.
