How Do I Stream Zoom, Meet or Teams Audio to My LLM?

MeetStream is a meeting bot API for AI agents: one API call sends a bot into Zoom, Google Meet or Microsoft Teams that can hear the room, speak and act, and stream audio and transcripts live. To feed an LLM, set live_audio_required.websocket_url. The bot connects to your server and streams 48 kHz PCM16 frames you resample and forward.

Updated October 2026. By Navaneeth Jawahar, CTO of MeetStream. MeetStream has run about 80,000 meeting bots and processed over a million meeting minutes. This guide covers the whole bridge: the exact wire format, the conversions OpenAI's Realtime API and Google's Gemini Live API need, how the agent talks back, and what breaks in long, busy meetings.

What does the path from a meeting to an LLM look like?

Two WebSockets and one small server. MeetStream connects to your bridge server twice: once to stream meeting audio in, and once to take commands out. Your bridge resamples the audio, forwards it to the model, and sends the model's spoken reply back into the meeting.

In the create_bot request (POST https://api.meetstream.ai/api/v1/bots/create_bot, header Authorization: Token <key>), set two fields:

• live_audio_required: {"websocket_url": "wss://bridge.example.com/audio"}. This is the inbound meeting audio.
• socket_connection_url: {"websocket_url": "wss://bridge.example.com/control"}. This is the control channel your bridge uses to make the bot speak, chat or stop talking.

MeetStream is the WebSocket client on both. You host the server, so it needs a public wss:// address; a tunnel works for local testing. Your server is the client when it talks to the model. The pattern is called a bridge server, and MeetStream's bridge server guide walks through it in more depth.

What exactly does MeetStream send over the audio WebSocket?

First a JSON text handshake, {"type": "ready", "bot_id": "..."}, and then binary frames. Each frame is a 1-byte message type (0x01), a length-prefixed speaker ID, a length-prefixed speaker name, and raw PCM16 little-endian audio at 48 kHz mono with no container. Both length prefixes are 2-byte unsigned integers, little-endian.

Four details matter in practice:

• The audio is mixed. Every frame contains the whole room in one mono stream. The speaker fields name the dominant speaker when the frame was captured, which is enough to attribute a question to a person. When attribution is not available, both fields read NoSpeaker.
• Frame sizes vary. Do not assume 20 ms. Compute duration from the bytes: len(pcm) / 2 / 48000 seconds. Twenty milliseconds is 1,920 bytes.
• Names are UTF-8. A name in Japanese or Hindi takes more bytes than characters, which is exactly why the length prefixes exist.
• Per-participant audio is separate. If you need one clean file per person, set audio_separate_streams: true (it is off by default) and fetch the per-participant files after the call. Zoom isolates each participant fully; Google Meet and Teams are partial.

The format is documented in MeetStream's real-time audio streaming guide, and the product overview is on the real-time audio API page, which puts live audio about 200 ms behind the room.

How do I parse the frames in Python?

Read the type byte, then each 2-byte little-endian length, then exactly that many bytes. Validate lengths before slicing, skip unknown message types, and hand the audio to a bounded queue so the socket read loop never waits on your model.

# pip install websockets
import asyncio, json, struct
from websockets.asyncio.server import serve

sessions = {}  # bot_id -> asyncio.Queue of (speaker_id, speaker_name, pcm)

def parse_frame(buf: bytes):
    """Return (speaker_id, speaker_name, pcm) or None if the frame is malformed."""
    if len(buf) < 5 or buf[0] != 0x01:        # 0x01 = PCM audio; skip unknown types
        return None
    (n,) = struct.unpack_from("<H", buf, 1)    # uint16, little-endian
    off = 3 + n
    if off + 2 > len(buf):
        return None
    speaker_id = buf[3:off].decode("utf-8", "replace")
    (m,) = struct.unpack_from("<H", buf, off)
    off += 2
    if off + m > len(buf):
        return None
    speaker_name = buf[off:off + m].decode("utf-8", "replace")
    pcm = buf[off + m:]
    return speaker_id, speaker_name, pcm[: len(pcm) - len(pcm) % 2]  # whole samples only

async def handler(ws):
    hello = json.loads(await ws.recv())        # {"type": "ready", "bot_id": "..."}
    q = sessions.setdefault(hello["bot_id"], asyncio.Queue(maxsize=200))
    if ws.request.path == "/control":           # MeetStream's control socket
        await run_agent(hello["bot_id"], q, ws)  # defined in the next sample
        return
    async for msg in ws:                        # the audio socket: binary frames
        frame = parse_frame(msg) if isinstance(msg, bytes) else None
        if frame is None:
            continue
        if q.full():
            q.get_nowait()                      # drop the oldest audio, never block
        q.put_nowait(frame)

async def main():
    async with serve(handler, "0.0.0.0", 8765, max_size=2**22):
        await asyncio.get_running_loop().create_future()

One server handles both sockets and tells them apart by path. Both start with the same handshake, so the bot_id ties the audio socket and the control socket to the same session. When the queue is full, the oldest audio is dropped. For a live agent, audio that is a few seconds stale is worth less than keeping up.

What audio format does each model expect?

MeetStream sends 48 kHz. OpenAI's Realtime API accepts PCM only at 24 kHz. Gemini Live is natively 16 kHz, but it resamples any rate you declare. Both models reply with 24 kHz PCM16, which you resample to 48 kHz for sendaudio, the rate MeetStream's control docs recommend.

Hop	Format and rate	Transport	What your bridge does	Bytes per 20 ms
MeetStream to bridge	PCM16 LE mono, 48 kHz, after the speaker header	Binary WebSocket frame	Strip the header	1,920
Bridge to OpenAI Realtime	audio/pcm, 16-bit mono, 24 kHz only	Base64 in input_audio_buffer.append	Resample 2:1, base64	960
Bridge to Gemini Live	16-bit PCM LE, audio/pcm;rate=16000	send_realtime_input with a Blob	Resample 3:1, or send 48 kHz and declare it	640
OpenAI Realtime to bridge	audio/pcm, 16-bit mono, 24 kHz	Base64 in response.output_audio.delta	Decode, resample 1:2	960
Gemini Live to bridge	16-bit PCM LE, 24 kHz	inline_data in the server content	Resample 1:2	960
Bridge to MeetStream	Raw PCM16 LE mono, 48 kHz	Base64 in a sendaudio command	Base64, pace near real time	1,920

Sources: OpenAI's Realtime client events reference ("Only a 24kHz sample rate is supported"), Google's Live API capabilities page, and MeetStream's control command docs. Base64 adds a third to every payload (RFC 4648), so a 48 kHz sendaudio stream is about 128,000 characters a second.

Resample with a real filter, and keep its state across chunks. Taking every second or third sample folds high frequencies back into the speech band as noise. A stateless resampler run chunk by chunk clicks at every boundary. The streaming resampler in python-soxr (soxr.ResampleStream) handles both. Python's old audioop.ratecv is gone: the module was removed in Python 3.13.

How do I pipe the audio into OpenAI Realtime or Gemini Live?

Open a WebSocket to the model when the control socket arrives. Configure 24 kHz PCM in and out, append resampled audio as it comes, and turn each audio delta into a paced sendaudio command. The code below continues the file above and was tested against a mock meeting and a mock model.

# pip install numpy soxr   (continues the file above)
import base64, os
import numpy as np, soxr
from websockets.asyncio.client import connect

OPENAI_URL = "wss://api.openai.com/v1/realtime?model=gpt-realtime-2.1"
AHEAD_S = 0.3  # most reply audio we let the meeting queue up

async def run_agent(bot_id, audio_q, control):
    down = soxr.ResampleStream(48000, 24000, 1, dtype="int16")
    up = soxr.ResampleStream(24000, 48000, 1, dtype="int16")
    out_q, st = asyncio.Queue(), {"gen": 0, "ends": 0.0}
    auth = {"Authorization": "Bearer " + os.environ["OPENAI_API_KEY"]}  # OpenAI's auth scheme
    async with connect(OPENAI_URL, additional_headers=auth, max_size=2**24) as oai:
        await oai.send(json.dumps({"type": "session.update", "session": {
            "type": "realtime", "instructions": "You are a meeting assistant. Be brief.",
            "audio": {"input": {"format": {"type": "audio/pcm", "rate": 24000},
                                "turn_detection": {"type": "server_vad"}},
                      "output": {"format": {"type": "audio/pcm", "rate": 24000}}}}}))

        async def to_model():
            while True:
                _, _, pcm = await audio_q.get()
                x = down.resample_chunk(np.frombuffer(pcm, dtype=np.int16))
                if len(x):
                    await oai.send(json.dumps({"type": "input_audio_buffer.append",
                                               "audio": base64.b64encode(x.tobytes()).decode()}))

        async def from_model():
            async for raw in oai:
                ev = json.loads(raw)
                if ev["type"] == "response.output_audio.delta":
                    y = up.resample_chunk(np.frombuffer(base64.b64decode(ev["delta"]), dtype=np.int16))
                    if len(y):
                        out_q.put_nowait(y.tobytes())
                elif ev["type"] == "input_audio_buffer.speech_started":  # someone talks over it
                    st["gen"] += 1
                    st["ends"] = 0.0
                    while not out_q.empty():
                        out_q.get_nowait()
                    await control.send(json.dumps({"command": "interrupt", "bot_id": bot_id,
                                                   "action": "clear_audio_queue"}))

        async def to_meeting():
            loop = asyncio.get_running_loop()
            while True:
                pcm48, gen = await out_q.get(), st["gen"]
                await asyncio.sleep(max(0.0, st["ends"] - loop.time() - AHEAD_S))
                if gen != st["gen"]:
                    continue                    # barge-in happened while we waited
                await control.send(json.dumps({"command": "sendaudio", "bot_id": bot_id,
                    "audiochunk": base64.b64encode(pcm48).decode(),
                    "sample_rate": 48000, "encoding": "pcm16"}))
                st["ends"] = max(st["ends"], loop.time()) + len(pcm48) / 96000  # 48 kHz x 2 bytes

        tasks = [asyncio.create_task(c) for c in
                 (to_model(), from_model(), to_meeting(), control.wait_closed())]
        _, pending = await asyncio.wait(tasks, return_when=asyncio.FIRST_COMPLETED)
        for t in pending:
            t.cancel()                          # bot left, or the model hung up

if __name__ == "__main__":
    asyncio.run(main())

A few choices in that code are deliberate. The two resamplers live for the whole session, so there are no clicks between chunks. The reply is paced so the meeting never has more than about 0.3 seconds queued, which keeps barge-in fast on every platform. The agent stops when the control socket closes, which happens when the bot leaves, so a finished meeting does not leave a model session running. The session shape follows OpenAI's Realtime conversations guide; model names change often, so check the current one there.

Gemini Live follows the same shape with different calls. Open a session with client.aio.live.connect(...) and send each chunk with session.send_realtime_input(audio=types.Blob(data=chunk, mime_type="audio/pcm;rate=16000")). Replies come from session.receive(): audio is in server_content.model_turn.parts[].inline_data.data at 24 kHz, and server_content.interrupted tells you a person cut in. Gemini resamples any input rate you declare, so you can send the 48 kHz audio as it arrives with rate=48000. Downsampling to 16 kHz first cuts upload bandwidth by two thirds.

How does the agent talk back into the meeting?

Send sendaudio commands over the control WebSocket: base64-encoded raw PCM16 little-endian mono audio, with its sample rate. MeetStream plays it through the bot's microphone and everyone in the meeting hears it. There is no documented way to play audio privately to one participant.

The command carries command, bot_id, audiochunk (the base64 audio), sample_rate and encoding ("pcm16"). Send long replies as a series of chunks rather than one large command, and pace them. Audio you have already handed to the bot is harder to take back than audio still sitting in your own queue.

Voice is not the only way to answer. The same control socket takes chat commands, and the REST API has POST /api/v1/bots/{bot_id}/send_message for the meeting chat and send_image to show an image or animated GIF as the bot's video. A bot does not share its own screen. A useful pattern is to speak a short answer and post the details, such as a link or a table, in chat.

How do turn-taking and interruption work in a meeting?

Let the model's voice activity detection find turns, but decide yourself when the agent should answer, and stop playback the moment a person talks over it. In a group call, an agent that replies to every pause will talk over people who are speaking to each other.

On OpenAI, server_vad defaults to a 0.5 threshold, 300 ms of prefix padding and 500 ms of silence, and semantic_vad judges turn ends from meaning instead. Set turn_detection.create_response to false and the model still detects turns but waits for you to send response.create. On Gemini, automatic activity detection is on by default, and its start and end sensitivities are configurable under realtimeInputConfig.automaticActivityDetection.

Good gates in a meeting are specific to meetings:

• The agent's name in MeetStream's live transcript, acted on when end_of_turn is true.
• A chat command, read with GET /api/v1/bots/{bot_id}/get_chats.
• The speaker tag on the frames, so only the meeting owner can trigger a reply.

When a person starts talking over the agent, do three things. Stop sending your queued audio. Send {"command": "interrupt", "bot_id": "...", "action": "clear_audio_queue"} on the control socket. Tell the model how much it actually said: on OpenAI, send conversation.item.truncate with the item_id, content_index and audio_end_ms of the cut, so its memory matches what the room heard. With server VAD on, OpenAI cancels the in-progress response itself.

Check platform support before you rely on interrupt alone. MeetStream's control docs list it as fully supported on Google Meet. On Zoom and Teams the command is accepted, but it does not yet clear audio already queued. Pacing is your guarantee on every platform: if the meeting never holds more than a few hundred milliseconds of reply, a barge-in costs at most that much.

If speech_started fires while the agent is talking and nobody else is, check whether the agent's own voice is reaching the model. If it is, pause the uplink while the agent speaks.

How do I handle backpressure and reconnection?

Never block the socket read loop, bound every queue, and treat stale audio as worthless. Key everything by the bot_id from the handshake, so a socket that reconnects rejoins its session instead of starting a new one, and plan for model session limits in long meetings.

• Know the rates. Inbound audio is 96,000 bytes a second per bot (48,000 samples of 2 bytes). A reader that falls behind lets buffers grow until something gives. Move all model I/O off the read loop, as the samples do.
• Bound by time. The samples cap the queue by frame count for brevity. In production, cap it by seconds of audio, since frames vary in size, and log every drop.
• Expect reconnects. MeetStream is the client, so your server must accept a new connection for a bot_id it already knows and reattach the queue and model session. Keep the model session alive for a short grace period after a socket drops.
• Plan for model limits. OpenAI caps a Realtime session at 60 minutes (OpenAI). Gemini limits audio-only sessions to 15 minutes without context window compression, and one connection lasts about 10 minutes, with a GoAway message first (Google). Turn on session resumption and compression, and carry a short summary into each new session.
• Follow the lifecycle. Set callback_url and branch on bot_event. Sockets connect when the bot joins, so bot.in_waiting_room means wait. bot.inmeeting means expect audio. bot.stopped, bot.kicked and bot.failed mean tear down.
• Zoom specifics. Zoom needs a one-time Marketplace app setup. Recording is gated on host consent, so watch for bot.recording_permission_denied. Bots joining meetings hosted by other accounts use an OBF token, and the bot is removed when the authorizing user leaves. See the Zoom meeting bot API page.

When should you use MIA voice agents instead of building this?

Use MeetStream's MIA voice agents when you want an agent that talks in the meeting without running a bridge server. MIA runs inside MeetStream, with wake words and interruption handling built in, and it is priced separately from bot hours. Build your own bridge when you need your own model, your own tools and data path, or turn-taking rules of your own.

	Your own bridge	MIA voice agent
What you run	A WebSocket server plus model connections	Nothing; it runs inside MeetStream
How you attach it	live_audio_required and socket_connection_url	A saved agent, referenced with agent_config_id in create_bot
Turn-taking	You build gating, pacing and barge-in	Wake words and interruption handling included
Cost	Bot time at $0.35 per hour, plus your model provider's bill	Priced separately
Best for	Custom models, tools and data paths	Shipping a voice agent without new infrastructure

The MIA docs cover agent setup. Either way, the agent joins through the same meeting bot API, so you can start with MIA and move to your own bridge later without changing how bots join.

Frequently asked questions

What audio format does MeetStream stream over WebSocket?

Raw PCM16, little-endian, 48 kHz, mono, with no container. Each binary frame starts with a 1-byte type (0x01) and two length-prefixed strings, the speaker ID and speaker name, and the first message on the socket is a JSON handshake with the bot_id.

Is the audio per speaker or mixed?

Mixed. Each frame is tagged with the dominant speaker, or NoSpeaker when attribution is not available. For one file per participant, enable audio_separate_streams and fetch the files after the call.

Do I need to resample for OpenAI Realtime or Gemini Live?

For OpenAI, yes: its Realtime API takes PCM only at 24 kHz, so downsample 2:1. Gemini Live accepts any rate you declare in the MIME type, but 16 kHz is its native rate and saves bandwidth. Both reply at 24 kHz, which you upsample to 48 kHz for sendaudio.

Can the bot speak in Zoom, Google Meet and Teams?

Yes, with sendaudio over the control socket on all three, and the whole meeting hears it. The interrupt command clears queued audio on Google Meet today, so pace your replies to keep barge-in fast on Zoom and Teams.

What does it cost?

Bot time is $0.35 per hour, prorated to the second, and $0.25 per hour for select startups and at volume. Your model provider bills its own usage. MIA voice agents are priced separately; see pricing.