Voice and Real-Time Agent Design
Parent: AI Agent Ecosystems · researched 2026-06-01T02:26:30.180Z· 50 sources · 9 concepts · skill ai-agent-engineering
AI & agent-engineering family ROUTER. Split into: ai-agents-orchestration (agent frameworks, multi-agent, memory, planning, guardrails, coding/GUI agents, autonomous loops, eval); ai-rag-retrieval (RA
ai-agent-engineering
- AI & agent-engineering family ROUTER. Split into: ai-agents-orchestration (agent frameworks, multi-agent, memory, planning, guardrails, coding/GUI agents, autonomous loops, eval); ai-rag-retrieval (RAG, iterative retrieval, vector/graph datastores); ai-llm-model-layer (training, fine-tuning, alignment/RLHF, compression, inference serving, transformer/multimodal architecture, model selection, observability); ai-mcp-sdk-prompting (MCP servers/builder, Anthropic SDK, prompt engineering, context engineering, LLM frameworks, tool-search, prompt lookup). Route to the matching sub-hub. [source]
- This hub routes to on-demand reference files under references/. See each spoke for depth. [source]
Children
- Cascaded vs speech-to-speech architecture (frontier)
- Real-time APIs (OpenAI Realtime gpt-realtime, Gemini Live) (frontier)
- Turn detection & barge-in (VAD, semantic, Deepgram Flux, backchannel) (frontier)
- Voice latency budgeting (<800ms, LLM TTFT ~70%) (frontier)
- Streaming STT/TTS components (Deepgram, ElevenLabs, Cartesia) (frontier)
- Orchestration & telephony (LiveKit, Pipecat, Vapi/Retell/Bland, SIP) (frontier)
- Voice UX (confidence-tiered confirmation, error recovery, consent) (frontier)
- Voice-agent evaluation (4-layer, WER limits, interruption accuracy, MOS, TSR/FCR) (frontier)
Frontier under this node: Cascaded vs speech-to-speech architecture, Orchestration & telephony (LiveKit, Pipecat, Vapi/Retell/Bland, SIP), Real-time APIs (OpenAI Realtime gpt-realtime, Gemini Live), Streaming STT/TTS components (Deepgram, ElevenLabs, Cartesia), Turn detection & barge-in (VAD, semantic, Deepgram Flux, backchannel), Voice UX (confidence-tiered confirmation, error recovery, consent), Voice latency budgeting (<800ms, LLM TTFT ~70%), Voice-agent evaluation (4-layer, WER limits, interruption accuracy, MOS, TSR/FCR)