Hyderabad · Los Angeles (I work US hours)last edited 28 Oct 2026

Abdul Rasheed Esa

Voice AI engineer, Hyderabad.

I'm interested in voice agents, latency, telephony and pretty much anything that has to happen in real time. Since February 2025 I've been the tech lead on FlowForge, the voice agent platform at Realbotics, on contract.

FlowForge Feb 2025 to now

We built it from zero with a team of four, and I built basically all of app.realbotics.ai. It does around 20k calls a day now, for LegalZoom, 1‑800Accountant and a couple of others. We run our own LiveKit (server, 4 SIP nodes, egress, TURN) with Twilio and customer SIP trunks on the phone side.

A lot of the work is figuring out where a call is losing time. Some of what I fixed:

• Handing a caller from one agent to another mid call used to take 5.5 seconds, with the caller on the line the whole time. It's 31ms now.
• A worker used to hold about 6 calls at once, now it's about 31. That took four bottlenecks, one after the other, and cold start p95 went from 4.6s to 1.7s on the way.
• Under campaign load SIP calls would go from 20% failing to 100% failing in 8 minutes. Found why and fixed it.
• Voicemail detection was treating 3.7% of real people as voicemail. Tuned it against 13k+ real calls and it's at 1%.
• Prompt cache hit rate went from 68% to 98%. On a phone call that matters, a miss is 2.1s to first token and a hit is 0.6s.
• We had issues with AWS, so I moved production to Hetzner and had it running there in under 24 hours. Deploys are blue/green and drain live calls before anything goes down.
• Load tested at 200 concurrent calls, 402 of 402 completed.
• Also a test bench for agents, with simulated callers that have personas, each tool mocked, and LLM judges.

Composer inside FlowForge

The copilot in FlowForge that builds agents with you. I designed it. The first version in Feb 2026 was a PydanticAI agent with 21 tools that could change pretty much anything on an agent: settings, voice, tools, knowledge base, flows, test chat. Now it sees whatever page you're on, can drive the UI for you and asks before it does anything consequential. It has a skills system too (SKILL.md files loaded from the DB, same shape as Claude Code skills). Around 440 of my commits mention it.

On my own GPU May 2026

A fully self-hosted voice agent on one RTX 6000 Ada on RunPod. Sesame CSM-1B fine-tuned with LoRA on a custom voice, next to a LiveKit worker. First audio went from 4.9s to about 315ms warm and live calls come in under 400ms end to end (my own measurement).

Same month I moved the end-of-turn model onto the GPU in fp16 PyTorch. p50 is 23ms, it used to be 300 to 400ms.

Open source Oct 2026

Four fixes in Mastra's realtime voice providers, each with tests, merged by their CTO.

• #26623 Nova Sonic: only the actual interrupted marker counts as a barge-in now
• #26624 OpenAI Realtime and Inworld always send tool output as a string
• #26697 OpenAI Realtime ends the speaker stream when a response has no audio
• #26698 Gemini Live cleans up speaker streams and session timers when the server closes

Before

• 2024 · KOKO AI, tech lead. Took a voice AI receptionist from zero to production, RAG over each business's own data, replies under 500ms (Twilio, Deepgram, ElevenLabs).
• 2023 · ML contract at an insurtech. Got vehicle damage detection from 65% to 94% accuracy and shipped it as an API doing 100+ claims a day.
• 2021–25 · B.E. in CS (AI & ML), MJCET, Osmania University. ICPC Asia Regionals. Smart India Hackathon 2022 grand finalist, out of 15,000+ teams.

If you've got a voice agent in production that's slower than it should be, email me, happy to look at it. rasheed@esa.bz