Building Use Cortana's Multi-Tenant AI Agent Studio - Drag-and-Drop Agents Across Voice, SMS, Email & Web

Use Cortana is a multi-tenant AI SaaS for teams that want agentic growth - track what ads actually make money, generate more winning creatives, and follow up on every channel without hiring a bigger ops team.

The product has two major sides:

1. Ad tracking / media systems - attribution, creative loops, platform sync
2. AI-powered agents - configurable sales and support agents across voice and messaging

My responsibility sat firmly on the agent side: a drag-and-drop style frontend studio where operators create, configure, and ship chat and voice agents for their own organizations - including white-label-friendly multi-tenant workspaces.

Product walkthrough

Two short Loom demos show the agent studio end to end - multi-channel setup, live messaging with memory, Temporal/Restack orchestration, and the embeddable web widget.

Multi-tenant by design

Use Cortana is not a single-tenant chatbot. Multiple customers run the platform at once:

• Create organizations / workspaces
• Isolate configuration and data per tenant
• Support white-label-style presentation of the product for partners
• Attach agents to a specific business with its own knowledge base

That tenancy model shapes every UI decision - agent lists, KB uploads, Twilio numbers, and GHL connections all resolve in org context.

What the agent studio lets you build

Operators open the studio and configure an agent without waiting on engineering tickets.

Identity and conversation shape

• Give the agent a name
• Choose chat, voice, or multi-channel deployment
• Pick conversation style: milestone-based flows (structured stages) or single-note conversations (freer dialog anchored on one system prompt / note)
• Set first message, official prompt, and backend context

Channels

One agent configuration can span the channels sales teams actually use:

• Voice (phone)
• SMS
• Email
• WhatsApp
• Web chat (embeddable widget)

Knowledge base + RAG

Each connected business owns a knowledge base. Operators upload documents and official org material - the policies, offers, and FAQs the agent must respect.

A RAG layer on the backend:

• Retrieves business content at answer time
• Remembers conversation context so the agent does not re-ask or invent policy

Same discipline as my RAG production write-up, applied inside a multi-tenant SaaS.

Voice: from cascaded chain to Hume speech-to-speech

v1 - STT → LLM → TTS (LiveKit chain)

The first voice path was a classic cascade:

• LiveKit for realtime media
• STT / TTS providers (including ElevenLabs)
• LLM reasoning (including Grok and related model options in the stack)

It worked - but callers felt delay. Cascaded pipelines stack time-to-first-audio across three network hops. For sales speed-to-lead, that friction kills conversions.

v2 - Speech-to-speech with Hume

We switched the voice runtime to Hume speech-to-speech. The frontend still owns configuration; Hume owns low-latency conversational audio and the tool surface operators already expect.

In the studio, users can:

• Clone a voice or pick from a predefined list
• Add characteristics / personality knobs for the voice agent
• Configure first message, prompt, and context
• Enable system tools (Hume-native capabilities) and custom tools they define

The product lesson: when latency is the product, change the pipeline - do not only tune prompts.

In code terms, the studio maps onto a Hume EVI config - voice, prompt, built-ins, and custom tools:

Sessions on the client look like a thin island around @humeai/voice-react - connect with an access token + configId, let Hume own the speech-to-speech loop:

// Client island - Hume speech-to-speech session
'use client'

import { VoiceProvider, useVoice } from '@humeai/voice-react'

export function HumeCallShell({
  accessToken,
  configId,
}: {
  accessToken: string
  configId: string
}) {
  return (
    <VoiceProvider>
      <StartHumeCall accessToken={accessToken} configId={configId} />
    </VoiceProvider>
  )
}

function StartHumeCall({
  accessToken,
  configId,
}: {
  accessToken: string
  configId: string
}) {
  const { connect, disconnect, status } = useVoice()
  const live = status.value === 'connected'

  return (
    <button
      type="button"
      onClick={() =>
        live
          ? disconnect()
          : connect({
              auth: { type: 'accessToken', value: accessToken },
              configId,
            })
      }
    >
      {live ? 'End call' : 'Start Hume call'}
    </button>
  )
}

Custom tools (CRM writes, lead stages) stay in your backend. EVI emits tool_call; you run Restack → Python → GHL, then send tool_response:

For the deeper telephony and migration narrative, see production voice AI at Use Cortana.

Call path: Frontend → Restack → Python

When a call is invoked, orchestration is deliberate:

1. Frontend initiates the call / session
2. Restack - a middle layer built on a Temporal-style durable engine - owns reliable workflow steps
3. Python backend executes agent logic, tools, and provider calls

That separation keeps the UI snappy while Restack absorbs retries, long-running tool work, and failure recovery - the same class of problem covered in Temporal for AI agents. Walkthrough Part 2 shows messaging resuming mid-conversation because Temporal/Restack pulls history through RAG before the backend responds.

Twilio + GoHighLevel

Telephony

Voice agents wire to Twilio for real phone traffic:

• Inbound call handling via configured webhooks / communication types
• Outbound campaigns with the same agent definition

Operators choose how the agent should be reached - they are not editing raw trunk config by hand for every experiment.

CRM

GoHighLevel (GHL) is the primary CRM. Users connect their own GHL account so voice and text agents stay in sync with pipeline stages, contacts, and follow-ups - critical when the rest of Cortana is optimizing paid acquisition into those same leads.

Webchat embed

Text agents are not trapped inside the SaaS. Operators generate a script / embed for webchat and place it on their own marketing site - so the same knowledge base and tools that power SMS or voice also power on-site conversations.

The embed supports floating popup, button-triggered modal, and inline iframe, with light/dark themes and custom colors - see walkthrough Part 2.

Stack snapshot (agent side)

Why this matters

Most “AI agent” demos are a single prompt in a playground. Use Cortana had to be a tenant-safe product: orgs, knowledge isolation, channels, CRM, telephony, and a studio UI non-engineers can operate.

Shipping that studio - and helping the platform move voice from a laggy cascade to Hume speech-to-speech - is the work that turns agentic growth from a slide into something sales teams run every day.

Related work

• Vibe coding: Claude + Cursor on Use Cortana · Loom
• Product walkthrough Part 1 (Loom) · Part 2
• Voice pipeline deep dive (LiveKit → Hume)
• Temporal / Restack-style durable workflows
• RAG memory for agents
• Hire Full Stack AI · Contact