Every time I shipped an agent to production the same thing happened: it'd crash halfway through a multi-step task, lose its state, and either do nothing or — worse — repeat a side effect it had already run. "Did it already charge the card? Send the email? No idea."

So I ended up building Relay, a cloud runtime for AI agents. You write your agent once with the SDK (TypeScript or Python) and Relay handles all the infra underneath. Turns out "the infra underneath" is a lot of things, so here's the real list of what actually works today:

Agent execution

• Run an agent with real-time token streaming (SSE), tool calls and results — all as an event stream.

• Multi-model: Anthropic (Opus/Sonnet/Haiku), OpenAI (gpt-4o, gpt-4.1, o3…) and any OpenAI-compatible endpoint, via provider:model syntax.

Multi-agent / workflows

• subagent() — wrap an agent as a tool another agent can call and delegate to.

• createOrchestrator() — supervisor + team pattern: you hand it named agents with descriptions, and it auto-generates the system prompt that routes each request to the right teammate.

• Graph API — declarative multi-step workflows with .step(), .agent(), .edge(), .conditional(): shared state, conditional branches, fan-out, cycles.

• Every run in a workflow shares a workflow_id, with aggregated cost (tokens summed across all sub-runs).

Your own tools

• Define a function in your code (tool({ name, description, inputSchema, handler })) and the agent calls it. The handler runs in your process via a long-poll callback; we validate the input against the schema before it fires.

Semantic memory

• Just set memory: { namespace }: after each run it stores the (input, output) pair, embeds it (text-embedding-3-small), and on the next run does a vector search and injects what's relevant into the system prompt. No vector DB to wire up yourself.

• API to inspect and delete memories by namespace.

Voice

• Speech-to-text (Whisper) and text-to-speech (OpenAI TTS, 11 voices, multiple formats) straight from the SDK. Audio is streamed, never persisted.

Observability

• Every run and every event is persisted (status, tokens, duration, errors, tool calls).

• Dashboard with a runs list and a detail view: full execution trace, system prompt, input/output, tool calls side by side.

• Optional ClickHouse mirror for 10k+ events/sec volumes.

Security / multi-tenant (this one became an obsession)

• BYOK: bring your own OpenAI/Anthropic keys, encrypted per tenant with AES-256-GCM, never logged. We don't resell tokens.

• Row-Level Security in Postgres on every table: no tenant_id, no rows.

• API key rotation, zero-downtime master-key rotation, per-tenant rate limiting, and an audit log of every sensitive action.

Infra / deploy

• Self-host with one docker-compose.yml (control plane, runtime, Postgres, Redis, optional NATS and ClickHouse).

• Or the managed cloud version. Scales horizontally with a NATS JetStream broker + RLS.

Stack, for the curious: TS + Python SDKs (the TS one has zero runtime dependencies), a Go runtime for the workers, Postgres + pgvector, Next.js dashboard.

Where it's at: it's early, but the core is complete and running — there are no "fake" features in the main loop. SDKs published, dashboard live, docs written. I'm basically a solo founder building this mostly in the open.

I'd genuinely love feedback from anyone who's fought with agents in production: which piece of this is most useful to you, what's missing, and would you pay for something like this or just build it yourself?

Link: relaygh.dev