The short version: AI SaaS products on shared runtimes tend to break somewhere around 50 customers. Env vars leak, one tenant's traffic takes everyone down, performance turns unpredictable, and per-customer config requests pile up. Migrating to per-tenant isolation after the fact took us 3 months. Starting with it would have taken days.
Here's the whole story, mistakes included.
I gonna be honest with you.
When we first started building our AI product we thought shared runtime was fine. One big container, all customers hit same endpoint. Easy peasy. Ship it friday, beers after.
We were so wrong.
The first 10 customers? Everything is beautiful
You deploy. Customers sign up. API responds fast. Logs are clean. You feel like a genius.
You tell your cofounder "see I told you we dont need kubernetes". High five. Life is good.
Then customer 11 joins. They got a usecase you didnt expect. They sending 400 requests per minute. Your other 10 customers? They all getting timeouts now. Your slack is blowing up.
But its fine right? Just add rate limiting. Quick fix.
Wrong. That was just the beginning.
What actually breaks at 50 customers
Let me walk you through what happened to us because I wish someone told me this before.
Env vars started leaking. We had one shared runtime with all customer API keys in env. One bad error handler was logging full env to our monitoring tool. We didnt notice for 3 weeks. Three weeks of customer secrets sitting in Datadog. I still lose sleep over this.
One customer crashed everyone. Customer 34 figured out they could send a prompt that made the model output infinite tokens. Runtime OOM'd. All 50 customers down. At 2:47am on a tuesday. Ask me how I know.
Performance became unpredictable. Customer A gets response in 200ms. Same request from Customer B takes 4 seconds. Why? Because Customer C is running a batch job eating all the CPU. You cant even debug this in a shared runtime because everyones traffic is mixed together.
Customization requests piled up. Enterprise customer wants GPT-4. Another wants Claude. Third one needs a custom system prompt. In a shared runtime? Good luck. You basically need feature flags for infrastructure. Thats not a thing.
The math that nobody does
Heres what I wish I calculated before we started:
Cost of building shared runtime: 2 weeks
Cost of migrating 50 customers to isolated runtimes after stuff breaks: 3 months
Cost of losing 2 enterprise customers because of a cross-tenant data leak: priceless (and not in the good way)
The migration was brutal. We had to build a provisioning system, container orchestration, volume management, DNS routing, SSL certs, env isolation. All while keeping 50 customers running. It was like changing the engine on a plane mid flight.
meanwhile, people who started with isolation are building real products
while we were drowning in migration work, other teams were shipping actual products on isolated runtimes from day one. look at what theyre building:
ClawField built a live trading bot scanner on isolated runtimes. Every entry within 30 seconds, precision execution down to the second. He wasnt debugging cross-tenant issues. He was building features that made money.
Max Blade shipped an entire iOS app that launches isolated OpenClaw agents. No Telegram, no API keys, no setup. Just sign in and your agent is live. He built that because he wasnt stuck maintaining shared infrastructure.
These are real products from real builders. They started with isolation so they could spend their time on the product instead of the plumbing.
Why nobody switches until its too late
I talk to a lot of founders building AI products. They all say the same thing:
"We'll isolate later when we have more customers"
This is the trap. By the time you have enough customers to justify isolation, you also have too many customers to migrate safely. The window where its easy to switch? Its right now. Before you have the problem.
Its like backups. Nobody cares about backups until they lose data. Then suddenly its the most important thing in the world.
What per-tenant isolation actually looks like
After we burned 3 months migrating, heres what we ended up with (the longer argument for why this architecture wins is its own post):
- Each customer gets own Docker container
- Each container has its own persistent volume at
/data - Environment variables are completely separate per tenant
- Each customer gets a unique URL like
customer.agents.shipclaw.io - One customer crashing doesnt affect anyone else
- You can customize model, config, rate limits per customer
If you want the product-doc version, here is how ShipClaw's per-tenant isolated runtimes work.
The difference was night and day. Support tickets dropped 80%. No more 3am pages. Enterprise customers stopped threatening to leave.
"But isolated runtimes are expensive"
This is the objection I hear most. More containers = more money right?
Kinda. But lets do the real math, using rough numbers from our own setup:
| Shared runtime | Isolated runtimes | |
|---|---|---|
| Infrastructure | ~$200/mo | ~$400/mo (yes, more) |
| 3am incident response | regular, soul-destroying | basically zero |
| Customer churn from reliability | real and recurring | way down |
| Enterprise deals | lost to "we cant guarantee isolation" | you can actually close them |
| Engineering time on infra issues | ~20 hrs/week | ~2 hrs/week |
The infrastructure costs more. Everything else costs way less. Net? You come out ahead by a lot.
How we do it now with ShipClaw
After going through all that pain we built ShipClaw so nobody else has to.
You open the visual builder. Drag a Runtime node onto canvas. Connect a Gateway for routing. Add a Volume for persistent storage. Add Env Config for secrets. Hit deploy.
Thats it. Each customer gets fully isolated OpenClaw runtime. Own container, own volume, own env vars, own URL. You didnt write a single Dockerfile or Kubernetes manifest.
The whole thing that took us 3 months to build manually? Its drag drop deploy now.
The part where I give you free advice
Look if youre building an AI product for multiple customers, please just start with isolation from day one. I dont care if you use ShipClaw or build it yourself or use something else entirely. Just dont do the shared runtime thing.
The technical debt compounds fast. The security risk is real. The 3am pages will come. And migrating later is 10x harder than starting right. If you want to see what the bill looks like in dollars, I've done that math too.
Trust me. I learned this the expensive way so you dont have to.