Logical Agents as Data, Not Infrastructure: The Multi-Tenant 'Let Customers Build Agents' Decision

A while back I wired an enterprise agent control plane to a customer-facing app on Amazon Bedrock AgentCore, using two open-source projects: Loom for the platform team’s control plane, and the FAST template for the customer-facing data plane. That post was the full evaluation: how the pieces connect, where they break, the JWT authorizer contract between them.

This post is about one decision inside that work that I keep coming back to, because it is the decision that quietly determines whether a multi-tenant agent product is a weekend prototype or a two-year rewrite.

The decision is this: when a customer wants to “create an agent,” what exactly are you creating?

Two words hiding two very different systems

“Create an agent” sounds like one feature. It is two, and they share almost nothing.

One meaning is a logical agent: a name, a system prompt, a model choice, a selection of tools. That is configuration. It is a row in a table.

The other meaning is a physical agent: a new runtime, its own container, its own IAM role, its own authorizer. That is infrastructure. It is minutes of provisioning, an account quota you will eventually hit, and one more identity in a pile of IAM roles that someone has to reason about later.

Almost every “build your own agent” product you have ever used sells the first and never provisions the second per customer. When you create a custom GPT, or a custom assistant in some SaaS tool, you are not getting a container. You are getting a row. Somebody’s shared runtime reads that row and behaves accordingly.

I find the cleanest way to hold this in my head is the old distinction between data and processes. A logical agent is data: cheap to create, cheap to destroy, trivially multi-tenant because it is just bytes with an owner. A physical agent is a process with its own address space: isolated, yes, but every one you spawn costs something, and now you have a fleet to schedule, monitor, and secure.

You would not launch a new server every time a user saves a filter preset. An agent config, for the overwhelming majority of use cases, is a filter preset with a prompt attached.

Why the row is almost always the right answer

Cost of creation is the obvious argument. A row is a PutItem. A runtime is a deployment.

But the argument that actually matters is tenancy. A shared runtime that reads per-tenant config has exactly one execution identity to reason about. A runtime-per-agent model gives every tenant their own infrastructure, and now the question “can tenant A reach tenant B’s data?” has as many answers as you have runtimes. Isolation feels like the safe choice, and for a specific class of compliance requirement it is the necessary one. But as a default, per-tenant infrastructure multiplies the surface you have to secure by the number of customers you have. That is backwards. Success should not make your security review longer.

There is also a governance argument, and it is the one people miss. If agents are data, then governance is a property of the runtime that interprets the data. You constrain the building blocks once: an allowed-model catalog, an approved tool registry. Customers compose freely inside that box. When you tighten a guardrail, every logical agent inherits it on the next request, because none of them own any infrastructure that could drift. Governance that lives in one runtime is governance you can actually change.

The fork in the road, and the two branches that hurt

When customer-driven creation lands on your desk, there are only three ways to wire it. Two of them look like shortcuts and are actually detours.

Branch one: expose the control plane’s API to customers. The control plane already has a “create agent” endpoint. Why not let customers call it? Because a control plane has no tenancy model, and was never supposed to. In a system like Loom, the call that writes an agent lives in the same backend as the calls that write IAM roles, credential providers, and secrets. agent:write and security:write are neighbors. Bridge customer identities into that surface and every authorization bug becomes a tenant-to-platform escalation. You are not adding a feature; you are removing a wall.

Branch two: use the control plane as a provisioning broker. Keep the control plane’s API internal, but have your product backend call it machine-to-machine to spin up a runtime per customer agent. This is more tempting because it looks disciplined. It is still wrong, because the control plane has no idempotency contract for your product, no per-tenant quotas, and a shared execution role, which means every tenant’s freshly provisioned agent runs as the same IAM identity. You would have to rebuild half the control plane to make it safely multi-tenant. That is not integration. That is a fork you now maintain.

Branch three: logical agents on a shared, governed runtime. Tenant agents are rows. One multi-tenant runtime, deployed and governed by the platform, interprets a tenant’s configuration on each request. This is the one that holds, and it is worth being precise about why.

The shape that works

Here is the architecture I prototyped and then tried to break.

   Customer browser
        |  prompt + JWT
        v
   API Gateway  --(Cognito authorizer: JWT verified at the edge)-->
        |
        v
   BFF (backend-for-frontend)
        |  1. tenant = verified JWT 'sub'  (never request body)
        |  2. load tenant's agent config row
        |  3. enforce allowed-model catalog
        |  4. write per-tenant usage record
        |
        |  SigV4 (BFF's own IAM role)
        v
   Shared AgentCore runtime  (one brain, governed by the control plane)
        ^
        |  reads
   DynamoDB: tenant-agents
      pk = TENANT#<jwt sub>
      { name, prompt, model, tools, usage }

Three things in that diagram are load-bearing.

The partition key comes from the verified token, not the request. The tenant’s DynamoDB partition key is the sub claim from the JWT that the edge already validated. A tenant listing agents sees only rows under their own key. A tenant trying to invoke another tenant’s agent gets a 404, not because someone remembered to write an ownership check, but because the key they are allowed to read from is derived from who they provably are. Isolation by construction beats isolation by code review.

The runtime is invocable only by the BFF. The shared runtime authenticates the BFF’s IAM role via SigV4. Customers never hold AWS credentials and never reach the runtime directly. The browser talks to your API; your API talks to the runtime. That indirection is not ceremony.

The BFF is the chokepoint you own. This is the part that makes the BFF non-optional the moment you go from demo to product. In the baseline FAST template, the browser calls the runtime directly with the customer’s JWT, which is elegant for a single-tenant app and useless for a multi-tenant business, because you have no place to meter. Per-tenant billing needs a point every request passes through, where you can attribute tokens and duration to a tenant and enforce their configuration before the model ever runs. A chokepoint you control is the price of productization.

What I verified, not just asserted

Slideware architectures always work. So I built this one and tested the claims with real requests.

A customer created a “CostBot” persona — FinOps instructions, a Haiku-class model — as a single POST /agents. A row appeared. No infrastructure was provisioned. Creation was a database write and nothing more.

Creating an agent on a model outside the approved catalog returned 403 with the allowed list attached. That is governance by catalog: the platform constrains the building blocks, the customer composes inside them, and crucially there is no per-agent human approval gate, because a human gate on self-service creation is the same as not having self-service.

Tenant isolation held structurally. Tenant two listing agents got an empty array. Tenant two invoking tenant one’s agent got a 404. The isolation came from the partition key, so there was no code path where “forgot to check ownership” was even possible.

Every invocation wrote a per-tenant usage record. That is the metering the direct-to-runtime pattern can never give you, sitting exactly where it belongs — at the chokepoint.

One finding I did not expect: the platform’s system prompt and the tenant’s persona instructions layered better than I feared. When my test persona tried to make cost the only consideration, the platform prompt pushed back in the output — the answer stayed cost-focused, but did not throw reliability and security overboard to get there. Guardrails outranking customization, visible in the response. That is the behavior you want, and it is a property of putting the governed prompt in the runtime rather than trusting each config to behave.

The exception, stated honestly

Some tenants genuinely need a dedicated runtime: a hard data-isolation requirement, a compliance regime that will not accept logical separation, a workload big enough to warrant its own scaling envelope. For those, the answer is not to bend the shared runtime.

It is product-owned provisioning of a physical runtime inside an IAM permission boundary, with mandatory tenant tags, auto-registered into the AWS Agent Registry so the control plane still sees it as part of the fleet even though it did not create it. The registry, not the control plane’s write API, is the meeting point. The control plane stays the observatory; it does not become a provisioning broker with customers on the other end.

The tell that you are drifting into trouble is when you find yourself wanting the control plane to provision infrastructure on behalf of customer actions. That is the smell. When you feel it, reach for a product-owned provisioning path and registry auto-registration instead.

The security lesson I earned the hard way

I took a shortcut in the prototype: I put the BFF on a public endpoint and validated the JWT inside the Lambda handler. Within hours, our enterprise security tooling flagged the function as world-accessible and stripped the public-access statement off it.

It was right. Authentication belongs at the edge, not in your handler. The compliant shape is API Gateway with a Cognito authorizer, so the request never reaches your code unauthenticated, with WAF in front, or a VPC-internal ALB if the callers are private. “Auth in code behind a public URL” is precisely the pattern automated scanners exist to kill. When one kills yours, do not argue with it — move the auth to the edge and thank it.

An honest recommendation

Default to logical agents. When someone asks you to let customers create agents, your first design should be a table with a per-tenant partition key, a shared governed runtime, and a BFF that verifies identity at the edge, enforces a model and tool catalog, and meters every call. That covers the overwhelming majority of what “create your own agent” actually means to customers, and it scales without turning every new customer into new infrastructure to secure.

Reach for physical, per-tenant runtimes only when a real isolation or compliance requirement forces it, and when you do, provision them from your product inside permission boundaries and register them into the fleet registry — do not route customer creation through your control plane’s API.

The property that makes this worth the discipline: the day you swap the model, tighten a guardrail, or pull a tool, every customer’s agent picks up the change on its next request, because none of them own anything that could drift. Agents as data means governance as a single, changeable fact. Agents as infrastructure means governance as a migration project you run N times. Pick the row.