> For the complete documentation index, see [llms.txt](https://docs.tomorrowx.com/cap/vZHo2144m1e79o3WX0RU/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.tomorrowx.com/cap/vZHo2144m1e79o3WX0RU/deployment-scenarios/openshell-supervisor-middleware.md). # OpenShell Supervisor Middleware NVIDIA OpenShell runs AI agents in sandboxes and governs where they may reach: which host, port, method and path, with credentials injected by its supervisor rather than held by the agent. The supervisor can also consult an external decision service, a supervisor middleware, before a sandboxed agent's request leaves and before a response returns to it. In this scenario a space.vars.X\_Agent\_Name\_Single is that decision service: it inspects what each request carries and each response returns, redacts or refuses by policy, and keeps the evidence record. It never holds the tool credential and never changes the destination; OpenShell keeps those. The agent runs as a standalone Multi-Protocol agent whose start-up rule set opens a gRPC listener with the Start gRPC Server rule and the OpenShell supervisor middleware binding. Every OpenShell event (request, response preflight, response body, WebSocket session) runs once through the agent's rule set as one set of variables, and the decision the rules leave is returned to OpenShell. A rule set that leaves no decision is answered fail-closed. See [Multi-Protocol Agents](/cap/vZHo2144m1e79o3WX0RU/concepts/multi-protocol-agents.md) and [Installing a Stand-alone Multi-Protocol Agent](/cap/vZHo2144m1e79o3WX0RU/deploy-and-configure/installation-and-configuration/installing-a-stand-alone-multi-protocol-agent.md). ```mermaid %%{init: {"theme": "base", "themeVariables": {"fontFamily": "Roboto, sans-serif", "primaryColor": "#e6f2c7", "primaryTextColor": "#000000", "primaryBorderColor": "#89a842", "lineColor": "#999999", "edgeLabelBackground": "#e8e8e8"}, "flowchart": {"padding": 22}}}%% flowchart LR A["AI agent in an OpenShell sandbox"] S["OpenShell supervisor checks reach, injects credentials"] M["Multi-Protocol agent (supervisor middleware) inspect, redact, deny, record"] T[" Tools, MCP servers, APIs "] G["HTTP gateway agent (model gateway)"] L[" Models "] A --> S S -.->|"gRPC: evaluate request, evaluate response"| M S -->|"admitted request"| T S -->|"admitted request, stamped with the sandbox id and request id"| G G --> L ``` ## What the middleware does * Request path: the rules see the request's host, method, path, headers and body. Guardrail Pack scans (identifier lists, patterns with card-number validation) run on the body; the rules return Allow with an edited body, or Deny with a reason code that OpenShell returns to the sandbox as a 403. * Response path: at preflight the rules choose whether to inspect the whole response body; on the body they return Pass, Transform (an edited body) or Block. * Header changes: the rules can queue header writes and removals for OpenShell to apply, for visible end-to-end headers only. The identity stamp below uses this. * Findings: counts only, never the matched values, returned to OpenShell so its own audit event shows a non-zero finding count. * Evidence: one capture record per decision, carrying OpenShell's request id as the exchange id and its sandbox id, so the record and OpenShell's audit event share keys. See [AI Audit Evidence](/cap/vZHo2144m1e79o3WX0RU/concepts/ai-audit-evidence.md). ## Two attachments, one agent The same registered middleware is attached twice in the OpenShell policy, selected by host: * Inline on tool, MCP and API hosts: the scans and decisions above. The evidence record names the destination host and the call: for an MCP request the method and tool name (`tools/call lookup_customer`), otherwise the HTTP method and path. * Identity stamp on the model gateway host: no scans; the request gains the sandbox id and request id as headers, so the model gateway's own evidence rows carry them and the admission is recorded without content. Model traffic then flows through the [AI API Gateway](/cap/vZHo2144m1e79o3WX0RU/concepts/ai-api-gateway.md) untouched by the middleware, and the two records of one model call share the same exchange id. ## Registration and policy OpenShell registers the middleware in its gateway configuration by name and gRPC endpoint. For anything beyond a laptop the endpoint is TLS with a certificate the gateway trusts, and the gateway signs every call with a bearer token the agent verifies against the gateway's public key, audience and gateway id, which the Start gRPC Server rule reads from the SupervisorMiddleware credential vault entry. The attachment in the policy is set to fail closed: if the agent is unreachable the call is refused, and the gateway will not start while the middleware cannot be described. Leave the agent running before sandboxes are created, and give the registration a timeout above OpenShell's 500 ms default (5 s has served) so that an agent's first call after a reload, on a busy node, is not refused. The rules behind the decisions are ordinary rule sets: which lists are armed, whether a match is observed, redacted or denied, and the capture mode are live settings changed in the Console without a redeploy. The observe setting counts and records matches and returns the finding count to OpenShell while the call leaves untouched, so the same deployment can run first as an observer of what agents send and then be switched to mediation with one setting. ## Running the middleware as a Kubernetes fleet OpenShell itself runs in a Kubernetes cluster, and so does the middleware in anything beyond a laptop: a fleet of identical Multi-Protocol agent pods behind one Service, managed by a cluster slave console inside the cluster on behalf of the master console wherever the operators work. The shape reuses the cluster model of space.vars.X\_Product\_Name (see [Managing Large Clusters](/cap/vZHo2144m1e79o3WX0RU/deploy-and-configure/administration-functions/managing-large-clusters.md) and [Creating a Cluster Slave Console Instance](/cap/vZHo2144m1e79o3WX0RU/deploy-and-configure/installation-and-configuration/creating-a-cluster-slave-console-instance.md)); nothing in it is specific to OpenShell except the port the gateway registers. ```mermaid %%{init: {"theme": "base", "themeVariables": {"fontFamily": "Roboto, sans-serif", "fontSize": "12px", "primaryColor": "#e6f2c7", "primaryTextColor": "#000000", "primaryBorderColor": "#89a842", "lineColor": "#999999", "edgeLabelBackground": "#e8e8e8"}, "flowchart": {"padding": 16, "nodeSpacing": 24, "rankSpacing": 32}}}%% flowchart TB MC[" CAP master console   run by the customer or partner,   outside the cluster "] subgraph K[" Customer Kubernetes cluster: the checks run inside this boundary "] subgraph OS[" NVIDIA OpenShell "] GW[" OpenShell gateway   policy, middleware registration "] SBX[" Sandbox pods   AI agent + OpenShell supervisor "] end SC[" CAP cluster   slave console "] SV[" Service sm:50051 "] subgraph F[" CAP supervisor middleware "] direction TB P0[" sm-0   Multi-Protocol PDA "] P1[" sm-1   Multi-Protocol PDA "] PN[" sm-N-1   Multi-Protocol PDA "] end end MC <-.->|"feed, heartbeats, counts; dials, evidence, logs"| SC SC -->|"deploy, roll one pod at a time"| F MC ~~~ GW GW -->|"creates sandboxes, attaches policy"| SBX SBX -.->|"OpenShell supervisor middleware API (gRPC): evaluate request, response"| SV SV --> F style MC fill:#d7e9a4,stroke:#89a842,stroke-width:3px style SC fill:#d7e9a4,stroke:#89a842,stroke-width:3px style P0 fill:#d7e9a4,stroke:#89a842,stroke-width:3px style P1 fill:#d7e9a4,stroke:#89a842,stroke-width:3px style PN fill:#d7e9a4,stroke:#89a842,stroke-width:3px ``` Read the diagram as two paths that meet at the supervisor middleware pods. On the right is OpenShell: its gateway creates the sandboxes and attaches their policy, and the supervisor in each sandbox calls the middleware through the Service for every request and response the policy sends to it. On the left is management: the master console reaches the pods through the cluster slave console. The darker boxes are the space.vars.X\_Product\_Name pieces: the master console, the cluster slave console and the pods of the supervisor middleware, each a Multi-Protocol space.vars.X\_Agent\_Name\_Single (space.vars.X\_Agent\_Short\_Single). The Service is Kubernetes; the gateway and the sandboxes are OpenShell. The middleware plugs into OpenShell's own supervisor middleware slot and is deployed and owned by the customer beside OpenShell, not inside it. Every check runs inside the customer's cluster: the rules run in the pods and keep their evidence in each pod's own store, and they call nothing outside the cluster unless the customer chooses to, for example to consult a classifier model. The master console is run by the customer or a partner, never as a TomorrowX service; it receives heartbeats and counts, and evidence rows reach it only when an operator reads them through the cluster slave console. How the pieces behave: * **One engine per pod, N pods as a StatefulSet.** The pods have stable names, so the slave console defines them once, up to the maximum replica count, from a declared list given in its environment; a scale-up or a replaced pod needs no change on the master. Each pod keeps its engine on its own volume and comes back serving after a restart without the slave's help. * **The slave console is not in the traffic path.** It is born headless from a ConfigMap (master URL, node name, declared agents) and a Secret (the node password), attaches to the master, pulls only the extensions the feed resolves to, deploys a pod when it comes online and rolls a feed change across the pods one at a time, waiting for each pod's gRPC health port (50052) to answer SERVING before the next. If the slave pod dies the fleet keeps serving as last deployed. * **The gateway holds one address.** It registers the middleware against the Service in front of the fleet, over TLS with a certificate that carries the Service name, and signs every call with its own key; the pods verify the token. Before a pod reloads it drains: its health port reports not ready while the pod keeps serving, the readiness probe takes it out of the Service, and the pod then asks the gateway to reconnect, which lands on a pod that is ready. * **The master sees the whole estate.** The node's agents appear in the master's API and the AI Control Plane as `node/agent` names; dials, evidence and logs are read and written through the node, and a dial can be applied to every pod of the fleet in one action. Evidence rows are read from each pod's own store over its management port, never from disk. * **A Console version is an image tag.** The slave image carries the Console build and installs it into the persistent volume on start; the agent image is the standalone Multi-Protocol agent with no Console in it. What a rolling change costs the caller: nothing. The Start gRPC Server rule in the SM Startup rule set sets five seconds to report not ready before closing. That is longer than a readiness probe of two seconds with a failure threshold of one, plus the time the Service takes to notice. Calls already on the draining pod finish there, and new calls go to the other pods. At least N-1 pods serve at every moment of the change, and a feed change rolled across two pods under one call per second refused no calls. Without the drain setting, the gateway's open connection breaks when a pod reloads, and with the policy set to fail closed, the requests in the two seconds it takes to reconnect are refused. Use cases for this scenario include: * Sensitive data kept out of tool calls and MCP requests made by autonomous agents * One evidence trail across an agent's tool calls and model calls * Regulated environments where the agent runtime and the content controls must be separate enforcement points --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation by asking a question. Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters: ``` GET https://docs.tomorrowx.com/cap/vZHo2144m1e79o3WX0RU/deployment-scenarios/openshell-supervisor-middleware.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.