5. The Case for Runtime Security¶
After this module you will be able to¶
- Distinguish between design-time, deployment-time, and runtime security controls
- Describe the three-layer defence model for AI runtime security
- Explain why runtime is where most AI failures actually occur
- Identify what comes next in the AIRS training programme
What "runtime" means¶
It helps to think about AI security in three phases:
| Design-time | Deployment-time | Runtime | |
|---|---|---|---|
| When | Before the system exists | Before the system goes live | While the system is processing real requests |
| Activities | Model selection, training data curation, prompt engineering, architecture decisions | Access controls, rate limiting, API gateway configuration, environment hardening | Input validation, output verification, behavioural monitoring, escalation |
| Who owns it | ML engineers, data scientists | Platform and DevOps teams | Security operations, application teams, governance functions |
| Limitation | Cannot anticipate every real-world input | Cannot inspect the content of model reasoning | Must operate under latency and scale constraints |
Most organisations concentrate their AI security effort at design-time and deployment-time. That makes intuitive sense: you want to build the system well and lock it down before users touch it.
But here is the problem. Every failure mode we have explored in this track, hallucination, prompt injection, unsafe tool use, data leakage, happens at runtime. These failures emerge when the system encounters real-world inputs, novel context, and edge cases that no amount of upfront engineering can fully predict.
Runtime security is the practice of verifying that AI systems are behaving soundly while they are operating, not just before they are deployed. It treats the live behaviour of the model as something that must be continuously monitored, evaluated, and constrained.
The three-layer defence model¶
The AIRS approach to runtime security uses three complementary layers. The role-based tracks cover these in depth; here is the overview.
Layer 1: Guardrails (~10ms latency). Fast, rule-based checks on inputs and outputs. These block known-bad patterns, enforce format constraints, and catch obvious policy violations. Guardrails are necessary but insufficient on their own. They are shallow by design.
Layer 2: Model-as-Judge (~500ms to 5s latency). An independent model evaluates whether the primary model's output is sound. This layer can catch semantic issues that pattern-matching misses, such as a response that is technically well-formed but factually wrong or subtly manipulative. It is slower, but deeper.
Layer 3: Human Oversight (minutes to hours). Humans review cases that automated controls cannot resolve with sufficient confidence. This is the backstop, thorough but unable to scale to every request.
The key insight is that these layers work together. No single layer is sufficient. Guardrails are fast but shallow. Model-as-Judge is deeper but slower. Human oversight is thorough but does not scale. A robust runtime security posture layers all three, routing decisions to the appropriate level based on risk and confidence.
The MASO framework¶
How do you decide which controls to apply and where? The AIRS framework organises runtime controls into eight domains under the MASO (Multi-Agent Security Operations) framework. We will not enumerate all eight here. The important point is that there is a structured way to map failure modes to specific control domains. The role-based tracks go deep on this.
What comes next¶
You have now completed the Foundations track. From here, the training branches:
- The Scenario (Phantom Compliance): a concrete case study that shows a runtime failure unfolding in a realistic enterprise setting. This is where the concepts become tangible.
- Role-based tracks: deep dives from the perspective of security architects, risk and governance professionals, and engineering leads. Each track covers the same core material through a different lens.
- Convergence exercise: a cross-functional collaboration where all three roles work together on a shared problem.
Why this matters now¶
AI agents are being deployed in production at increasing scale and autonomy. They are making decisions, taking actions, and generating content that organisations are accountable for.
The gap between what these systems can do and what security controls can verify is widening. Every new capability, tool use, multi-step reasoning, autonomous action, creates new surface area that traditional controls were never designed to cover.
Runtime security is not optional for production AI. It is the missing layer.
Reflection
Think about your own organisation's AI security efforts. Where is the majority of that effort concentrated: design-time, deployment-time, or runtime? What would need to change to shift more attention to runtime controls?
Consider
Count the controls. How many of your AI-related security measures operate before the system is live (model cards, bias assessments, red-teaming exercises) versus during live operation? If the balance is heavily weighted toward pre-deployment, that's the gap this training is designed to address.