Agents Security Boundaries
Specification
Establish security boundaries for agents.
Threat coverage
Architectural relevance
Lifecycle
Data collection, Data storage, Resource provisioning
Training
Validation/Red Teaming
Orchestration, AI Services supply chain, AI applications
Continuous monitoring
Data deletion
Ownership / SSRM
PI
Shared across the supply chain
Shared control ownership refers to responsibilities and activities related to LLM security that are distributed across multiple stakeholders within the AI supply chain, including the Cloud Service Provider (CSP), Model Provider (MP), Orchestrated Service Provider (OSP), Application Provider (AP), and Customer (AIC). These controls require coordinated actions, communication, and governance across all involved parties to ensure their effectiveness.
Model
Owned by the Model Provider (MP)
The model provider (MP) designs, develops, and implements the control as part of their services or products to mitigate security, privacy, or compliance risks associated with the Large Language Model (LLM). Model Providers are entities that develop, train, and distribute foundational and fine-tuned AI models for various applications. They create the underlying AI capabilities that other actors build upon. Model Providers are responsible for model architecture, training methodologies, performance characteristics, and documentation of capabilities and limitations. They operate at the foundation layer of the AI stack and may provide direct API access to their models. Examples: OpenAI (GPT, DALL-E, Whisper), Anthropic(Claude), Google(Gemini), Meta(Llama), as well as any customized model.
Orchestrated
Shared Model Provider-Orchestrated Service Provider (Shared MP-OSP)
The MP and OSP are jointly responsible and accountable for the design, development, implementation, and enforcement of the control to mitigate security, privacy, or compliance risks associated with Large Language Model (LLM)/GenAI technologies in the context of the services or products they develop and offer.
Application
Owned by the Application Provider (AP)
The Application Provider (AP) is responsible for the design, development, implementation, and enforcement of the control to mitigate security, privacy, or compliance risks associated with Large Language Model (LLM)/GenAI technologies in the context of the services or products they develop and offer. The AP is responsible and accountable for the implementation of the control within its own infrastructure/environment. If the control has downstream implications on the users/customers, the AP is responsible for enabling the customer and/or upstream partner in the implementation/configuration of the control within their risk management approach. The AP is accountable for carrying out the due diligence on its upstream providers (e.g MPs, Orchestrated Services) to verify that they implement the control as it relates to the service/product develop and offered by the AP. These providers build and offer end-user applications that leverage generative AI models for specific tasks such as content creation, chatbots, code generation, and enterprise automation. These applications are often delivered as software-as-a-service (SaaS) solutions. These providers focus on user interfaces, application logic, domain-specific functionality, and overall user experience rather than underlying model development. Example: OpenAI (GPTs,Assistants), Zapier, CustomGPT, Microsoft Copilot (integrated into Office products), Jasper (AI-driven content generation), Notion AI (AI-enhanced productivity tools), Adobe Firefly (AI-generated media), and AI-powered customer service solutions like Amazon Rufus, as well as any organization that develops its AI-based application internally.
Implementation guidelines
Auditing guidelines
1. Review CSP documentation and templates defining secure hosting guidelines for AI agents (e.g., sandbox profiles, scoped IAM roles, API throttling), and confirm that guidance is up‑to‑date and communicated to customers. 2. Inspect network and identity isolation controls (e.g., VPCs, service identities, scoped IAM roles, or equivalent mechanisms) to confirm that agent deployments are securely separated in shared environments. 3. Verify that the CSP’s API gateways or equivalent controls can enforce agent‑specific permission sets, filter unauthorized calls, and produce auditable access logs. 4. Evaluate vendor validation processes and security requirements for any third‑party agent offerings in the CSP’s marketplace or service catalog. 5. Confirm that the CSP offers telemetry and behavior analysis services that enable customers to monitor agent activity, detect abnormal behavior, and investigate elevated access patterns.
Standards mappings
42001: A.6.1.3 Processes for responsible AI system design and development 42001: A.6.2.2 AI system requirements and specification 42001: A.6.2.3 Documentation of AI system design and development 42001: A.6.2.5 AI system deployment 42001: A.6.2.7 AI system technical documentation 27001: 8.26 Application security requirements 27001: 8.27 Secure system architecture and engineering principles 27001: A.5.18 Access rights
Addendum
New or revised controls would need to: Define what constitutes an “agent” in AI systems Require clear separation and monitoring between agents Enforce isolation, access control, and communication boundaries
No Mapping
Addendum
Include security boundaries for agents, for both AI System and AI Model, in Article 3.
MS-2.7-001
Addendum
N/A
C4 PF-4 C5 PSS-08
Addendum
There could be some explicit definition of agents in the intro text of document to categorize them as roles within the ruleset.
AI-CAIQ questions (1)
Are the security boundaries for agents established?