Output Modification and Special Authorization
Specification
When allowing model output modification of AI generated output, establish a role for this access and allow changes only by authorized identities.
Threat coverage
Architectural relevance
Lifecycle
Resource provisioning, Data collection
Supply Chain, Design
Evaluation, Validation/Red Teaming
Orchestration, AI Services supply chain, AI applications
Operations, Maintenance
Archiving, Data deletion, Model disposal
Ownership / SSRM
PI
Owned by the Cloud Service Provider (CSP)
The Cloud Service Provider (CSP) is responsible for the design, development, implementation, and enforcement of the control to mitigate security, privacy, or compliance risks associated with cloud computing (processing, storage, and networking) technologies in the context of the services or products they develop and offer. The CSP is responsible and accountable for implementing the control within its own infrastructure/environment. The CSP is responsible for enabling the customer and/or upstream partner to implement/configure the control within their risk management approach. The CSP is accountable for ensuring that its providers upstream implement the control related to the service/product developed and offered by the CSP.
Model
Owned by the Model Provider (MP)
The model provider (MP) designs, develops, and implements the control as part of their services or products to mitigate security, privacy, or compliance risks associated with the Large Language Model (LLM). Model Providers are entities that develop, train, and distribute foundational and fine-tuned AI models for various applications. They create the underlying AI capabilities that other actors build upon. Model Providers are responsible for model architecture, training methodologies, performance characteristics, and documentation of capabilities and limitations. They operate at the foundation layer of the AI stack and may provide direct API access to their models. Examples: OpenAI (GPT, DALL-E, Whisper), Anthropic(Claude), Google(Gemini), Meta(Llama), as well as any customized model.
Orchestrated
Owned by the Orchestrated Service Provider (OSP)
The Orchestrated Service Provider (OSP) is responsible for the design, development, implementation, and enforcement of the control to mitigate security, privacy, or compliance risks associated with Large Language Model (LLM)/GenAI technologies in the context of the services or products they develop and offer. The OSP is responsible and accountable for the implementation of the control within its own infrastructure/environment. If the control has downstream implications on the users/customers, the OSP is responsible for enabling the customer and/or upstream partner in the implementation/configuration of the control within their risk management approach. The OSP is accountable for ensuring that its providers upstream (e.g MPs) implement the control as it relates to the service/product the develop and offered by the OSP. This refers to entities that create the technical building blocks and management tools that enable AI implementation. This can include platforms, frameworks, and tools that facilitate the integration, deployment, and management of AI models within enterprise workflows. These providers focus on model orchestration and offer services like API access, automated scaling, prompt management, workflow automation, monitoring, and governance rather than end-user functionality or raw infrastructure. They help businesses implement AI in a structured and efficient manner. Examples: AWS, Azure, GCP, OpenAI, Anthropic, LangChain (for AI workflow orchestration), Anyscale (Ray for distributed AI workloads), Databricks (MLflow), IBM Watson Orchestrate, and developer platforms like Google AI Studio.
Application
Shared Application Provider-AI Customer (Shared AP-AIC)
The AP and AIC both share responsibility and accountability for the design, development, implementation, and enforcement of the control to mitigate security, privacy, or compliance risks associated with Large Language Model (LLM)/GenAI technologies in the context of the services or products they offer and consume.
Implementation guidelines
Auditing guidelines
1. Ensure CSP-managed AI platforms restrict post-inference output modifications to administrative roles. 2. Verify role-specific logging of any changes to AI inference result storage or redirection. 3. Confirm support for output hashing or versioning mechanisms as part of audit requirements. 4. Validate CSP’s change control procedures cover model endpoint outputs used in production pipelines. 5. Check if customers are provided with mechanisms to lock output fields or enforce immutability where necessary.
Standards mappings
42001: A.3.2 / B.3.2 42001: A.2.4 / B.2.4 27001: A.5.18 27001: A.8.3 27001: A.8.32
Addendum
N/A
Article 9 Article 14 Article 15 Article 16 Article 17 Annex IV
Addendum
The AICM IAM-18 control is partially covered in the EU AI Act through general security and risk management principles but lacks explicit requirements for privileged access management.
No Mapping
Addendum
No (explicit/implicit) reference to the requirement set by the AICM control is made in the NIST AI 600-1 standard.
C4 DM-02 C4 RE-02
Addendum
N/A
AI-CAIQ questions (1)
Are role for access when allowing model output modification of AI-generated output established to ensure changes are made only by authorized identities?