SEF · Security Incident Management, E-Discovery, & Cloud Forensics

SEF-04Cloud & AI Related

Incident Response Testing

Specification

Follow a structured approach to evaluate the effectiveness of incident response plans at planned intervals or upon significant changes.

Threat coverage

Model manipulation

Data poisoning

Sensitive data disclosure

Model theft

Model/Service Failure

Insecure supply chain

Insecure apps/plugins

Denial of Service

Loss of governance

Architectural relevance

Physical infrastructure

Network

Compute

Storage

Application

Data

Lifecycle

Preparation

Data storage, Resource provisioning, Team and expertise

Development

Guardrails

Evaluation

Evaluation, Validation/Red Teaming, Re-evaluation

Deployment

Orchestration, AI Services supply chain, AI applications

Delivery

Operations, Maintenance, Continuous monitoring, Continuous improvement

Retirement

Archiving

Ownership / SSRM

Shared across the supply chain

Shared control ownership refers to responsibilities and activities related to LLM security that are distributed across multiple stakeholders within the AI supply chain, including the Cloud Service Provider (CSP), Model Provider (MP), Orchestrated Service Provider (OSP), Application Provider (AP), and Customer (AIC). These controls require coordinated actions, communication, and governance across all involved parties to ensure their effectiveness.

Model

Owned by the Model Provider (MP)

The model provider (MP) designs, develops, and implements the control as part of their services or products to mitigate security, privacy, or compliance risks associated with the Large Language Model (LLM). Model Providers are entities that develop, train, and distribute foundational and fine-tuned AI models for various applications. They create the underlying AI capabilities that other actors build upon. Model Providers are responsible for model architecture, training methodologies, performance characteristics, and documentation of capabilities and limitations. They operate at the foundation layer of the AI stack and may provide direct API access to their models. Examples: OpenAI (GPT, DALL-E, Whisper), Anthropic(Claude), Google(Gemini), Meta(Llama), as well as any customized model.

Orchestrated

Shared Model Provider-Orchestrated Service Provider (Shared MP-OSP)

The MP and OSP are jointly responsible and accountable for the design, development, implementation, and enforcement of the control to mitigate security, privacy, or compliance risks associated with Large Language Model (LLM)/GenAI technologies in the context of the services or products they develop and offer.

Application

Shared Orchestrated Service Provider-Application Provider (Shared OSP-AP)

The OSP and AP are jointly responsible and accountable for the design, development, implementation, and enforcement of the control to mitigate security, privacy, or compliance risks associated with Large Language Model (LLM)/GenAI technologies in the context of the services or products they develop and offer.

Implementation guidelines

[All Actors]
1. Regularly test AI-specific incident response plans through structured exercises, such as tabletop simulations or red team engagements.

2. Validate response team readiness for handling AI-centric threats, including those targeting training data, inference pipelines, or model interfaces.

3. Include multi-party coordination with vendors and third parties to simulate complex real-world scenarios.

4. Capture and track lessons learned to improve playbooks and cross-functional workflows.

Auditing guidelines

1. Confirm CSP regular and systematic testing of incident response plans.

2. Ensure incident response plans can state of the environment for the incident. 

3. Verify the AIC can update the plans based on determined intervals, key events (e.g., changes to infrastructure supporting the model, changes to model).

4. Verify comprehensive documentation of testing activities and outcomes.

5. Check implementation of improvements based on test results.

6. Ensure relevant stakeholders participate in response testing.

7. Validate incident response testing scenarios reflect realistic threat environments.

Standards mappings

ISO 42001Partial Gap

42001: A.6.2.4 / B.6.2.4
42001: A.6.2.4 / B.6.2.6
42001: Clause 9.1
27001: A.5.27
27002: 5.27

Addendum

Require structured incident response testing, with defined intervals, scope, and evaluation criteria.

EU AI ActFull Gap

No Mapping

Addendum

Include specific requirement for testing and updating incident response plans at planned intervals or after significant changes.

NIST AI 600-1No Gap

GV-1.5-002
MG-4.2-002
MG-4.3-001

Addendum

N/A

BSI AIC4No Gap

C4 RE-05
C5 SIM-05

Addendum

N/A

AI-CAIQ questions (1)

SEF-04.1

Is a structured approach followed, to evaluate the effectiveness of incident response plans at planned intervals or upon significant changes?