SEF · Security Incident Management, E-Discovery, & Cloud Forensics

SEF-05Cloud & AI Related

Incident Response Metrics

Specification

Establish, monitor and report information security incident metrics.

Threat coverage

Model manipulation

Data poisoning

Sensitive data disclosure

Model theft

Model/Service Failure

Insecure supply chain

Insecure apps/plugins

Denial of Service

Loss of governance

Architectural relevance

Physical infrastructure

Network

Compute

Storage

Application

Data

Lifecycle

Preparation

Data storage

Development

Guardrails

Evaluation

Re-evaluation

Deployment

Orchestration, AI Services supply chain, AI applications

Delivery

Operations, Maintenance, Continuous monitoring, Continuous improvement

Retirement

Data deletion

Ownership / SSRM

Shared across the supply chain

Shared control ownership refers to responsibilities and activities related to LLM security that are distributed across multiple stakeholders within the AI supply chain, including the Cloud Service Provider (CSP), Model Provider (MP), Orchestrated Service Provider (OSP), Application Provider (AP), and Customer (AIC). These controls require coordinated actions, communication, and governance across all involved parties to ensure their effectiveness.

Model

Owned by the Model Provider (MP)

The model provider (MP) designs, develops, and implements the control as part of their services or products to mitigate security, privacy, or compliance risks associated with the Large Language Model (LLM). Model Providers are entities that develop, train, and distribute foundational and fine-tuned AI models for various applications. They create the underlying AI capabilities that other actors build upon. Model Providers are responsible for model architecture, training methodologies, performance characteristics, and documentation of capabilities and limitations. They operate at the foundation layer of the AI stack and may provide direct API access to their models. Examples: OpenAI (GPT, DALL-E, Whisper), Anthropic(Claude), Google(Gemini), Meta(Llama), as well as any customized model.

Orchestrated

Shared Model Provider-Orchestrated Service Provider (Shared MP-OSP)

The MP and OSP are jointly responsible and accountable for the design, development, implementation, and enforcement of the control to mitigate security, privacy, or compliance risks associated with Large Language Model (LLM)/GenAI technologies in the context of the services or products they develop and offer.

Application

Shared Orchestrated Service Provider-Application Provider (Shared OSP-AP)

The OSP and AP are jointly responsible and accountable for the design, development, implementation, and enforcement of the control to mitigate security, privacy, or compliance risks associated with Large Language Model (LLM)/GenAI technologies in the context of the services or products they develop and offer.

Implementation guidelines

[All Actors]
1. Define a baseline set of incident response metrics such as Mean Time to Detect (MTTD), Mean Time to Respond (MTTR), containment time, and incident closure rate.

2. Track and report severity distribution (e.g., critical, high, medium, low) of security incidents over time to identify systemic issues.

3. Measure the volume of incidents escalated vs. resolved at first triage level to assess effectiveness of frontline defenses.

4. Monitor adherence to Service Level Objectives (SLOs) and regulatory deadlines (e.g., breach notification windows under GDPR, HIPAA).

5. Analyze incident recurrence frequency and use root cause tracking metrics to drive continuous improvements.

6. Maintain metric dashboards accessible to stakeholders for transparency and trend analysis.

Auditing guidelines

1. Verify CSP has documented metrics for evaluating incident response effectiveness.

2. Confirm metrics align with cloud service level agreements, organizational goals and industry best practices (e.g., Cloud Mean Time to Detect (CMTTD), Cloud Alert Fidelity (true positives from AWS GuardDuty, Auzre Defender), Anomalous Behavior Detection Rate).

3. Check regular collection, analysis, and reporting of response metrics.

4. Ensure documentation of actions taken based on metrics analysis.

5. Confirm clear accountability for monitoring incident response metrics.

Standards mappings

ISO 42001Partial Gap

42001: A.6.2.6
42001: B.6.2.6
27001: A.5.24
27001: A.5.27
27001: Clause 9.3
27002: 5.24 (b)

Addendum

Define AI-specific incident metrics, Require active monitoring and logging, Integrate with performance evaluation and management review.

EU AI ActFull Gap

No Mapping

Addendum

No requirement for the full range of documentation and review processes specified in the AICM requirement.

NIST AI 600-1No Gap

MS-2.7-004
MS-2.7-006

Addendum

N/A

BSI AIC4No Gap

C4 RE-05
C5 SIM-05

Addendum

N/A

AI-CAIQ questions (1)

SEF-05.1

Are information security incident metrics established, monitored and reported?