MDS-06AI-Specific

Adversarial Attack Analysis

Specification

Define, implement, and evaluate processes and technical measures to assess adversarial threats specific to each AI model.

Threat coverage

Model manipulation

Data poisoning

Sensitive data disclosure

Model theft

Model/Service Failure

Insecure supply chain

Insecure apps/plugins

Denial of Service

Loss of governance

Architectural relevance

Physical infrastructure

Network

Compute

Storage

Application

Data

Lifecycle

Preparation

Data collection, Data curation, Data storage, Team and expertise

Development

Design, Training, Guardrails

Evaluation

Evaluation, Validation/Red Teaming, Re-evaluation

Deployment

Orchestration, AI Services supply chain, AI applications

Delivery

Operations, Maintenance, Continuous improvement, Continuous monitoring

Retirement

Data deletion, Model disposal

Ownership / SSRM

Owned by the Cloud Service Provider (CSP)

The Cloud Service Provider (CSP) is responsible for the design, development, implementation, and enforcement of the control to mitigate security, privacy, or compliance risks associated with cloud computing (processing, storage, and networking) technologies in the context of the services or products they develop and offer. The CSP is responsible and accountable for implementing the control within its own infrastructure/environment. The CSP is responsible for enabling the customer and/or upstream partner to implement/configure the control within their risk management approach. The CSP is accountable for ensuring that its providers upstream implement the control related to the service/product developed and offered by the CSP.

Model

Owned by the Model Provider (MP)

The model provider (MP) designs, develops, and implements the control as part of their services or products to mitigate security, privacy, or compliance risks associated with the Large Language Model (LLM). Model Providers are entities that develop, train, and distribute foundational and fine-tuned AI models for various applications. They create the underlying AI capabilities that other actors build upon. Model Providers are responsible for model architecture, training methodologies, performance characteristics, and documentation of capabilities and limitations. They operate at the foundation layer of the AI stack and may provide direct API access to their models. Examples: OpenAI (GPT, DALL-E, Whisper), Anthropic(Claude), Google(Gemini), Meta(Llama), as well as any customized model.

Orchestrated

Shared Model Provider-Orchestrated Service Provider (Shared MP-OSP)

The MP and OSP are jointly responsible and accountable for the design, development, implementation, and enforcement of the control to mitigate security, privacy, or compliance risks associated with Large Language Model (LLM)/GenAI technologies in the context of the services or products they develop and offer.

Application

Shared Orchestrated Service Provider-Application Provider (Shared OSP-AP)

The OSP and AP are jointly responsible and accountable for the design, development, implementation, and enforcement of the control to mitigate security, privacy, or compliance risks associated with Large Language Model (LLM)/GenAI technologies in the context of the services or products they develop and offer.

Implementation guidelines

[Shared Responsibilities (Applicable to MP, AP)]
1. Ensure Adequate Training for Security Teams:
If an internal Vulnerability Assessment and Penetration Testing (VA/PT) service is available, ensure that operators and security professionals undergo specific training programs to assess AI model vulnerabilities and adversarial threats effectively.

2. Analyze and Score Vulnerabilities:
Analyze identified vulnerabilities based on their likelihood and potential impact. Assign scores to help prioritize the most significant risks.

3. Prioritize Mitigation Efforts:
Focus on addressing high-risk threats first, ensuring that critical vulnerabilities are mitigated in a timely manner.

4. Establish Periodic Reporting on VA/PT Activities:
Generate regular reports—at least annually—summarizing Vulnerability Assessment and Penetration Testing (VA/PT) activities. These reports should highlight key security findings, provide actionable insights, and outline recommended remediation strategies to enhance AI model security.

5. Regular Reviews:
Organizations should schedule regular reviews of their models’ security posture, at minimum on a semi-annual basis. These reviews should include updated risk assessments that take into account newly deployed models and emerging threat landscapes.

Auditing guidelines

1. Review network security controls in place to protect hosted AI models from adversarial attacks. 

2. Assess intrusion detection and prevention systems specific to detecting AI-related attacks. 

3. Verify logging and monitoring of network traffic for suspicious activity related to model interaction. 

4. Evaluate security measures to protect APIs used for accessing hosted models. 

5. Assess procedures for incident response to detected adversarial attacks. 

6. Review procedures for patching vulnerabilities related to adversarial attacks.

Standards mappings

ISO 42001No Gap

ISO 42001 A.6.2.3 - Documentation of AI system design and development
ISO 42001 B.6.2.3 - Documentation of AI system design and development
ISO 42001 A.6.2.6 - AI system operation and monitoring
ISO 42001 B.6.2.6 - AI system operation and monitoring
ISO 42001 B.6.2.7 - AI system technical documentation
ISO 42001 B.6.2.7 - AI system technical documentation

Addendum

N/A

EU AI ActNo Gap

Article 15 (1)
Article 15 (5)

Addendum

N/A

NIST AI 600-1No Gap

GV-3.2-002
GV-3.2-005
MP-5.1-005
MP-5.1-006
MS-1.1-008

Addendum

N/A

BSI AIC4No Gap

C4 SR-01
C4 SR-02

Addendum

N/A

AI-CAIQ questions (1)

MDS-06.1

Are processes and technical measures defined, implemented, and evaluated to regularly assess adversarial threats specific to each AI model?