Robustness against Adversarial Attack / Model Hardening
Specification
Define, implement, and evaluate processes, procedures, and technical measures for Model Hardening to mitigate relevant adversarial attacks as identified in the Threat Analysis and Adversarial Threat Analysis.
Threat coverage
Architectural relevance
Lifecycle
Data collection, Data curation, Data storage, Team and expertise
Design, Training
Evaluation, Validation/Red Teaming, Re-evaluation
Orchestration, AI Services supply chain, AI applications
Operations, Maintenance, Continuous improvement
Model disposal, Data deletion
Ownership / SSRM
PI
Owned by the Cloud Service Provider (CSP)
The Cloud Service Provider (CSP) is responsible for the design, development, implementation, and enforcement of the control to mitigate security, privacy, or compliance risks associated with cloud computing (processing, storage, and networking) technologies in the context of the services or products they develop and offer. The CSP is responsible and accountable for implementing the control within its own infrastructure/environment. The CSP is responsible for enabling the customer and/or upstream partner to implement/configure the control within their risk management approach. The CSP is accountable for ensuring that its providers upstream implement the control related to the service/product developed and offered by the CSP.
Model
Owned by the Model Provider (MP)
The model provider (MP) designs, develops, and implements the control as part of their services or products to mitigate security, privacy, or compliance risks associated with the Large Language Model (LLM). Model Providers are entities that develop, train, and distribute foundational and fine-tuned AI models for various applications. They create the underlying AI capabilities that other actors build upon. Model Providers are responsible for model architecture, training methodologies, performance characteristics, and documentation of capabilities and limitations. They operate at the foundation layer of the AI stack and may provide direct API access to their models. Examples: OpenAI (GPT, DALL-E, Whisper), Anthropic(Claude), Google(Gemini), Meta(Llama), as well as any customized model.
Orchestrated
Shared Model Provider-Orchestrated Service Provider (Shared MP-OSP)
The MP and OSP are jointly responsible and accountable for the design, development, implementation, and enforcement of the control to mitigate security, privacy, or compliance risks associated with Large Language Model (LLM)/GenAI technologies in the context of the services or products they develop and offer.
Application
Shared Orchestrated Service Provider-Application Provider (Shared OSP-AP)
The OSP and AP are jointly responsible and accountable for the design, development, implementation, and enforcement of the control to mitigate security, privacy, or compliance risks associated with Large Language Model (LLM)/GenAI technologies in the context of the services or products they develop and offer.
Implementation guidelines
Auditing guidelines
1. Examine the CSP's service-level protections that complement model hardening, such as input validation, rate limiting, and anomaly detection at the infrastructure level. 2. Assess monitoring mechanisms detect potential adversarial attacks on hosted models. 3. Verify documentation of how CSP infrastructure supports customer's model hardening strategies. 4. Review the process for testing and validating the effectiveness of infrastructure-level defenses.
Standards mappings
ISO 42001 A.6.2.3 - Documentation of AI system design and development ISO 42001 B.6.2.3 - Documentation of AI system design and development
Addendum
ISO 42001 does not mention specifically the MDS-7 topic of hardening a model.
Article 15 (5)
Addendum
N/A
MS-2.3-001 MS-4.2-001
Addendum
Reference how threat analysis and adversarial threat analysis can be incorporated into model baseline or hardening. No mention of "hardening" or "threat analysis" but it does provide guidance to "address attempts to deceive or manipulate" that helps to harden the mode.
C4 SR-04 C4 SR-05 C4 SR-06 C4 SR-07
Addendum
N/A
AI-CAIQ questions (1)
Are processes, procedures, and technical measures defined, implemented, and evaluated for Model Hardening to mitigate relevant adversarial attacks as identified in the Threat Analysis and Adversarial Threat Analysis?