MDS-12AI-Specific

Open Model Risk Assessment

Specification

Establish a process to evaluate risk associated with open models. Periodically review these risk factors, and implement a process to monitor and mitigate any determined vulnerabilities.

Threat coverage

Model manipulation

Data poisoning

Sensitive data disclosure

Model theft

Model/Service Failure

Insecure supply chain

Insecure apps/plugins

Denial of Service

Loss of governance

Architectural relevance

Physical infrastructure

Network

Compute

Storage

Application

Data

Lifecycle

Preparation

Resource provisioning, Team and expertise

Development

Training, Guardrails

Evaluation

Evaluation, Validation/Red Teaming

Deployment

AI Services supply chain, AI applications

Delivery

Operations, Maintenance, Continuous improvement

Retirement

Archiving, Model disposal

Ownership / SSRM

Shared Cloud Service Provider-Model Provider (Shared CSP-MP)

The CSP and MP are jointly responsible and accountable for the design, development, implementation, and enforcement of the control to mitigate security, privacy, or compliance risks associated with Large Language Model (LLM)/GenAI technologies in the context of the services or products they develop and offer.

Model

Owned by the Model Provider (MP)

The model provider (MP) designs, develops, and implements the control as part of their services or products to mitigate security, privacy, or compliance risks associated with the Large Language Model (LLM). Model Providers are entities that develop, train, and distribute foundational and fine-tuned AI models for various applications. They create the underlying AI capabilities that other actors build upon. Model Providers are responsible for model architecture, training methodologies, performance characteristics, and documentation of capabilities and limitations. They operate at the foundation layer of the AI stack and may provide direct API access to their models. Examples: OpenAI (GPT, DALL-E, Whisper), Anthropic(Claude), Google(Gemini), Meta(Llama), as well as any customized model.

Orchestrated

Shared Model Provider-Orchestrated Service Provider (Shared MP-OSP)

The MP and OSP are jointly responsible and accountable for the design, development, implementation, and enforcement of the control to mitigate security, privacy, or compliance risks associated with Large Language Model (LLM)/GenAI technologies in the context of the services or products they develop and offer.

Application

Shared Orchestrated Service Provider-Application Provider (Shared OSP-AP)

The OSP and AP are jointly responsible and accountable for the design, development, implementation, and enforcement of the control to mitigate security, privacy, or compliance risks associated with Large Language Model (LLM)/GenAI technologies in the context of the services or products they develop and offer.

Implementation guidelines

[Shared Responsibilities (OSP, AP, AIC)]
1. Continuous Monitoring and Updating: Regularly monitor the model for new vulnerabilities and update it accordingly.

2. Community and Peer Review: Eventually engage with the broader AI community for peer reviews and feedback.

Auditing guidelines

1. Verify that the CSP offers infrastructure security measures to protect open weight models from unauthorized access. 

2. Review processes conducted when integrating open weight models into service offerings, regarding the potential security flaws. 

3. Assess the monitoring of potential vulnerabilities as part of the CSP integration security testing.

4. Confirm CSP security requirements comply with any security rules and guidance from the government or industry regulation.

Standards mappings

ISO 42001Partial Gap

ISO 42001 6.1.2 AI risk assessment
ISO 42001 6.1.3 AI risk treatment
ISO 42001 A.6.2.6 - AI system operation and monitoring
ISO 42001 B.6.2.6 - AI system operation and monitoring

Addendum

However, ISO 42001 does not mention specifically the MDS-12 topic of open weight models ("weights" are publicly accessible).

EU AI ActNo Gap

Article 9

Addendum

N/A

NIST AI 600-1Partial Gap

MS-2.7-001

Addendum

NIST AI 600-1 should reference establishing a process to monitor and mitigate any determined vulnerabilities and periodically review the risk factors.

BSI AIC4No Gap

SR-01
SR-02
SR-03
SR-06
PF-07

Addendum

N/A

AI-CAIQ questions (2)

MDS-12.1

Are processes established to evaluate the risk associated with open models?

MDS-12.2

Are risk factors periodically reviewed, and is a process implemented to monitor and mitigate any determined vulnerabilities?