GRC-14AI-Specific

Explainability Evaluation

Specification

Evaluate, document, and communicate the degree of explainability of the AI Services, including possible limitations and exceptions.

Threat coverage

Model manipulation

Data poisoning

Sensitive data disclosure

Model theft

Model/Service Failure

Insecure supply chain

Insecure apps/plugins

Denial of Service

Loss of governance

Architectural relevance

Physical infrastructure

Network

Compute

Storage

Application

Data

Lifecycle

Preparation

Data collection, Data curation, Data storage, Team and expertise

Development

Design, Training, Guardrails, Supply Chain

Evaluation

Evaluation, Validation/Red Teaming, Re-evaluation

Deployment

Orchestration, AI Services supply chain, AI applications

Delivery

Operations, Maintenance, Continuous monitoring, Continuous improvement

Retirement

Archiving, Data deletion, Model disposal

Ownership / SSRM

Owned by the Orchestrated Service Provider (OSP)

The Orchestrated Service Provider (OSP) is responsible for the design, development, implementation, and enforcement of the control to mitigate security, privacy, or compliance risks associated with Large Language Model (LLM)/GenAI technologies in the context of the services or products they develop and offer. The OSP is responsible and accountable for the implementation of the control within its own infrastructure/environment. If the control has downstream implications on the users/customers, the OSP is responsible for enabling the customer and/or upstream partner in the implementation/configuration of the control within their risk management approach. The OSP is accountable for ensuring that its providers upstream (e.g MPs) implement the control as it relates to the service/product the develop and offered by the OSP. This refers to entities that create the technical building blocks and management tools that enable AI implementation. This can include platforms, frameworks, and tools that facilitate the integration, deployment, and management of AI models within enterprise workflows. These providers focus on model orchestration and offer services like API access, automated scaling, prompt management, workflow automation, monitoring, and governance rather than end-user functionality or raw infrastructure. They help businesses implement AI in a structured and efficient manner. Examples: AWS, Azure, GCP, OpenAI, Anthropic, LangChain (for AI workflow orchestration), Anyscale (Ray for distributed AI workloads), Databricks (MLflow), IBM Watson Orchestrate, and developer platforms like Google AI Studio.

Model

Owned by the Model Provider (MP)

The model provider (MP) designs, develops, and implements the control as part of their services or products to mitigate security, privacy, or compliance risks associated with the Large Language Model (LLM). Model Providers are entities that develop, train, and distribute foundational and fine-tuned AI models for various applications. They create the underlying AI capabilities that other actors build upon. Model Providers are responsible for model architecture, training methodologies, performance characteristics, and documentation of capabilities and limitations. They operate at the foundation layer of the AI stack and may provide direct API access to their models. Examples: OpenAI (GPT, DALL-E, Whisper), Anthropic(Claude), Google(Gemini), Meta(Llama), as well as any customized model.

Orchestrated

Shared Model Provider-Orchestrated Service Provider (Shared MP-OSP)

The MP and OSP are jointly responsible and accountable for the design, development, implementation, and enforcement of the control to mitigate security, privacy, or compliance risks associated with Large Language Model (LLM)/GenAI technologies in the context of the services or products they develop and offer.

Application

Shared Application Provider-AI Customer (Shared AP-AIC)

The AP and AIC both share responsibility and accountability for the design, development, implementation, and enforcement of the control to mitigate security, privacy, or compliance risks associated with Large Language Model (LLM)/GenAI technologies in the context of the services or products they offer and consume.

Implementation guidelines

[All Actors]
1. Define what AI model attributes, performance metrics, datasets, risks, and limitations must be disclosed to stakeholders.

2. Create documentation templates (e.g., model cards, system cards) to standardize disclosures.

3. Publish transparency reports or disclosures when AI models are used in decision-making roles.

4. Update transparency documentation with each major model revision or retraining.

5. Ensure disclosures are accessible, understandable, and tailored to stakeholder needs.

Auditing guidelines

1. Verify that CSPs define explainability requirements for AI services hosted on their platforms, in alignment with regulatory and compliance expectations of customers.

2. Verify that CSPs provide tools or APIs to evaluate the explainability of AI models deployed in their environment.

3. Verify that any limitations (e.g., restrictions due to proprietary models) and exceptions (e.g., limited observability) are documented and communicated to customers.

4. Ensure that CSP service documentation includes guidance on how to assess and document the explainability level of hosted AI services.

5. Verify that any native explainability outputs are accessible and interpretable to non-technical users.

Standards mappings

ISO 42001No Gap

42001: B.8.3 (External reporting)
42001 B.9.3 (Objectives for responsible use of AI system)

Addendum

N/A

EU AI ActPartial Gap

Article 11 (1)
Article 13
Article 52

Addendum

Evaluate the "degree of explainability" and "including possible limitations and exceptions."

NIST AI 600-1No Gap

MG-3.2-001
GV-4.1-001

Addendum

N/A

BSI AIC4No Gap

EX-01

Addendum

N/A

AI-CAIQ questions (1)

GRC-14.1

Is the degree of explainability of the AI Services evaluated, documented, and communicated, including possible limitations and exceptions?