TVM-06Cloud & AI Related

Penetration Testing

Specification

Define, implement and evaluate processes, procedures and technical measures for the periodic performance of penetration testing by independent third parties.

Threat coverage

Model manipulation

Data poisoning

Sensitive data disclosure

Model theft

Model/Service Failure

Insecure supply chain

Insecure apps/plugins

Denial of Service

Loss of governance

Architectural relevance

Physical infrastructure

Network

Compute

Storage

Application

Data

Lifecycle

Preparation

Data storage, Resource provisioning

Development

Guardrails

Evaluation

Evaluation, Validation/Red Teaming, Re-evaluation

Deployment

Orchestration, AI Services supply chain, AI applications

Delivery

Operations, Maintenance, Continuous monitoring, Continuous improvement

Retirement

Data deletion, Model disposal

Ownership / SSRM

Shared Cloud Service Provider-Model Provider (Shared CSP-MP)

The CSP and MP are jointly responsible and accountable for the design, development, implementation, and enforcement of the control to mitigate security, privacy, or compliance risks associated with Large Language Model (LLM)/GenAI technologies in the context of the services or products they develop and offer.

Model

Owned by the Model Provider (MP)

The model provider (MP) designs, develops, and implements the control as part of their services or products to mitigate security, privacy, or compliance risks associated with the Large Language Model (LLM). Model Providers are entities that develop, train, and distribute foundational and fine-tuned AI models for various applications. They create the underlying AI capabilities that other actors build upon. Model Providers are responsible for model architecture, training methodologies, performance characteristics, and documentation of capabilities and limitations. They operate at the foundation layer of the AI stack and may provide direct API access to their models. Examples: OpenAI (GPT, DALL-E, Whisper), Anthropic(Claude), Google(Gemini), Meta(Llama), as well as any customized model.

Orchestrated

Shared Model Provider-Orchestrated Service Provider (Shared MP-OSP)

The MP and OSP are jointly responsible and accountable for the design, development, implementation, and enforcement of the control to mitigate security, privacy, or compliance risks associated with Large Language Model (LLM)/GenAI technologies in the context of the services or products they develop and offer.

Application

Shared Orchestrated Service Provider-Application Provider (Shared OSP-AP)

The OSP and AP are jointly responsible and accountable for the design, development, implementation, and enforcement of the control to mitigate security, privacy, or compliance risks associated with Large Language Model (LLM)/GenAI technologies in the context of the services or products they develop and offer.

Implementation guidelines

[All Actors]
1. Assess the potential impact of each vulnerability on the organization’s systems, services, and data.

2. Classify vulnerabilities based on their potential to cause disruption to critical business functions, such as model integrity, service availability or data privacy.

3. Conduct impact assessments for high-priority vulnerabilities to understand potential consequences and prioritize remediation efforts.

4. Use risk management frameworks to guide impact assessments and ensure they align with organizational goals and compliance requirements.

5. Ensure that impact assessments are communicated effectively to all stakeholders to inform decision-making around remediation efforts.

6. Regularly review and update impact-assessment procedures to reflect new threats, regulatory changes and technology shifts.

7. Schedule penetration tests at defined intervals and after major system changes; ensure tests are executed by qualified, independent third parties and that results feed into the impact-assessment workflow above.

Auditing guidelines

1. Verify that the CSP has defined and documented processes, procedures, and technical measures for periodic penetration testing by independent third parties. Documentation must include scope, objectives, roles, and responsibilities.

2. Examine whether these processes comply with regulatory requirements and industry best practices.

3. Inspect alignment of the processes with the relevant threat scenarios specific to the CSP’s infrastructure.

4. Confirm that these processes are implemented and adhered to.

5. Verify that findings from penetration tests are reviewed and translated into concrete remediation actions.

6. Inspect whether metrics and indicators are monitored to evaluate the efficacy and efficiency of the penetration testing program.

7. Inspect evidence that the processes are reviewed and updated at least annually or upon significant changes.

8. Verify that the CSP has a formal, documented policy permitting customers to conduct penetration testing of their own workloads on the CSP platform, with clear processes for authorization and scoping.

9. Review the CSP’s own independent penetration testing reports and third‑party attestations (e.g., SOC 2 Type II, ISO 27001) covering its infrastructure and confirm they are kept current and made available to customers

From CCM:
1. Examine policy for adequacy, currency, and effectiveness.
2. Determine if the process for defining frequency of penetration testing is defined.
3. Determine if the process for selection of independent third parties is defined, and evaluated.

Standards mappings

ISO 42001Partial Gap

6. Planning  (27001)
9. Performance Evaluation (27001)
A.8.8 Management of technical vulnerabilities (27001)
A.6.2.6 AI System Operation and Monitoring (42001)

Addendum

Missing mention to penetration test performed by a third party

EU AI ActNo Gap

Article 43

Addendum

N/A

NIST AI 600-1No Gap

MP-5.1-005
MS-4.2-001

Addendum

N/A

BSI AIC4Partial Gap

C4 SR-05
C5 OPS-19

Addendum

BSI C4 SR-05 does support penetration testing but not within the TVM-06 topic of 'by independent third parties'.

AI-CAIQ questions (1)

TVM-06.1

Are processes, procedures, and technical measures defined, implemented, and evaluated for the periodic performance of penetration testing by independent third parties?