Logging Scope
Specification
Establish, document and implement which information meta/data system events should be logged. Review and update the scope at least annually or whenever there is a change in the threat environment.
Threat coverage
Architectural relevance
Lifecycle
Data collection, Data curation, Data storage, Resource provisioning
Design, Training, Guardrails
Evaluation, Validation/Red Teaming, Re-evaluation
Orchestration, AI Services supply chain, AI applications
Operations, Maintenance, Continuous monitoring, Continuous improvement
Data deletion
Ownership / SSRM
PI
Shared Cloud Service Provider-Model Provider (Shared CSP-MP)
The CSP and MP are jointly responsible and accountable for the design, development, implementation, and enforcement of the control to mitigate security, privacy, or compliance risks associated with Large Language Model (LLM)/GenAI technologies in the context of the services or products they develop and offer.
Model
Owned by the Model Provider (MP)
The model provider (MP) designs, develops, and implements the control as part of their services or products to mitigate security, privacy, or compliance risks associated with the Large Language Model (LLM). Model Providers are entities that develop, train, and distribute foundational and fine-tuned AI models for various applications. They create the underlying AI capabilities that other actors build upon. Model Providers are responsible for model architecture, training methodologies, performance characteristics, and documentation of capabilities and limitations. They operate at the foundation layer of the AI stack and may provide direct API access to their models. Examples: OpenAI (GPT, DALL-E, Whisper), Anthropic(Claude), Google(Gemini), Meta(Llama), as well as any customized model.
Orchestrated
Shared Model Provider-Orchestrated Service Provider (Shared MP-OSP)
The MP and OSP are jointly responsible and accountable for the design, development, implementation, and enforcement of the control to mitigate security, privacy, or compliance risks associated with Large Language Model (LLM)/GenAI technologies in the context of the services or products they develop and offer.
Application
Owned by the Application Provider (AP)
The Application Provider (AP) is responsible for the design, development, implementation, and enforcement of the control to mitigate security, privacy, or compliance risks associated with Large Language Model (LLM)/GenAI technologies in the context of the services or products they develop and offer. The AP is responsible and accountable for the implementation of the control within its own infrastructure/environment. If the control has downstream implications on the users/customers, the AP is responsible for enabling the customer and/or upstream partner in the implementation/configuration of the control within their risk management approach. The AP is accountable for carrying out the due diligence on its upstream providers (e.g MPs, Orchestrated Services) to verify that they implement the control as it relates to the service/product develop and offered by the AP. These providers build and offer end-user applications that leverage generative AI models for specific tasks such as content creation, chatbots, code generation, and enterprise automation. These applications are often delivered as software-as-a-service (SaaS) solutions. These providers focus on user interfaces, application logic, domain-specific functionality, and overall user experience rather than underlying model development. Example: OpenAI (GPTs,Assistants), Zapier, CustomGPT, Microsoft Copilot (integrated into Office products), Jasper (AI-driven content generation), Notion AI (AI-enhanced productivity tools), Adobe Firefly (AI-generated media), and AI-powered customer service solutions like Amazon Rufus, as well as any organization that develops its AI-based application internally.
Implementation guidelines
Auditing guidelines
1. Inquiring with Control Owners 1.1 Conduct interviews with personnel responsible for establishing, documenting, and implementing logging scope for cloud infrastructure events and compute resource metadata to understand their process for defining which AI processing and customer workload events should be logged and their procedures for annual reviews. Verify their understanding of infrastructure security threat environment changes that trigger scope updates and their implementation of documented logging requirements across compute clusters, storage systems, and customer tenant environments. 2. Inspecting Records and Documents 2.1 Confirm documentation specifies which cloud infrastructure events must be logged (e.g., compute resource allocation, customer workload execution, storage access, network traffic, hypervisor activities). 2.2 Validate inclusion of both success and failure events in the cloud infrastructure logging scope, including successful resource provisioning and failed authentication attempts. 2.3 Ensure regular reviews of cloud infrastructure logging scope to capture evolving infrastructure security threats such as hypervisor attacks, side-channel exploits, and customer isolation breaches. 2.4 Check scope alignment with customer SLA requirements, regulatory compliance standards, and cloud service contractual obligations. 2.5 Assess procedures for adjusting logging scope when deploying new infrastructure services, AI accelerators, or customer environments. 2.6 Confirm stakeholder approval for the defined cloud infrastructure logging scope, including input from cloud operations teams, security architects, and compliance officers. 2.7 Verify logs reflect real-world cloud infrastructure events as specified in scope documents, including resource utilization, customer activities, and system operations. 2.8 Examine evidence of annual cloud infrastructure logging scope reviews and documentation of any scope updates driven by new infrastructure threats or regulatory changes. 2.9 Review procedures for monitoring and responding to infrastructure threat environment changes that may require logging scope adjustments for emerging attack vectors. 2.10 Validate that implementation of cloud infrastructure logging scope requirements is consistently applied across all compute resources, storage systems, network infrastructure, and customer tenant isolation mechanisms. 2.11 Confirm that the scope includes all infrastructure-level logs (compute, storage, network). 2.12 Validate services automatically log user actions, configuration changes, and API calls. 2.13 Assess whether default logging scope can be customized per tenant. 2.14 Ensure regular scope reviews as services or customer requirements evolve. 2.15 Verify logging of control plane and data plane events. 2.16 Review service documentation to ensure it defines and enforces consistent logging scope.
Standards mappings
ISO 42001 A.6.2.8
Addendum
The ISO 42001 does not cover the LOG-07 topic of documenting which systems are to be logged and does not cover reviewing the list of logged items annually.
Article 12
Addendum
Contrarily to the AICM control that provides an increased level of detail, the EU AI Act establishes more general requirements that may be interpreted as encompassing the aspects tackled in the control.
GV-1.5-001 MG-4.2-001
Addendum
There doesn't seem to be NIST AI 600-1 requirements that provide guidance on defining "system events" for monitoring or logging. There are a number of requirements, however, that provide guidance on monitoring other important events in the GAI system. As when monitoring is occurring, recording of that information in system logs is "usually" part of the configuration.
C4 PC-02 C5 OPS-10 C5 OPS-11
Addendum
N/A
AI-CAIQ questions (2)
Are information metadata system events that should be logged, established, documented, and implemented?
Is the scope reviewed and updated at least annually, or whenever there is a change in the threat environment?