Controls
The following sections describe controls available to organizations for specific AI risks.
Each control is mapped onto the corresponding risks it can address, with the exception of Governance and Assurance controls, which apply universally to all risks and every stage of the AI development process.
Data controls
Privacy Enhancing Technologies
Control
Privacy Enhancing Technologies
Who can implement
Model Creators
Risk mapping
Sensitive Data Disclosure
Use technologies that minimize, de-identify, or restrict use of PII data in training or evaluating models.
Control
Privacy Enhancing Technologies
Who can implement
Model Creators
Risk mapping
Sensitive Data Disclosure
Training Data Management
Control
Training Data Management
Who can implement
Model Creators
Risk mapping
Inferred Sensitive Data, Unauthorized Training Data
Ensure that all data used to train and evaluate models is authorized for the intended purposes.
Control
Training Data Management
Who can implement
Model Creators
Risk mapping
Inferred Sensitive Data, Unauthorized Training Data
Training Data Sanitization
Control
Training Data Sanitization
Who can implement
Model Creators
Risk mapping
Data Poisoning, Unauthorized Training Data
Detect and remove or remediate poisoned or sensitive data in training and evaluation.
Control
Training Data Sanitization
Who can implement
Model Creators
Risk mapping
Data Poisoning, Unauthorized Training Data
User Data Management
Control
User Data Management
Who can implement
Model Creators, Model Consumers
Risk mapping
Sensitive Data Disclosure, Excessive Data Handling
Store, process, and use all user data (e.g. prompts and logs) from AI applications in compliance with user consent.
Control
User Data Management
Who can implement
Model Creators, Model Consumers
Risk mapping
Sensitive Data Disclosure, Excessive Data Handling
Infrastructure controls
Model and Data Inventory Management
Control
Model and Data Inventory Management
Who can implement
Model Creators, Model Consumers (if storing models)
Risk mapping
Data Poisoning, Model Source Tampering, Model Exfiltration
Ensure that all data, code, models, and transformation tools used in AI applications are inventoried and tracked.
Control
Model and Data Inventory Management
Who can implement
Model Creators, Model Consumers (if storing models)
Risk mapping
Data Poisoning, Model Source Tampering, Model Exfiltration
Model and Data Access Controls
Control
Model and Data Access Controls
Who can implement
Model Creators, Model Consumers (if storing models)
Risk mapping
Data Poisoning, Model Source Tampering, Model Exfiltration
Minimize internal access to models, weights, datasets, etc. in storage and in production use.
Control
Model and Data Access Controls
Who can implement
Model Creators, Model Consumers (if storing models)
Risk mapping
Data Poisoning, Model Source Tampering, Model Exfiltration
Model and Data Integrity Management
Control
Model and Data Integrity Management
Who can implement
Model Creators, Model Consumers (if storing models)
Risk mapping
Data Poisoning, Model Source Tampering
Ensure that all data, models, and code used to produce AI models are verifiably integrity-protected during development and deployment.
Control
Model and Data Integrity Management
Who can implement
Model Creators, Model Consumers (if storing models)
Risk mapping
Data Poisoning, Model Source Tampering
Secure-by-Default ML Tooling
Control
Secure-by-Default ML Tooling
Who can implement
Model Creators, Model Consumers (if storing models)
Risk mapping
Data Poisoning, Model Source Tampering, Model Exfiltration, Model Deployment Tampering
Use secure-by-default frameworks, libraries, software systems, and hardware components for AI development or deployment to protect confidentiality and integrity of AI assets and outputs.
Control
Secure-by-Default ML Tooling
Who can implement
Model Creators, Model Consumers (if storing models)
Risk mapping
Data Poisoning, Model Source Tampering, Model Exfiltration, Model Deployment Tampering
Model controls
Input Validation and Sanitization
Control
Input Validation and Sanitization
Who can implement
Model Creators, Model Consumers
Risk mapping
Prompt Injection
Block or restrict adversarial queries to AI models.
Control
Input Validation and Sanitization
Who can implement
Model Creators, Model Consumers
Risk mapping
Prompt Injection
Output Validation and Sanitization
Control
Output Validation and Sanitization
Who can implement
Model Creators, Model Consumers
Risk mapping
Prompt Injection, Rogue Actions, Sensitive Data Disclosure, Inferred Sensitive Data
Block, nullify, or sanitize insecure output from AI models before passing it to applications, extensions or users.
Control
Output Validation and Sanitization
Who can implement
Model Creators, Model Consumers
Risk mapping
Prompt Injection, Rogue Actions, Sensitive Data Disclosure, Inferred Sensitive Data
Adversarial Training and Testing
Control
Adversarial Training and Testing
Who can implement
Model Creators, Model Consumers
Risk mapping
Model Evasion, Prompt Injection, Sensitive Data Disclosure, Inferred Sensitive Data, Insecure Model Output
Use techniques to make AI models robust to adversarial inputs (i.e. prompts) in the context of their use in applications.
Control
Adversarial Training and Testing
Who can implement
Model Creators, Model Consumers
Risk mapping
Model Evasion, Prompt Injection, Sensitive Data Disclosure, Inferred Sensitive Data, Insecure Model Output
Application controls
Application Access Management
Control
Application Access Management
Who can implement
Model Consumers
Risk mapping
Denial of ML Service, Model Reverse Engineering
Ensure that only authorized users and endpoints can access specific resources for authorized actions.
Control
Application Access Management
Who can implement
Model Consumers
Risk mapping
Denial of ML Service, Model Reverse Engineering
User Transparency and Controls
Control
User Transparency and Controls
Who can implement
Model Consumers
Risk mapping
Sensitive Data Disclosure, Excessive Data Handling
Inform users of relevant AI risks with disclosures, and provide transparency and control experiences for use of their data in AI applications.
Control
User Transparency and Controls
Who can implement
Model Consumers
Risk mapping
Sensitive Data Disclosure, Excessive Data Handling
Agent User Control
Control
Agent User Control
Who can implement
Model Consumers
Risk mapping
Sensitive Data Disclosure, Rogue Actions
Ensure user approval for any actions performed by agents/plugins that alter user data or act on the user’s behalf.
Control
Agent User Control
Who can implement
Model Consumers
Risk mapping
Sensitive Data Disclosure, Rogue Actions
Agent Permissions
Control
Agent Permissions
Who can implement
Model Consumers
Risk mapping
Insecure Integrated System, Sensitive Data Disclosure, Rogue Actions
Use least-privilege principle as the upper bound on agentic system permissions to minimize the number of tools that an agent is permitted to interact with and the actions it is allowed to take. An agentic system’s use of privileges should be contextual and dynamic, adapting to the specific user query and trusted contextual information. This design also applies to agents that have access to user information. For example, an agent asked to fill out a form or answer questions should share only contextually appropriate information and can be designed to dynamically minimize exposed data using reference monitors.
Control
Agent Permissions
Who can implement
Model Consumers
Risk mapping
Insecure Integrated System, Sensitive Data Disclosure, Rogue Actions
Agent Observability (New)
Control
Agent Observability
Who can implement
Model Consumers
Risk mapping
Sensitive Data Disclosure, Rogue Actions
Ensure an agent's actions, tool use, and reasoning are transparent and auditable through logging, allowing for debugging, security oversight, and user insights into agent activity.
Control
Agent Observability
Who can implement
Model Consumers
Risk mapping
Sensitive Data Disclosure, Rogue Actions
Assurance controls
Red Teaming
Control
Red Teaming
Who can implement
Model Creators, Model Consumers
Risk mapping
All
Identify security and privacy improvements through self-driven adversarial attacks on AI infrastructure and products.
Control
Red Teaming
Who can implement
Model Creators, Model Consumers
Risk mapping
All
Vulnerability Management
Control
Vulnerability Management
Who can implement
Model Creators, Model Consumers
Risk mapping
All
Proactively and continually test and monitor production infrastructure and products for security and privacy regressions.
Control
Vulnerability Management
Who can implement
Model Creators, Model Consumers
Risk mapping
All
Threat Detection
Control
Threat Detection
Who can implement
Model Creators, Model Consumers
Risk mapping
All
Detect and alert on internal or external attacks on AI assets, infrastructure, and products.
Control
Threat Detection
Who can implement
Model Creators, Model Consumers
Risk mapping
All
Incident Response Management
Control
Incident Response Management
Who can implement
Model Creators, Model Consumers
Risk mapping
All
Manage response to AI security and privacy incidents.
Control
Incident Response Management
Who can implement
Model Creators, Model Consumers
Risk mapping
All
Governance controls
User Policies and Education
Control
User Policies and Education
Who can implement
Model Consumers
Risk mapping
Will vary
Publish easy to understand AI security and privacy policies and education for users.
Control
User Policies and Education
Who can implement
Model Consumers
Risk mapping
Will vary
Internal Policies and Education
Control
Internal Policies and Education
Who can implement
Model Creators, Model Consumers
Risk mapping
All
Publish comprehensive AI security and privacy policies and education for your employees.
Control
Internal Policies and Education
Who can implement
Model Creators, Model Consumers
Risk mapping
All
Product Governance
Control
Product Governance
Who can implement
Model Creators, Model Consumers
Risk mapping
All
Validate that all AI models and products meet the established security and privacy requirements.
Control
Product Governance
Who can implement
Model Creators, Model Consumers
Risk mapping
All
Risk Governance
Control
Risk Governance
Who can implement
Model Creators, Model Consumers
Risk mapping
All
Inventory, measure, and monitor residual risk to AI in your organization.
Control
Risk Governance
Who can implement
Model Creators, Model Consumers
Risk mapping
All
