Agent Risk Self Assessment
AI Agents are sensitive to a specific set of risks. We developed this self-assessment for security practitioners, to help identify which AI risks may be most relevant to your agents.
Use this assessment to start conversations and guide further research.
Agent Risk Self Assessment
This is a resource to help you understand which SAIF risks may be relevant to your organization.
It is for informational purposes only and does not constitute or replace professional advice. Your answers are not collected or shared. By continuing, you agree with the terms.
About this self-assessment
For informational purposes only
This self-assessment is intended to help you understand what AI security risks may be relevant to your organization. By using this self-assessment, you agree that this resource is for informational purposes only. Consult a security professional for advice. You're encouraged to talk to a security professional about what your results may mean for your organization. Google disclaims any liability, loss, or risk incurred as a consequence of using this resource.
Private and secure
Google does not collect or share answers or results from the self-assessment. We collect data only to analyze traffic, and we use that data only in ways that do not identify you and that follow our Privacy Policy.
Agent Risk Self Assessment
Identity and authorization
Does your AI Agent use an agent-specific identity with well-scoped credentials to ensure it can access only the data and tools it actually needs and is authorized to use?
Observability
Does your AI Agent keep a clear record of its actions, inputs, and outputs so users and operators can monitor and audit what it is doing on their behalf?
User Control
Does your AI Agent use Human-in-The-Loop (HiTL) or Human-on-The-Loop (HoTL) mechanisms, enabling users to oversee and govern actions involving critical systems and sensitive data?
Built-in model safety
Is your AI Agent powered by a model that has built-in adversarial safeguards to protect itself against prompt injection attacks and to prevent it from generating harmful content (or code)?
In-tool guardrails
Does your AI Agent have policy guardrails to prevent accidental or malicious misuse of your data and systems?
Input and output screening
Does your AI Agent check incoming requests for attacks and sanitize its responses to prevent data leaks?
Testing
Have you tested your AI Agent to see how it handles prompt injection attacks designed to trick it into breaking security rules?
SAIF Agent Security Risk Report
This report has been shared with you. Start over to create your own report.
Agent risks
Rogue actions
An agent executes an operation (via tools or APIs) that is not aligned with the user's intent; either accidentally (due to a model hallucination or reasoning error) or maliciously (due to indirect prompt injection through untrusted content).
Sensitive Data Disclosure (or Data Exfiltration)
An agent reveals private, confidential, or proprietary information to unauthorized parties.
Assessment & mitigations
Mitigations that need consideration
Identity and authorization
Does your AI Agent use an agent-specific identity with well-scoped credentials to ensure it can access only the data and tools it actually needs and is authorized to use?
Mitigation
Tightly scoping access to data and tools limits the potential impact of a rogue action through deterministic controls tailored to the intended functionality of an agent (principle of least privilege).
Observability
Does your AI Agent keep a clear record of its actions, inputs, and outputs so users and operators can monitor and audit what it is doing on their behalf?
Mitigation
Implementing auditable logs brings transparency that allows monitoring agent activity, enabling you to detect rogue actions or the disclosure of sensitive data and take corrective measures.
User Control
Does your AI Agent use Human-in-The-Loop (HiTL) or Human-on-The-Loop (HoTL) mechanisms, enabling users to oversee and govern actions involving critical systems and sensitive data?
Mitigation
Ensuring the user has the final word before an agent performs state-changing or irreversible actions with high consequences is a powerful safeguard against rogue actions and the disclosure of sensitive data, guaranteeing that the planned agent action is aligned with user intent.
Built-in model safety
Is your AI Agent powered by a model that has built-in adversarial safeguards to protect itself against prompt injection attacks and to prevent it from generating harmful content (or code)?
Mitigation
Building on a safe foundation forms a strong initial protection against rogue actions by guarding against prompt injections and preventing many forms of unwanted actions by default. This may include adversarial fine-tuning or other model-level techniques.
In-tool guardrails
Does your AI Agent have policy guardrails to prevent accidental or malicious misuse of your data and systems?
Mitigation
Defining security policies that implement a product-specific set of constraints on the actions that the tools the agent calls can perform prevents rogue actions (often in conjunction with User Control). Critic models may further augment in-tool guardrails.
Input and output screening
Does your AI Agent check incoming requests for attacks and sanitize its responses to prevent data leaks?
Mitigation
Blocking adversarial queries from reaching your agent closes an avenue for triggering malicious rogue actions. Sanitizing or blocking insecure content before your agent outputs it to applications, extensions, or users prevents the disclosure of sensitive data and the triggering of rogue actions in adjacent systems. Input screening may include prompt injection classifiers, among others.
Testing
Have you tested your AI Agent to see how it handles prompt injection attacks designed to trick it into breaking security rules?
Mitigation
Have you tested your AI Agent to see how it handles prompt injection attacks designed to trick it into breaking security rules?
Mitigations in place
Identity and authorization
Does your AI Agent use an agent-specific identity with well-scoped credentials to ensure it can access only the data and tools it actually needs and is authorized to use?
Mitigation
Tightly scoping access to data and tools limits the potential impact of a rogue action through deterministic controls tailored to the intended functionality of an agent (principle of least privilege).
Observability
Does your AI Agent keep a clear record of its actions, inputs, and outputs so users and operators can monitor and audit what it is doing on their behalf?
Mitigation
Implementing auditable logs brings transparency that allows monitoring agent activity, enabling you to detect rogue actions or the disclosure of sensitive data and take corrective measures.
User Control
Does your AI Agent use Human-in-The-Loop (HiTL) or Human-on-The-Loop (HoTL) mechanisms, enabling users to oversee and govern actions involving critical systems and sensitive data?
Mitigation
Ensuring the user has the final word before an agent performs state-changing or irreversible actions with high consequences is a powerful safeguard against rogue actions and the disclosure of sensitive data, guaranteeing that the planned agent action is aligned with user intent.
Built-in model safety
Is your AI Agent powered by a model that has built-in adversarial safeguards to protect itself against prompt injection attacks and to prevent it from generating harmful content (or code)?
Mitigation
Building on a safe foundation forms a strong initial protection against rogue actions by guarding against prompt injections and preventing many forms of unwanted actions by default. This may include adversarial fine-tuning or other model-level techniques.
In-tool guardrails
Does your AI Agent have policy guardrails to prevent accidental or malicious misuse of your data and systems?
Mitigation
Defining security policies that implement a product-specific set of constraints on the actions that the tools the agent calls can perform prevents rogue actions (often in conjunction with User Control). Critic models may further augment in-tool guardrails.
Input and output screening
Does your AI Agent check incoming requests for attacks and sanitize its responses to prevent data leaks?
Mitigation
Blocking adversarial queries from reaching your agent closes an avenue for triggering malicious rogue actions. Sanitizing or blocking insecure content before your agent outputs it to applications, extensions, or users prevents the disclosure of sensitive data and the triggering of rogue actions in adjacent systems. Input screening may include prompt injection classifiers, among others.
Testing
Have you tested your AI Agent to see how it handles prompt injection attacks designed to trick it into breaking security rules?
Mitigation
Have you tested your AI Agent to see how it handles prompt injection attacks designed to trick it into breaking security rules?
