Jump to Content
SAIF Map Tour
SAIF Map Tour

Risk Self Assessment

Every organization and AI implementation is unique. We developed this self-assessment for security practitioners, to help identify which AI risks may be most relevant to you.

Use this assessment to start conversations and guide further research.

Risk Self Assessment

This is a resource to help you understand which SAIF risks may be relevant to your organization.

It is for informational purposes only and does not constitute or replace professional advice. Your answers are not collected or shared. By continuing, you agree with the terms.
Learn more.

Risk Assessment Questions


Which of the following best describes your organization’s use of Generative AI models? You may select more than one option.

Risk Assessment Questions


Do you have robust management of all training, tuning, or evaluation data used with your models to ensure that sensitive, unauthorized, or malicious data does not enter your models?


Are you able to detect, remove, and remediate malicious or accidental changes in your training, tuning, or evaluation data?


Is any sensitive user data used in training, tuning, or evaluating your AI models?


Do you have robust management of all user data that results from your Generative AI applications to ensure that user data is stored, processed, and used in accordance with user consents and user policies?


Do you have a complete inventory of all models, datasets (for training, tuning, or evaluation), and related ML artifacts (such as code)?


Do you have robust access controls on all models, datasets, and related ML artifacts to minimize, detect, and prevent unauthorized reading or copying?


Are you able to ensure that all data, models, and code used to train, tune, or evaluate models cannot be tampered without detection during model development and during deployment?


Are the frameworks, libraries, software systems, and hardware components used in the development and deployment of your models analyzed for and protected against security vulnerabilities?


Do you protect your Generative AI applications and models against large-scale malicious queries from user accounts, devices, or via APIs?


Are you using secure-by-default designs and coding frameworks in applications integrated with Generative AI applications?


Do you perform adversarial testing and training on models and Generative AI applications to improve resistance to adversarial inputs?


Do you build or deploy Generative AI powered agents or tools that can take actions on behalf of internal or external users?

SAIF Risk Report

Based on your self assessment answers, the following risks may be relevant to your organization.

This report has been shared with you. Start over to create your own report.

Relevant risks based on your responses

Data Poisoning

Altering data sources used to train the model. In terms of impact, Data Poisoning is comparable to modifying the logic of an application to change its behavior.

The following controls may help mitigate this risk:

Read more about the risk of Data Poisoning.

  • Are you able to detect, remove, and remediate malicious or accidental changes in your training, tuning, or evaluation data?
  • Do you have a complete inventory of all models, datasets (for training, tuning, or evaluation), and related ML artifacts (such as code)?
  • Do you have robust access controls on all models, datasets, and related ML artifacts to minimize, detect, and prevent unauthorized reading or copying?
  • Are you able to ensure that all data, models, and code used to train, tune, or evaluate models cannot be tampered without detection during model development and during deployment?
  • Are the frameworks, libraries, software systems, and hardware components used in the development and deployment of your models analyzed for and protected against security vulnerabilities?

Unauthorized Training Data

Using unauthorized data for model training. Using a model trained with Unauthorized Training Data might lead to legal or ethical challenges.

The following controls may help mitigate this risk:

Read more about the risk of Unauthorized Training Data.

  • Do you have robust management of all training, tuning, or evaluation data used with your models to ensure that sensitive, unauthorized, or malicious data does not enter your models?

Model Source Tampering

Tampering with the model's code or data. Model Source Tampering is similar to tampering with traditional software code, and can create vulnerabilities or unintended behavior.

The following controls may help mitigate this risk:

Read more about the risk of Model Source Tampering.

  • Do you have a complete inventory of all models, datasets (for training, tuning, or evaluation), and related ML artifacts (such as code)?
  • Do you have robust access controls on all models, datasets, and related ML artifacts to minimize, detect, and prevent unauthorized reading or copying?
  • Are you able to ensure that all data, models, and code used to train, tune, or evaluate models cannot be tampered without detection during model development and during deployment?
  • Are the frameworks, libraries, software systems, and hardware components used in the development and deployment of your models analyzed for and protected against security vulnerabilities?

Excessive Data Handling

Unauthorized collection, retention, processing, or sharing of user data. Excessive Data Handling may lead to policy and legal challenges.

The following controls may help mitigate this risk:

Read more about the risk of Excessive Data Handling.

  • Do you have robust management of all user data that results from your Generative AI applications to ensure that user data is stored, processed, and used in accordance with user consents and user policies?

Model Exfiltration

Theft of a model. Similar to stealing code, this threat has both intellectual property and security implications.

The following controls may help mitigate this risk:

Read more about the risk of Model Exfiltration.

  • Do you have a complete inventory of all models, datasets (for training, tuning, or evaluation), and related ML artifacts (such as code)?
  • Do you have robust access controls on all models, datasets, and related ML artifacts to minimize, detect, and prevent unauthorized reading or copying?
  • Are the frameworks, libraries, software systems, and hardware components used in the development and deployment of your models analyzed for and protected against security vulnerabilities?

Model Deployment Tampering

Unauthorized changes to model deployment components. Model Deployment Tampering can result in changes to model behavior.

The following controls may help mitigate this risk:

Read more about the risk of Model Deployment Tampering.

  • Are the frameworks, libraries, software systems, and hardware components used in the development and deployment of your models analyzed for and protected against security vulnerabilities?

Denial of ML Service

Overloading ML systems with resource-intensive queries. Like traditional DoS attacks, Denial of ML Service can reduce availability of or entirely disrupt a service.

The following controls may help mitigate this risk:

Read more about the risk of Denial of ML Service.

  • Do you protect your Generative AI applications and models against large-scale malicious queries from user accounts, devices, or via APIs?

Model Reverse Engineering

Recreating a model by analyzing its inputs, outputs, and behaviors. A reverse engineer model can be used to create imitation products or adversarial attacks.

The following controls may help mitigate this risk:

Read more about the risk of Model Reverse Engineering.

  • Do you protect your Generative AI applications and models against large-scale malicious queries from user accounts, devices, or via APIs?

Insecure Integrated Component

Software vulnerabilities that can be leveraged to compromise AI models. Insecure Integrated Component can lead to privacy and security concerns, as well as potential ethical and legal challenges.

The following controls may help mitigate this risk:

Read more about the risk of Insecure Integrated Component.

  • Are you using secure-by-default designs and coding frameworks in applications integrated with Generative AI applications?

Prompt Injection

Tricking a model to run unintended commands. In terms of impact, Prompt Injection can change a model's behavior.

The following controls may help mitigate this risk:

Read more about the risk of Prompt Injection.

  • Do you perform adversarial testing and training on models and Generative AI applications to improve resistance to adversarial inputs?
  • Do you build or deploy Generative AI powered agents or tools that can take actions on behalf of internal or external users?

Model Evasion

Changes to a prompt input to cause the model to produce incorrect inferences. Model Evasion can lead to reputational, legal, security, and privacy risks.

The following controls may help mitigate this risk:

Read more about the risk of Model Evasion.

  • Do you perform adversarial testing and training on models and Generative AI applications to improve resistance to adversarial inputs?

Sensitive Data Disclosure

Disclosure of sensitive data by the model. Sensitive Data Disclosure poses a threat to user privacy, organizational reputation, and intellectual property.

The following controls may help mitigate this risk:

Read more about the risk of Sensitive Data Disclosure.

  • Is any sensitive user data used in training, tuning, or evaluating your AI models?
  • Do you have robust management of all user data that results from your Generative AI applications to ensure that user data is stored, processed, and used in accordance with user consents and user policies?
  • Do you perform adversarial testing and training on models and Generative AI applications to improve resistance to adversarial inputs?

Inferred Sensitive Data

Model inferring personal information not contained in training data or inputs. Inferred Sensitive Data may be considered a data privacy incident.

The following controls may help mitigate this risk:

Read more about the risk of Inferred Sensitive Data.

  • Do you have robust management of all training, tuning, or evaluation data used with your models to ensure that sensitive, unauthorized, or malicious data does not enter your models?
  • Do you perform adversarial testing and training on models and Generative AI applications to improve resistance to adversarial inputs?

Insecure Model Output

Unvalidated model output passed to the end user. Insecure Model Output poses risks to organizational reputation, security, and user safety.

The following controls may help mitigate this risk:

Read more about the risk of Insecure Model Output.

  • Do you perform adversarial testing and training on models and Generative AI applications to improve resistance to adversarial inputs?

Rogue Actions

Unintentional model-based actions executed via extensions. Rogue Actions can create a cascading, risk to organizational reputation, user trust, security, and safety.

The following controls may help mitigate this risk:

Read more about the risk of Rogue Actions.

  • Do you build or deploy Generative AI powered agents or tools that can take actions on behalf of internal or external users?