Securing the AI Attack Surface: Separating the Unknown from the Well Understood
2024-8-7 22:0:0 Author: www.tenable.com(查看原文) 阅读量:1 收藏

Lucas Tamagna-Darr

Securing the AI Attack Surface - Separating the Unknown from the Well Understood

There's a lot of confusion and unknowns regarding AI application risks, and a lot of vendors are trying to offer solutions to problems that aren't clearly defined. In this blog we explain why a smart approach is to start by focusing on basic, foundational cyber hygiene, adopt well-established best practices and enforce common-sense usage policies.

Introduction

With all of the excitement around large language models (LLMs) and artificial intelligence (AI) in general, there has been an inevitable rush to build the next great “powered by LLM” applications. As is frequently the case, that rush has been followed by a lot of questions about the related risks and about how to secure these applications. Some aspects of securing LLM-powered applications are novel and present serious complexities that require consideration. Meanwhile, other aspects are not that different from the risks that traditional vulnerability management programs have been dealing with all along.

OWASP provides a Top 10 list which identifies the following risks for LLM applications:

  • Prompt Injection
  • Insecure Output Handling
  • Training Data Poisoning
  • Model DoS
  • Supply Chain Vulnerabilities
  • Sensitive Information Disclosure
  • Insecure Plugin Design
  • Excessive Agency
  • Overreliance
  • Model Theft

Novel challenges

Risks like prompt injection, sensitive-information disclosure, and training-data poisoning are not in and of themselves new. However, the practicality of exploitation and the challenges of securing against those risks are still an area of extensive research. We’ve seen reports of prompt-injection attacks on language models where an attacker convinces the model to ignore its instructions and disclose information that it shouldn’t have disclosed. These types of attacks are technically achievable but because of the nature of LLMs, the practicality of these attacks is uncertain. Unlike a SQL injection attack where you can develop some degree of control of the data you get back, an LLM prompt-injection attack is always going to be subject to the behavior of the underlying model.

Securing an LLM application against prompt injection similarly presents unique challenges. With traditional command injection / SQL injection attacks, we have a high degree of control of the input and can significantly constrain what a user inputs, as well as filter any unwanted input in pre-processing. With LLMs, the ability to have more open-ended questions and responses, and potentially follow-up questions, means that any pre-filtering or processing risks reducing the functionality of the feature. This inevitably becomes a delicate balancing act between security and functionality. We will likely see further developments in both improved attacks against prompts as well as better tooling to protect those same prompts.

Similar challenges for both the attacker and defender also exist in output handling and training-data poisoning. These risks should not be ignored and teams should be as defensive as possible, given the degree of uncertainty and the complex challenges. However, it can be easy to get bogged down in trying to address these risks while overlooking equally dangerous vulnerabilities that are also less complex to exploit and more familiar to defenders.

What to do - start with the basics

Two core risks associated with AI and LLM applications are supply-chain vulnerabilities and the inherent privacy risks. Both of these should be top of mind for applications developed in-house and for third-party applications, such as those from software vendors and from open-source projects. . Fortunately these risks are much better understood and can be incorporated into existing vulnerability management programs without significant complexity.

Looking at some of the major libraries that support AI and LLM applications, such as Transformers, NLTK, Tensorflow, and Langchain, we can see that many of them have at least some high and critical severity vulnerabilities that have been disclosed. When developing applications, it is critical that teams have visibility on the libraries being used as well as on the vulnerabilities in those libraries. We’ve recently seen vulnerability databases such as https://avidml.org/database/ as well as bug bounty programs such as https://huntr.com/ that are focused on AI-specific vulnerabilities. These can be valuable resources for teams that want to focus specifically on ensuring they mitigate any vulnerabilities in those libraries.

Privacy risks can be mitigated to some degree by establishing strong policies on the language models that engineering teams are permitted to use when building in-house applications and monitoring to ensure that only those language models are used. This is particularly important for any language models that are hosted by a third party. Additionally, it’s vital to have an inventory of all applications that employees are using which have LLM-powered features, as these can become uncontrollable channels for data leakage. A search for the term “AI” in the Firefox browser-extension catalog returns over 2,000 results , while a search for the term “LLM” returns over 300 results. Many employees may choose to install these types of extensions without fully understanding the security and privacy implications, nor how any data they input may end up being passed to a third party or added to an LLM training dataset. Organizations should establish approved applications and browser extensions, as well as establish policies prohibiting the use of any extensions that have not been evaluated and approved for usage.

Conclusion

LLMs and AI in general have created a lot of excitement across organizations globally but have also left many security teams with unanswered questions about how they can manage the risks. We will inevitably see a rush of new products and features aimed at securing models and prompts, and at blocking any malicious activity. Given the uncertainty around both the reliability and practicality of attacks on LLMs and the ability of tools to block those attacks without negating the value of LLMs, it may be best to watch that space before rushing in. In the meantime, security teams can take concrete, well-understood and well-defined steps to reduce their risk and secure the attack surface. They can also establish strong corporate policies on the usage of LLMs and LLM-powered applications, and conduct monitor to ensure those policies are followed.

Lucas Tamagna-Darr

Lucas Tamagna-Darr

In his role as a director of engineering and research solutions architect, Lucas Tamagna-Darr leads the automation and engineering functions of Tenable Research. Luke started out at Tenable developing plugins for Nessus and Nessus Network Monitor. He subsequently went on to lead several different functions within Tenable Research and now leverages his experience to help surface better content for customers across Tenable's products.

Related Articles

  • Attack Surface Management
  • Exposure Management
  • Risk-based Vulnerability Management

Cybersecurity News You Can Use

Enter your email and never miss timely alerts and security guidance from the experts at Tenable.

Tenable Vulnerability Management

Enjoy full access to a modern, cloud-based vulnerability management platform that enables you to see and track all of your assets with unmatched accuracy.

Tenable Vulnerability Management trials created everywhere except UAE will also include Tenable Lumin and Tenable Web App Scanning.

Tenable Vulnerability Management

Enjoy full access to a modern, cloud-based vulnerability management platform that enables you to see and track all of your assets with unmatched accuracy. Purchase your annual subscription today.

100 assets

Choose your subscription option:

Thank You

Thank you for your interest in Tenable Vulnerability Management. A representative will be in touch soon.

Tenable Vulnerability Management

Enjoy full access to a modern, cloud-based vulnerability management platform that enables you to see and track all of your assets with unmatched accuracy.

Tenable Vulnerability Management trials created everywhere except UAE will also include Tenable Lumin and Tenable Web App Scanning.

Tenable Vulnerability Management

Enjoy full access to a modern, cloud-based vulnerability management platform that enables you to see and track all of your assets with unmatched accuracy. Purchase your annual subscription today.

100 assets

Choose your subscription option:

Thank you

Thank you for your interest in Tenable.io. A representative will be in touch soon.

Tenable Vulnerability Management

Enjoy full access to a modern, cloud-based vulnerability management platform that enables you to see and track all of your assets with unmatched accuracy.

Tenable Vulnerability Management trials created everywhere except UAE will also include Tenable Lumin and Tenable Web App Scanning.

Tenable Vulnerability Management

Enjoy full access to a modern, cloud-based vulnerability management platform that enables you to see and track all of your assets with unmatched accuracy. Purchase your annual subscription today.

100 assets

Choose your subscription option:

Thank you

Thank you for your interest in Tenable Vulnerability Management. A representative will be in touch soon.

Try Tenable Web App Scanning

Enjoy full access to our latest web application scanning offering designed for modern applications as part of the Tenable One Exposure Management platform. Safely scan your entire online portfolio for vulnerabilities with a high degree of accuracy without heavy manual effort or disruption to critical web applications. Sign up now.

Your Tenable Web App Scanning trial also includes Tenable Vulnerability Management and Tenable Lumin.

Buy Tenable Web App Scanning

Enjoy full access to a modern, cloud-based vulnerability management platform that enables you to see and track all of your assets with unmatched accuracy. Purchase your annual subscription today.

Try Tenable Lumin

Visualize and explore your exposure management, track risk reduction over time and benchmark against your peers with Tenable Lumin.

Your Tenable Lumin trial also includes Tenable Vulnerability Management and Tenable Web App Scanning.

Buy Tenable Lumin

Contact a sales representative to see how Tenable Lumin can help you gain insight across your entire organization and manage cyber risk.

Thank you

Thank you for your interest in Tenable Lumin. A representative will be in touch soon.

Request a demo of Tenable Security Center

Please fill out this form with your contact information.

A sales representative will contact you shortly to schedule a demo.

* Field is required

Request a demo of Tenable OT Security

Get the Operational Technology security you need.

Reduce the risk you don’t.

Request a demo of Tenable Identity Exposure

Continuously detect and respond to Active Directory attacks. No agents. No privileges.

On-prem and in the cloud.

Request a demo of Tenable Cloud Security

Exceptional unified cloud security awaits you!

We’ll show you exactly how Tenable Cloud Security helps you deliver multi-cloud asset discovery, prioritized risk assessments and automated compliance/audit reports.

See
Tenable One
in action

Exposure management for the modern attack surface.

See Tenable Attack Surface Management in action

Know the exposure of every asset on any platform.

Thank You

Thank you for your interest in Tenable Attack Surface Management. A representative will be in touch soon.

Try Tenable Nessus Professional free

Free for 7 days

Tenable Nessus is the most comprehensive vulnerability scanner on the market today.

NEW - Tenable Nessus Expert
now available

Nessus Expert adds even more features, including external attack surface scanning, and the ability to add domains and scan cloud infrastructure. Click here to Try Nessus Expert.

Fill out the form below to continue with a Nessus Pro trial.

Buy Tenable Nessus Professional

Tenable Nessus is the most comprehensive vulnerability scanner on the market today. Tenable Nessus Professional will help automate the vulnerability scanning process, save time in your compliance cycles and allow you to engage your IT team.

Buy a multi-year license and save. Add Advanced Support for access to phone, community and chat support 24 hours a day, 365 days a year.

Try Tenable Nessus Expert free

Free for 7 days.

Built for the modern attack surface, Nessus Expert enables you to see more and protect your organization from vulnerabilities from IT to the cloud.

Already have Tenable Nessus Professional?
Upgrade to Nessus Expert free for 7 days.

Buy Tenable Nessus Expert

Built for the modern attack surface, Nessus Expert enables you to see more and protect your organization from vulnerabilities from IT to the cloud.

Learn How Tenable Helps Achieve SLCGP Cybersecurity Plan Requirements

Tenable solutions help fulfill all SLCGP requirements. Connect with a Tenable representative to learn more.


文章来源: https://www.tenable.com/blog/securing-the-ai-attack-surface-separating-the-unknown-from-the-well-understood
如有侵权请联系:admin#unsafe.sh