The AI Security Newsletter #1 | Looking back at our favourite content

The inaugural edition!

October 06, 2023

Greetings AI and Security Enthusiasts,

Welcome to the inaugural edition of The AI Security Newsletter - your update on all things AI security!

Whether you're an aspiring white-hat hacker, leading the charge on defence, concerned about the risk of AI in your organisation, or leveraging AI to enhance security, we've got you covered. We'll deliver the most useful updates in AI security directly to your inbox.

In this first post, we're highlighting some of the foundational insights that have shaped the AI security landscape as we know it today. Consider it a little background reading to get us all on the same page before we dive into the latest developments in the next issue.

Let's dive in!

The What

OWASP recently released their Top 10 list of security risks specific to large language models like ChatGPT. This is a must-read guide to the unique vulnerabilities of this emerging tech.

Why We Liked It

OWASP's involvement fills a critical gap in standardizing security measures for LLMs.

The What

Building on the OWASP top threat to LLM’s being Prompt Injection, Simon Willison's Weblog offers a detailed breakdown, including a webinar organised by LangChain.

Why We Liked It

Prompt injection is your first line of defence and your worst vulnerability, all rolled into one. This resource breaks it down into digestible bits and shares opinions on why proposed solutions will be not be effective, making it accessible without dumbing it down.

The What

This paper introduces an efficient and simple method to execute adversarial attacks on aligned language models. These attacks can force models to generate inappropriate or harmful content, and they're not limited to just one model; they can be transferred across various platforms.

Why We Liked It

The research stands out for its focus on the systemic weaknesses in AI language models. It effectively pushes the need for more advanced prevention mechanisms to the forefront.

The What

This illuminating post reveals how vulnerabilities within ChatGPT's plugin infrastructure can be exploited for malicious ends. A sinister website could potentially take control of a ChatGPT chat session and exfiltrate the conversation history.

Why We Liked It

The article serves as a stark reminder that even areas often considered peripheral, like plugins and third-party integrations, can be high-risk zones for data exfiltration. The piece emphasizes the need for greater scrutiny on what data is being sent to these plugins and calls for more robust security controls. It's a timely wakeup call for those who may have overlooked this vector in their security plans.

The What

This post investigates vulnerabilities in OpenAI's browsing plugin for ChatGPT, specifically focusing on two prompt injection techniques. These methods exploit the input-dependent nature of the conversational AI model, enabling an attacker to extract sensitive data. The study reveals significant risks associated with user privacy and data security in AI conversational platforms.

Why We Liked It

A little self promotion here, our post underscores the acute vulnerabilities that can arise in AI-driven conversational platforms, particularly when it comes to user privacy and data security. By focusing on two specific types of prompt injection techniques, we offer a targeted view into the risks inherent in these systems

The What

This insightful paper delves into the intriguing concept of Indirect Prompt Injection attacks targeted at Large Language Models (LLMs) when integrated with different applications. It identifies a host of security risks, such as remote data theft and ecosystem contamination, that are relevant not just in a theoretical context but also in real-world applications.

Why We Liked It

This paper doesn't just stop at identifying vulnerabilities; it serves as a call for the tech community to reassess their security strategies with regard to LLMs. The real-world implications of these findings, especially in the context of remote data theft and ecosystem contamination, are significant. If you're leveraging LLMs in your stack, this is more than a good read—it's essential for understanding the broader security implications.

The What

Google's red team offers a detailed guide on hacking AI systems. They've developed the Secure AI Framework (SAIF), combining traditional cybersecurity methods with challenges unique to AI, such as model theft and data poisoning.

Why We Liked It

Google's SAIF is an ambitious and holistic approach to AI security. It takes into account both classic and AI-specific vulnerabilities, serving as an invaluable guide for the tech community.

The What

Dropbox is experimenting with large language models for potential product backends. The Security team is particularly focused on preventing abuse from user-controlled inputs, working diligently to harden internal infrastructure against such vulnerabilities.

Why We Liked It

Dropbox's approach impresses us for its foresight and commitment to security. They're taking the initiative to understand the risks inherent in AI technology and are actively working to mitigate them.

That wraps up our inaugural edition of the AI Security Newsletter. We hope you found these deep dives into the world of AI and security as enlightening as we did. In our next issue, look forward to cutting-edge content that keeps you updated on the most recent developments in AI security so make sure to subscribe. Until then, stay secure and keep questioning.