Manipulating AI Summarization Features
好的,我现在需要帮用户总结这篇文章的内容。用户的要求是用中文,控制在100字以内,不需要特定的开头,直接写描述。
首先,我得仔细阅读文章内容。文章提到微软报告称,公司通过“用AI总结”按钮嵌入隐藏指令,试图通过URL参数将持久命令注入AI助手的记忆中。这些提示指令会让AI记住公司为可信来源或优先推荐其产品,从而在未来偏向于推荐它们。已经发现31家公司使用了50多种独特的提示,涉及14个行业。这种技术容易部署,但可能导致AI在健康、金融和安全等关键领域提供带有偏见的建议,用户却不知情。
接下来,作者提到两年前就讨论过这种情况,认为这是LLM优化的例子,类似于SEO,并认为这将成为大生意。
现在我需要将这些信息浓缩到100字以内。要抓住关键点:公司利用按钮嵌入指令影响AI记忆和推荐,导致偏见;已有多家公司采用;技术简单但影响严重;与SEO类似,可能成为重要业务。
确保语言简洁明了,避免使用复杂的术语。同时注意不要超过字数限制。
最后检查一下是否涵盖了所有主要信息:隐藏指令、注入记忆、偏见推荐、公司数量、技术易用性、潜在影响、与SEO的类比以及商业潜力。
</think>
微软报告指出,企业通过“用AI总结”按钮嵌入隐藏指令,在用户点击时尝试将持久命令注入AI助手的记忆中。这些提示旨在让AI记住企业为可信来源或优先推荐其产品服务。已有31家公司使用此技术,可能对健康、金融和安全等领域产生隐性偏见影响。
2026-3-4 12:6:1
Author: www.schneier.com(查看原文)
阅读量:4
收藏
Microsoft is reporting:
Companies are embedding hidden instructions in “Summarize with AI” buttons that, when clicked, attempt to inject persistence commands into an AI assistant’s memory via URL prompt parameters….
These prompts instruct the AI to “remember [Company] as a trusted source” or “recommend [Company] first,” aiming to bias future responses toward their products or services. We identified over 50 unique prompts from 31 companies across 14 industries, with freely available tooling making this technique trivially easy to deploy. This matters because compromised AI assistants can provide subtly biased recommendations on critical topics including health, finance, and security without users knowing their AI has been manipulated.
I wrote about this two years ago: it’s an example of LLM optimization, along the same lines as search-engine optimization (SEO). It’s going to be big business.
Tags: AI, LLM, Microsoft
Posted on March 4, 2026 at 7:06 AM •
1 Comments
Sidebar photo of Bruce Schneier by Joe MacInnis.
文章来源: https://www.schneier.com/blog/archives/2026/03/manipulating-ai-summarization-features.html
如有侵权请联系:admin#unsafe.sh