Here at RSA, the hype is on “high”, including dune buggies driving the streets wrapped in high-tech banners claiming to have solved all things AI. Even before you get downtown you are greeted at the airport with big budget AI splashed all over the walls with outsized claims.
But what is real?
We here at SecureIQLab are trying to bring some of those claims back down to earth, in the form of neutral third party tools and reports to give normal businesses and organizations a way to sort through the fluff and get to what real information you can use.
Here are a few ways to cut the clutter and get to what you can trust about AI.
- Show me the failure cases, not the demo – Right now everyone’s talking about fantastic capabilities, but what happens in the first contact with adversarial attack traffic? What happened under heavy load, not just a few test prompts? How does a vendor tune their systems against false positives and false negatives? If they don’t know – that’s not a good sign.
- Map claims to specific attack classes – A vendor claiming to “stop AI threats” is vague, at best. Best to ask “Which of the OWASP LLM Top 10 do you actually stop—and how do you prove it?” For example:
- Prompt Injection – How did you test it?
- Data exfiltration – Did you monitor, or block it?
- Model extraction – How did you measure it?
- Separate detection from prevention – Right now, many AI “security” tools are little more than telemetry engines. Better ask “is your tool blocking, or just monitoring”, and “how many attacks are automatically blocked”.
- Ask for an independent assessment – Right now, many vendors are basically grading their own papers – and producing “surprisingly” good results. But self-testing is an awful lot like not testing, and just as likely to survive when faced with real adversarial traffic. If there’s no third-party validation, assume you’re still looking at a demo.
At RSA, AI is everywhere, but clarity is not. It’s a good time to gather information, with a healthy dose of skepticism, and ask some specific, pointed questions. Solid, tested vendors won’t just show you their successes, but also failures, where they’re learning, and how they’re evolving their products to face real-world threats.
If you’re at RSAC 2026, book a meeting with me to see our SOCx + AI validation demo or to learn more about our AI Security CyberRisk Validation Methodology v1.0–releasing this week.
*** This is a Security Bloggers Network syndicated blog from SecureIQ Lab authored by Cameron Camp. Read the original post at: https://secureiqlab.com/rsa-2026-ai-oozing-out-of-every-pore/