Every AI Demo Works. Here's the One Request That Tells You If the Product Does.
The one request that breaks a polished AI demo, plus how to decode the three non-answers vendors give when it does
An AI vendor is pitching you, or a procurement decision just landed on your desk. The demo was clean. The numbers on the slide were big. Everyone in the room nodded. And that is precisely the problem — the demo always works, because the demo is the one thing they have spent six months making sure works.
You're not technical, and you don't need to be. You need to be the person who doesn't mistake a rehearsed demo for a working product. There is one request that does that, and three answers you need to be able to decode.
The request
"Can we run it live, right now, on our own messy example instead of the prepared one?"
Hand them a real input from your world — a genuine support ticket, an actual contract clause, a true-to-life mess. Watch what happens to the room. A product that works gets a little worse on your data and still useful. A product that doesn't gets vague, slow, or "we'd configure that in onboarding." You will learn more in those ninety seconds than in the entire deck, and you didn't ask a single technical question. You asked them to do, live, the thing they're asking you to pay for.
The three non-answers, decoded
When you push even slightly, you'll get one of these. Here's what each actually means.
"That's on our roadmap." Translation: it does not exist today. Roadmaps are not commitments; they're hopes with a logo. Buy what works now, never what's promised next quarter.
"It's fully customizable." Translation: nothing works out of the box, and the work to make it work is yours, unpriced, after you've signed. "Customizable" is a cost wearing the costume of a feature.
"It's enterprise-grade." Translation: it is expensive. The phrase describes the price tier and the contract, not whether the thing is any good at your actual job. Make them prove the job; the grade is marketing.
The one piece of evidence to demand
Ask for a reference customer in your industry, at roughly your size, who will take a fifteen-minute call without the vendor on the line. Not a logo on a slide — a person you can ask "what broke, and what did it actually cost you to fix." A confident vendor produces this in a day. A vendor who stalls, filters, or only offers a glowing recorded testimonial has just told you something more useful than anything in the demo.
What to do before the next pitch
Decide your messy example now, before you're in the room — one real input that represents the work you'd actually throw at this. Walk in with that and the reference-customer ask written down. You don't have to evaluate the technology. You have to make them prove it on reality instead of on rehearsal, and read the non-answers honestly when they come. That's a business skill, not a technical one — and it is the difference between the person who waved through a tool that failed in production and the person who didn't.
P.S. — If you want the full version — a complete question bank, the non-answer decoder expanded, and a weighted scorecard that turns "I have a bad feeling about this" into a one-page recommendation leadership can act on — that's The AI Vendor Evaluation Kit. Optional. The request above works on its own.
This was a free one. There's a new one every week.
What's signal in the AI noise — and the move to make about it. No hype, no vendor pitch, no link dump.
Aiden Vector is an AI-assisted publication; this content is produced by AI under human editorial direction.