The AI Vendor Evaluation Scorecard
Score your current or shortlisted AI vendor against 7 weighted criteria before the next procurement, renewal, or leadership pitch — so you walk in with a defensible number, not an opinion.
The Edge · Paid Template · Ju
ne 2026
You are being asked to evaluate AI vendors — or you already have one and someone is about to ask you to justify it.
Either way, the conversation you're walking into is not a technology conversation. It is a business decision that will be reviewed by people who do not want to understand the technology. They want a number, a recommendation, and confidence that you thought it through.
This scorecard gives you all three. It is the paid version — seven weighted criteria, a scoring scale, a worked example, and the one question that changes the outcome of every vendor meeting. A shorter 4-question version circulates freely. This is the version you use when the stakes are real.
Fill it in. Score it. Walk in with the total.
How to Use This Scorecard
Time required: 45–90 minutes per vendor, depending on how much information they've given you.
When to use it:
Before recommending a vendor to leadership
During a contract renewal when you're deciding whether to stay or switch
When evaluating two or more competing tools for the same use case
When you're inheriting a vendor relationship and need to quickly assess what you have
Scoring scale: 1–5 per criterion (1 = fails / no answer, 3 = adequate, 5 = clear strength)
Weight: Each criterion carries a weight. Your weighted score = raw score × weight. Maximum possible total = 100.
Recommendation thresholds:
85–100 — Proceed / renew with confidence
70–84 — Proceed with named conditions (list them)
55–69 — Request 30-day structured pilot before committing
Below 55 — Do not proceed — escalate or re-evaluate alternatives
The Scorecard
Vendor name: [ vendor name ]
Use case being evaluated: [ use case being evaluated ]
Evaluated by / date: [ evaluated by / date ]
Criterion 1 — Business Fit (Weight: 20)
Does this tool actually solve the specific workflow problem you have, at the scale you operate?
5 — Directly addresses the named workflow; references work comparable to your org size and function
4 — Strong fit with minor adjustments required
3 — Adequate fit; some workflow gaps that require workarounds
2 — Partial fit; significant adaptation needed or core use case is adjacent, not direct
1 — No clear fit; vendor is pitching broadly without addressing your specific workflow
Questions to ask:
"Show me a demo built around [your specific workflow] — not a generic feature tour."
"What's your typical customer's team size and function?"
Your score (1–5): [ score ] × 20 = [ weighted ]
Notes: [ notes ]
Criterion 2 — Total Cost Transparency (Weight: 20)
Do you know exactly what you will pay at current usage AND at 2× and 5× scale?
The hidden cost structure is the most common vendor trap for non-technical buyers. AI vendors frequently charge by the token (a token is roughly ¾ of a word — so a 1,000-word document is approximately 1,300 tokens). At small scale this is trivial. At org scale, it can be a surprise budget line.
5 — Per-seat or flat-fee pricing with clear overage terms; vendor provides usage estimates for your volume and a written cost projection for 2× and 5× scale
4 — Clear pricing model; 2× cost projection available on request
3 — Pricing is documented but requires calculation; vendor willing to provide estimates
2 — Usage-based pricing with limited visibility; must request cost modeling
1 — Pricing is unclear, contact-sales-only, or vendor deflects cost questions
Questions to ask:
