How to evaluate an AI consultancy: 10 questions to ask

The best way to evaluate an AI consultancy is to ask how they ship (working software or strategy decks?), whether they’re truly vendor-neutral (any kickbacks?), how they measure ROI, how they handle your data, and what they’ve actually delivered. A strong partner answers concretely and shows measured results; a weak one talks in trends and hype.

The market is crowded and noisy right now, so a sharp set of questions is your best filter.

The 10 questions — and what a good answer sounds like

1. Do you ship working software, or strategy decks? Good: working systems in production; the deck is a means, not the deliverable.
2. Are you vendor-neutral, or do you earn commissions? Good: no kickbacks; recommendations based only on your needs.
3. How do you measure ROI? Good: a specific number agreed up front, tied to your business.
4. How long until something is in production? Good: weeks for a focused use case — not “it depends” with no range.
5. How do you handle our data and security? Good: concrete controls designed in from the start.
6. Can you show measured results from past work? Good: real, attributable outcomes — not vague logos.
7. Who actually does the work? Good: senior practitioners, not a junior team behind a senior pitch.
8. What’s your risk-reversal if it doesn’t work? Good: a guarantee or clear accountability, not “no refunds.”
9. How do you avoid vendor lock-in? Good: built so you can change models/tools later without a rewrite.
10. What do we own at the end? Good: you own the code, data, and IP — full stop.

[Add a first-hand note: the question you wish more buyers asked you, and why.]

Red flags to watch for

Hype and buzzwords with no specifics; a deck instead of a shipped example; every recommendation pointing to one vendor; no ROI logic; no clear answer on who does the work; and reluctance to put any guarantee or accountability in writing.

Big firm or boutique?

Big firms bring brand and scale but often bill six figures and months for strategy. A focused boutique can ship faster and cheaper and stay vendor-neutral — the trade-off is bench depth. Ask the 10 questions of both; the answers matter more than the logo.

How ProvenForge answers these

Briefly: we ship working software (not decks), we’re vendor-neutral (no kickbacks), we tie every engagement to an ROI number, we design security in, our Assessment is guaranteed, and you own everything we build. We’d rather show you than tell you — start with a free AI Opportunity Scan.