How to Read an AI Consulting Case Study
Read an AI consulting case study as a list of claims to check, not as proof. For each claim, ask what was measured, against what starting point, over what period and who did which part of the work. If the case study can't answer, ask the firm directly or ask to speak to the client.
Case studies are marketing, and that is fine. They show what a firm wants to be known for. The useful skill is turning each polished line into a question that gets you facts. Below are common case study phrases, followed by what to ask instead.
Claim: "Reduced processing time dramatically"
Ask: Processing time for which task, measured how? What was the time before, and how many weeks after go-live was it measured? Did the number hold after the first few months?
A result measured in the first week, while the firm was still on site, tells you less than one measured after handover.
Claim: "Deployed an AI assistant across the organization"
Ask: How many people use it each week now, and how many were meant to? Which teams stopped using it, and why?
"Deployed" means installed. It doesn't mean used.
Claim: "Built a custom model"
Ask: Was a model trained or fine-tuned, or was an existing model connected to the client's documents with prompts and retrieval? Who maintains it now?
Neither approach is better by default. You just want to know which one you would be buying, because maintenance and running costs differ.
Claim: "Delivered in eight weeks"
Ask: Eight weeks from what to what? Does that include discovery, integration, testing and training, or only the build?
Timelines shrink when the start and end points move.
Claim: "Integrated with the client's existing systems"
Ask: Which systems, and was the connection live in production or a one-off data export? Who handled the client-side IT work?
Integration can take a large part of the effort. A case study that lists systems by name is more credible than one that says "existing systems."
Claim: "Trusted by a global brand"
Ask: What was the scope of the work for that client? Was this firm the main supplier or one of several? Is the team that did that work still at the firm?
A well-known logo can come from a small piece of a large program.
Claim: A Quote From the Client
Ask: Can we speak to the person quoted, or someone in the same role? What would they do differently next time?
If a firm can't arrange any conversation with the client behind a case study, weigh the case study lightly.
Three Things That Make a Case Study Credible
- A before-and-after measure with a period. Not just a percentage, but what it was a percentage of and when it was measured.
- What went wrong. Real projects hit problems. A case study that mentions one and how it was handled reads as honest.
- What the client does now. Who runs the system today, and whether the firm is still involved.
Using This in Practice
Pick the two case studies closest to your own project from each firm on your shortlist. Write the questions above next to each claim, and use them on your reference calls. You will learn quickly which firms are comfortable with specifics.
If you are still building that shortlist, a buyer's guide such as https://bestaiconsultingfirms.com/ can give you firms to start with before you dig into their case studies.