How to Choose an AI Consultant for Custom AI Systems.

"AI consultant" is currently the least informative job title in professional services. It covers strategy shops that never ship code, agencies that resell someone else's platform under their own logo, contract developers who build precisely what you specify and nothing more, and firms that design, build, and then actually run the system in production. All four charge similar rates and use similar language. Only one of them is the right answer for building and running a custom AI system — and which one depends on what you actually need. This is how to tell them apart before you sign.

What are the four business models hiding behind one title?

1. The strategy shop. Produces a roadmap, a maturity assessment, and a slide deck. Genuinely useful if your problem is executive alignment. Useless if your problem is that nothing is in production. The tell: they cannot name the systems their last three clients now run.
2. The reseller agency. Configures a third-party platform and bills it as a custom build. Fine for commodity workflows, and often the cheapest correct answer — but you own nothing, and your run cost is whatever the platform decides it is next year. The tell: the "custom" system cannot be exported or migrated.
3. The contract dev shop. Builds exactly the spec you hand them. Excellent when you have a competent internal technical owner writing that spec. Dangerous when you do not, because they will faithfully build the wrong system.
4. The design-build-run firm. Owns the design decisions, ships the system, and stays accountable for its behavior in production. Most expensive up front, and the only model where the consultant carries the consequences of a bad design decision.

None of these is fraudulent. The failure mode is buying one and believing you bought another. For the closely related question of firm structure, see AI Development Firm vs. AI Agency.

What questions expose which model you are talking to?

Ask these in the first conversation. They are not gotchas — a good partner answers all of them without hesitation, and the hesitation itself is the signal.

• "Show me a Decision Log from a live system." Redacted is fine. A firm that runs systems in production has this. A firm that ships demos does not know what you are asking for.
• "Who reviews the system after launch, and how often?" A named role and a stated cadence, or the answer is nobody.
• "What happens if the model vendor deprecates the version you built on?" A designed system answers "configuration change plus regression run." An assembled one answers with a rebuild quote.
• "What do I own at the end?" Code, prompts, evaluation sets, logs, and infrastructure definitions — enumerated, in writing. "You own your data" is a non-answer; everyone says that.
• "What is the monthly run cost, and which parts of it are yours versus passed through?" Model inference, telephony, hosting, and their margin should be separable line items.
• "Which project of yours failed, and why?"Anyone who has shipped enough AI systems has one. A firm with no failures has no production history, or is not being candid.

How should you evaluate technical credibility without being technical?

You do not need to assess model architecture. You need to assess whether the firm makes decisions deliberately. Three proxies work reliably:

• Do they push back on your brief? A firm that agrees with everything in the first call is selling, not designing. The most valuable early meetings we have are the ones where we tell a business that the system they asked for is the third thing they should build, not the first.
• Do they distinguish between what the system decides and what it recommends? This distinction is the core of responsible AI design, and firms that have not thought about it will not have language for it. The govern and manage functions of the NIST AI Risk Management Framework exist precisely because that boundary is where AI systems cause harm.
• Do they talk about measurement before capability? Research on where organizations capture value from AI — including ongoing work published by MIT Sloan Management Review — consistently points at organizational and process factors rather than model selection as the differentiator. A firm that leads with which model it uses is answering the least important question.

What should the commercial terms actually say?

Most disputes we hear about are not about quality. They are about terms nobody read carefully. Five clauses are worth negotiating properly:

1. Ownership and escrow. Enumerate the artifacts. If the firm hosts the system, specify what you receive if the relationship ends — ideally a working export, not a zip file of source you cannot deploy.
2. Run-cost transparency. Pass-through costs (inference, telephony, hosting) itemized separately from the management fee, with a stated policy for what happens when usage exceeds the assumed volume.
3. Change control. Who can change the system's behavior, and does that change get logged? An AI system whose instructions can be edited silently by anyone with dashboard access is unauditable regardless of what the contract says.
4. Performance definition. The primary metric, its measurement method, and the review cadence — in the statement of work, not in an email.
5. Exit. Notice period, transition support, and the state the system is left in. A partner confident in the work makes this easy.

When is a consultant the wrong answer entirely?

Sometimes the honest recommendation is not to hire one. In our engagements, the cases where outside help is the wrong call cluster tightly:

• The workflow is genuinely commodity. If an off-the-shelf product does 90% of what you need, buy the product. Custom is for the part of your business that is actually different.
• The data is not ready. If the system of record is contested — two CRMs, three price lists — no AI system will resolve that, and building one on top of the ambiguity just makes the ambiguity faster.
• Nobody internally will own it. Without a named human in command, a custom system degrades quietly. Hire that person first.
• You already have the team. If you have engineers with production AI experience and the capacity to use them, the trade-off is different — we lay it out in Strategic AI Consultants vs. In-House AI Team.

The short version

Decide first whether you need a strategy, a build, or an owner. Then buy that specific thing from a firm that can prove it has done it in production — a Decision Log from a live system, a named reviewer, an itemized run cost, and a candid failure story. The firms that clear that bar are a small subset of the ones that will pitch you, and the filtering takes one conversation.

For the architecture side of this decision, see Bespoke AI System Design: The NURO Approach. For the failure patterns that make this choice consequential, see Why AI Projects Fail at $5M–$50M Businesses.