- Choosing Your First Step
AI Assessment vs. AI Pilot: Which Do You Need?
Some businesses should study before they build. Others should build one small thing and learn from it. Choosing wrong wastes either months or money, here is how we decide.
The Real Difference
One Answers "Where"; the Other Answers "Whether"
An assessment is a structured review across your business that produces a ranked roadmap. It answers the question where should AI go first, and what has to be true before it can. It is the right first step when you have several candidate problems and no confident way to rank them.
A pilot is a single narrow build against one process, run for a defined period with a defined measure. It answers the question does this actually work here. It is the right first step when you already know exactly which process hurts, the data is reachable, and the argument in the room is about feasibility rather than priority.
The failure mode in each direction is predictable. Businesses that pilot without assessing build something impressive against the wrong process. Businesses that assess when they should have piloted spend two months producing a document to answer a question one afternoon of building would have settled.
This distinction comes up on almost every first call we take from a Dallas business owner. The pattern is consistent enough that we can usually tell within twenty minutes which of the two a company needs, and we say so plainly from our Plano office rather than selling the larger engagement.
Side by Side
How the Two Compare
AI Assessment
ANSWERS
Where AI belongs in this business, ranked, and what has to be fixed first.
OUTPUT
Six written deliverables including a phased roadmap you own outright.
RISK IT REMOVES
Building the right thing in the wrong place, or on a foundation that cannot hold it.
CHOOSE IT WHEN
You have multiple candidate problems, several systems, or compliance exposure.
AI Pilot
ANSWERS
Whether this specific approach works on this specific process, here.
OUTPUT
A working narrow build plus a measured result against a number set in advance.
RISK IT REMOVES
Committing budget and staff time to something that was never going to hold up in practice.
CHOOSE IT WHEN
You know the process, the data is reachable, and the disagreement is about feasibility.
The most common sequence
Assessment, then a pilot against the top-ranked item on the roadmap, then a wider rollout if the pilot’s number holds. Most businesses we work with do not need to choose between them. They need to do them in the right order.
Rules of Thumb
Four Signals That Settle It
You cannot name the process → assess
If the answer to “which workflow” is a category rather than a specific sequence of steps, there is nothing to pilot yet.
Leadership disagrees on priority → assess
A pilot built to settle a political argument gets judged politically. A ranked roadmap moves the conversation onto evidence.
One process, obvious pain → pilot
Everyone in the building names the same bottleneck without prompting, and the data lives in a system we can reach. Build the small version.
Regulated data is involved → assess
Patient records, client files, or donor data mean the compliance question has to be answered before anything touches production. See HIPAA & AI.
How long should a pilot run?
Long enough to cover a full cycle of the work, for most back-office processes that is four to six weeks. Shorter and you are measuring novelty; longer and it stops being a pilot and quietly becomes production without anyone deciding it should.
What makes a pilot fail even when the technology works?
No measure agreed in advance. If nobody wrote down what success looks like before starting, the readout becomes a debate about impressions, and impressions of new tools are always mixed. Set the number first.
Can a pilot run on data we are not allowed to move?
Often yes, using tooling that stays inside your existing tenant and de-identified samples for testing. But that is a design decision made before the pilot starts, not a problem discovered halfway through, which is exactly why regulated businesses assess first.
We already ran a pilot and it fizzled. Now what?
Diagnose why before running another. In our experience it is almost alwaysone of four causes: no named owner, training that was never tied to real work, an undocumented process, or data the tool could not reach. Each has a different fix.
Is a vendor demo a pilot?
No. A demo runs on the vendor’s data, in the vendor’s configuration, with the vendor driving. A pilot runs on your data, in your environment, with your staff doing the work. The gap between those two is where most disappointment lives.
BK
Written and reviewed by Bob Kehr, President & Founder of Kehr Technologies, Plano City Council member, and Plano Chamber Small Business Person of the Year 2022. He writes the scope documents these figures come from, and has quoted assessments for practices, firms, and non-profits across the Dallas area.