Start Your AI Feature From One Problem You Can Measure

The demo that never became a feature

The pattern is familiar. A team connects a language model to their data, builds a demo in a week, and everyone in the room is impressed. Three months later the demo is still a demo. Nobody can say whether it would help customers, and nobody wants to be the one to turn it off.

The model was not the problem. The project started from the technology instead of the work, so there was never a clear answer to the only question that matters: did it make something better?

Start from the work, not the model

A useful AI feature begins with a task people already do, repeatedly, and would gladly do less of. Good candidates share a few traits:

• The task happens often enough that small improvements add up.
• There is a clear input and a clear output, such as a support ticket and a reply, or an invoice and its extracted fields.
• A person can tell quickly whether the output is right.

"Use AI to improve customer support" is not a starting point. "Draft a first reply to billing questions so agents start from something instead of a blank box" is.

Define better before you build

Before any code is written, write down what success looks like in a way you can check. Not a target pulled from the air, but a definition the team agrees on:

• What gets faster, cheaper or more accurate?
• How will you measure it today, before the feature exists?
• What would make you switch the feature off?

This is the step most teams skip, and it is the reason so many pilots end in a meeting where everyone has a different opinion of how it went. If you cannot measure the current state, measure that first. It is often the most useful thing the project produces.

Ship a small version and measure it

The first release should be narrow enough to finish quickly and real enough to learn from. In practice that means:

• one workflow, for one group of users;
• the AI output shown as a suggestion that a person accepts, edits or rejects;
• every outcome logged, so you can see how often the suggestion was used as-is.

That log is your evaluation. It tells you where the feature helps, where it fails, and what to improve next, using real cases instead of a test set someone wrote in an afternoon.

We work in two-week sprints that each end with a demo, which suits this kind of feature well. Each sprint adds one improvement, and each demo answers whether the numbers moved.

Keep people in the loop where it matters

Not every AI feature should act on its own. Drafting, summarising and classifying are good places to let the model do the first pass. Sending, deleting, paying or changing records are good places to keep a person's click in the flow, at least until the logs show the feature can be trusted with more.

This is not caution for its own sake. It is how you collect the evidence that lets you automate further later.

Questions to answer before your first AI sprint

1. Which single workflow are we improving?

2. How do we measure that workflow today?

3. What result would make this worth keeping?

4. Where does a person review the output before it takes effect?

5. What will we log so we can tell whether it is working?

We build AI features, from LLM apps to automation, aimed at a measurable problem. If you have a workflow in mind and want help scoping the first step, [start a conversation](/contact).