The most valuable AI skill may be knowing when to use something else.
Part 3 of the series: When Building Becomes Easy, What Becomes Hard?
“We should build an AI agent for this.”
It’s a sentence I hear increasingly often.
Sometimes it’s an excellent idea. AI agents can help organizations work with unstructured information, interpret complex requests, coordinate activities and automate tasks that previously required human judgment.
But sometimes the proposed AI agent is a surprisingly complicated answer to a much simpler problem.
A missing integration. An outdated process. Poorly structured information. A feature that already exists in an application nobody knows how to use.
As AI-assisted development becomes faster and more accessible, it’s tempting to jump straight from identifying a frustrating task to imagining an intelligent solution.
And that’s understandable. When you have a powerful new tool, you naturally start noticing opportunities to use it.
But there’s a difference between recognizing an opportunity for improvement and choosing the right way to deliver it.
Not every problem that can be solved with AI should be solved with AI.
When every improvement idea becomes an AI use case
In AI ambassador programmes and innovation workshops I’ve facilitated, participants often start by identifying problems in their everyday work.
They describe time-consuming reporting, information scattered across different systems, repetitive administrative tasks, difficulty finding reliable answers and manual steps between applications.
These are exactly the kinds of problems organizations should be looking for.
The challenge comes when the solution is selected too early.
If the workshop is about AI, people naturally start describing their problems as AI use cases.
“We need an AI assistant to find project information.”
“We need an agent to transfer information between systems.”
“We need AI to automate our reporting.”
Sometimes these ideas genuinely benefit from AI. But the initial description rarely tells us enough to make that decision.
For example, an employee might spend an hour every week copying customer information from one application to another.
An AI agent could potentially automate that work.
But if the source and destination systems have reliable APIs, a conventional integration might be simpler, cheaper, more predictable and easier to maintain.
Or perhaps the information shouldn’t be duplicated in the first place.
The problem isn’t necessarily a lack of intelligence in the workflow.
It may be a lack of connectivity or a poorly designed process.
Four different kinds of problems hiding behind AI ideas
When evaluating AI opportunities, I find it useful to distinguish between four overlapping types of problems.
1. An information and interpretation problem
People need to understand, summarize, classify or extract meaning from information that isn’t already structured.
Examples include interpreting customer requests, extracting relevant clauses from documents, identifying themes in feedback or summarizing meeting discussions.
This is often where AI can add significant value.
Large language models are particularly useful when the input is variable, expressed in natural language or difficult to describe through fixed rules.
But even here, we need to consider the reliability of the output, the sensitivity of the information and the consequences of errors.
AI can help interpret information. It doesn’t automatically make that information accurate.
2. A connectivity problem
Information exists in one system but is needed in another.
An employee manually transfers data between applications, updates the same information in multiple places or struggles because different systems don’t share a consistent view of a customer, project or transaction.
This is often an integration or data architecture problem.
APIs, events, data pipelines or existing integration capabilities may be better suited to solving it.
AI might help interpret unstructured inputs or decide which workflow should be triggered.
But moving known, structured information reliably between systems usually doesn’t require a language model to invent the next step.
3. A workflow and automation problem
The necessary information is available, and the steps are relatively well understood, but people still perform them manually.
Perhaps an approval is routed through email. A report is generated by copying figures into a spreadsheet. A recurring task requires someone to remember a sequence of actions.
These problems may be solved through conventional automation, workflow engines, business rules or better configuration of existing software.
AI becomes useful when part of the workflow requires interpreting variable information or handling situations that are difficult to express through deterministic rules.
The most effective solution may combine both.
4. A process, ownership or adoption problem
Sometimes the technology already exists.
People don’t use it consistently. Responsibilities are unclear. The process has unnecessary steps. Data quality is poor because nobody owns its maintenance.
Building an AI agent on top of this situation may make the symptoms less visible without addressing the underlying cause.
In some cases, the right intervention is training, clearer responsibilities, process redesign or improving an existing application.
This may not produce an exciting demo.
But it can produce significant business value.
These four categories aren’t mutually exclusive. Most real business problems involve more than one.
That’s precisely why understanding the problem matters before choosing the technology.
An example: The AI assistant that doesn’t solve the whole problem
Consider a project manager who wants to automate the work that follows meetings.
The initial idea sounds straightforward:
“Let’s build an AI assistant that summarizes meetings and creates follow-up tasks.”
A prototype could analyze meeting transcripts, identify decisions, extract action items and suggest responsible people.
This is a reasonable use of AI.
But let’s examine the actual workflow.
The project manager needs the decisions to be recorded in the appropriate project documentation. Customer-related information may need to be updated in the CRM. Tasks must be assigned to the right people and tracked until completion.
Some actions may require human confirmation before they become official commitments.
Now the solution looks different.
AI can interpret the meeting content.
Integrations can transfer approved information into the relevant systems.
Workflow automation can create tasks, notifications and reminders.
Existing applications can provide the interfaces for managing those tasks.
And people remain responsible for confirming important decisions and commitments.
The real solution isn’t necessarily an AI agent.
It’s a well-designed business workflow that uses AI where AI adds value.
AI should be part of the solution architecture, not a substitute for understanding it.
Why deterministic automation still matters
There’s a tendency to associate AI agents with more advanced or sophisticated automation.
But sophistication isn’t the same as suitability.
If a task follows clear rules, has structured inputs and requires predictable outcomes, deterministic automation is often a strong choice.
It is generally easier to test, explain and monitor.
For example, when an approved invoice reaches a certain status, the system may need to notify a responsible person or update another application.
There’s usually little value in asking a language model to decide whether that predefined action should happen.
On the other hand, if the system first needs to interpret an unstructured email or document to determine what kind of request has arrived, AI may be useful.
A practical architecture could look like this:
Unstructured input → AI interpretation → validation → deterministic workflow → business systems
The AI handles the ambiguity.
The workflow handles the predictable execution.
And where the consequences are significant, a human reviews the result before the next action.
This division of responsibilities can make solutions easier to understand, test and maintain.
It also helps avoid giving AI unnecessary authority over processes that don’t require it.
The hidden problem of information quality
Another common AI opportunity is making organizational knowledge easier to find.
Employees want to ask questions in natural language instead of searching through folders, documents and internal systems.
Retrieval-augmented generation, or RAG, can be useful here. It allows an AI application to retrieve relevant information and use it when generating answers.
But what happens if the underlying information is outdated?
Or if several documents contradict each other?
Or if nobody knows which version is authoritative?
An AI assistant may make the information easier to access without making it more trustworthy.
In some situations, it may even make the problem more difficult to notice by presenting uncertain information in a convincing form.
Before building a knowledge assistant, it’s worth asking:
Who owns the information? Which sources are authoritative? How is outdated content identified? What access restrictions apply? How will we evaluate the quality of the answers?
Sometimes the first useful experiment isn’t building a sophisticated RAG architecture.
It’s selecting a small, representative set of documents, checking their quality and testing whether they can support the intended questions.
This is another example of how AI shifts the development bottleneck.
The model may be available. The code may be relatively easy to generate.
But reliable information and clear ownership still require work.
The most useful AI experiment might not involve AI
This may sound contradictory in an article about AI innovation.
But if the goal is to learn how to improve a business process, the smallest useful experiment might be something else entirely.
Suppose a team believes an AI assistant could reduce the time needed to prepare customer reports.
Before building anything, they could test a standardized report template with a few users.
Perhaps the main problem is inconsistent input data.
Or perhaps a simple integration could eliminate most of the manual copying.
Alternatively, a small AI experiment might reveal that generating the narrative sections is where the greatest benefit lies.
Each of these findings helps the organization make a better decision.
The purpose of experimentation is not to prove that AI can be used.
It’s to discover what creates value.
A good experiment reduces uncertainty. A good solution addresses the underlying problem.
They are related, but they are not the same thing.
A lightweight way to choose the right approach
In practice, I would encourage teams to ask five questions before committing to an AI solution.
1. What is the actual job to be done?
Describe the desired business outcome without naming a technology.
Instead of “build an AI agent for reporting,” try “reduce the time needed to produce accurate monthly reports.”
2. What makes the current work difficult?
Is the problem caused by unstructured information, missing connections between systems, repetitive rules, unclear responsibilities or poor information quality?
There may be several causes.
3. What capabilities already exist?
Look at existing applications, APIs, automation tools, data products and previous experiments.
The organization may already own much of what it needs.
4. Where does AI genuinely add value?
Identify the parts that benefit from language understanding, interpretation, classification, generation or flexible reasoning.
Don’t assume the entire workflow needs to be AI-driven.
5. What is the smallest useful way to test the hypothesis?
Choose an experiment that provides evidence about the important uncertainty.
That could be a prototype, an integration test, a manual process trial or a comparison with an existing solution.
The goal isn’t to make every idea pass through an architecture committee.
It’s to help teams make better decisions before investing time and money.
AI ambassadors need problem-solving skills, not just prompting skills
This has important implications for AI literacy and organizational learning.
Many AI training programmes focus on teaching people how to use tools: writing prompts, creating assistants, generating content or automating tasks.
Those skills are useful.
But if we want AI ambassadors to help their organizations innovate, they need something more.
They need to recognize different types of problems, understand when existing capabilities should be reused, formulate useful hypotheses and identify the risks and dependencies of proposed solutions.
They should also know when to involve specialists in data, integrations, architecture, security or process development.
An AI ambassador doesn’t need to become an expert in all these disciplines.
Their role is to help connect business needs with the right expertise and the right kind of experiment.
This is where I see a particularly valuable opportunity for organizations.
Rather than simply encouraging employees to build more AI applications, we can help them become better at identifying and validating opportunities for improvement.
That capability scales beyond any individual AI tool or model.
The future isn’t AI everywhere. It’s AI where it matters.
AI-assisted development is making it easier than ever to create new applications and automate work.
That’s a powerful change.
But the availability of a technology doesn’t make it the best answer to every problem.
Sometimes the right solution is an AI agent.
Sometimes it’s an API, an integration, a workflow, a data quality improvement or a change in how people work.
Often, it’s a combination.
The real skill is knowing which parts of the problem require which capabilities — and resisting the temptation to build something new simply because we can.
As AI makes implementation easier, that judgment becomes increasingly valuable.
The future of intelligent organizations isn’t about putting AI into every process. It’s about making better decisions about where intelligence is needed.
This is Part 3 of When Building Becomes Easy, What Becomes Hard?, a series exploring how AI is changing software development, experimentation and digital innovation.
In Part 1, From Hill to Valley: How AI Is Moving the Software Development Bottleneck, we explored how AI changes the distribution of development effort.
In Part 2, Ten AI Experiments. One Business Problem., we examined why more prototypes don’t necessarily mean more innovation.
Next: The Invisible Work Between a Demo and a Service — what it really takes to turn a promising AI prototype into something an organization can safely rely on.