Author: Matt Daum

I’ve been seeing a lot of people and projects trying to use AI for everything, even if it’s not really needed, even for “Generate a random password”. Sure it can do it, I guess, but is it really the right use case for it? Sometimes people are going overboard in their use of AI just so they can say “it’s AI enabled!”.

Drawing the line between a good use of AI and when another tool, simple function, or a different approach is better can be difficult since it has such broad capabilities. I’ve started to try to come up with some general rules to follow. When you need something that is very predictable, i.e. business rules, validation, record keeping, etc., generally AI isn’t going to be the best fit. On the other hand, AI is great for when you need to deal with messy inputs from an email, PDF, image, handwriting, slightly varying formats, etc.

That is not a limitation of AI. It is what makes a good implementation work. The goal is not to turn every step into an AI decision. It is to give AI the part of the problem where ambiguity is real and use other tools where the organization needs a repeatable answer.

A couple of recent projects we’ve had can highlight this distinction.

Check processing: use AI to read the check, not run the process

Consider a check-processing application that accepts uploaded images and prepares records for an accounting or fundraising platform.

Checks are a good use case for AI. They come in different formats, handwriting can be hard to read, and important information may be in different places depending on the check. AI can identify a likely payer or donor, pull out the amount and check number, read a memo line, and return structured candidate data.

That removes a lot of manual entry. It turns a document into a useful starting point.

But recognizing the document is not the same thing as deciding what should happen next. The application still needs deterministic software to validate required fields and expected formats, check whether the check was already processed, match the payer against known records, apply account or fund mappings, and determine whether the item needs review.

Only after those checks pass should the workflow submit a record to the downstream platform. It should retain the original image, the extracted values, any corrections, and a record of what was submitted.

In this case, AI is the right tool for interpretation. A rules-based workflow is the right tool for deciding whether the interpretation is ready to become a trusted record.

Purchase-order intake: use AI to handle variation, not enforce the contract

Purchase orders have the same shape. They often arrive by email or as PDFs, and every customer has their own template. Line items, pricing, shipping details, purchase-order numbers, and customer information may all be presented differently.

AI can turn those inconsistent documents into a draft order. It can identify the likely customer, extract line items, quantities, and prices, and point out fields where it is uncertain.

That is exactly the type of variation AI is useful for. It saves someone from manually typing information from a PDF into an ERP or order-management system.

But the AI should not be responsible for enforcing the rules around the order. Deterministic software should validate that the customer exists, the SKUs are real, quantities are valid, and the price matches the customer’s contract or approved price list. It can verify that the purchase-order number has not already been used, make sure required shipping and billing information is present, and route exceptions to the right person.

Once those rules pass, the application can create the order in the ERP. If they do not pass, the system should say why it stopped and what needs to happen next.

AI helps make the document usable. The rest of the application makes the order trustworthy.

Choosing the boundary is part of the implementation

The mistake isn’t using AI, it is treating AI as the whole application because it can produce an answer that looks structured.

A good implementation gives each tool a job it is suited for:

• AI interprets ambiguity. It reads documents, extracts likely data, classifies inputs, and identifies uncertainty.
• Deterministic software verifies facts. It checks business rules, required fields, duplicates, permissions, mappings, and known reference data.
• Workflow software coordinates action. It routes exceptions, requests approvals, submits to the correct downstream system, and keeps an audit trail.

That division makes the system easier to operate too. AI output can change when a model, prompt, or document format changes. Business rules may change, but they should be explicit, testable, and owned by the people responsible for the process.

When an extracted amount is wrong, someone should be able to tell whether the problem came from the document, the AI output, a validation rule, or a mapping. When a downstream system receives a record, someone should be able to explain why it was submitted.

Those details are not exciting demo features, but they are what make a workflow usable in a real organization.

Start with the problem, not the model

When evaluating an AI workflow, start by asking where the uncertainty is.

If the problem is understanding an email, PDF, image, or conversation, AI may be the right tool. If the problem is enforcing a policy, validating a known fact, selecting a system of record, or creating an auditable action, conventional software is often the better tool.

The best AI implementations are not built around the idea that the model should do everything. They are built around the idea that a workflow can use many tools well. AI handles the ambiguous part. The rest of the application provides the boundaries that make its output useful.

Long story short, choose the right tool for the part of the problem you are trying to solve versus treating everything like a nail that you hammer on with AI.

A big thing happened the other week: Anthropic helped launch a $1.5 billion AI services company. OpenAI is also deploying enterprise agents with its own team and selected systems integrators.

$1.5 billion is not chump change. These companies are thinking about where the market is going, and they have realized that a capable model is not the only thing that matters. The implementation around the model is equally important.

Ode with Anthropic was announced in July as a standalone AI services firm. A week later, OpenAI introduced Presence, an enterprise deployment model that starts with a specific workflow and includes system access, policies, approvals, testing, and ongoing support.

While massive companies may be able to work directly with OpenAI or Anthropic, or at least afford to, most of us do not have that luxury. What is clear is that getting an impressive demo or proof of concept is not the hard part. The hard part is taking it to the next step, where the complex edge cases, data, users, and exceptions show up.

Small choices can turn into bigger problems

We were brought into an AI project where a fairly simple decision became a bigger problem later: the application was not locked to a specific model and release version.

At first, the AI application looked fine. It produced structured output and the rest of the automation could use it. As the provider changed the model’s behavior over time, the format changed and it began classifying some inputs differently than it had when the application was originally built. The output still looked reasonable to a person, but the rest of the automation relied on those expected formats and classifications, so it became unreliable.

This is easy to miss because the AI still appears to work. The problem only shows up when another system expects the output to be consistent, and there is no test in place to catch the change before it becomes an issue.

A model should be treated like any other production dependency. Pin the model and version where possible, keep a set of real test cases, test changes before they go live, and have a plan for upgrades. It is not exciting work, but it keeps a useful application from becoming unreliable over time.

A demo is not a workflow

We have seen the same thing with agents. An agent can look great in a demo because it can answer questions, summarize documents, or draft an action. Once it is used in a real business process, though, it needs to know where the right data lives, what to do when data is missing, when it should ask a person for help, who can approve an action, and what needs to be recorded later.

If that work is not planned ahead of time, the agent usually does one of two things. It takes actions it should not take, or it hands so much back to people that it does not save much time. In both cases, the model may be fine. The larger workflow was not thought through.

Building an AI agent is not the same as building a working process. The agent needs clear boundaries, the right access, a way to handle exceptions, and someone responsible for it when the process changes.

What good implementation looks like

A good implementation partner does more than connect a model to an API and call it innovation. They should help a company decide which workflow is worth changing, where the right data is, what the system is allowed to do, when people need to step in, and how to tell if the work is actually improving the process.

They should also be asking what happens when the model changes, the company changes a policy, or the data and workflow change. Those are normal parts of operating a business, and the AI application needs to keep up with them.

Sometimes an LLM is the right tool. Other times, ordinary software, better data integration, or a rules-based process makes more sense. Someone who has worked on these projects should know the difference.

AI makes it much faster to go from an idea to something that works, and that is valuable. But the problems with a weak workflow, bad permissions, unreliable output, or missing monitoring often do not show up until after the demo works. That is why implementation experience matters.

If an AI initiative is stuck between an impressive demo and something useful in production, do not assume it needs a bigger model or another tool. It may need a clearer workflow and people who know where these systems tend to fail.

A lot of small applications are getting built right now. Someone has a frustrating, repetitive problem, perhaps they need to clean up a report, move information from one system to another, or give customers a simpler way to complete a task. They use AI, a low-code tool, or just write a little code, and in a surprisingly short amount of time they have something useful. Then people start using it every day.

That is a huge win! The application proved that there is a real problem worth solving, which is often the hardest thing to figure out before building a larger project. The trouble is that once people depend on it, it has quietly stopped being just a quick experiment, even if it is still deployed like one.

Maybe it started as an internal tool that only a few people used, but now it is how a team gets a report out every week. Maybe it is a customer-facing feature that got enough use to become part of the product. Either way, when it stops working, someone notices pretty quickly, and the business usually has to work around it until it is fixed.

The app works until it doesn’t

This is a familiar situation. One person builds something useful and they know all the details: where it runs, how to deploy it, which API key it needs, and what to do when it gets weird input. Nobody else really has to think about it because the app has worked so far, and there are always other things to work on.

Then a dependency updates, an integration changes, or a user does something the original builder did not expect. The app might start throwing an error, or it might do something worse and quietly give the wrong answer. The person who built it could be on vacation, busy with a different project, or no longer at the company. Now a tool that people rely on has no clear owner and no obvious way for someone else to figure out what went wrong.

The original builder did not do anything wrong. Getting a useful version into people’s hands quickly is usually the right move, especially when nobody knows yet whether the idea will be valuable. However, proving that an idea works and operating an application that other people depend on are two different things, and they require a different level of support.

The small things are usually the big things

Most teams do not decide that error handling, monitoring, or security reviews are unimportant. They simply have not needed them yet, or they have been focused on getting the useful part of the application in front of people. When five people use a tool, it is easy to hear about a problem and fix it directly. When fifty people use it, or when customers use it outside of business hours, those shortcuts start to become real problems.

For example, what happens when an integration fails halfway through a process? If the app has already updated one system but has not updated the next system, can someone tell what happened and safely retry it, or do they have to start manually checking records to figure out what needs to be fixed? There may not be one right answer, but it is worth thinking through before the first time it fails in a way that affects the business.

Some other common things we see are:

• An error happens, but the user gets an unhelpful message and there is no log that explains what actually failed.
• Nobody knows whether the application is healthy until someone says it is broken in Slack.
• Deploying a change means following a series of manual steps that live in one person’s head.
• API keys and other credentials are stored wherever was convenient when the application was first built.
• The application has its own username and password, even though everyone already signs in through the company’s identity provider.
• It handles company or customer information, but nobody has taken the time to review who can see it, where the information goes, or how long it is kept.
• No one besides the original builder knows how the pieces fit together or how to get the application back up after a problem.

None of these things are especially exciting or sexy to build, but they are the things that make a useful application either dependable or stressful to rely on. They are also usually much easier to address while the person who built the app is still around and the application is otherwise working well.

It does not mean starting over

The answer is not to take a working application away from the people using it and disappear for six months to rewrite it. Usually the application already has the most important thing: a real use case, people who can explain what matters, and an understanding of where it actually helps the business. Starting from that point is much better than starting from a blank page and guessing.

A better approach is to look at what is there, understand how it is being used, fix the biggest risks first, and add the support the application needs as it grows. The questions are fairly straightforward, but the answers tend to point to the work that matters most:

• Where does it run, and how does a change get from code to users?
• What happens when one of its services or integrations fails?
• Can the team see errors, usage, and the health of the application without guessing?
• Who has access to the application, its data, and its credentials?
• Can it use the company’s existing login and access controls rather than adding another account for everyone to manage?
• Could someone besides the original builder understand it, make a change, and respond to an issue?

Giving a useful app the support it needs

A production application does not need to be complicated, and not every small tool needs the same amount of process as a large customer-facing system. It does need a few basics that match the importance of the job it is doing.

A repeatable deployment process means changes can be reviewed, tested, and released in a predictable way instead of requiring a set of commands someone hopes they remember correctly. It also gives the team a reasonable way to roll back a bad change when one inevitably gets through.

Good error handling and visibility means useful errors for users, logs that explain what happened, and enough metrics, monitoring, and alerting to notice a problem before it turns into a long Slack thread. This is not about collecting every possible number; it is about being able to answer the basic question of whether the application is working and why it is not when it fails.

Security should fit how the company already works. That includes being deliberate about secrets, permissions, and data flows, as well as reviewing who should have access to the application. For an internal tool, using the company’s existing identity provider often avoids one more username and password for employees to manage, and makes it easier to remove access when somebody changes roles or leaves.

Finally, the application needs some documentation and shared ownership. A short overview of how the pieces fit together, a runbook for common problems, and more than one person who can work on it go a long way. The person who first built the application should not have to be the permanent support team just because they had the initiative to solve a problem.

If AI is part of the application, there are a few additional questions around testing prompt or model changes, handling unusual input, and deciding when a person should review an output. Those are important questions, but they sit alongside the same operational basics that every useful application needs.

Make the app more dependable while it is going well

The best time to make an application more supportable is when it is working and people are getting value from it. There is less pressure, the original builder can explain why the application works the way it does, and the team can make improvements without disrupting the workflow it supports.

This is the type of work Setfive can help with: understanding the application that exists today, stabilizing the areas most likely to cause trouble, reviewing security and access, putting appropriate infrastructure and monitoring in place, and making sure the business is not dependent on one person. The goal is not to turn every small application into a huge project. It is to make sure the useful thing stays useful as more people start relying on it.

AI code generation is one of the most useful things to happen to software development in a long time. We use it. It gets people from an idea to a working application much faster than they could have a few years ago.

That is a big deal. It is also not the same thing as getting from an idea to a well-operated application.

This week, one of our clients’ Snowflake cost alerts went off. A new application had spent more than $4,000 in a couple of days. The application had been built with Claude and it was doing what its owner intended it to do. The problem was how it was doing it: repeatedly polling Snowflake with a larger warehouse than the work required.

The code worked. The bill did too.

The alert was the important part

The client had anomaly alerts in place long before this application existed. Over years of normal use, those alerts had established a useful picture of what ordinary compute usage looked like. When the new application’s usage departed from that pattern, it stood out quickly.

That monitoring was not glamorous, and it was not new. It was operational knowledge turned into a guardrail. Without it, the polling loop could have continued until someone happened to notice an unusually large bill.

This is worth emphasizing because it is easy to see AI as the whole story. The model helped create the application. The monitoring, the historical baseline, and the people who responded to the alert are what limited the damage.

Working is not the same as economical

AI is good at getting to a plausible solution. It can write the query, connect the service, add a loop, and return the result. But it does not naturally care whether a process runs every minute instead of every hour, whether data can be cached, whether an existing system already solves part of the problem, or whether a warehouse is sized appropriately for the query.

Sometimes the generated solution will rebuild something that already exists. Sometimes it will choose a direct approach that is perfectly functional but wasteful at production scale. A polling loop is a simple example: it may make a feature feel responsive while quietly paying for repeated work that is unnecessary.

None of that makes the application useless, or AI a bad tool. A few years ago, a person without deep technical experience might not have been able to build and deploy this application at all. Now they can. That is real leverage.

But the leverage changes where the risk sits. Development time may go down while cloud spend, maintenance, security exposure, or reliability risk goes up. Those costs often arrive after the demo is working and the application is in use.

A short review can be a very good investment

The answer is not to ban AI-generated code or require every idea to go through a long development process. The answer is to put experienced eyes on the parts that determine how software behaves in the real world.

For a data-backed application, that review can be straightforward:

• What runs on a schedule, and how often does it actually need to run?
• Which queries execute, on what warehouse, and how much data do they scan?
• Can the application cache results, react to an event, or reuse an existing data set instead of polling?
• What is the expected cost at normal usage and at a failure mode?
• Which alerts will tell us when the application behaves differently from expected?

A review like this does not need to take longer than the work it is reviewing. In this case, it could have prevented a four-figure surprise. More importantly, it creates a habit of treating an AI-generated application as software that will be operated, not just code that needs to run once.

Keep the human in the loop

There is a familiar parallel with outsourcing. Lower-cost implementation can be a good trade when the work is understood and the output is reviewed. It becomes expensive when the apparent savings mean nobody owns the architecture, the quality, or the ongoing consequences.

AI assistance is similar. It can make capable builders out of more people, and that is something to embrace. But it can also produce slop, inefficiencies, security risks, and bugs that are easy to miss because the first version looks complete.

The goal should not be to slow people down. It should be to pair the speed of AI with monitoring that catches surprises and with people who understand the systems, costs, and tradeoffs behind the code.

This is the first in a series of examples from the gap between shipping software quickly and operating it well. AI can help you build faster. Make sure someone is also asking what the resulting system will cost to run.

Have you ever tried pitching an upgrade to management? Odds are, you probably didn’t find yourself walking away with a blank check. Maybe you’re a network administrator for a real estate company whose boss doesn’t understand why the cheaper network infrastructure isn’t always the best option for scalability; or maybe you need to request an upgrade for an application that takes up a significant amount of your time every day to troubleshoot because it’s incompatible with other operating systems. The conversation goes something like:

You: “Boss, we really need to upgrade [xyz] software package.”

Them: “Why do we need the upgrade? If it ain’t broke, don’t fix it.”

Your: “Well, it’s creating a number of issues for our team. The manufacturer no longer supports the version we use, because it’s been obsolete for 10 years. Whenever an issue comes up we have to come up with a workaround.”

Them: “How much is the upgrade?”

You: “It’ll be $ X for a shared license for all team members.”

Them: “I just don’t think we don’t have the money in the budget for that kind of upgrade. We have a lot more pressing projects requiring capital right now, and I can’t see us justifying that expense to our board.”

Such a request may not be well received because of difference in perception of the situation –of the cost-reward assessment of the solution. The management team may not speak the same language, so to speak, as the technical support or engineers, so it’s crucial to put the request into terms they will understand and listen to. Better yet, frame that request as an offer.

Here’s three ways here that you can sell to that point, in language even your boss can understand.

1. Security

Technology has never had an obsolescence rate as fast as it is today. Failure to keep up to date is not just a matter of having the best and newest techy toys, though; it can lead to a security breach of personal information (like the Target PIN data breach of 2013), stolen identities and stolen money.

Cyber security expenses are perhaps the hardest sell to make, considering failure to upgrade presents a latent risk rather than an active one. It works until it doesn’t. Earlier in 2020, the stock market witnessed a reactionary boost in security spending in companies like FireEye, after celebrities like Elon Musk’s Twitter accounts were hacked.

As an engineer pitching an upgrade to management, convey the risk of not getting it and the potential fallout.

2. Developer productivity

A software version upgrade can be money in the bank if it saves man hours by making tasks less labor intensive and more efficient. Use terms like “faster,” “leaner,” or the military favorite “force multiplier.” If you’ve got estimates on time allotted to a given project that can be broken down into hourly direct labor savings, that’s always a great selling point.

3. Hiring and Retention

The success of any project depends not just on the tools, but on the people using those tools; to a large degree, the more cutting edge your tools are, the more cutting edge the people in your employ will be. When it comes to hiring and retaining employees, one deciding factor will surely be how modern your operating environment is.

If your team uses obsolete tools, it may even be more difficult to find someone with that skill. If you use Python 2 instead of Python 3, the syntax and many features are quite different, but all modern users are taught the most recent version, so it will present a small challenge to hire someone who’s willing to use (or learn) an obsolete version of that language.

Whatever the case, it’s safe to say meeting the bottom line is among any company’s top priorities, when it comes to spending. The more you can appeal to that end and sell a tangible ROI for the cost of the upgrade, the more likely you are to hear a ‘yes.’

Another tip is to present multiple options: a good, better, and best option with #1 being the most expensive. People are often more likely to choose to do something when it is presented as one of multiple options than alone.