Category: General

I’ve been seeing a lot of people and projects trying to use AI for everything, even if it’s not really needed, even for “Generate a random password”. Sure it can do it, I guess, but is it really the right use case for it? Sometimes people are going overboard in their use of AI just so they can say “it’s AI enabled!”.

Drawing the line between a good use of AI and when another tool, simple function, or a different approach is better can be difficult since it has such broad capabilities. I’ve started to try to come up with some general rules to follow. When you need something that is very predictable, i.e. business rules, validation, record keeping, etc., generally AI isn’t going to be the best fit. On the other hand, AI is great for when you need to deal with messy inputs from an email, PDF, image, handwriting, slightly varying formats, etc.

That is not a limitation of AI. It is what makes a good implementation work. The goal is not to turn every step into an AI decision. It is to give AI the part of the problem where ambiguity is real and use other tools where the organization needs a repeatable answer.

A couple of recent projects we’ve had can highlight this distinction.

Check processing: use AI to read the check, not run the process

Consider a check-processing application that accepts uploaded images and prepares records for an accounting or fundraising platform.

Checks are a good use case for AI. They come in different formats, handwriting can be hard to read, and important information may be in different places depending on the check. AI can identify a likely payer or donor, pull out the amount and check number, read a memo line, and return structured candidate data.

That removes a lot of manual entry. It turns a document into a useful starting point.

But recognizing the document is not the same thing as deciding what should happen next. The application still needs deterministic software to validate required fields and expected formats, check whether the check was already processed, match the payer against known records, apply account or fund mappings, and determine whether the item needs review.

Only after those checks pass should the workflow submit a record to the downstream platform. It should retain the original image, the extracted values, any corrections, and a record of what was submitted.

In this case, AI is the right tool for interpretation. A rules-based workflow is the right tool for deciding whether the interpretation is ready to become a trusted record.

Purchase-order intake: use AI to handle variation, not enforce the contract

Purchase orders have the same shape. They often arrive by email or as PDFs, and every customer has their own template. Line items, pricing, shipping details, purchase-order numbers, and customer information may all be presented differently.

AI can turn those inconsistent documents into a draft order. It can identify the likely customer, extract line items, quantities, and prices, and point out fields where it is uncertain.

That is exactly the type of variation AI is useful for. It saves someone from manually typing information from a PDF into an ERP or order-management system.

But the AI should not be responsible for enforcing the rules around the order. Deterministic software should validate that the customer exists, the SKUs are real, quantities are valid, and the price matches the customer’s contract or approved price list. It can verify that the purchase-order number has not already been used, make sure required shipping and billing information is present, and route exceptions to the right person.

Once those rules pass, the application can create the order in the ERP. If they do not pass, the system should say why it stopped and what needs to happen next.

AI helps make the document usable. The rest of the application makes the order trustworthy.

Choosing the boundary is part of the implementation

The mistake isn’t using AI, it is treating AI as the whole application because it can produce an answer that looks structured.

A good implementation gives each tool a job it is suited for:

• AI interprets ambiguity. It reads documents, extracts likely data, classifies inputs, and identifies uncertainty.
• Deterministic software verifies facts. It checks business rules, required fields, duplicates, permissions, mappings, and known reference data.
• Workflow software coordinates action. It routes exceptions, requests approvals, submits to the correct downstream system, and keeps an audit trail.

That division makes the system easier to operate too. AI output can change when a model, prompt, or document format changes. Business rules may change, but they should be explicit, testable, and owned by the people responsible for the process.

When an extracted amount is wrong, someone should be able to tell whether the problem came from the document, the AI output, a validation rule, or a mapping. When a downstream system receives a record, someone should be able to explain why it was submitted.

Those details are not exciting demo features, but they are what make a workflow usable in a real organization.

Start with the problem, not the model

When evaluating an AI workflow, start by asking where the uncertainty is.

If the problem is understanding an email, PDF, image, or conversation, AI may be the right tool. If the problem is enforcing a policy, validating a known fact, selecting a system of record, or creating an auditable action, conventional software is often the better tool.

The best AI implementations are not built around the idea that the model should do everything. They are built around the idea that a workflow can use many tools well. AI handles the ambiguous part. The rest of the application provides the boundaries that make its output useful.

Long story short, choose the right tool for the part of the problem you are trying to solve versus treating everything like a nail that you hammer on with AI.

A lot of small applications are getting built right now. Someone has a frustrating, repetitive problem, perhaps they need to clean up a report, move information from one system to another, or give customers a simpler way to complete a task. They use AI, a low-code tool, or just write a little code, and in a surprisingly short amount of time they have something useful. Then people start using it every day.

That is a huge win! The application proved that there is a real problem worth solving, which is often the hardest thing to figure out before building a larger project. The trouble is that once people depend on it, it has quietly stopped being just a quick experiment, even if it is still deployed like one.

Maybe it started as an internal tool that only a few people used, but now it is how a team gets a report out every week. Maybe it is a customer-facing feature that got enough use to become part of the product. Either way, when it stops working, someone notices pretty quickly, and the business usually has to work around it until it is fixed.

The app works until it doesn’t

This is a familiar situation. One person builds something useful and they know all the details: where it runs, how to deploy it, which API key it needs, and what to do when it gets weird input. Nobody else really has to think about it because the app has worked so far, and there are always other things to work on.

Then a dependency updates, an integration changes, or a user does something the original builder did not expect. The app might start throwing an error, or it might do something worse and quietly give the wrong answer. The person who built it could be on vacation, busy with a different project, or no longer at the company. Now a tool that people rely on has no clear owner and no obvious way for someone else to figure out what went wrong.

The original builder did not do anything wrong. Getting a useful version into people’s hands quickly is usually the right move, especially when nobody knows yet whether the idea will be valuable. However, proving that an idea works and operating an application that other people depend on are two different things, and they require a different level of support.

The small things are usually the big things

Most teams do not decide that error handling, monitoring, or security reviews are unimportant. They simply have not needed them yet, or they have been focused on getting the useful part of the application in front of people. When five people use a tool, it is easy to hear about a problem and fix it directly. When fifty people use it, or when customers use it outside of business hours, those shortcuts start to become real problems.

For example, what happens when an integration fails halfway through a process? If the app has already updated one system but has not updated the next system, can someone tell what happened and safely retry it, or do they have to start manually checking records to figure out what needs to be fixed? There may not be one right answer, but it is worth thinking through before the first time it fails in a way that affects the business.

Some other common things we see are:

• An error happens, but the user gets an unhelpful message and there is no log that explains what actually failed.
• Nobody knows whether the application is healthy until someone says it is broken in Slack.
• Deploying a change means following a series of manual steps that live in one person’s head.
• API keys and other credentials are stored wherever was convenient when the application was first built.
• The application has its own username and password, even though everyone already signs in through the company’s identity provider.
• It handles company or customer information, but nobody has taken the time to review who can see it, where the information goes, or how long it is kept.
• No one besides the original builder knows how the pieces fit together or how to get the application back up after a problem.

None of these things are especially exciting or sexy to build, but they are the things that make a useful application either dependable or stressful to rely on. They are also usually much easier to address while the person who built the app is still around and the application is otherwise working well.

It does not mean starting over

The answer is not to take a working application away from the people using it and disappear for six months to rewrite it. Usually the application already has the most important thing: a real use case, people who can explain what matters, and an understanding of where it actually helps the business. Starting from that point is much better than starting from a blank page and guessing.

A better approach is to look at what is there, understand how it is being used, fix the biggest risks first, and add the support the application needs as it grows. The questions are fairly straightforward, but the answers tend to point to the work that matters most:

• Where does it run, and how does a change get from code to users?
• What happens when one of its services or integrations fails?
• Can the team see errors, usage, and the health of the application without guessing?
• Who has access to the application, its data, and its credentials?
• Can it use the company’s existing login and access controls rather than adding another account for everyone to manage?
• Could someone besides the original builder understand it, make a change, and respond to an issue?

Giving a useful app the support it needs

A production application does not need to be complicated, and not every small tool needs the same amount of process as a large customer-facing system. It does need a few basics that match the importance of the job it is doing.

A repeatable deployment process means changes can be reviewed, tested, and released in a predictable way instead of requiring a set of commands someone hopes they remember correctly. It also gives the team a reasonable way to roll back a bad change when one inevitably gets through.

Good error handling and visibility means useful errors for users, logs that explain what happened, and enough metrics, monitoring, and alerting to notice a problem before it turns into a long Slack thread. This is not about collecting every possible number; it is about being able to answer the basic question of whether the application is working and why it is not when it fails.

Security should fit how the company already works. That includes being deliberate about secrets, permissions, and data flows, as well as reviewing who should have access to the application. For an internal tool, using the company’s existing identity provider often avoids one more username and password for employees to manage, and makes it easier to remove access when somebody changes roles or leaves.

Finally, the application needs some documentation and shared ownership. A short overview of how the pieces fit together, a runbook for common problems, and more than one person who can work on it go a long way. The person who first built the application should not have to be the permanent support team just because they had the initiative to solve a problem.

If AI is part of the application, there are a few additional questions around testing prompt or model changes, handling unusual input, and deciding when a person should review an output. Those are important questions, but they sit alongside the same operational basics that every useful application needs.

Make the app more dependable while it is going well

The best time to make an application more supportable is when it is working and people are getting value from it. There is less pressure, the original builder can explain why the application works the way it does, and the team can make improvements without disrupting the workflow it supports.

This is the type of work Setfive can help with: understanding the application that exists today, stabilizing the areas most likely to cause trouble, reviewing security and access, putting appropriate infrastructure and monitoring in place, and making sure the business is not dependent on one person. The goal is not to turn every small application into a huge project. It is to make sure the useful thing stays useful as more people start relying on it.

A while ago, a brochure website was a meaningful software project. Somebody needed to lay out the pages, create navigation, make a contact form work, and get it all deployed. Website builders did not make a good website automatic, but they made that particular layer of work cheap enough that it stopped being the main thing most companies paid for.

Something similar is happening to CRUD applications.

A competent engineer with current tools can get surprisingly far, surprisingly quickly: a schema, basic APIs, forms, table views, search, permissions, validation, an admin screen, and a handful of ordinary integrations. LLMs help produce that code faster and make the usual implementation details less expensive to iterate on.

That is real progress. It is also easy to draw the wrong conclusion from it.

The fact that it is getting easier to build a system of record does not mean the business problem is solved. It means the database-shaped part of the problem is less scarce.

The valuable question was rarely just “where do we put the records?” It was “what should we do next, who needs to do it, and how do we know it worked?”

From recording work to improving work

CRUD is still necessary. Organizations need a place to record customers, jobs, invoices, inventory, cases, and the rest of the nouns that make up their work. They need people to be able to find and correct those records.

But a record is not an outcome.

A useful distinction is between a system of record and a system of action. The first stores what happened. The second helps decide what deserves attention, coordinates action across people and systems, and learns from the result.

System of record	System of action
Stores customers, jobs, invoices, and cases	Prioritizes work and moves it forward
Lets people enter, search, and update data	Coordinates people, systems, and exceptions
Reports what happened	Forecasts, recommends, and optimizes what to do next
Uses broadly reusable patterns	Encodes domain-specific constraints and tradeoffs

The CRUD layer is often part of a system of action. It is just not usually the part that makes the system valuable.

The floor is moving to workflow

Take a customer-success tool. The commodity version has accounts, contacts, renewal dates, health-score fields, notes, and tasks. That is useful, and it is also a familiar application shape.

The harder version combines product usage, unresolved support issues, contract terms, champion turnover, and outcomes from similar accounts to answer a more useful question: which accounts need attention this week, and what action is most likely to change the outcome?

Then it needs to make that action practical. Perhaps a support issue needs escalation, a CSM needs a meeting, sales needs to be involved before a renewal date, and the team needs a shared view of what happened next. The value is not a nicer account page. It is reducing the chance that an important customer falls through the cracks.

This pattern appears everywhere. Field-service software can store work orders, technicians, addresses, and status updates. The differentiated work is scheduling and re-scheduling against technician skills, promised windows, parts availability, geography, overtime rules, uncertain job duration, and emergency calls. A credible schedule at 8:00 AM is not enough if the system cannot respond when a job takes twice as long as expected at 10:30.

That is workflow orchestration: the messy part involving handoffs, timing, exceptions, policy, and people. It is not glamorous, but it is where a lot of operational software earns its keep.

Not everything valuable is an LLM

LLMs are part of this shift, but “CRUD to AI” is too narrow a description.

LLMs are particularly useful when a workflow begins with unstructured information: an email, a document, a call transcript, an image, or a request written in normal language. They can help extract information, classify incoming work, summarize context, or give a person a natural-language interface to a system.

Other valuable systems may have no LLM in the critical path at all. They may use a rules engine, a forecast, a statistical process-control chart, a constraint solver, a simulation, or a carefully constructed report. Many systems will combine several of these approaches.

The common thread is not the model. The software does more than preserve a record of work. It helps make a better decision, execute it, and learn from the outcome.

Analytics turns data into a question worth answering

Consider revenue operations. A CRM stores leads, opportunities, stages, activity, and quotas. The useful analysis is often above that layer: pipeline coverage by segment, conversion rates between stages, typical cycle times, and the difference between a healthy-looking pipeline and one that is unlikely to close in time.

Those views support real decisions. Is a territory short on coverage? Is a segment converting differently? Does the organization need more sales capacity, a different territory design, or a different target? The implementation might be straightforward cohort analysis or a forecast based on historical data. It does not need to be generative AI to be valuable.

Product analytics has a similar trap. A dashboard can show that activation or retention moved. An experiment, with a clear metric and a credible comparison group, helps answer whether a product change caused the movement. That difference matters when deciding what to ship to everyone.

The system of record supplies the events. The analytical layer makes them useful for a decision.

Optimization makes tradeoffs explicit

Some of the highest-value software is not about generating text or predicting a label. It is about choosing among competing, constrained options.

A logistics application might store shipments, vehicles, drivers, stops, service windows, and delivery status. The difficult work is assigning loads and planning routes while respecting vehicle capacity, driver-hours rules, pickup timing, delivery promises, and cost. This is an optimization problem. There may be no chat interface and no LLM involved.

Inventory is another familiar example. A basic app can show that stock is low. A more useful system estimates demand and lead-time uncertainty, accounts for storage limits and the differing cost of stockouts, and recommends what to order, from whom, and when. It makes the tradeoff visible instead of leaving a person to infer it from a table of quantities.

Workforce scheduling has the same shape. Employee records, certifications, availability, and shifts are CRUD. Building a workable schedule means balancing coverage, labor rules, preferences, fairness, qualifications, and overtime. The value is a schedule that an operation can actually run.

Operations research, forecasting, and constraint solving have been doing this work for a long time. Cheaper application development does not replace them. It makes it more feasible to spend effort on the part that changes the outcome.

Reliable execution is part of the product

A recommendation that cannot be acted on is just another dashboard.

Useful systems need to connect to the places where work happens, create or route the next task, explain why an item was prioritized, and handle cases that do not fit the normal path. They need audit trails where the decision matters. They need safe fallbacks and a clear way for a person to take over.

This is especially important when an LLM is involved. A model can help read an invoice, summarize a case, or classify an incoming request. It should not turn uncertainty into an invisible decision. The system needs confidence thresholds, validation, exception queues, permissions, and a way to correct mistakes. Those are not incidental implementation details. They are what make automation usable in a real operation.

The same is true for non-AI logic. A routing optimizer needs to expose the constraints it used. A forecast needs to show when its assumptions no longer resemble reality. An approval workflow needs a path for the unusual case. Dependability is not separate from the product; it is part of the value proposition.

Start with the bottleneck, not the screen

For builders, the practical implication is simple: start with the recurring decision or bottleneck.

Ask what people are repeatedly deciding, what information they have to assemble to decide it, which constraints they are balancing, and what happens after they make the call. Then work backward to the data, integrations, analysis, and interface required.

That approach may still produce a CRUD application. Most useful systems need records. But the record pages become infrastructure for a more specific outcome: fewer missed renewals, better route utilization, more disciplined purchasing, faster resolution, or a decision that used to require several people and a spreadsheet.

LLMs have made the CRUD shell cheaper to produce. That should be good news. It lets teams spend more of their attention on the work that has always been difficult: understanding an operation well enough to remove delays, make tradeoffs explicit, and reliably move work forward.

Inside most companies right now, AI is already at work. Even if you have not rolled out ChatGPT Team or Copilot, people are using their personal subscriptions to speed things up. That is great for initiative, not so great for consistency. It looks a lot like the early Excel era: clever workarounds, duplicate effort, and new questions about data governance.

Sherpa is our way to bring order to that energy. We analyze real tasks from Asana or monday.com, group similar work with an LLM, and point to the places where automation will pay off. You get a clear plan you can act on, without buying another stack of licenses first.

What Sherpa is

Think of Sherpa as an AI audit for your task data. It connects to your workspace, reads tasks with your permission, and maps the repetitive patterns that eat time. Then it scores where automation is likely to win, explains why, and recommends how to build it. The output is practical and specific: plays, tools, prompts, and an effort estimate so you can prioritize.

How it works

You start by connecting Asana or monday.com with OAuth. Access is read only and under your control at all times. We do not change or write tasks.

Next, a large language model groups related work and finds recurring patterns. That includes obvious repeats, quiet duplicates that happen across teams, and tasks that often move together in a process.

Finally, we deliver a short report that tells you what to automate, in what order, and how. Each recommendation includes expected time savings, suggested connectors or integrations, and sample prompts so your team can move quickly.

Typical turnaround is about a week from connection to findings.

Why scan tasks now

Personal AI usage is already shaping how work gets done. Sherpa helps you see what is working, what is risky, and what should be standardized. It replaces guesswork with a picture of real workflows, so you can invest in the right automations and avoid paying for licenses that will not get used.

Leaders also get a common view of where hours are going. That makes process conversations easier. Instead of debating tools in the abstract, you can point to specific clusters of tasks and decide how to fix them.

What you get in the report

• An automation scorecard with high, medium, and low opportunities, each with a short rationale.
• A top 3 list of automation plays with exact steps, recommended tools, and integration notes.
• An impact section that translates hours into dollars using your inputs.

You also get a recurring task map, duplicate detection across teams, suggested prompts and connectors, and a next step build plan that you can implement with your team or with Setfive.

A sample finding

Manual reporting shows up in almost every audit. A team exports CSVs every Friday, merges them by hand, and posts a summary. The play is straightforward: schedule the extract, load it to a source of truth, and send a templated summary to Slack or email.

• Impact: High
• Effort: Medium
• Estimated savings: 6 hours per week

If 10 people each save 6 hours per week at an average loaded rate of 75 dollars per hour, that is 6×10×75=4500 dollars of capacity back every week.

Where Sherpa fits with ChatGPT Team and Copilot

Already have licenses? Sherpa shows where to deploy them and turns ad hoc prompts into repeatable, auditable workflows.

Still evaluating? Run Sherpa first to find the highest value use cases, then buy only what you need.

Not ready to buy seats? Many plays use tools you already have, so you can capture savings now and expand later.

Security and privacy

Sherpa uses OAuth with scoped, read only access. You can revoke access at any time. We follow your data retention requirements, and your findings are your IP. We do not use your data to train public models.

Who benefits

Ops and RevOps leaders with checklist heavy processes. PMOs juggling handoffs. CS and Support teams producing weekly reports. Marketing ops moving content through approvals. Finance and People teams closing the loop on routine reconciliations. If the same task shows up again and again, Sherpa will find it.

FAQs

Do we need to change how we work to try it? No. Sherpa analyzes the work you already do.

Will this replace people? The goal is to remove low leverage, repetitive tasks so your team can focus on higher value work.

Can you help implement the plays? Yes. Implementation projects are scoped after the audit.

Try the Free AI Task Audit

Stop guessing where AI will help. Measure it. Sherpa shows you the work your team should not be doing and how to automate it, fast.

Get your Free AI Task Audit, a concise scorecard, and a prioritized plan with savings you can defend.
Ready to see your opportunities? Get in touch at contact@setfive.com or read more about Sherpa at sherpa.setfive.com

A lot of information these days is just a Google search away, but there is still a surprising number of businesses out there that keep information like pricing locked behind phone lines. Oftentimes, this is deliberate, and they may do this for a variety of reasons:

• Fluctuating prices that change based on demand, inventory, or seasonality.
• Sales psychology that converts curious callers into customers.
• A competitive advantage in keeping pricing opaque to competitors.
• Personalized quotes that change based on customer need.
• Old school businesses that just never went digital.

Traditionally, to gather information from these businesses, you would need someone or even multiple people to work through an endless call list, navigating phone menu trees, waiting on hold, and manually transcribing conversations into spreadsheets. This is tedious, expensive, slow, and doesn't scale.

At Setfive, we decided to look into how we could automate this.

OpenAI Realtime API

The timing couldn't have been better. As we were exploring ways to do this, OpenAI released its Realtime API, a game-changer for voice-based AI applications. Unlike conventional text-based APIs that require separate speed-to-text and text-to-speech steps, the Realtime API combines these and enables:

• Low-latency native voice conversations.
• Natural interruptions for more human-like interactions.
• Built-in function calling for triggering actions mid-conversation.

This was an AI capable of having an actual over-the-phone conversation.

Building The Bridge

With the brain of the operation sorted, it was time to find a way to actually make phone calls. For this, we chose Twilio, a well-regarded platform for telecommunications for almost two decades.

Twilio's Media Streams API made it simple to pipe audio directly to and from the OpenAI Realtime API, creating a seamless conversation flow. The business on the other end hears a responsive customer who can handle unexpected conversational turns.

Navigating The Maze

One of the first challenges we ran into? Phone trees. You know them: "Press 1 for appointments, Press 2 to speak to a customer service representative, …" These interactive voice response (IVR) systems are designed for touch-tone input, not voice commands.

We solved this by building AI tools that can simulate DTMF (Dual-Tone Multi-Frequency) signals using Twilio's API - which required some trial and error with their callback and TwiML architecture - so that our AI can listen to menu options, simulate button presses, navigate complex multi-level menus, and find the fastest path to reach a customer service representative or a front desk.

From Conversations To Structured Data

Getting through to the right person is only half the battle. The real magic happens when our AI finally gets into a conversation. From there, we are able to extract structured information from free-flowing conversations in real time. Using carefully crafted prompts, our system can:

• Identify key information even when it's mentioned casually
• Ask clarifying questions about discrepancies in the information received
• Extract additional valuable data like availability, pricing details (first-time customer, minimum orders, ect.), and more
• Create clean, structured data ready for your database, Excel spreadsheet, or whatever else you're using.

When Nobody Answers

Here's something we didn't anticipate: businesses that rely heavily on phone communication are often too busy to answer their phones. These are often small businesses that may not have dedicated staff for handling phones or may have employees who wear multiple hats. They're not sitting by the phone waiting for calls.

This was having a real effect on our success rate, and we didn't want to make multiple calls to the same business, hoping for someone to be available. The next step was obvious: voicemail. We enhanced our system to handle a full communication cycle:

• Intelligent voicemail detection to detect when we have reached a voicemail inbox.
• Leave a natural message requesting whatever information the AI is looking for.
• Callback handling that is able to naturally continue the conversation when a business calls back.

Ready to Build?

Interested in how this can help you? Email us at contact@setfive.com to find out more or check out
our demo at voice2data.setfive.com!