A lot of small applications are getting built right now. Someone has a frustrating, repetitive problem, perhaps they need to clean up a report, move information from one system to another, or give customers a simpler way to complete a task. They use AI, a low-code tool, or just write a little code, and in a surprisingly short amount of time they have something useful. Then people start using it every day.

That is a huge win! The application proved that there is a real problem worth solving, which is often the hardest thing to figure out before building a larger project. The trouble is that once people depend on it, it has quietly stopped being just a quick experiment, even if it is still deployed like one.

Maybe it started as an internal tool that only a few people used, but now it is how a team gets a report out every week. Maybe it is a customer-facing feature that got enough use to become part of the product. Either way, when it stops working, someone notices pretty quickly, and the business usually has to work around it until it is fixed.

The app works until it doesn’t

This is a familiar situation. One person builds something useful and they know all the details: where it runs, how to deploy it, which API key it needs, and what to do when it gets weird input. Nobody else really has to think about it because the app has worked so far, and there are always other things to work on.

Then a dependency updates, an integration changes, or a user does something the original builder did not expect. The app might start throwing an error, or it might do something worse and quietly give the wrong answer. The person who built it could be on vacation, busy with a different project, or no longer at the company. Now a tool that people rely on has no clear owner and no obvious way for someone else to figure out what went wrong.

The original builder did not do anything wrong. Getting a useful version into people’s hands quickly is usually the right move, especially when nobody knows yet whether the idea will be valuable. However, proving that an idea works and operating an application that other people depend on are two different things, and they require a different level of support.

The small things are usually the big things

Most teams do not decide that error handling, monitoring, or security reviews are unimportant. They simply have not needed them yet, or they have been focused on getting the useful part of the application in front of people. When five people use a tool, it is easy to hear about a problem and fix it directly. When fifty people use it, or when customers use it outside of business hours, those shortcuts start to become real problems.

For example, what happens when an integration fails halfway through a process? If the app has already updated one system but has not updated the next system, can someone tell what happened and safely retry it, or do they have to start manually checking records to figure out what needs to be fixed? There may not be one right answer, but it is worth thinking through before the first time it fails in a way that affects the business.

Some other common things we see are:

• An error happens, but the user gets an unhelpful message and there is no log that explains what actually failed.
• Nobody knows whether the application is healthy until someone says it is broken in Slack.
• Deploying a change means following a series of manual steps that live in one person’s head.
• API keys and other credentials are stored wherever was convenient when the application was first built.
• The application has its own username and password, even though everyone already signs in through the company’s identity provider.
• It handles company or customer information, but nobody has taken the time to review who can see it, where the information goes, or how long it is kept.
• No one besides the original builder knows how the pieces fit together or how to get the application back up after a problem.

None of these things are especially exciting or sexy to build, but they are the things that make a useful application either dependable or stressful to rely on. They are also usually much easier to address while the person who built the app is still around and the application is otherwise working well.

It does not mean starting over

The answer is not to take a working application away from the people using it and disappear for six months to rewrite it. Usually the application already has the most important thing: a real use case, people who can explain what matters, and an understanding of where it actually helps the business. Starting from that point is much better than starting from a blank page and guessing.

A better approach is to look at what is there, understand how it is being used, fix the biggest risks first, and add the support the application needs as it grows. The questions are fairly straightforward, but the answers tend to point to the work that matters most:

• Where does it run, and how does a change get from code to users?
• What happens when one of its services or integrations fails?
• Can the team see errors, usage, and the health of the application without guessing?
• Who has access to the application, its data, and its credentials?
• Can it use the company’s existing login and access controls rather than adding another account for everyone to manage?
• Could someone besides the original builder understand it, make a change, and respond to an issue?

Giving a useful app the support it needs

A production application does not need to be complicated, and not every small tool needs the same amount of process as a large customer-facing system. It does need a few basics that match the importance of the job it is doing.

A repeatable deployment process means changes can be reviewed, tested, and released in a predictable way instead of requiring a set of commands someone hopes they remember correctly. It also gives the team a reasonable way to roll back a bad change when one inevitably gets through.

Good error handling and visibility means useful errors for users, logs that explain what happened, and enough metrics, monitoring, and alerting to notice a problem before it turns into a long Slack thread. This is not about collecting every possible number; it is about being able to answer the basic question of whether the application is working and why it is not when it fails.

Security should fit how the company already works. That includes being deliberate about secrets, permissions, and data flows, as well as reviewing who should have access to the application. For an internal tool, using the company’s existing identity provider often avoids one more username and password for employees to manage, and makes it easier to remove access when somebody changes roles or leaves.

Finally, the application needs some documentation and shared ownership. A short overview of how the pieces fit together, a runbook for common problems, and more than one person who can work on it go a long way. The person who first built the application should not have to be the permanent support team just because they had the initiative to solve a problem.

If AI is part of the application, there are a few additional questions around testing prompt or model changes, handling unusual input, and deciding when a person should review an output. Those are important questions, but they sit alongside the same operational basics that every useful application needs.

Make the app more dependable while it is going well

The best time to make an application more supportable is when it is working and people are getting value from it. There is less pressure, the original builder can explain why the application works the way it does, and the team can make improvements without disrupting the workflow it supports.

This is the type of work Setfive can help with: understanding the application that exists today, stabilizing the areas most likely to cause trouble, reviewing security and access, putting appropriate infrastructure and monitoring in place, and making sure the business is not dependent on one person. The goal is not to turn every small application into a huge project. It is to make sure the useful thing stays useful as more people start relying on it.

A while ago, a brochure website was a meaningful software project. Somebody needed to lay out the pages, create navigation, make a contact form work, and get it all deployed. Website builders did not make a good website automatic, but they made that particular layer of work cheap enough that it stopped being the main thing most companies paid for.

Something similar is happening to CRUD applications.

A competent engineer with current tools can get surprisingly far, surprisingly quickly: a schema, basic APIs, forms, table views, search, permissions, validation, an admin screen, and a handful of ordinary integrations. LLMs help produce that code faster and make the usual implementation details less expensive to iterate on.

That is real progress. It is also easy to draw the wrong conclusion from it.

The fact that it is getting easier to build a system of record does not mean the business problem is solved. It means the database-shaped part of the problem is less scarce.

The valuable question was rarely just “where do we put the records?” It was “what should we do next, who needs to do it, and how do we know it worked?”

From recording work to improving work

CRUD is still necessary. Organizations need a place to record customers, jobs, invoices, inventory, cases, and the rest of the nouns that make up their work. They need people to be able to find and correct those records.

But a record is not an outcome.

A useful distinction is between a system of record and a system of action. The first stores what happened. The second helps decide what deserves attention, coordinates action across people and systems, and learns from the result.

System of record	System of action
Stores customers, jobs, invoices, and cases	Prioritizes work and moves it forward
Lets people enter, search, and update data	Coordinates people, systems, and exceptions
Reports what happened	Forecasts, recommends, and optimizes what to do next
Uses broadly reusable patterns	Encodes domain-specific constraints and tradeoffs

The CRUD layer is often part of a system of action. It is just not usually the part that makes the system valuable.

The floor is moving to workflow

Take a customer-success tool. The commodity version has accounts, contacts, renewal dates, health-score fields, notes, and tasks. That is useful, and it is also a familiar application shape.

The harder version combines product usage, unresolved support issues, contract terms, champion turnover, and outcomes from similar accounts to answer a more useful question: which accounts need attention this week, and what action is most likely to change the outcome?

Then it needs to make that action practical. Perhaps a support issue needs escalation, a CSM needs a meeting, sales needs to be involved before a renewal date, and the team needs a shared view of what happened next. The value is not a nicer account page. It is reducing the chance that an important customer falls through the cracks.

This pattern appears everywhere. Field-service software can store work orders, technicians, addresses, and status updates. The differentiated work is scheduling and re-scheduling against technician skills, promised windows, parts availability, geography, overtime rules, uncertain job duration, and emergency calls. A credible schedule at 8:00 AM is not enough if the system cannot respond when a job takes twice as long as expected at 10:30.

That is workflow orchestration: the messy part involving handoffs, timing, exceptions, policy, and people. It is not glamorous, but it is where a lot of operational software earns its keep.

Not everything valuable is an LLM

LLMs are part of this shift, but “CRUD to AI” is too narrow a description.

LLMs are particularly useful when a workflow begins with unstructured information: an email, a document, a call transcript, an image, or a request written in normal language. They can help extract information, classify incoming work, summarize context, or give a person a natural-language interface to a system.

Other valuable systems may have no LLM in the critical path at all. They may use a rules engine, a forecast, a statistical process-control chart, a constraint solver, a simulation, or a carefully constructed report. Many systems will combine several of these approaches.

The common thread is not the model. The software does more than preserve a record of work. It helps make a better decision, execute it, and learn from the outcome.

Analytics turns data into a question worth answering

Consider revenue operations. A CRM stores leads, opportunities, stages, activity, and quotas. The useful analysis is often above that layer: pipeline coverage by segment, conversion rates between stages, typical cycle times, and the difference between a healthy-looking pipeline and one that is unlikely to close in time.

Those views support real decisions. Is a territory short on coverage? Is a segment converting differently? Does the organization need more sales capacity, a different territory design, or a different target? The implementation might be straightforward cohort analysis or a forecast based on historical data. It does not need to be generative AI to be valuable.

Product analytics has a similar trap. A dashboard can show that activation or retention moved. An experiment, with a clear metric and a credible comparison group, helps answer whether a product change caused the movement. That difference matters when deciding what to ship to everyone.

The system of record supplies the events. The analytical layer makes them useful for a decision.

Optimization makes tradeoffs explicit

Some of the highest-value software is not about generating text or predicting a label. It is about choosing among competing, constrained options.

A logistics application might store shipments, vehicles, drivers, stops, service windows, and delivery status. The difficult work is assigning loads and planning routes while respecting vehicle capacity, driver-hours rules, pickup timing, delivery promises, and cost. This is an optimization problem. There may be no chat interface and no LLM involved.

Inventory is another familiar example. A basic app can show that stock is low. A more useful system estimates demand and lead-time uncertainty, accounts for storage limits and the differing cost of stockouts, and recommends what to order, from whom, and when. It makes the tradeoff visible instead of leaving a person to infer it from a table of quantities.

Workforce scheduling has the same shape. Employee records, certifications, availability, and shifts are CRUD. Building a workable schedule means balancing coverage, labor rules, preferences, fairness, qualifications, and overtime. The value is a schedule that an operation can actually run.

Operations research, forecasting, and constraint solving have been doing this work for a long time. Cheaper application development does not replace them. It makes it more feasible to spend effort on the part that changes the outcome.

Reliable execution is part of the product

A recommendation that cannot be acted on is just another dashboard.

Useful systems need to connect to the places where work happens, create or route the next task, explain why an item was prioritized, and handle cases that do not fit the normal path. They need audit trails where the decision matters. They need safe fallbacks and a clear way for a person to take over.

This is especially important when an LLM is involved. A model can help read an invoice, summarize a case, or classify an incoming request. It should not turn uncertainty into an invisible decision. The system needs confidence thresholds, validation, exception queues, permissions, and a way to correct mistakes. Those are not incidental implementation details. They are what make automation usable in a real operation.

The same is true for non-AI logic. A routing optimizer needs to expose the constraints it used. A forecast needs to show when its assumptions no longer resemble reality. An approval workflow needs a path for the unusual case. Dependability is not separate from the product; it is part of the value proposition.

Start with the bottleneck, not the screen

For builders, the practical implication is simple: start with the recurring decision or bottleneck.

Ask what people are repeatedly deciding, what information they have to assemble to decide it, which constraints they are balancing, and what happens after they make the call. Then work backward to the data, integrations, analysis, and interface required.

That approach may still produce a CRUD application. Most useful systems need records. But the record pages become infrastructure for a more specific outcome: fewer missed renewals, better route utilization, more disciplined purchasing, faster resolution, or a decision that used to require several people and a spreadsheet.

LLMs have made the CRUD shell cheaper to produce. That should be good news. It lets teams spend more of their attention on the work that has always been difficult: understanding an operation well enough to remove delays, make tradeoffs explicit, and reliably move work forward.

AI code generation is one of the most useful things to happen to software development in a long time. We use it. It gets people from an idea to a working application much faster than they could have a few years ago.

That is a big deal. It is also not the same thing as getting from an idea to a well-operated application.

This week, one of our clients’ Snowflake cost alerts went off. A new application had spent more than $4,000 in a couple of days. The application had been built with Claude and it was doing what its owner intended it to do. The problem was how it was doing it: repeatedly polling Snowflake with a larger warehouse than the work required.

The code worked. The bill did too.

The alert was the important part

The client had anomaly alerts in place long before this application existed. Over years of normal use, those alerts had established a useful picture of what ordinary compute usage looked like. When the new application’s usage departed from that pattern, it stood out quickly.

That monitoring was not glamorous, and it was not new. It was operational knowledge turned into a guardrail. Without it, the polling loop could have continued until someone happened to notice an unusually large bill.

This is worth emphasizing because it is easy to see AI as the whole story. The model helped create the application. The monitoring, the historical baseline, and the people who responded to the alert are what limited the damage.

Working is not the same as economical

AI is good at getting to a plausible solution. It can write the query, connect the service, add a loop, and return the result. But it does not naturally care whether a process runs every minute instead of every hour, whether data can be cached, whether an existing system already solves part of the problem, or whether a warehouse is sized appropriately for the query.

Sometimes the generated solution will rebuild something that already exists. Sometimes it will choose a direct approach that is perfectly functional but wasteful at production scale. A polling loop is a simple example: it may make a feature feel responsive while quietly paying for repeated work that is unnecessary.

None of that makes the application useless, or AI a bad tool. A few years ago, a person without deep technical experience might not have been able to build and deploy this application at all. Now they can. That is real leverage.

But the leverage changes where the risk sits. Development time may go down while cloud spend, maintenance, security exposure, or reliability risk goes up. Those costs often arrive after the demo is working and the application is in use.

A short review can be a very good investment

The answer is not to ban AI-generated code or require every idea to go through a long development process. The answer is to put experienced eyes on the parts that determine how software behaves in the real world.

For a data-backed application, that review can be straightforward:

• What runs on a schedule, and how often does it actually need to run?
• Which queries execute, on what warehouse, and how much data do they scan?
• Can the application cache results, react to an event, or reuse an existing data set instead of polling?
• What is the expected cost at normal usage and at a failure mode?
• Which alerts will tell us when the application behaves differently from expected?

A review like this does not need to take longer than the work it is reviewing. In this case, it could have prevented a four-figure surprise. More importantly, it creates a habit of treating an AI-generated application as software that will be operated, not just code that needs to run once.

Keep the human in the loop

There is a familiar parallel with outsourcing. Lower-cost implementation can be a good trade when the work is understood and the output is reviewed. It becomes expensive when the apparent savings mean nobody owns the architecture, the quality, or the ongoing consequences.

AI assistance is similar. It can make capable builders out of more people, and that is something to embrace. But it can also produce slop, inefficiencies, security risks, and bugs that are easy to miss because the first version looks complete.

The goal should not be to slow people down. It should be to pair the speed of AI with monitoring that catches surprises and with people who understand the systems, costs, and tradeoffs behind the code.

This is the first in a series of examples from the gap between shipping software quickly and operating it well. AI can help you build faster. Make sure someone is also asking what the resulting system will cost to run.

Claude, Codex, and the rest are useful debugging partners. They can suggest hypotheses quickly, explain unfamiliar systems, and keep an investigation moving when you are stuck.

They can also send you on a very convincing rabbit hole.

We ran into that with a client’s ASP.NET application. After it had been up for a while, the first request for a static JavaScript file could be very slow. Requests after that were fast. It was the kind of narrow, intermittent behavior that invites a long list of theories.

Claude’s initial diagnosis was that SSL certificate revocation checking was holding up the first request. That is a real thing worth knowing about, and it sounded plausible in the abstract. But the application was using a self-signed certificate. There was no certificate authority revocation check to perform. A small piece of domain knowledge ruled out a direction that otherwise could have consumed hours.

The problem was not that Claude mentioned certificate revocation. The problem would have been treating a confident, technically detailed answer as evidence.

Start with what the system is doing

Rather than follow the SSL theory, we tested the behavior we could observe. We read the assets directly from the filesystem and fetched the asset through the application. The pattern was consistent:

1. Fetch an asset and the first request is slow.
2. Fetch it again immediately and it is fast.
3. Edit the file, then fetch it again, and the next request is slow again.

That is a much more useful description of the issue than “static JavaScript is slow.” The expensive path was associated with first access to changed file content. It was not ordinary request handling, and it did not fit the TLS explanation.

The experiment strongly pointed to endpoint scanning. SentinelOne was running in that environment and was the most likely cause: changed content was likely being scanned on its first access, while the next read benefited from the result already being available. We did not treat that as a definitive vendor-level attribution, but it fit the observed behavior far better than revocation checking did.

Plausible is not proven

LLMs are especially good at producing plausible explanations. They have seen the vocabulary around a symptom, and they can connect it to a real mechanism. That is useful for generating a list of things to investigate.

But a diagnosis has to survive the details of the actual system:

• Does the proposed mechanism exist in this deployment?
• Does it explain the timing and repeatability of the symptom?
• What inexpensive test could distinguish it from the other hypotheses?
• What observation would prove it wrong?

A self-signed certificate was enough to make us stop and question the revocation theory. The cold-read, warm-read, and modified-file test gave us a better hypothesis to pursue. Neither step required an encyclopedic knowledge of every possible cause. They required knowing enough to check the assumptions and to design a small experiment.

Use the model as a partner, not an authority

Claude still helped with the investigation. The right use was not to ask it for the answer and implement the first response. It was to use it as a partner while we compared theories against the environment and the measurements.

A practical debugging loop looks like this:

1. State the observation precisely, including what changes between a slow request and a fast one.
2. Ask the model for competing hypotheses and a test that would separate each one.
3. Check its assumptions against the architecture, configuration, and operational environment.
4. Run the smallest useful experiment.
5. Feed the result back in and repeat.

This is also why domain expertise still matters when using LLMs. If you cannot tell whether an answer fits the system you are operating, confidence and detail are easy to mistake for correctness. Bring in someone who knows the domain, or slow down enough to validate the model’s premises before chasing its conclusion.

The model can make a good investigator faster. It cannot replace the judgment needed to decide whether a theory belongs in the investigation at all.

Everybody likes an API. It is the clean version of an integration: documented endpoints, structured responses, credentials meant for software, and hopefully somebody else’s problem when the implementation changes.

But plenty of useful systems do not have one. Or they have an API that covers part of the product but not the screen a team actually needs. The data is there, behind a normal login, and somebody is opening a browser and copying it into a spreadsheet.

That is a good place for browser automation.

We have been looking at a practical combination of Playwright and Codex for this kind of work. Playwright drives a real Chrome browser. Codex can work with a browser exposed over the Chrome DevTools Protocol (CDP), which makes it useful for exploring an application and helping build the automation. The important piece is that the login stays human: someone who is authorized to use the site signs in and completes MFA themselves. The automated job uses the resulting browser session; it does not try to get around the login.

A concrete example is recording live odds from sportsbooks such as FanDuel or BetMGM. If an organization is allowed to collect and use the data, that is a much better job for a computer than for someone watching pages and updating a sheet all day.

The browser is sometimes the integration

This is not an argument to scrape everything. If there is an official API that does the job, use it. It is almost always less fragile and easier to support.

But sometimes the browser is the only interface available to the user. A person can log in, look at a live market, and see the numbers, but there is no supported endpoint for getting the same information into an internal system. In that situation, the browser can be the boundary between the site and your workflow.

There are obvious limits. The account needs to be authorized, and the intended use needs to comply with the site’s terms, contracts, and applicable law. That matters especially for sportsbooks, where access and permitted use can vary by operator and jurisdiction. This is not a way to bypass MFA, CAPTCHAs, rate limits, or other controls.

Getting a logged-in browser session

Playwright is a browser automation framework. It can launch Chromium or Chrome, navigate pages, click buttons, fill forms, and read what the page renders.

The feature that makes authenticated automation workable is a persistent browser profile. Instead of starting fresh every time, Playwright launches Chrome with the same profile directory. That lets it retain browser state such as cookies and sessions when the site permits it.

The first run is simple: launch a visible browser, log in normally, handle the two-factor prompt, and make sure the page you need is available. After that, the job can reopen that same profile. It can run visibly while somebody is building or debugging it, then run headlessly when it is ready to collect data on a schedule.

import { chromium } from "playwright";

const userDataDir = "/secure/path/to/browser-profile";
const context = await chromium.launchPersistentContext(userDataDir, {
  channel: "chrome",
  headless: false, // first run: let the account owner log in
});

const page = await context.newPage();
await page.goto("https://example.com/login");

// The authorized user completes sign-in and MFA in this browser window.

That profile directory is sensitive. It may contain an active session, so treat it like a credential: keep it in approved storage, restrict who can access it, and do not put it in source control or logs. Also plan for it to expire. The site decides how long a session lasts. If the job finds a login page again, it should stop and let a person reauthenticate.

Asking Codex to do the tedious part

The page is usually the part that makes these projects annoying. Modern web applications load data after the page appears, change the DOM as markets update, and use selectors that are not obvious until you can inspect the live application.

This is where Codex helps. With the authenticated browser connected over CDP, you can ask it to inspect the page, find the market and selection elements, and build the Playwright script around what is actually there. It is much faster than guessing at selectors from a screenshot or trying to reverse engineer an undocumented backend.

A reasonable first prompt is something like:

Set up Playwright with a persistent Chrome profile and launch it visibly so I can log in. Once I have an authenticated session, use that session to inspect the live odds page and build a script that records the event, market, selection, displayed odds, and collection time.

Codex is helping with the implementation; it is not replacing the account holder. Keep that line clear. It should not be asked to find credentials, solve MFA, or work around controls the site has put in place.

A live-odds collector

Say the job is to keep an internal, timestamped record of odds for a set of games and markets.

First decide what an observation is. At a minimum, it will probably include the sportsbook, event, market, selection, displayed odds, the source page, and the time the value was seen. That last field matters: a number collected at 2:00 PM is not the same thing as a number collected five minutes later.

Next, use the visible browser session to get to the right market and let Codex inspect the rendered page. The job needs to know when the market is actually loaded, how a suspended or unavailable price is represented, and which labels or attributes are stable enough to use as selectors. Prefer user-facing labels where possible over a long chain of generated CSS classes that will disappear in the next redesign.

Once that is understood, the collector can launch the same persistent profile in headless mode, visit only the pages it needs, validate what it finds, and write normalized records to a database or queue.

const context = await chromium.launchPersistentContext(userDataDir, {
  channel: "chrome",
  headless: true,
});
const page = await context.newPage();

await page.goto(targetMarketUrl, { waitUntil: "domcontentloaded" });
await page.getByRole("heading", { name: /live odds/i }).waitFor();

const collectedAt = new Date().toISOString();
const observations = await page.locator("[data-market]").evaluateAll((markets) =>
  markets.map((market) => ({
    market: market.getAttribute("data-market"),
    text: market.textContent?.trim(),
    collectedAt,
  }))
);

The code above is deliberately generic. The real selectors should come from the target site and should be tested against its actual states. The useful outcome is not just a script that reads a page once. It is a small internal data source that dashboards, reports, or models can rely on without each one having to understand the sportsbook’s UI.

The unglamorous stuff is what makes it work

A browser job will change when the website changes. That is normal. The difference between a useful integration and a fragile script is how it behaves on a bad day.

Keep the browser profile locked down. Save where each observation came from and when it was collected. Alert when the job suddenly gets no results, sees a login screen, or returns far fewer records than normal. And give somebody a straightforward way to rerun the visible browser and refresh the session.

That is enough to turn a manual task into something dependable without pretending the website is an API.

For companies sitting on useful data behind logins, this is a practical option: a person handles the authorization once, Playwright keeps the browser state, and Codex speeds up the work of turning what is on the screen into structured data.