Category: Machine learning

AI code generation is one of the most useful things to happen to software development in a long time. We use it. It gets people from an idea to a working application much faster than they could have a few years ago.

That is a big deal. It is also not the same thing as getting from an idea to a well-operated application.

This week, one of our clients’ Snowflake cost alerts went off. A new application had spent more than $4,000 in a couple of days. The application had been built with Claude and it was doing what its owner intended it to do. The problem was how it was doing it: repeatedly polling Snowflake with a larger warehouse than the work required.

The code worked. The bill did too.

The alert was the important part

The client had anomaly alerts in place long before this application existed. Over years of normal use, those alerts had established a useful picture of what ordinary compute usage looked like. When the new application’s usage departed from that pattern, it stood out quickly.

That monitoring was not glamorous, and it was not new. It was operational knowledge turned into a guardrail. Without it, the polling loop could have continued until someone happened to notice an unusually large bill.

This is worth emphasizing because it is easy to see AI as the whole story. The model helped create the application. The monitoring, the historical baseline, and the people who responded to the alert are what limited the damage.

Working is not the same as economical

AI is good at getting to a plausible solution. It can write the query, connect the service, add a loop, and return the result. But it does not naturally care whether a process runs every minute instead of every hour, whether data can be cached, whether an existing system already solves part of the problem, or whether a warehouse is sized appropriately for the query.

Sometimes the generated solution will rebuild something that already exists. Sometimes it will choose a direct approach that is perfectly functional but wasteful at production scale. A polling loop is a simple example: it may make a feature feel responsive while quietly paying for repeated work that is unnecessary.

None of that makes the application useless, or AI a bad tool. A few years ago, a person without deep technical experience might not have been able to build and deploy this application at all. Now they can. That is real leverage.

But the leverage changes where the risk sits. Development time may go down while cloud spend, maintenance, security exposure, or reliability risk goes up. Those costs often arrive after the demo is working and the application is in use.

A short review can be a very good investment

The answer is not to ban AI-generated code or require every idea to go through a long development process. The answer is to put experienced eyes on the parts that determine how software behaves in the real world.

For a data-backed application, that review can be straightforward:

• What runs on a schedule, and how often does it actually need to run?
• Which queries execute, on what warehouse, and how much data do they scan?
• Can the application cache results, react to an event, or reuse an existing data set instead of polling?
• What is the expected cost at normal usage and at a failure mode?
• Which alerts will tell us when the application behaves differently from expected?

A review like this does not need to take longer than the work it is reviewing. In this case, it could have prevented a four-figure surprise. More importantly, it creates a habit of treating an AI-generated application as software that will be operated, not just code that needs to run once.

Keep the human in the loop

There is a familiar parallel with outsourcing. Lower-cost implementation can be a good trade when the work is understood and the output is reviewed. It becomes expensive when the apparent savings mean nobody owns the architecture, the quality, or the ongoing consequences.

AI assistance is similar. It can make capable builders out of more people, and that is something to embrace. But it can also produce slop, inefficiencies, security risks, and bugs that are easy to miss because the first version looks complete.

The goal should not be to slow people down. It should be to pair the speed of AI with monitoring that catches surprises and with people who understand the systems, costs, and tradeoffs behind the code.

This is the first in a series of examples from the gap between shipping software quickly and operating it well. AI can help you build faster. Make sure someone is also asking what the resulting system will cost to run.

Claude, Codex, and the rest are useful debugging partners. They can suggest hypotheses quickly, explain unfamiliar systems, and keep an investigation moving when you are stuck.

They can also send you on a very convincing rabbit hole.

We ran into that with a client’s ASP.NET application. After it had been up for a while, the first request for a static JavaScript file could be very slow. Requests after that were fast. It was the kind of narrow, intermittent behavior that invites a long list of theories.

Claude’s initial diagnosis was that SSL certificate revocation checking was holding up the first request. That is a real thing worth knowing about, and it sounded plausible in the abstract. But the application was using a self-signed certificate. There was no certificate authority revocation check to perform. A small piece of domain knowledge ruled out a direction that otherwise could have consumed hours.

The problem was not that Claude mentioned certificate revocation. The problem would have been treating a confident, technically detailed answer as evidence.

Start with what the system is doing

Rather than follow the SSL theory, we tested the behavior we could observe. We read the assets directly from the filesystem and fetched the asset through the application. The pattern was consistent:

1. Fetch an asset and the first request is slow.
2. Fetch it again immediately and it is fast.
3. Edit the file, then fetch it again, and the next request is slow again.

That is a much more useful description of the issue than “static JavaScript is slow.” The expensive path was associated with first access to changed file content. It was not ordinary request handling, and it did not fit the TLS explanation.

The experiment strongly pointed to endpoint scanning. SentinelOne was running in that environment and was the most likely cause: changed content was likely being scanned on its first access, while the next read benefited from the result already being available. We did not treat that as a definitive vendor-level attribution, but it fit the observed behavior far better than revocation checking did.

Plausible is not proven

LLMs are especially good at producing plausible explanations. They have seen the vocabulary around a symptom, and they can connect it to a real mechanism. That is useful for generating a list of things to investigate.

But a diagnosis has to survive the details of the actual system:

• Does the proposed mechanism exist in this deployment?
• Does it explain the timing and repeatability of the symptom?
• What inexpensive test could distinguish it from the other hypotheses?
• What observation would prove it wrong?

A self-signed certificate was enough to make us stop and question the revocation theory. The cold-read, warm-read, and modified-file test gave us a better hypothesis to pursue. Neither step required an encyclopedic knowledge of every possible cause. They required knowing enough to check the assumptions and to design a small experiment.

Use the model as a partner, not an authority

Claude still helped with the investigation. The right use was not to ask it for the answer and implement the first response. It was to use it as a partner while we compared theories against the environment and the measurements.

A practical debugging loop looks like this:

1. State the observation precisely, including what changes between a slow request and a fast one.
2. Ask the model for competing hypotheses and a test that would separate each one.
3. Check its assumptions against the architecture, configuration, and operational environment.
4. Run the smallest useful experiment.
5. Feed the result back in and repeat.

This is also why domain expertise still matters when using LLMs. If you cannot tell whether an answer fits the system you are operating, confidence and detail are easy to mistake for correctness. Bring in someone who knows the domain, or slow down enough to validate the model’s premises before chasing its conclusion.

The model can make a good investigator faster. It cannot replace the judgment needed to decide whether a theory belongs in the investigation at all.

There has been a lot of discussion recently about companies using AI to build internal tools that replace SaaS licenses. That is interesting, but it misses a big category of software: the web apps you cannot replace.

Sometimes the constraint is technical. More often, it is not. An insurance company may require you to use its verification portal. A specialty vendor may only accept orders through a clunky ecommerce site. Or a marketplace may be where all of the demand for your product or service lives.

You can build a better internal tool, but you still have to use those sites.

The browser is the integration point

Chrome extensions have always been a way to change the experience of a site you do not control. An extension can read information from a page, add controls to it, and help guide a user through a workflow.

Historically, that was possible but often not practical. You needed to write and maintain custom code for every awkward workflow, and the payoff had to be large enough to justify it.

The latest AI models change that calculation. It is now much easier to build an extension that augments a legacy site, whether that means changing a workflow, extracting information from a page, or adding LLM capabilities directly where people are already working.

Instead of asking someone to copy information from one system into another, you can put the assistance in the browser tab where the work already happens.

A bike search on Facebook Marketplace

I recently had a good excuse to try this out. I was looking for a bike on Facebook Marketplace with a specific set of requirements. The hard part was not finding listings. It was reviewing the photos for each listing to determine whether a bike was actually a fit.

Doing that manually meant opening and reviewing dozens of listings every day. That is exactly the sort of repetitive visual task that an AI model can help with.

So I built a Chrome extension that uses OpenAI to review listing photos and flag the listings that match what I was looking for. Rather than replacing Facebook Marketplace, the extension improves the part of the Marketplace workflow that was taking the most time.

The result is not a fully autonomous bike buyer. It is a faster way to narrow down a large list of listings so I can spend my time looking at the promising ones.

Where this approach works

The Marketplace example is personal, but the pattern applies to business workflows too. Look for web-based processes where a person repeatedly has to review, classify, summarize, or move information before they can make a decision.

A Chrome extension can be a practical place to add help to:

• an insurer’s required portal
• a vendor ordering site
• a marketplace your team depends on
• an internal legacy application that is difficult to change

The goal is not necessarily to replace the site. It is to remove the tedious steps around it while keeping people in the workflow they already need to use.

See it in action

Check out the demo below to see the bike finder at work:

Watch the video on YouTube

Interested in building something similar for a workflow your team cannot avoid? Get in touch with Setfive.

I’ll preface this by saying that I know just enough about machine learning to be dangerous and get myself into trouble. That said, if anything is inaccurate or misleading let me know in the comments and I’ll update it. Last April Amazon announced Amazon Machine Learning, a new AWS service aimed at developers to help them build and deploy machine learning solutions. We’ve been excited to experiment with AWS ML since it launched but haven’t had a chance until just now.

A bit of background

So what is “machine learning”? Looking at Wikipedia’s definition machine learning is ‘is a subfield of computer science that evolved from the study of pattern recognition and computational learning theory in artificial intelligence. In 1959, Arthur Samuel defined machine learning as a “Field of study that gives computers the ability to learn without being explicitly programmed”.’ That definition in turn translates to using a computer to solve problems like regression or classification. Machine learning powers dozens of the products that internet users interact with everyday from spam filtering to product recommendations to Siri and Google Now.

Looking at the Wikipedia article, ML as a field has existed since the late 1980s so what’s been driving its recent growth in popularity? I’d argue key driving factors have been compute resources getting cheaper, especially storage, which has allowed companies to store orders of magnitude more data than they were 5 or 10 years ago. This data along with elastic public cloud resources and the increasing maturity of open source packages has made ML accessible and worthwhile for an increasingly large number of companies. Additionally, there’s been an explosion of venture capital funding into ML focussed startups which has certainly also helped boost its popularity.

Kicking the tires

The first thing we need to do before testing out Amazon ML was to pick a good machine learning problem to tackle. Unfortunately, we didn’t have any internal data to test with so I headed over to Kaggle to find a good problem to tackle. After some exploring I settled on Digit Recognizer since its a “known problem”, the Kaggle challenge had benchmark solutions, and no additional data transformations would be neccessary. The goal of the Digit Recognizer problem is to accept bitmap representations of handwritten numerals and then correctly output what number was written.

The dataset is a modified version of the Mixed National Institute of Standards and Technology which is a well known dataset often used for training image processing systems. Unlike the original MNIST images, the Kaggle dataset has already been converted to a grayscale bitmap array so individual pixels are represented by an integer from 0-255. In ML parlance, the “Digit Recognizer” challenge would fall under the umbrella of a classification problem since the goal would be to correctly “classify” unknown inputs with a label, in this case a 0-9 digit. Another interesting feature of the MNIST dataset is that the Wikipedia provides benchmark performance for a variety of approaches so we can have a sense of how AWS ML stacks up.

At a high level, the big steps we’re going to take are to train our model using “train.csv”, evaluate it against a subset of known data, and then predict labels for the rows in “test.csv”. Amazon ML makes this whole process pretty easy using the AWS Console UI so there’s not really any magic. One thing worth noting is that Amazon doesn’t let you select which algorithm will be used in the model you build, it selects it automatically based on the type of ML problem. After around 30 minutes your model should be built and you’ll be able to explore the model’s performance. This is actually a really interesting feature of Amazon ML since you wouldn’t get these insights with visualizations “out of the box” from most open source packages.

Performance

With the model built the last step is to use it to predict unknown values from the “test.csv” dataset. Similar to generating the model, running a “batch prediction” is pretty straightforward on the AWS ML UI. After the prediction finishes you’ll end up with a results file in your specified S3 bucket that looks similar to:

1,6,0,4,7,3,5,8,9,2
1.544274E-10,3.736493E-6,1.298402E-4,3.529298E-8,2.738585E-7,1.814797E-5,3.520103E-6,7.861468E-6,1.17829E-6,9.998354E-1
2.196675E-11,7.322348E-6,9.969799E-1,3.165914E-10,1.485307E-5,4.171782E-6,2.970602E-3,6.487699E-6,1.930486E-7,1.643579E-5
1.345541E-3,1.968907E-4,1.209908E-5,7.132479E-2,1.572009E-3,7.926991E-4,3.921966E-2,2.677833E-2,8.559376E-1,2.820424E-3
8.210548E-6,2.666948E-3,6.711699E-3,5.835555E-2,2.407767E-1,3.312858E-5,1.107812E-3,5.861267E-4,6.673768E-1,2.237697E-2
1.205649E-3,1.128586E-2,4.179959E-4,4.942253E-5,2.8488E-4,7.751592E-1,6.43877E-3,8.812359E-3,1.996288E-4,1.961462E-1

Because there are several possible classifications of a digit the ML model generates a probability per classification with the largest number being the most likely. Individual probabilities are great but what we really want is a single digit per input sample. Running the input through the following PHP will produce that along with a header for Kaggle:

<?php

$lines = explode("\n", file_get_contents("results_test.csv"));
$header = str_getcsv($lines[0]);

unset($lines[0]);

echo "ImageId,Label\n";
$imageId = 1;

foreach($lines as $ln){
    $ln = str_getcsv($ln);
    for($i = 0; $i < count($ln); $i++){
        $ln[$i] = (float) $ln[$i];        
    }
        
    $maxIndex = array_search(max($ln), $ln);
    
    echo $imageId . "," . $header[$maxIndex];
    
    if($imageId < count($lines)){
        echo "\n";
    }
    
    $imageId += 1;    
}

And finally the last step of the evaluation is uploading our results file to Kaggle to see how our model stacks up. Uploading my results produced a score of 0.91671 so right around 92% accuracy. Interestingly, looking at the Wikipedia entry for MNIST a 8% error rate is right around what was academically achieved using a linear classifier. So overall, not a bad showing!

Takeaways

Comparing the model’s performance to the Kaggle leaderboard and Wikipedia benchmarks, AWS ML performanced decently well especially considering we took the defaults and didn’t pre-process the data. One of the downside of AWS ML is the lack of visibility into what algorithms are being used and additionally not being able to select specific algorithms. In my experience, solutions that mask complexity like this work great for “typical” use cases but then quickly breakdown for more complicated tasks. Another downside of AWS ML is that it can currently only process text data that’s formatted into CSVs with one record per row. The result of this is that you’ll have to do any data transformations with your own code running on your own compute infrastructure or AWS EC2.

Anyway, all in all I think Amazon’s Machine Learning product is definitely an interesting addition to the AWS suite. At the very least, I can see it being a powerful tool to be able to quickly test out ML hypothesis which can then be implemented and refined using an open source package like skit-learn or Apache Mahout.