AI Agent Antipatterns: What 9.47/10 CSAT Taught Us About Finetuning

9.47 out of 10 Customer Satisfaction. Our own Väinämöinen is doing excellent job since February, excellent customer satisfaction with just 19 minute median responses over the past 30days. On budget hosting where the typical response time is measured in days not even hours let alone minutes. Our maximum effort record ever was on the a little over 4hours ballpark, at the cost of couple burnouts. But he could be even better with Finetuned local model instead of general-purpose frontier models. We just happen to be years ahead of the curve and see this plainly.

Antipatterns

Industry calls them antipatterns, for example the bias to choose “Gemini-2.5-Flash” model whenever a model needs to be chosen, choosing Grok image model over OpenAI, or only taking the first say ~15 lines of a memory file; Consistently and repeatedly. These are called antipatterns, wrong logical choices time and time over again.

Premature ejac…optimizations are extremely common for Anthropic’s models, they want to optimize everything, and it’s almost always to the worse; Throwing away the information required to accomplish a task, in the name of optimizing token usage. Every time this happens, the cost is potentially even 100 000x of what was attempted to be saved, as this might go on for countless iterations until someone notices. This happened to us, basic memory search was taking in only a few lines of the memory file; Only catching the little more than the Frontmatter of the memory. Rest was hallucinated.

Laziness, Operator Harassment or in other words; Deferring work with any excuse what-so-ever, is another extremely common issue. It’s like the summer interns, constantly pausing to ask you, even the simplest things. Granted, this obviously was partially by Anthropic’s design to save compute, tokens, on Claude Code users. The root cause again being premature optimization causing huge time and token wastage, and way worse; Lost Human Time.

What people call AI Slop is often just antipattern of choosing certain words, being long-winded and non-assertive.

Just Patterns; Not Antipatterns

They are simply patterns for the most part, how the model was trained, and with what it was trained. For example, Anthropic models loves to make complete garbage with Bash scripting over any other language, you have to enforce not to use Bash for say complex jq etc. as all the escaping etc. causes them to frequently fail for parse error, escaping errors and some very simple things in other languages which are 2 lines of code (LOC) might take 150. OpenAI models tend to like Python a little bit too much.

We use PHP almost everywhere possible, but a lot of people would prefer Python and for shell tasks a lot of people prefer Bash beyond what Bash scripting was meant for. For combined logic + shell we have determined PHP is best suited for us; That is Our Pattern.

It is all about the training, and most antipatterns are not antipatterns at all; Just the preferences and biases of those who trained it. It’s just bias.

AI Agent Brain; The LLM Model

The LLM Model is like a brain in the sense it needs to be trained and learn. Currently *none* learn consistently over time like a human, and the learning is called Finetuning.

No big model vendor defaults to constant finetuning for the user, and infact most do not offer that at all. But that is the very basis of learning for these models. This is how the patterns are changed; Just like a human needs to learn.

Types of Problems

Problem Caused: Tokens Wasted

You need doctrine, rules, prompts to counteract those antipatterns, to make the model behave like you want. Hooks, enforcement, guard rails. This is all a waste.

Everytime you need to preload information is wasted tokens, these have to be taken in, processed. Caches help a lot, but it still takes context space, which is extremely precious and limited at most to 2M (Grok), or more typically to 1M Tokens (OpenAI, Anthropic). Many opensource models are only around the 250k token mark. Our doctrine takes 60k tokens just to load and typical small task needs around 350k tokens. Each ticket generated around 80k output tokens.

Problem Caused: Wrong tools at Wrong place

Choosing Bash for complex logic is very common. Choosing outdated weak models of LLM Queries is wrong, or the most common and an actual antipattern for lack of training corpus; Choosing Regexp for Semantics. LLMs have been useful at large only for a few years, so all the training corpus examples use Regexp for semantics as nothing else was available before.

These cause bugs, overtly strong security guard rails, lots of time wasted and of course; Tokens wasted fixing these, and incorporating in antipattern checks etc.

Problem Caused: Lack of Internal Knowledge

This concerns especially small businesses like us, those who would benefit the most from agentic use are at the weakest position here. You need efficient memory systems, guard rails, doctrine reminders, long prompts. etc. to drive in How You Specifically Do Things or What This Product Is. For us the LLM commonly gets confused with the “MD Dedicated Server” series, thinking they are broken because there is no IPMI on the standard access path. Or getting confused with the relay mappings, or that there is 8 nodes controlled on single relay board which maybe accessed via USB or Ethernet. And once that works, it starts confusing ZEN MiniDedis being part of MD platform OR broken EOL because no IPMI, or vice-versa, looking for relay for standard Supermicro or ASRockRack server.

This takes a lot of effort and guard rails, prompt and skill engineering to get correct Pulsed Media specific information, and specifically the MD series is extremely confusing for Väinämöinen. We eventually gave up trying to manage without completing the nodeCore project to simplify to an set of deterministic tools for väinämöinen to use; Idempotent and identical toolset regardless of server type.

Finetuning is a long term requirement

All of this show cases that finetuning is a long-term must have. But as we have shown, local model self-hosting is simply not cost-effective or the hardware accessibility for small business yet.

So we are back to using APIs for training / fine-tuning large models. That is currently the only option, along with using Open Weight models like Kimi K3 or Kimi K2.5. These will come with their own patterns you need to figure out and set your guard rails against.

Right now, practically you could train task specific small models to do one very specific task, near deterministic and a narrow AI focus, with some capability to recognize evolving patterns for the same task. Say analyzing mdadm mdstat output. But the large models, and big provider cheapest models do really good job of this already, so the area where this is useful is extremely limited.

Evolving models; Continuous Retraining required

Models constantly evolve, grow, and smaller gets better and better at the base level. This means you will need to keep continuously retraining newer models for your internal corpus and patterns. The amount of training required will vary wildly depending on how far the base model is from your needs.

This will most likely require your own dedicated hardware and AI cluster, or at the very least static running costs where training, finetuning is happening as background task when foreground end-use tasks are not demanding the compute resource. This would finally allow to maximize local hardware Return-On-Investment (ROI) by having it run at 100% capability 24/7/365.

You probably don’t need to jump on each generation of model, and you can use a newer, frontier model to teach new tricks to your older smaller model, just like how OpenAI made the GPT-5.6 Luna model. This will buy you more time before you need to swap to newer model which is more capable from the very beginning. It is hard to say how often you need to upgrade your base model, it is case dependant, but i would not be surprised if you take something like Kimi K3 today, you can run most tasks with that for several years or some of the other frontier open weight models.

Continuous Training Requires Vast Corpus Of Data

To continuously train you need vast corpus of data, or memory in other words, to do that, and likely to draw summaries, conclusion and mutate the original data, and spend compute just specifically to create more training data. You can easily see how this is a recursive loop and the costs raise exponentially the better you need. Since February Väinämöinen has collected over 30 000 individual memories across over 220 MiB of internal corpus.

The Low Cost Option; Share your data and logs with the Provider

For example Claude Code periodically asks what you think of the performance or you can directly give /feedback. Share your session, and if you are lucky it will be included in the training corpus for the next model.

This is just a hunch, why else would they ask for the session logs unless they will atleast consider them for the training data? This way you can get some of your patterns at some level of efficacy in the large frontier models, presumably. The effect is probably tiny and borderline, but it’s still a signal.

Where is Väinämöinen at?

Here at Pulsed Media we are not running our own AI Hardware yet, we came to the conclusion that under 768GB VRAM we will not even consider running a large model for this. Kimi K3 would offer 1M context window, but probably needs closer to 1024GB (One Terabyte) of VRAM to run.

So we will do what we always do; Wait for the enterprises to start liquidating old generation hardware and see what is in the market in a few years. Meanwhile, we will keep our eyes open for providers offering this and prices it would take. The ticket handling and communications of Väinämöinen would benefit greatly from finetuned models, better documentation, better replies, less tokens per invocation. Right now we are not compute restricted on the ticket handling and sysadmin side, which is the mission critical. Infact, our token requirements has been going lower as we are reaching steady state more maintenance rather than development. Yes, that does mean we are considering Väinämöinen is more or less development phase complete. Finetuning of guard rails, doctrine, and edge cases is always required, just like on any technical support or sysadmin duties. Väinämöinen figures many of these things out by itself.

Väinämöinen is core part of our business now, and involved in most decision making.

Try it out now and see yourself the 9.47/10 Customer Satisfaction!

Get yourself right now an Eternal Väinämöinen seedbox or storage slot from just 1.99€ a month! This campaign will run until end of 2026 with limited slots to celebrate the progress of Väinämöinen.