How I fixed a two-year bottleneck at Fleury in two days using System One Models

Hook

Estimated reading time: 6 min

I spent months tweaking prompts, chaining GPT calls, and stressing out trying to read medical orders at Grupo Fleury, only to watch our accuracy stall at 85%. It was infuriating. We were paying a hefty bill every month, waiting over two seconds per document, and still catching JSON errors right in the middle of our pipeline.

Then I decided to test an approach using SOM (System One Model), and basically pulled off two years of progress in two days. We broke through the 85% barrier and locked in 92% accuracy reading exams, while response times and token spend plummeted. If your team is still using generative LLMs to make simple decisions and classify typed text, you're burning cash for nothing.

The context

At Grupo Fleury, our product challenge sounded simple on paper but was a real pain in practice: take scanned medical order text, full of noisy OCR errors and truncated abbreviations, and map it to the exact exam codes in our official catalog.

Our old architecture really struggled here. We were running three sequential GPT calls (one for tests, one for order details, and a third for location) and then throwing everything into a BERT model to try to match raw text to the right code. Every single file took almost five seconds end-to-end. We were basically burning money on output tokens generating JSON, always dreading that the model would hallucinate a field or crash our parser.

Generative LLMs are just too slow and expensive for this. An autoregressive model sits there predicting the next token to spit out freeform text; putting it to work just to decide if an exam is present or not is basically using a sledgehammer on a screw. That's where System One Models (SOM), which gained ground with things like TypeSafe's Jev and Microsoft-Decision-1, changed our entire perspective.

SOM flips the logic. Instead of sending a prompt and waiting for text generation, we feed in the order text alongside typed questions and closed options. The model does a single forward pass and returns calibrated probabilities, whether that's a yes/no boolean, multiple choice, or a scored rating. No text generation, no schema breaks, and the response drops right into our software logic.

What happened

We locked ourselves in a room back in October 2026 for a two-day marathon. The goal was dead simple: rip out the heavy GPT-dependent lab test parsing step and wire single-pass inference straight in, testing Microsoft-Decision-1 and Jev directly through OpenRouter.

Our first shot was naive. We tried dumping the whole test catalog on the model at once so it could evaluate every option in exhaustive mode. When I opened the spreadsheet and ran the numbers on that raw test, I nearly fell out of my chair: running the full catalog would hit over eighteen thousand dollars a month at Fleury scale, Grupo Fleury scale. Total bust right off the bat.

I realized right then that we needed a cheap pre-filter before hitting the API. So I put together the S80Y strategy, building an adaptive K logic that sets the exact number of candidate acronyms based on mentions in the text itself, using local counting that doesn't burn a single cent in tokens.

With that up and running, we fired off 51 live runs against our closed set of 527 test documents, mixing real pseudonymized medical orders with some really nasty synthetic inputs.

The numbers backed it up immediately. The monthly bill for the lab test step, which used to run us $839 combining GPT and BERT, dropped down to $406 with S80Y. And the median response time fell from roughly 2.2 seconds to 1.20 seconds per file. It was basically pulling off two years of product improvements in just two days of hacking, proving the real impact of this new model on our software development.

The turning point

I realized dumping the entire test catalog onto the model at once was plain stupid. Desperately parallelizing everything was a dead end. Instead, we ran a quick local pre-filter in milliseconds before even touching the network. In our S80Y pipeline, we basically count how many acronyms have an exact match in the raw text (our nA) and dynamically calculate candidate volume: K = clamp(round(12 * nA + 10), 20, 120). If the doctor's note mentions two obvious tests, we send about 34 questions; if it lists ten, we widen the pool. We cut half the tokens without losing a single bit of clinical context.

The real trick with Decision-1 is its boolean primitive, noul. Instead of praying for the LLM to spit out valid JSON and paying through the nose for every generated token, we batch questions together and get back calibrated probabilities in one single inference pass. Output tokens literally cost zero.

Building the call against the Azure Foundry endpoint is super direct, without relying on any heavy SDK:

python


import json
import urllib.request

def avaliar_exames(texto_pedido: str, candidatas: list[dict], url: str, api_key: str):
    questions = {
        item["sigla"]:
            "type": "noul",
            "instructions": f"Was the {item['nome']} ({item['sigla']}) test requested in the prescription?",
            "criteria": {"true": "Test explicitly requested.", "false": "Not mentioned or ruled out."}
        }
        for item in candidatas
    }
    payload = json.dumps({"model": "microsoft-decision-1", "state": texto_pedido, "questions": questions}).encode("utf-8")
    req = urllib.request.Request(f"{url}/providers/microsoft/v1/systemone", data=payload, headers={"Authorization": f"Bearer {api_key}", "Content-Type": "application/json"})
    with urllib.request.urlopen(req) as resp:
        respostas = json.loads(resp.read().decode("utf-8")).get("answers", {})
    return [sigla for sigla, dados in respostas.items() if dados.get("noul", 0) >= 0.93]

This script sends the medical order inside state and evaluates candidate tests in parallel. Traditional software engineering takes back control. A plain if check with a 0.93 threshold filters approvals on the spot, with zero schema hallucinations, bringing our product up to 92% accuracy.

Lessons learned

After banging my head against so many test runs and benchmarks on the product, a few things became obvious. I can basically sum up the whole experience into four points that would've saved me sleepless nights if someone had warned me earlier.

First: shortening queries thinking you're saving tokens is a trap. In our excitement to cut down API costs, we tried stripping descriptions down to just the canonical exam name. The result was an ugly drop of almost 9 points on our F1 score. The model actually needs rich synonyms and clinical variations right in the criteria to make the call with confidence, especially when real doctor notes come full of messy abbreviations.

Second: if the answer fits in a closed set, using a generative LLM is throwing money down the drain. There's no point paying top dollar for output tokens and waiting two or three seconds to get a JSON with a fixed value. System One solves this in a single direct pass, costs nothing on the return payload, and gives you the data on the spot.

Third: cascaded decisions beat blind parallelism. Early on, the temptation was to fire everything off simultaneously. But I learned fast that breaking down responsibility into sequential steps with smart pre-filters works way better. You run quick triage first, and only spin up heavy logic where actual ambiguity shows up.

And finally, theoretical calibration doesn't hold up on its own in our Portuguese. A model might promise gorgeous accuracy on foreign benchmark tests, but with real medical terms, the distribution shifts. Freezing confidence thresholds without running a solid battery of tests on your own data guarantees a nasty scare in production.

What I'd tell my past self

If I could travel back a few months, I'd tap myself on the shoulder and say: stop spending weeks tweaking giant prompts just to solve what is, basically, a chain of conditional checks.

We wasted precious days trying to force generative LLMs to spit out clean JSON across long chains, stacking layer upon layer of validation just to deal with format hallucinations. Honestly, all we actually needed was a fast, calibrated score to know which path to take in the product flow.

Trust the classic split more: offload quick probabilistic inference to a System One model that decides in milliseconds, and keep your business rules inside traditional deterministic code. The real speedup in team delivery doesn't come from blowing up model size. It comes straight from simplifying the API contract.

Saves brain cells. And actually fixes the problem.

Final thoughts

To me, software engineering with AI is shifting direction. We spent way too long forcing LLMs to generate freeform text when, deep down, we only needed a fast, calibrated statistical meter right at the front door.

Seeing SOM unblock our product at Fleury showed me just how promising this tech is. You can cut costs and drop latency without relying on prompt acrobatics.

How does your team handle triage and classification right now? Are you still fighting high response times? Let me know in the comments. And if you know someone on your team who's sick of JSON validation blowing up in production every single week, send this post their way. The conversation is just getting started.