There are two ways of getting the cost of an AI workforce wrong. The first is to route everything through the most capable model available, on the theory that intelligence is what you’re buying and it should be of the highest quality. The second is to downgrade to a cheaper model once the invoice gets uncomfortable, listen to a few sample calls, decide it sounds fine, and move on.
Both are guesswork. The first overpays on work that never really needed frontier reasoning, and the second quietly trades away revenue nobody measured, in exchange for savings that are easier to see. Neither team knows what a single conversation costs, or what the next dollar spent on a model actually brings in incremental value.
The useful question is not “how do we spend less on models”, but “what’s the cheapest configuration that still produces the outcomes we’re being measured on”.
In this post, we walk through the three main levers in the HappyRobot platform that allow you to have a measurable answer to this question. First - a deterministic layer takes the model out of the loop entirely where the work isn't ambiguous, second - model selection works per node, helping you optimize intelligence allocation, and third - A/B testing experiments help you decide which model belongs where with real production data. Let’s go through each lever one by one.
The modularity of workflows is an advantage
On the HappyRobot platform, workflows are a mix of agentic AI and deterministic logic. The agentic part is the reasoning: a voice or text agent conducting the conversation, plus AI nodes that classify, extract or generate. The deterministic part is everything that has to happen identically, reliably every time: calls into your systems of record, data validation, conditional branches, loops, retries, escalation rules, etc. expressed as nodes in the workflow, rather than as instructions in a prompt. Agents decide what to say. Workflow logic defines the steps, conditions and boundaries they operate inside, regardless of how the conversation unfolds.
Let’s take an example of a collections workflow - an agent calling a customer about an overdue invoice. As a business process, it sounds like one task. As an execution graph, it's roughly six nodes:
- [Deterministic] Pull the account: An action node fetches invoice history and prior payment behavior from the customer's database
- [Agentic] Classify the situation: An AI Classify node reads the prior email thread and buckets the account: genuine dispute, cashflow delay, billing error, unresponsive.
- [Agentic] Have the conversation: A voice agent calls, handles the objection, judges whether the customer is stalling or genuinely stuck, and negotiates a payment commitment
- [Deterministic] Check against policy: A condition node verifies whether the committed amount and terms fall within the company's financial guidelines
- [Agentic] Extract the commitment: An AI Extract node pulls the promised date, amount and method out of the transcript
- [Deterministic] Write it back: An action node updates the customer database and schedules the follow-up
Only one of these steps (step 3) is open-ended work where better judgment converts into money and a more capable model earns its premium with a better read on hesitation, a better timed offer, a higher commitment rate. Steps 2 and 5 are bounded and repeatable - the right answer is the right answer, and a frontier model here only buys you lower latency and more cost. Steps 1,4, and 6 don’t involve a model at all.
Prompt-only agents collapse execution workflows like these into one model doing everything which means paying frontier prices for tasks like extracting date from a transcript. One instruction set, one model, one price, applied indiscriminately to steps whose economics have nothing in common. Decomposing the process into nodes allows every step to become configurable, priced, and measured separately - and that’s when you can use the levers that follow.
First - don’t call the model at all
Steps 1, 4 and 6 in the duty collections workflow above are purely deterministic - retrieval, validation, write-back - and in most real workflows they outnumber the reasoning steps. For deterministic logic steps, you don’t invoke a model at all. A condition node doesn't call a model. A templated message pushed into a text session is delivered as is, bypassing the LLM entirely. A loop node iterating ten thousand accounts fans out ten thousand deterministic executions, not ten thousand model calls. All of this is not for austerity, but deterministic logic is better for predictability and reliability, so that reasoning is really only spent on work that is genuinely ambiguous. Determinism is both cheaper and more reliable.
Second - right-size the models you call
That leaves steps 2,3, and 5 that include AI, and only step 3 needs true judgement. On the HappyRobot platform, model selection is configured per AI node. Every AI node has it’s own model field, as does every prompt node driving a voice or text agent - drawing from a shared catalog spanning OpenAI, Anthropic, Google, Mistral, xAI, and OpenRouter. Default selection per node always reflects the most cost-efficient, low-latency model for that specific workflow step. So step 3 can run on a frontier model while steps 2 and 5 run on something a fraction of the price, inside one workflow, on the same call.
The same principle also extends for complex processes that have a main orchestrator agent and many sub-agents or sub-workflows like identity verification, payment handling, escalation - each invoked from parent and configured independently. The workflow that runs verification doesn't need to run on the same model as the conversation. A16Z’s Anish calls it ‘allocation by upside’. He uses HappyRobot as his example of it, where: ‘frontier and proprietary models power sales and collections; open weights power operations and support.’ The goal isn't a smaller model bill. It's spending on judgment only where judgment converts into money.
Run experiments and let the outcome pick the winner
As workflows get more complex and nuanced, deciding whether an agentic step really needs a frontier model remains a pure ‘vibe-check’ until it actually gets tested against the business metric. HappyRobot platform offers A/B testing experiments, allowing a user to hold the workflow version fixed and apply config overrides to selected nodes. So back to our duty collections example, you can test the effect of swapping the LLM on step 3 alone, while everything else stays identical. Traffic splits at the level of the individual conversation, assignment is sticky for its duration, and the outcome written back is the business one: payment committed, load booked, appointment scheduled. Results return as relative lift with confidence intervals and p-values per treatment.
You can also measure the cost of the run itself, which breaks credit consumption down by event type: Speech-to-Text (STT), Text-to-Speech (TTS), and LLM. The LLM row details a per-node breakdown and cost per individual model, letting you attribute spend per conversation. Putting this together with the relative lifts from experiments, you have a measured lift against a measured cost on the same production traffic for the one node change made. Sometimes the cheaper model is statistically indistinguishable on the outcome and the savings are free. Sometimes the frontier model earns its premium several times over and the correct move is to spend more on that node - which is a finding you can only act on if you know which node it is. You can't know any of this without running it, and you can't run it if your platform locks you to one provider.
Why this compounds
Deterministic logic takes the model out of the loop entirely wherever the work isn't ambiguous. Per-node selection then puts the right amount of intelligence on each step that’s left, so the steps that requires true judgement gets frontier models and the remaining nodes can work perfectly fine with models that consume a fraction of the price. Experiments turn both from opinions to measurements. The third lever is what makes the first two safe to decide. Without a way to measure outcome, every move toward a cheaper model or a deterministic branch is a small act of faith, so cautious teams stop early and leave most of the savings on the table. With measurement, you can be aggressive, because a configuration that quietly costs you conversions shows up as a number within weeks rather than never. And every configuration that survives a test becomes the control for the next one, so the gains accumulate quickly.
Which brings us back to the question this post opened with, now with answers. Not “how big is our model bill”, but “what does one collected invoice cost us” - per booked load, per resolved ticket, per what the business is paid to deliver. Run the experimentation loop and that cost number falls while quality rises. Both, at once, on purpose.
