← Back to Insights

Insight

The Serial Fraction

Ariel Agor
•
The Serial Fraction

Listen · Read by Ariel · click any word to jump

0:00 / n/a
· loading…

In September a San Francisco startup called TypeSafe AI released a model that refuses to write. It is called Jev. You give it a question and a fixed set of answers, it hands back one of those answers with a confidence number, and it adds nothing else. According to Fortune, nearly 13% of Vercel's paid teams had tried it within a day of it landing on Vercel's AI Gateway. Cloudflare released two open-weight rivals, Clef and Clef-flash, on October 1. OpenAI followed on October 6 with the Decisions API, and OpenAI's Nikunj Handa credited Jev with "inspiring this whole thing." In under a month the industry built a new category of software whose whole output is a choice.

Two days after OpenAI's launch, VentureBeat published survey numbers that point the other way. Among respondents at organizations that run autonomous agents, the share that allows production changes on automated checks alone, or is building toward it, fell from 75% in July to 56% in August. The share expecting to keep a human reviewer for the foreseeable future rose from 20% to 42%.

Read those two facts side by side and you have the most useful lesson of the year about increasing decision velocity with AI, even though nobody has framed it that way. The machines got faster at deciding while the companies got slower at letting them. A company decides only as fast as the slowest step it runs in sequence, and in most firms that step is a person with a full inbox. Engineers already have a name for this, and executives should borrow it.

A model that only answers the question

Decision models deserve a careful look, because they change what a single decision costs.

A chat model writes. You ask whether a support ticket is urgent and it gives you a paragraph, which your code then has to parse, hoping the paragraph contains a word it recognizes. A decision model skips the paragraph. OpenAI's announcement on its developer forum on October 6 described the product plainly: "Let your app choose the right model, tool, or action in near real-time with Decisions API, now available to all developers in public beta." It supports three kinds of question. A predicate estimates the probability that a statement is true. A choice picks from options the developer defines, with confidence scores. A score rates an input on a numeric range.

Cloudflare's launch post, written by Michelle Chen, puts it in one line: "A decision model makes classifications to help agents decide how to act, based on certain probabilities." Cloudflare published its own latency figures. Clef-flash, the 9 billion parameter model, had a median latency of 38.8 milliseconds. The larger Clef came in at 209.3 milliseconds and Jev at 524.1. In an internal threat intelligence test, Clef fetched, rendered and classified a domain in 2.2 seconds, while gpt-oss-120b, Cloudflare's fastest general model, took 4.7 seconds and returned only two classifications.

The prices are close to nothing. Fortune reports that OpenAI charges $0.10 per million input tokens and Jev charges $0.042, and neither charges for output. TechTarget set Jev's price beside Claude Fable 5.1 at $10 per million input tokens and $50 per million output tokens. Paolo Rosson of Dext, quoted in Fortune, put the economics in a single sentence: "you pay for a full written answer when all you need is a label." He estimated Jev's calls cost his company seven cents per 1,000 changes. Ashutosh Mathore of Atlensa named the habit the new category is designed to break, describing places "where we may be using LLMs out of habit even when the task is not really a language problem."

So the cost and delay of one machine judgment have dropped below what any executive would bother measuring. Torsten Volk of Omdia told TechTarget that Jev can run entire decision loops about 10 times per second, where an LLM would need several seconds.

Most companies will read this as good news about speed, and they will be disappointed, for reasons a computer architect worked out decades ago.

Amdahl's law has an org chart

Amdahl's law says that when you speed up one part of a system, the overall gain is capped by the share of time the system spends in the parts you did not touch. Make one stage infinitely fast and the work still waits on every other stage. The part that has to run in sequence, the serial fraction, sets the ceiling.

The decision model market has already run this experiment on itself. TypeSafe markets Jev as "193.6x faster." GreenNode, a cloud provider, published an analysis describing a TypeSafe employee who swapped one step of a real pipeline for Jev. The whole system got 15.9% faster. GreenNode does not give the pipeline's details, and the vendor number and the test number measure different things, which is the point. A step can become a couple of hundred times faster while the system around it barely moves, because the other steps still take most of the time.

Now apply the same arithmetic to a company. Suppose a distributor approves credit for a new trade customer in five business days. Suppose the analysis inside that process, the part a model could do, takes four hours of an analyst's time, and everything else is waiting: for a manager to read the memo, for finance to sign, for the weekly credit meeting to come round. Suppose you make the analysis instant. The cycle falls from five days to a little under five days. The customer, who was choosing between you and a competitor that answers by Thursday, notices nothing.

Most enterprise AI programs are in this position. They spend their budget on the parallel part of the work, the reading, drafting and classifying that a model can do, and leave the serial part exactly as it was. That serial part is the chain of people who must each say yes before anything happens. The board then asks why the AI investment produced no visible change in cycle time, and the honest answer is that nobody touched the denominator. (I am using the word loosely here. The formula has its own denominator; mine is the queue of signatures.)

Why the humans came back

The VentureBeat survey would be easy to read as timidity. It is better read as a rational response to a measurement problem.

Taryn Plumb's write up of the VentureBeat Intelligence data, published October 8, gives the reasons. Of the respondents whose organizations run pre-deployment evaluations, 61% reported at least one case in the past 12 months where a feature "passed internal testing but subsequently caused a customer-facing failure." The most cited trust concern was misalignment with real-world outcomes, at 27%, followed by lack of explainability at 24%. When respondents named the area where they expected reliability spending to grow most, human review workflows came first at 30%, ahead of production observability tooling at 26% and automated evaluation pipelines at 21%. VentureBeat's own summary says that for a growing share of respondents, "that threshold has not yet been met."

The sample is small, 140 respondents in August, and self-selected, and VentureBeat says so. The direction still matters. Companies have watched agents pass their tests and then fail in front of customers, and their response has been to add a person back into the sequence. Each added person increases the serial fraction, so these companies are buying safety with speed at a fixed exchange rate and paying for it in cycle time.

They do this because they cannot price the risk of leaving the person out. If you do not know how often the automated judgment is wrong, or what a wrong judgment costs, a human signature is the only control you can explain to an auditor. A human reviewer is expensive and slow, and it is also the only control most companies know how to staff.

Decision models matter because they give companies a way to price that risk, though most buyers have not yet noticed.

Increasing decision velocity with AI is a threshold problem

A chat model gives you prose, and prose comes with no error rate attached. A decision model gives you a probability. Probabilities can be calibrated, and a calibrated probability can be turned into a policy.

Cloudflare's launch post lists its training recipe, and one item stands out to anyone who has worked in risk: a Brier loss. The Brier score punishes a model for being confidently wrong, so a model trained against it learns to make its stated confidence match how often it turns out to be right. When a calibrated model says 0.93, roughly 93 of every 100 such calls should be correct. That property is what lets you remove a human from part of the sequence without lying to yourself.

Once you have calibrated probabilities, every decision flow splits at a threshold. Above the line, the system acts. Below it, a person looks. The threshold is a single number, and it encodes your whole appetite for that category of error. Raise it and more decisions go to people while fewer mistakes get through. Lower it and the reverse happens. Increasing decision velocity with AI comes down to moving that number on purpose, one decision type at a time, with evidence.

The dial belongs to whoever eats the loss

In most companies running these systems today, the engineer who wrote the integration set the threshold, often by picking a round number that felt safe. Engineers should not own that call, any more than a junior analyst should set a bank's credit limits.

The threshold on a refund decision is a statement about how much fraud the business will tolerate to make honest customers happier. The threshold on a vendor onboarding check is a statement about supply chain exposure. These are risk appetite decisions, and they belong with whoever answers for the loss: the CFO, the head of risk, the general manager of the line. That person should see the threshold as a named number in a register, with the date it was last changed and the evidence for the change.

When the threshold sits in a register, the human review step is no longer a habit but a priced control. You can tell the audit committee how much error the company accepts in exchange for how much speed, and you can change the exchange rate in a meeting instead of in a sprint.

Size the queue before you build it

A threshold also tells you how many people you need. Suppose your refund flow handles 10,000 decisions a month. Suppose that at a threshold of 0.9, a calibrated model clears 8,200 of them automatically and sends 1,800 to people. Suppose each review takes six minutes. That gives you 180 hours of review a month, which you can staff, schedule and hold to a service level, the way a call center plans for Monday morning.

Most companies that "add human review" never do this sum. They put a person in the loop with no service level, the queue grows, and the median decision time ends up set by vacation calendars. A queue you can size is a queue you can manage. A queue that simply happens to you becomes your serial fraction.

Move the human behind the decision

The move that changes cycle time most is to change where the person stands. A reviewer who approves every item before it ships is a serial step. A reviewer who samples items after they ship, checks them against outcomes and adjusts the threshold works in parallel. The decision goes out at machine speed, and the human still governs it, through the policy and through the record of what happened.

The VentureBeat data shows a few companies already working this way. Of the 38 respondents who allow unreviewed production changes, 10 named automated quality checks on live outputs as their main monitoring approach. That group is small. They have stopped asking a person to approve each decision in advance and started asking a person to own the error rate afterward.

Not every decision should move. A decision you cannot undo, such as a wire transfer, a termination or a regulatory filing, should keep a human in front of it no matter how good the model is. A decision you can undo cheaply, such as routing a ticket, tagging an invoice or offering a discount below a ceiling, should almost never wait on a person. The work is to sort your decisions by how reversible they are and how costly an error would be, then let the reversible ones go fast.

The calibration record is the asset

A threshold you cannot defend will be pushed back up the first time something goes wrong. A threshold is defensible only if someone recorded what actually happened after each automated decision: the refund that turned out to be fraud, the ticket marked low priority that became a churned account.

Cloudflare's launch describes the plumbing for a version of this. Its new reinforcement learning service, run at first by a forward deployed engineering team, uses AI Gateway to capture traffic as a dataset, Containers as a sandbox to score agent actions, and a new Trainer to update weights before the model is redeployed on Workers AI. Cloudflare says it does not "read, store, or train on your requests or responses" unless the customer opts in to fine-tuning. The tooling exists. What most companies lack is the habit of joining a decision to its outcome weeks later, in a table someone owns.

That table is the most valuable thing a company can build in this category, and it is the one thing a vendor cannot sell you. The model is a commodity priced in fractions of a cent per million tokens, and next quarter there will be a cheaper one. Your record of which of your decisions turned out right, at which confidence, under which threshold, belongs only to you. It is the evidence that lets a risk committee lower a threshold without feeling reckless, so it is also what removes people from the serial path.

Executives who want faster decisions should ask for this record before they ask for another model. If nobody can show the error rate of last month's automated decisions, the human reviewers are staying, and they should.

What fast deciding will not fix

Decision models have limits, and the people closest to them say so. Ned Bellavance told TechTarget, "if you need a deterministic flow, Jev is not going to give you that." The same experts noted that Jev is not a drop-in replacement for an LLM and that it is harder to explain than one. Aaron Roy of GlutenOrNot told Fortune that getting good outcomes requires shaping questions and prompts differently from traditional LLMs. The VentureBeat respondents ranked explainability as their second biggest trust concern.

These limits fall hardest on the decisions that matter most. A model choosing among five options cannot tell you that the right answer was a sixth option nobody wrote down. It cannot tell you that the question was wrong. Strategy, pricing architecture, which market to leave: none of these can be squeezed into a typed question, and a company that tries will make its biggest choices quickly and badly.

The opportunity is in the other decisions, the thousands of small ones each day that have always been typed questions dressed up as judgment calls. Should this ticket go up a level? Does this invoice match this purchase order? Is this login suspicious? Those decisions used to be expensive enough to batch into queues and important enough to need a signature. Now they cost a fraction of a cent and take less time than a page load. A company that still routes them through a person's inbox is choosing, by default, to be slow.

Where the speed actually lives

The vendors have done their part, and done it quickly. TypeSafe went from launch to two serious competitors in three weeks. Prices are falling so fast that the price on any given contract is close to irrelevant. Every capable engineering team can now buy a fast judgment.

None of them can buy a shorter sequence. The serial fraction lives in your approval policies, your delegation of authority matrix, your habit of waiting for the weekly meeting and your audit committee's comfort level. It lives in who has the right to say yes, and in whether that right is attached to a person or a number. None of it is software, so none of it improves when the software does.

The companies that turn October's decision models into real speed will do four unglamorous things. They will list their recurring decisions and sort them by reversibility. They will give each automated decision type a threshold owned by a named executive. They will move reviewers from in front of decisions to behind them wherever the decision can be undone. They will record outcomes so the thresholds can move. Everyone else will buy the fastest model on the market and see their cycle time improve by something like 15.9%.

Build the sequence before you buy the speed

Increasing decision velocity with AI is a redesign of who is allowed to decide what, at what confidence, with what evidence. Buying the tool takes an afternoon. Rewriting the delegation of authority, setting thresholds that a CFO will sign, sizing the review queue and building the outcome record takes months of work across finance, risk, operations and engineering, and it has to be designed as one system, because a threshold with no outcome record will not hold and a review queue with no service level becomes the new bottleneck.

Do this work now, while the models are cheap and your competitors are still celebrating milliseconds. The firms that redraw their serial path this quarter will answer customers in minutes, and the firms that do not will answer at the speed of their slowest signature, however fast the model behind it runs.

Agor AI Advisory does this architectural work: we map your decision chains, find the serial fraction, write the threshold policy with the executives who own the loss, and build the outcome record that lets you take people out of the sequence without taking them out of control. Schedule a strategic consultation with us today.

Sources

The step got 193.6x faster. The system got 15.9% faster.

This verifies the post's central claim that a system's speedup is capped by the steps still run in sequence, using the only published pairing of a vendor speedup and a whole-pipeline result. After 15 seconds the reader should see that machine judgment got dramatically faster while the human-review path grew, so end-to-end speed barely moved.

  • A step can get 193.6x faster while the system around it gets 15.9% faster.
  • Model latency fell to milliseconds in the same months that the share of companies expecting a human reviewer doubled, from 20% to 42%.
  • The caveats are shown on purpose: the numbers are small and loosely matched, but they all point the same way.
Artifact type trace is staged but not yet rendered. The data is captured in the sidecar JSON.

Source: Figures and quotes taken from the post's cited sources: Cloudflare Blog (Oct 1, 2026), TechTarget (Sep 22, 2026), GreenNode, and VentureBeat (Oct 8, 2026). · verified · as of 2026-10-28

Want this kind of automation working for your business?

Agor AI designs and ships the systems these posts describe, scoped in weeks, not quarters.

Book a Free Strategy Call