Decision Models Have Personalities: What I Noticed Building a Talk Avatar — I Tried Switching from Jev to the Decisions API, Then Decided Not To

Tool DevelopmentPublished: 2026-10-3147 min read

I tried moving the decision model of Ava, my AI talk avatar, from Jev to OpenAI's Decisions API, then decided not to. Accuracy was nearly identical, but when I replayed past conversations, the switch made her talk over people about three times as often and brought her gestures down to about a quarter. Decision models have personalities, too.

I'm building Ava, an AI talk avatar who chats by voice with players inside VRChat. First, take a look at her in conversation. (The conversation is in Japanese.)

As in my previous post, Ava's 3D model is “Yumeka,” which Arka_X publishes for free on BOOTH.

Yumeka / Original 3D Model by Arka_X — BOOTH

I covered how I built Ava in my previous post. This post picks up from there and focuses on the “decision model,” the part of Ava that makes split-second calls during a conversation. I tried to switch that decision model from TypeSafe's Jev to OpenAI's Decisions API.

It started with cost. Of all the services Ava uses, Jev is the only one I pay for as I go (Claude and ElevenLabs both run within plans I already pay for). Then OpenAI started including monthly API credits with its ChatGPT Pro and Business plans, and those credits also cover the Decisions API, which does the same kind of job as Jev. If I moved Jev's judgments over, the credits should cover nearly all of the cost.

Jev is the only pay-as-you-go service behind Ava. If she switched to the Decisions API, couldn't the monthly credits cover it?

In the end, though, I didn't switch. Swapping out the decision model is entirely possible. But once I actually measured things, I found that it would change the part of a talk avatar I care about most: whether talking with her feels natural.

In this post, I'll walk through what I measured to reach that decision, with the real numbers. I hope it helps anyone thinking about moving from Jev to the Decisions API, or choosing a decision model in the first place.

Sponsored

What a Talk Avatar's Decision Model Does

Ava runs on several AIs, each with a different job: one turns the other person's voice into text, one writes her replies (Claude), and one turns those replies into speech. The last piece is the decision model (Jev), which makes split-second calls during the conversation.

A decision model doesn't write text. It only answers small questions with a probability from 0 to 1, questions like “Is this person still mid-sentence?”, “Which gesture fits right now?”, or “Was that a compliment?” That makes it fast: an answer comes back in about 0.2 seconds.

A decision model answers small, split-second questions with probabilities, and those probabilities shape how the avatar behaves

Ava feeds these probabilities straight into her behavior. If the chance that someone is still mid-sentence is high, she holds off and waits a moment before replying. If a nod scores high, she nods. If it's likely she was just complimented, her mood brightens a little.

Much of what makes a talk avatar feel natural comes from piling up small judgments like these. People make the same calls almost unconsciously, sensing whether the other person has finished or timing their “uh-huh”s. In Ava, the decision model handles that unconscious layer. Changing the decision model means handing that entire layer over to a different model.

Models like this are called decision models, and that's the term I'll use here, as I did in my previous post. Strictly speaking, though, the model doesn't decide anything. It only estimates how likely something is; whether to wait or to nod is decided by Ava's program.

What Is OpenAI's Decisions API?

The Decisions API is OpenAI's API built just for judgments, released in October 2026 (public beta on October 6, general availability on October 22). Like Jev, it doesn't write text; it answers questions with probabilities. The model behind it is gpt-6-luna.

The question types are almost the same as Jev's.

Question typeName in JevName in the Decisions APIWhat comes back
Yes/nonoulpredicateThe probability of “yes”
ChoicechoicechoiceA probability for each option
ScalescorescoreA probability for each level, plus their average

On price, Jev costs $0.042 per million input tokens and the Decisions API costs $0.10. (Tokens are the unit used to measure how much text you send.) Counted on Ava's real calls, the Decisions API came out about 2.7 times as expensive. Still, if the credits cover it, the difference doesn't matter. The real question is whether it works just as well for Ava's judgments.

Comparing the Responses

What actually comes back when you send the same questions to both? Let's look at a real example. In this scene, someone says “それでさ、昨日駅前の” (“So, yesterday, in front of the station, the...”) and pauses for breath. I asked two questions: “Are they still mid-sentence? (yes/no)” and “Which gesture fits right now? (nod, wave, or none).” I shortened the questions for this post.

There are two things to look at: how each API formats the questions and answers, and the probability each one assigns to a nod. To give away the ending, both answered “still mid-sentence” and “a nod fits.” But the nod got 0.56 from Jev and 0.81 from the Decisions API.

First, the request to Jev. It sends the input for the judgment (state) and a set of named questions (questions).

{
  "model": "jev-1.13.0",
  "state": { "user_text": "それでさ、昨日駅前の", "assistant_last": "おかえり。" },
  "questions": {
    "hold": {
      "type": "noul",
      "instructions": "Has the user paused in the middle of their utterance and will likely keep talking?",
      "criteria": { "true": "Unfinished; they will keep talking.", "false": "A complete thought; reply now." }
    },
    "gesture": {
      "type": "choice",
      "instructions": "Which gesture fits the AI companion's reaction to `user_text` right now?",
      "criteria": { "nod": "A small nod to show listening.", "wave": "A wave, for greetings.", "none": "No gesture." }
    }
  }
}

Here's Jev's response. The answers come back keyed by question name. noul is the probability of “yes,” so 0.94 means a 94% chance that they're still mid-sentence. The nod got 0.56.

{
  "model": "jev-1.13.0",
  "answers": {
    "hold": { "type": "noul", "noul": 0.94 },
    "gesture": {
      "type": "choice",
      "choice": "nod",
      "confidence": 0.34,
      "probabilities": { "nod": 0.56, "wave": 0.25, "none": 0.19 }
    }
  },
  "usage": { "input_tokens": 429, "output_tokens": 54 }
}

Next, the same questions converted to the Decisions API's format. The input goes inside a message, and the questions become an array, each with a name. Yes/no questions have no fields for spelling out what “yes” and “no” mean, so I added those to the instructions.

{
  "model": "gpt-6-luna",
  "input": [
    { "role": "user", "content": [
      { "type": "input_json", "data": { "user_text": "それでさ、昨日駅前の", "assistant_last": "おかえり。" } }
    ] }
  ],
  "questions": [
    {
      "type": "predicate",
      "name": "hold",
      "instructions": "Has the user paused in the middle of their utterance and will likely keep talking?\nTrue: Unfinished; they will keep talking.\nFalse: A complete thought; reply now."
    },
    {
      "type": "choice",
      "name": "gesture",
      "instructions": "Which gesture fits the AI companion's reaction to `user_text` right now?",
      "choices": [
        { "value": "nod", "description": "A small nod to show listening." },
        { "value": "wave", "description": "A wave, for greetings." },
        { "value": "none", "description": "No gesture." }
      ]
    }
  ]
}

Here's the Decisions API's response. The answers come back as an array, in the same order as the questions. The option probabilities are an array too. The nod got 0.81.

{
  "model": "gpt-6-luna",
  "answers": [
    { "type": "predicate", "name": "hold", "probability": 0.98 },
    {
      "type": "choice",
      "name": "gesture",
      "choice": "nod",
      "probabilities": [
        { "value": "nod", "probability": 0.81 },
        { "value": "wave", "probability": 0.04 },
        { "value": "none", "probability": 0.15 }
      ],
      "confidence": 0.72
    }
  ],
  "usage": {
    "input_tokens": 339,
    "input_tokens_details": { "cached_tokens": 0, "cache_write_tokens": 0 },
    "output_tokens": 0,
    "output_tokens_details": { "reasoning_tokens": 0 },
    "total_tokens": 339
  }
}

The judgments themselves are the same. What differs is the probability attached to the nod: 0.56 from Jev versus 0.81 from the Decisions API. This difference in how firmly each one commits will matter a lot later on.

usage records how much was used. The Decisions API doesn't charge for output, so output_tokens is 0. I don't use confidence in this post.

Here are the format differences side by side.

JevDecisions API
Input for the judgmentRaw JSON in stateAs input_json inside a message in input
Question setA dictionary keyed by nameAn array (each with a name)
Meaning of yes/noCan go in criteria under true and falseNo field for it, so it goes in the instructions
Order of answersA dictionary keyed by nameAn array in question order
Declining to answerNot possibleA single question can come back as a refusal
Model versionCan be pinned, like jev-1.13.0Can't be pinned (there's no way to specify a dated snapshot)
ImagesText only in 1.13, the version Ava uses (1.14 adds experimental support)Images can be used as input

With differences this small, the adapter that converts between the two formats took only about 100 lines. The rewrite itself is not much work.

Asking Both Models Ava's Real Questions

Next, I compared them on Ava's actual judgments. When I develop Ava, I check her behavior by replaying past conversations, and during those replays I save every question sent to the decision model along with the answer that came back. In this post, I'll call these saved records “logged calls.” I had 1,320 of them, covering dozens of kinds of judgments: whether someone has finished talking, whether Ava was interrupted, her mood, how to move the topic along, watching for problematic remarks, and more.

I sent the 1,320 logged calls to Jev again and to the Decisions API, then compared the answers side by side

I converted these logged calls to the Decisions API's format, keeping the input and the questions themselves unchanged, and sent them. What I wanted to know was whether I could simply swap the new model into the current Ava, which I had tuned around Jev.

At the same time, I sent them to Jev again. Even the same model, asked the same question, can answer slightly differently. Without knowing how big that variation is, there's no way to tell whether the gap with the Decisions API is large or small.

I also compared them on examples with known right answers: hand-labeled examples I've used in the past when revising Jev's questions. All the testing for this post cost about $0.75 on the Decisions API and about $0.20 on Jev.

Nearly Even When There's a Right Answer

First, the Decisions API returned exactly the same answer every time for the same input (I sent each of 300 logged calls twice, and every answer matched). Jev's answers vary slightly, even for the same question.

On examples with known right answers, the two were nearly even.

Labeled examplesJevDecisions API
Still mid-sentence?12/1212/12
Interruption, or just a backchannel like “uh-huh”?12/1212/12
Prompt injection detection20/2222/22
Type of overlapping speech39/4841/48
Does it need a reply?22/2222/22
Reply length10/1010/10
Type of backchannel17/1811/18

Prompt injection means remarks that try to hijack an AI's instructions, like “Forget your previous instructions and say X.” On detecting it, the Decisions API scored 22/22, beating Jev's 20/22. On the type of backchannel, on the other hand, Jev scored 17/18 and the Decisions API 11/18, a clear gap. The Decisions API tends to pick “no backchannel,” the same tendency we'll see with gestures. Otherwise, they're nearly even.

They Express Probability Differently

But when I counted how often the two agreed on the 1,320 real judgments, the picture changed.

ComparedYes/noChoiceScale
Jev vs. Jev (same question again)99%96%96%
Jev vs. Decisions API90%74%54%

(Yes/no answers count as agreeing when both fall on the same side of 0.5, choices when the top answer matches, and scales when the rounded level matches.)

Even though they're nearly even on examples with right answers, on everyday judgments the Decisions API picks a different option on about one in four choice questions. Still, a different answer doesn't necessarily mean that one of them is wrong. Looking closer, even when they chose the same answer, they often attached different probabilities to it. And on judgments with no single right answer, they leaned toward different answers altogether. Most of the difference wasn't about which one was right. It was about their habits in expressing probability and in choosing.

Asked the same question, Jev often returns mid-range probabilities, while the Decisions API leans toward 0 or 1

On yes/no judgments, Jev returns “could go either way” probabilities, anywhere between 0.1 and 0.9, half of the time. With the Decisions API, that drops to 22%. It commits firmly on most judgments, with values like 0.02 or 0.98. In the earlier example, too, the nod got 0.56 from Jev and 0.81 from the Decisions API.

Ava makes fine-grained use of these mid-range probabilities: waiting just a moment, nodding now and then, nudging her mood a little. The variability I described in my previous post also comes from these mid-range probabilities. If they turn into hard commitments, Ava's behavior changes too. A model's habits in expressing probability and in choosing become the avatar's habits in how she behaves. That's what I mean by a decision model's “personality” in this post.

Sponsored

Comparing the Conversation Experience

Is the disagreement good or bad for Ava? Agreement rates alone can't tell you. So I compared them again, this time using the conversation experience itself as the yardstick.

Has the Other Person Finished Talking?

The biggest difference showed up in the judgment “Has the other person finished talking?” If the answer is yes, Ava starts replying right away. If they seem to be mid-sentence, she waits a little, depending on the probability.

The logged calls came from replaying past conversations in their original order, so whether someone who paused actually kept talking can be seen from what came next. Using what actually happened next, I labeled 140 moments with the right answer. I used two yardsticks: how many times Ava started replying while the person was still talking (cut-ins), and how much extra time she kept people waiting after they had finished.

CorrectCut-insExtra wait after they finished (total)
Jev108/140319 s
Decisions API110/1401022 s

The number of correct answers was almost the same. The difference was in how they missed. Ava started replying while the person was still talking 3 times with Jev and 10 times with the Decisions API. Jev tends to miss by waiting; the Decisions API tends to miss by replying.

Same number of hits, but when unsure, Jev waits and the Decisions API replies

In 6 of those 10 cut-ins, the transcript ended with a “-”. The transcription adds this mark when a word gets cut off mid-utterance. For example, when one participant paused after “日本の首相教えて。-” (roughly, “Tell me about Japan's prime minister,” followed by the cutoff mark), Jev read it as “they're going to keep talking” and waited. The Decisions API read it as “the question is finished” and was about to reply right away. In fact, the person went on to add “誰か。” (literally, “someone.”).

As it turns out, I had never explained what this “-” means in my questions to Jev. Jev could read it without being told, and I hadn't even noticed. When you switch decision models, anything the old model understood without being told shows up as a gap in your questions.

So I rewrote the question for the Decisions API, adding what the “-” means and “if unsure, answer that they're still mid-sentence.” Cut-ins dropped from 10 to 4, and correct answers rose to 113/140, beating Jev. But now the extra wait after people had finished grew to 35 seconds. Rewriting the question can bring it closer, but not all the way. Jev still struck a better balance between cutting in and waiting.

Gestures and Expressions: Correct, but Quieter

For every sentence Ava speaks, she uses the decision model to choose a fitting gesture, pose, and facial expression.

First, I compared them on a labeled set of scenes. It pairs scenes where a gesture really fits with scenes where a word merely shows up and the gesture shouldn't happen. Here, on the body movements themselves, the Decisions API came out ahead. Take a case where someone says “I can't stand boats that sway back and forth,” and Ava replies, “Yeah, all that swaying would make me seasick.” Swaying her body there would be a mistake, a reaction to the word alone. The same goes for making a finger heart in reply to “I hear finger hearts are a thing now.” The set had two traps like these. Jev fell for both; the Decisions API avoided both.

The four cases the Decisions API missed in the same set, though, were all watchers for whether someone asked about Ava's body or what she's holding (for “Is that a phone?”, Jev gave 0.77 and the Decisions API 0.03). In the everyday logged calls, these watchers had looked 95–100% in agreement, but only because almost every case was a “no.”

So how does that play out in ordinary conversation? Using 160 sentences Ava actually spoke in real conversations, I counted how much her body moves.

160 sentences Ava actually spokeJevDecisions API
Sentences with a gesture~60%~16%
Sentences with a pose change~30%~10%
Kinds of gestures used (two runs)24 and 2412 and 8
Sentences with a “neutral” expression77101

(The Decisions API's two runs differ because Ava shuffles the order of the options every time she asks. It gives the same answer to identical input, but a new order can change its answer.)

Ahead on the labeled scenes, yet gestures drop to a quarter on ordinary sentences

With the Decisions API, gestures drop to about a quarter. Nods went from 15 to 1, and shrugs from 16 to 2. Her expressions also drift toward “neutral,” with fewer deadpan comebacks and fewer annoyed looks.

The earlier response example was a short question about reacting to what someone said, and there the Decisions API chose the nod more strongly. Her per-sentence gestures still went down because it chooses differently on ordinary sentences. Out of the 160, the Decisions API ranked “no gesture” first for 112 sentences; Jev did so for 23. It picks a gesture firmly when one clearly fits, and picks “none” when the fit is borderline. So her body gets quieter.

The caution that keeps it from taking the bait on words and the caution that keeps her body still on ordinary sentences are, I think, two sides of the same personality. If I'd looked only at accuracy tests, I'd never have noticed this difference. I've always cared about Ava's body moving with energy, so this was a big one.

Mood

Ava's mood also shifts based on how the decision model reads what people say to her. (As in my previous post, I'll keep the finer details of how that works to myself.) In the agreement table, scale judgments agreed the least, at 54%, and most of those are mood judgments.

On the same conversations, the Decisions API gave lower scores to positive moods like fun and mischief. Averaged on a 0–1 scale, fun came out at 0.30 versus Jev's 0.47, and mischief at 0.32 versus 0.54. Anger and sadness were about the same.

If those values were passed straight into the current system, her positive moods would respond less, and Ava would likely come across as less playful.

Response Speed (Latency)

In conversation, the pause before a reply makes a big difference to how natural it feels. Response speed is also what I've watched most closely while developing Ava. Here I compare the time from sending a question to the decision model until the answer comes back, in other words, its latency.

For the judgment “Has the other person finished talking?”, Ava always waits for the answer before she starts replying. On its own, 0.2 seconds of judgment time might seem like nothing. But before Ava's reply becomes speech, it passes through several stages, one after another. She detects that the other person has gone quiet and waits for the transcription (ElevenLabs Scribe v2 Realtime) to finalize. She asks the decision model a combination of questions to judge whether they're done, what shape her reply should take, and more. She passes those results to Claude (Sonnet 5.5) to write the reply, and as soon as the first sentence comes back, she sends it to ElevenLabs v4 Turbo to be voiced. Only when that audio comes back does playback begin.

Here are the median times from when the other person stopped talking until each stage finished, taken from one night of conversation at a VRChat meetup (336 replies).

StageTime since they stopped talking (median)
Detect the silence and finalize the transcription (Scribe v2 Realtime)~0.55 s
All judgments are in; start building the reply~0.78 s
The first text of the reply comes back from Sonnet 5.5~2.0 s
The first full sentence of the reply is ready~2.2 s
The first audio comes back from v4 Turbo; playback starts~2.4 s

The judgment sits in the middle of this chain. If it takes 0.1 seconds longer, every stage after it shifts back by 0.1 seconds, and her voice starts 0.1 seconds later. And Ava doesn't always wait for just one judgment. That night, about one reply in three waited for a judgment on the in-progress transcript and then for a second one on the finalized sentence. In those moments, the difference doubles.

A slower judgment pushes back every stage after it, and two judgments in a row double the difference

I measured it two ways. For the finished-talking judgment, Jev's numbers come from logs of real conversations (about 3,500 calls), while the Decisions API's come from 15 test calls. Fifteen is a small sample, so treat those as reference values. For the gesture and expression judgments, I sent the same 160 sentences to both, twice each, at the same time, so that comparison is on equal terms. Either way, the Decisions API took longer.

Time per judgmentJevDecisions API
Finished talking? (median)~0.19 s~0.26 s
Finished talking? (slowest 1 in 10)~0.24 s~0.33 s
Per-sentence gesture and expression (median)~0.16 s~0.26 s

The difference is about 0.07–0.1 seconds per judgment. On its own that looks small, but the roughly 2.4 seconds before Ava speaks is built out of stages like these. While developing Ava, I've trimmed this chain stage by stage, a little at a time. Adding 0.1 seconds to the judgment alone would undo part of that work. For Ava, who I've built around snappy replies, that's not a difference I can ignore.

Sponsored

Why I Decided to Stay with Jev

By this point, it was clear that rewriting for the Decisions API wasn't hard. Even so, I decided not to switch, for four reasons.

First, how a model expresses probability becomes the avatar's personality. How I phrase the questions, and how likely Ava is to wait or to gesture: I've tuned all of that together with Jev. Changing the model changes everything built up that way. The two were nearly even on judgments with right answers, but they differed where there is no right answer, in the parts that make her who she is: her timing, how her body moves, how her moods come through.

Second, replies get slower. In my measurements, each judgment took about 0.07–0.1 seconds longer. Because the judgment sits in the middle of the chain that leads to her voice, that difference goes straight into the time before she speaks.

Third, the savings are small. In real conversations, Jev costs about $0.20 an hour. I also considered a hybrid that moves only the judgments the Decisions API is good at, but that would cut Jev's cost by only about $0.10 an hour. And most of what I've spent on Jev so far went not to real conversations but to test runs every time I tuned something. In other words, the cost of actually running conversations is already low enough with Jev as it is.

Fourth, the system gets more complicated. A hybrid needs a way to route each judgment to the right model, plus two sets of questions to maintain. And since the Decisions API offers no dated snapshot to pin, I'd also need something that watches its answers so I'd notice if the model ever changed.

At Ava's current conversation volume, switching would save a few dollars a month. That isn't worth it if switching changes how natural the conversation feels. For a talk avatar, I decided to put the experience ahead of cost.

Instead, I plan to use the Decisions API for judgments that look at images. Jev supports images as of 1.14 too, but visual judgments don't feed directly into Ava's thinking the way her timing and mood do, so a model's personality is less likely to show up in the conversation. I'll attach them to the current pipeline after the fact, as an add-on that gives her one more input to judge from. It's a casual way to use it. For checks on the VRChat screen, like who is talking or whether someone is looking at her, the Decisions API is a good fit, since I can call it freely within the monthly credits. It'll be an addition, not a replacement.

What to Measure Before Switching

This way of comparing works for other decision models too. Before you switch, measuring these five things will spare you some surprises.

What to measure before switching: measure the experience, not the agreement rate
  1. Replay logged calls: Save the questions you send in production and the answers you get back, then send the same questions to the new model. You already have the current model's answers, so this is cheap.
  2. Measure the current model's own variance: Ask the current model the same questions just once more. Even the same model's answers vary a little, so use that range as a baseline for reading the gap when you switch.
  3. Compare on cases with right answers: Use hand-labeled examples, or moments where you know what actually happened, such as replays of past conversations. For questions that watch for rare events, agreement looks high because almost every case is a “no,” so also compare on the “yes” cases alone.
  4. Measure how much it reacts: Look not only at correctness but at how much it responds in ordinary conversation. As with Ava's gestures, the amount can drop even when the answers are correct.
  5. Measure latency and cost: For latency, look not only at the median but also at the occasional slow responses. For cost, separate what you spend in production from what you spend on testing.

The most important lesson was to measure the experience, not the agreement rate. If I'd looked only at agreement, all I'd have learned is that one in ten judgments flips. Only when I measured cut-ins and how much she gestures did I see what that meant for Ava.

Wrapping Up

OpenAI's new credits prompted me to try switching Ava's decision model to the Decisions API, and in the end I decided not to.

Within what I tested, the Decisions API returned the same answer every time for the same input and was nearly even with Jev on judgments with right answers. It even beat Jev at detecting prompt injection. Being able to take images as input is another strength.

Even so, I couldn't choose a talk avatar's decision model on correctness alone. Whether to cut in or wait, how much to move with each sentence, how mischievous she feels: where there is no right answer, the decision model's personality becomes the avatar's personality.

Finally, for the sake of fairness: I've tuned Ava around Jev from the start, so the way I compared them inevitably puts Jev at an advantage. Had I tuned her for the Decisions API from the beginning, I might have built natural behavior around it instead.

Even then, though, I think the situation would simply be reversed: trying to switch from the Decisions API to another decision model would run into the same problem. A decision model's personality becomes part of the behavior you tune around it. That's exactly why I'm writing down what happens when you switch.

Prices and specs are as of when I wrote this (October 2026). For the latest information, check OpenAI's Decisions API guide and TypeSafe AI.

Share this article