A drafted support reply usually gets one round of scrutiny before it goes out, usually by the same person who wrote it, in a hurry. Tone problems, unauthorized promises, and replies that should have gone to a manager first tend to surface later, in a QA pass, well after the message is already sent.

Most AI writing assistants solve a different problem than this one. They generate text. Here, the agent already wrote the reply. What's needed is a fast, structured judgment about text that already exists. The system should be able to answer critical questions, such as is the tone right, is there a risky commitment buried in it, does this need another set of eyes before it sends.

Coercing a chat model into that kind of verdict means generating prose and parsing it back into something code can act on, adding latency and a parsing failure mode for something that should be a plain yes, no, or which-option answer.

Use code HACKERNOON for $5 in hosting credits if you want to try this yourself.

A model built to decide, not to talk

TypeSafe AI's Jev, released in September 2026, is built for this shape of problem specifically. It isn't a chat model. You send it a state and a set of typed questions, and it returns typed, calibrated answers directly in the form of a choice, a score, or a probability. There is no lengthy wall of text, and certainly no need to programmatically parse one to find a single-word answer.

We built a small live co-pilot on top of it. A support agent types a reply, and four separate judgments about that reply update while they type, before the message ever sends.

Four questions, one call

Each of the four signals maps onto one of Jev's three typed primitives, Choice, Score, and Noul. All four run as a single call against one shared state:

const response = await client.systemOne({
  state: {
    customer_message: customerMessage,
    draft_reply: draftReply,
  },
  questions: {
    tone: choice("What tone does `draft_reply` take toward the customer?", {
      empathetic: "Acknowledges the customer's frustration or situation directly",
      neutral: "Matter of fact, no emotional acknowledgment either way",
      curt: "Short and transactional, reads as dismissive",
      defensive: "Justifies the company's position rather than addressing the customer",
    }),
    makes_commitment: noul(
      "Does `draft_reply` promise a specific refund amount, replacement, or timeline the company may not be able to guarantee?"
    ),
    needs_review: noul(
      "Given `customer_message` and `draft_reply`, should a manager review this reply before it is sent?"
    ),
    predicted_satisfaction: score(
      "How satisfied is the customer likely to be after reading `draft_reply`, given `customer_message`?",
      [
        "Likely to escalate or complain further",
        "Neutral, may follow up again",
        "Satisfied, unlikely to need further contact",
      ]
    ),
  },
});

Bundling questions this way follows the pattern TypeSafe's own documentation recommends. Every question in a request runs in parallel and independently against the same state, so adding more questions barely changes response time.

One of TypeSafe's published cookbooks reports batching several questions into a single call at roughly ten times cheaper and ten times faster than firing them separately, with no change in the answers. A single yes-or-no flag doesn't demonstrate that. Four differently shaped answers from one call does.

The actual cost of running all four is small enough to stop thinking about. At $0.042 per million input tokens, with output free, a state and four questions built from a few hundred words costs a small fraction of a cent per evaluation, cheap enough to run on every keystroke pause rather than something to ration.

The instrument panel

The interface has two halves: a plain paper-toned panel where the agent writes, and a dark instrument rail where the four judgments live. Two numbers sit at the bottom of that rail, round trip time from keystroke pause to answer on screen, and Jev's own response time. The difference between them is what the hosting is responsible for. Jev's number is fixed by TypeSafe's infrastructure regardless of where the app runs.

Watching it work

A reply that sounds apologetic but still makes a specific promise is a good starting test, since it separates tone from risk cleanly. This one reads as empathetic and still gets flagged:

A more explicit version of the same promise, dollar amount included, reads cooler in tone, but the risk signals hold:

The more interesting result came from a reply that makes no promise at all:

Commitment risk is correctly clear here. Nothing in this reply promises anything. But needs-review reads higher than either message that actually made a promise. That points to the model weighing more than surface-level risk phrases: telling a customer who's already been charged twice to go argue with their own bank is review-worthy on its substance, whether or not it contains a specific risky sentence. A keyword-based check would have missed this reply entirely.

Reading confidence, not just the answer

Choice and Score answers carry a second value beyond the winning option itself. The confidence score from 0 to 1, describing how concentrated the whole probability distribution is, not just which option came out on top. Noul deliberately has no separate confidence field, since its own probability already carries that meaning. A value near 0.5 already means uncertain, a value near 0 or 1 already means confident.

The dismissive reply above is a good example of why this matters. Tone read Curt at only 54 percent, close to a coin flip against whatever came in second. Right now the interface just shows the winning label. A sharper version treats a low confidence value as its own signal, independent of what any single answer says:

if (answers.tone.confidence < 0.6) {
  // Flag for a second look regardless of what tone itself came back as.
  // A confident "curt" and an unconfident "curt" aren't the same finding.
}

This is the mechanism TypeSafe's own docs describe as confidence-gated routing. This lets high-confidence answers pass straight through, and route low-confidence ones to a person, rather than treating every answer as equally certain just because it came back with a label attached.

What the cache bug taught us

The first version of the in-memory cache checked whether the current draft matched the last settled result. That breaks down when two requests for identical text land close together, one while the first is still waiting on a real network call to Jev. A settled-only check misses the overlap: neither request sees a matching settled result, both fire independent calls to the real API, and if those two calls land with slightly different confidence values (real model variance, not a defect), the result is two different answers for what was meant to be one evaluation.

The logs from testing showed exactly this happening:

[req 4] text: "...Thanks"
[req 4] cache MISS, calling Jev
[req 5] text: "...Thanks"
[req 5] joining in-flight call
The fix tracks in-flight calls as a separate state from settled ones. A second identical request arriving while the first is still running now waits on that same pending call instead of starting a new one, so both resolve to the same result by construction:
js
if (draftReply === pendingDraft && pendingPromise) {
  const result = await pendingPromise;
  socket.send(JSON.stringify({ requestId, ...result, cached: true }));
  return;
}

The gap that's supposed to be there

Locally, on a loopback connection with no real network between browser and server, round trip and Jev's own response time sat within one to four milliseconds of each other. In production, that gap runs between 150 and 250 milliseconds across every message tested. This isn't a regression, and the hosting architecture isn't adding it. It's the real network path between a visitor's browser and Cloudways' servers, something a localhost test can't surface. Stating that plainly here keeps a reader from mistaking a wider gap for slower hosting.

Getting it into production

We will use Cloudways Velocity to run this demo. The idea is to use a managed environment so that we can focus on testing the app rather than wasting an hour setting up the test server environment and getting the tooling ready.

Push the repo to GitHub with the following commands:

git init
git add .
git commit -m "Initial commit"
git push -u origin main

Connect it to Velocity:

and it detects the Express and ws setup from package.json on its own:

The build log confirms it: dependencies installed, PM2 starts the app.

The real API key never touches the repository. It lives only in Velocity's environment variables:

Why this behaves differently on typical serverless hosting

A fresh function instance behind a cold start adds latency variance on top of whatever the model itself takes, and a long-lived connection either can't be held open at all or gets recycled between messages. This isn't an argument against serverless in general, plenty of workloads are genuinely spiky and fit it well. It's specific to an app that needs to feel instant while someone types, repeatedly, for as long as the page stays open.

Who this is for

A support team that wants a live quality signal without standing up a full review queue is a good fit, as is a team building internal tools who wants a working template for keeping a call to Jev warm between requests.

A single broad, vague prompt trying to catch everything at once is not a good fit, nor is a state that needs to survive across multiple servers rather than one process's memory. Jev's own guidance applies directly here. Ask for the kind of judgment a knowledgeable person could make in a few seconds given the right context, and decompose anything larger into separate questions combined in code. 

Extending the state: judging the whole conversation, not just the last line

Right now the state sent to Jev is just two fields: the customer's most recent message, and the draft reply. That's enough for the four questions here, but not for a real life implementation of this tool that has to deal with conversation threads rather than single messages. TypeSafe’s allows you to reference specific parts of a larger, structured state by path.

So if you wish to set this up for a more practical support scenario, instead of a flat customer_message string, the state can carry the full ticket, and each question can point at exactly the part of it that matters:

export async function evaluateDraft(ticketMessages, draftReply) {
  const response = await client.systemOne({
    state: {
      ticket: { messages: ticketMessages },
      draft_reply: draftReply,
    },
    questions: {
      tone: choice(
        "What tone does `draft_reply` take toward the customer, given the full conversation in `ticket.messages`?",
        {
          empathetic: "Acknowledges the customer's frustration or situation directly",
          neutral: "Matter of fact, no emotional acknowledgment either way",
          curt: "Short and transactional, reads as dismissive",
          defensive: "Justifies the company's position rather than addressing the customer",
        }
      ),
      makes_commitment: noul(
        "Does `draft_reply` promise a specific refund amount, replacement, or timeline the company may not be able to guarantee?"
      ),
      needs_review: noul(
        "Given the conversation in `ticket.messages` and `draft_reply`, should a manager review this reply before it is sent? Consider whether this is a repeat issue for this customer."
      ),
      predicted_satisfaction: score(
        "How satisfied is the customer likely to be after reading `draft_reply`, given the full conversation in `ticket.messages`?",
        [
          "Likely to escalate or complain further",
          "Neutral, may follow up again",
          "Satisfied, unlikely to need further contact",
        ]
      ),
    },
  });

  return response.answers;
}

ticketMessages here is just an array shaped like [{ from: "customer", text: "..." }, { from: "agent", text: "..." }], built from whatever ticketing system or conversation log the app already has. Jev doesn't require any particular format beyond valid JSON.

Note that needs_review can now weigh whether this is a repeat issue, because the model can see the customer already said so earlier in the thread, not because the question itself changed to look for that phrase. The judgment gets sharper by giving it more of the real conversation, not by writing a longer instruction.

The one thing this changes upstream is the server needs to receive and forward a message history instead of a single string, a small change to the WebSocket payload and whatever builds it client-side. That part isn't shown here, since it depends entirely on where that history already lives (usually a ticketing system or a database). The snippet above is the whole of what changes inside the Jev integration itself. As with every LLM-based application, remember that a longer state also means more input tokens per call, not just more context.

Closing thoughts

Four typed questions, one call, one process holding a WebSocket open so nothing sits between a browser and Jev but code under direct control. The gap that matters is the one the hosting is responsible for, and on Velocity, that gap stays close to what the model itself takes.

The code for this demo can be found at the Github repository we connected to Cloudways Velocity.

Use code HACKERNOON for $5 in hosting credits if you want to try this yourself.