TurnWise targets credit errors in streaming video

TurnWise applies turn-level Group Relative Policy Optimization (GRPO) to streaming video, where a model repeatedly chooses to answer or wait as new frames arrive. Its central claim is specific: silence-completed prefix scoring estimates each decision’s group-relative remaining return, so a later reply is not trained using rewards earned before that decision.

The problem is that standard trajectory-level GRPO broadcasts one advantage across every turn. A redundant reply after an early success can inherit positive credit, while a useful follow-up after earlier failures can inherit negative credit. TurnWise separates the return from each turn onward from the reward already accrued, then weights response and waiting turns equally in the policy loss.

The method is evaluated on ProactiveVQA, OVO-Timing, and StreamingBench using Qwen2.5-VL-3B and Qwen3-VL-4B backbones. The authors report the best aggregate scores on all three benchmarks with both models, and say that turn-mean aggregation improves response timing while reducing repetition relative to credit-only optimization. A related video-RL direction emphasizes focused video reasoning, whereas TurnWise focuses on when to respond during a stream.

Streaming interaction differs from offline video question answering because the model acts on partial observations and must account for what it has already told the user. A response can be timely and useful at one point, while another response at a later decision adds no information. The learning signal therefore needs to reflect both answer quality and the timing of each response-or-wait choice.

The paper positions TurnWise against MMDuet2, ThinkStream, and StreamPro, which improve interaction rewards or assess response-level and trajectory-level quality. It argues that these reward designs still leave a key GRPO issue: a single trajectory advantage is shared across turns. TurnWise changes credit assignment and loss weighting rather than introducing a new trajectory reward. The broader distinction between shared sequence feedback and per-step credit also appears in turn-level credit assignment, although TurnWise applies it to streaming video response and waiting decisions.

The concrete failure case is a trajectory with a successful early answer followed by a redundant reply: the later action may still receive positive shared credit. The reverse can happen when a useful follow-up follows earlier failures. The authors’ correction subtracts group-relative accrued reward before each decision, allowing later turns to be judged by their remaining return.

Illustrative dialogue and advantage plot contrasting shared GRPO credit with TurnWise credit across turns.

The example contrasts a shared trajectory advantage with TurnWise’s remaining-return credit, showing how a later redundant reply can inherit earlier success under GRPO. The distinction motivates turn-specific learning signals.

Remaining-return advantages and equal turn weights

For each sampled trajectory, TurnWise keeps the first t actions and replaces every later action with silence, then scores that prefix with the same observation plan and evaluation horizon. The difference between this score and the full trajectory score gives the return from turn t onward. Group-relative normalization produces a turn-specific advantage without a learned critic. The remaining-return construction is related in granularity to Turn-PPO advantage estimation, but TurnWise derives its signal from silence-completed video prefixes.

The prefix scores reuse the existing trajectory evaluations: the authors state that a silent suffix changes neither retained responses nor their preceding replies. This avoids additional rollouts and judge calls. A limitation of the method’s credit signal is that it depends on the fixed trajectory scorer, including its response-quality, timing, repetition, and out-of-window penalties.

TurnWise also changes the GRPO objective. It averages token losses within each turn and then averages across turns, so a long textual answer does not outweigh a short waiting action merely because it contains more tokens. The two mechanisms address separate issues: remaining-return advantages determine what credit a turn receives, while turn-mean aggregation determines how much that turn contributes to optimization.

TurnWise pipeline from streaming inputs and sampled trajectories to turn advantages and turn-mean loss.

The overview connects silence-completed prefix scoring to group-relative remaining-return advantages, then applies turn-mean aggregation. It summarizes how TurnWise aligns credit and policy weighting with streaming decisions.

Benchmarks, comparisons, and reported evidence

Training starts from supervised fine-tuning on 100,874 instruction examples, then uses 3,983 streaming instances with groups of four sampled trajectories. The reward is PAUC-based and judged by gpt-oss-20b; the evaluation covers proactive answer quality, event-triggered timing, and real-time visual understanding. The paper reports one epoch of RL training for each configuration.

The named RL comparisons are MMDuet2, ThinkStream, and StreamPro, implemented in the same streaming GRPO framework, alongside the supervised models. TurnWise achieves the best aggregate scores on ProactiveVQA, OVO-Timing, and StreamingBench for both Qwen backbones. The supplied paper text does not include the numerical benchmark scores, so the size of its gains cannot be quantified here. A related line of work also studies turn-level credit policy optimization, while the comparisons reported here are MMDuet2, ThinkStream, and StreamPro.

The authors also report a component-level result: turn-mean aggregation improves response timing and reduces repetition compared with credit-only optimization. This supports their claim that turn-specific credit alone does not fix length-dependent weighting. The paper text provided does not specify a numerical repetition reduction or timing improvement.

What the method establishes—and what it leaves unspecified

TurnWise’s technical contribution is the pairing of silence-completed prefix scoring with turn-mean policy optimization. The first removes earlier accrued reward from a decision’s group-relative signal; the second gives each sampled response-or-wait decision equal loss weight. Both operate within GRPO and use the existing rollout group rather than a critic or extra trajectories.

The reported evidence is broad across three streaming benchmarks and two Qwen backbones, but the supplied results section gives aggregate rankings rather than score tables. It also does not provide the numerical gains, per-category results, or detailed ablation values. Those omissions limit how precisely a reader can compare effect sizes from the available text.

The paper’s stated edge case is the mismatch between trajectory quality and individual turn quality: a good trajectory can contain a bad turn, and a poor trajectory can contain a useful one. TurnWise is designed to address that mismatch under the paper’s PAUC-based scorer and streaming setup. Its reported findings do not establish performance beyond the listed benchmarks and configurations. Unlike difficulty-aware video RL, which targets video generation, TurnWise evaluates streaming video understanding and interaction timing.

This is a Plain English Papers summary of a research paper called Turns Matter: Advancing Streaming Video RL via Turn-Level Credit.