WorldMind arXiv · 2026

arXiv preprint · 2026

WorldMind Decoupled Game World Model for State-Aware NPC Behavior

Zhiyang Deng1,2,†, Boran Zhang1,†, Danze Chen1,2,†, Yeying Jin1,2,‡,*

1Tencent    2National University of Singapore

†Work done during research internships at Tencent.   ‡Project lead.   *Corresponding author.

WorldMind decouples compact-state construction, NPC decision-making, action control, and video generation—then reconnects them in a closed loop.

Construct the state. Plan the response. Render the outcome. Repeat.

arXiv Code Checkpoints Dataset

01 / Paper summary

Abstract

Game world models have recently demonstrated promising capabilities in generating visually coherent and action-controllable gameplay videos. However, non-player character (NPC) behavior in existing models is either implicitly entangled with video generation or explicitly prescribed through external control signals. Consequently, a game world model has to jointly understand the state, plan the NPC's response and render its visual outcome, limiting its ability to produce responsive and state-aware NPC behavior. The challenge lies in the lack of an explicit interface for state-grounded decision-making. To this end, we introduce WorldMind, to our knowledge the first decoupled framework for state-aware NPC behavior in game world models. WorldMind separates interactive world modeling into four layers: an Understanding Layer that constructs a compact state from generated frames; a Decision Layer that reasons over the compact state to plan the NPC's next action; a Control Layer that translates the actions into temporally aligned conditions; and a Generation Layer that synthesizes their visual outcomes. By reconnecting layers in a closed interaction loop, WorldMind grounds NPC behavior in the evolving game state. We further introduce BOSS-140K, a dataset of gameplay videos paired with rich internal game states, together with an agent that automates the collection at scale. Experiments on BOSS-140K demonstrate reliable compact state reconstruction and mechanics-grounded planning, with WorldMind preferred over the baselines in approximately 70% of pairwise comparisons for its more tactically appropriate and coherent NPC behavior.

02 / System at a glance

Overview

Plausible frames are not enough.

A boss can look convincing while choosing the wrong move. Existing game world models either bury NPC behavior inside video prediction or ask an external controller to prescribe it; WorldMind reconstructs the current state, decides the boss action, turns it into a control condition, and renders the next segment.

Comparison of implicit NPC behavior, externally controlled NPC behavior, and WorldMind's state-aware boss behavior Open full figure ↗
Paper figure 1 The baselines either leave NPC behavior implicit in video generation or expose it as external control. WorldMind instead plans from a compact state.

  • Implicit behavior

    The action stays hidden in pixels.

    The model must jointly infer the current state, reason about the NPC action, and render its visual outcome.

  • External control

    The action is explicit—but prescribed.

    The world model renders a supplied boss action without deciding whether it fits the evolving state.

  • WorldMind

    The action is explicit and state-aware.

    A compact state drives the boss decision before the visual consequence is synthesized.

03 / Four-layer closed loop

Architecture

One interaction. Four distinct responsibilities.

WorldMind separates state understanding, decision-making, action control, and video generation, then feeds the rendered result back into the loop.

  1. L1 What is happening?

    Understanding

    Constructs a compact state from generated frames and issued boss-action history, combining a learned geometry branch with a deterministic skill branch.

    • boss identity
    • previous skill + elapsed time + cooldowns
    • distance + angle + facing
  2. L2 What should the boss do?

    Decision

    A general-purpose language model reasons over the compact state, the full skill set, and cooldown information to form a short-horizon plan.

    • mechanics-aware skill choice
    • skill choice + movement
    • deterministic duration + legality check
    • issue the first action, then replan
  3. L3 How is it expressed?

    Control

    Converts player and boss actions into temporally aligned natural-language conditions consumed by the video generator.

    • separate entity action slots
    • direct player control
    • high-level director control
  4. L4 What happens next?

    Generation

    Renders the selected actions with a Wan 2.2 TI2V-5B generator distilled through causal AR training, consistency initialization, and asymmetric DMD.

    • temporally aligned synthesis
    • autoregressive video
    • distilled generator at ≈20 FPS

Then the rendered frames return to L1. The boss can react again as the interaction evolves.

Overview of the WorldMind framework with Understanding, Decision, Control, and Generation layers in a closed loop Open full figure ↗
Paper figure 3 WorldMind separates state reconstruction, decision-making, temporally aligned control, and video generation, then reconnects them in a closed loop.

Direct control

You play. WorldMind runs the boss.

Keyboard and mouse events become symbolic player actions, while L2 chooses the boss response from the current compact state.

PLAYER: move left + dodge
BOSS: approach + heavy slash

Director control

You direct the scene at a higher level.

A natural-language instruction guides both entities while the state-aware decision process remains inside the loop.

“The player keeps evading while
the boss remains highly aggressive.”

04 / Across three games

Examples

One decision, end to end.

Across Game A, The Binding of Isaac, and Hollow Knight, the same L1 → L2 → L3 → L4 contract turns visual history into a state-aware action and then into the next generated segment.

Three qualitative examples showing state reconstruction, boss action selection, prompt construction, and generated output Open full figure ↗
Paper figure 4 Qualitative walkthrough. L2 may plan up to three actions, but only the first planned action passes to L3; its rationale remains analysis-only.

05 / System and interfaces

Results

The full system wins under both judges.

We compare the assembled closed-loop system against the implicit and explicit baselines under matched starts, the same backbone, and the same training corpus.

Full system

WorldMind ranks first under both judges.

71.5%

preferred over implicit NPC behavior

70.9%

preferred over explicit NPC control

System Action validity ↑ Sequence fit ↑
WorldMind 74.0% 3.85 / 5
Wan w/o NPC control 63.6% 3.28 / 5
Wan w/ NPC control 63.6% 3.22 / 5

One-minute Game A rollouts: 86 matched starts per system, 12 generated segments per rollout, and three independent judge passes. All systems use the same Wan 2.2 backbone and training corpus; the implicit baseline is player-only, while the explicit baseline and WorldMind are boss-conditioned.

  1. Decision 1

    distance 5.21

    jump_attack

    The player is at a far distance (5.2 units), so a gap-closer attack is necessary to initiate engagement.

  2. Decision 2

    distance 0.53

    mace_slam

    The player is point-blank, so a ground attack like mace_slam will maximize immediate damage and pressure.

06 / Training data

Dataset

BOSS-140K aligns pixels, controls, skills, and state on one timeline.

Boss-fight Observations with States and Skills provides aligned data for training and evaluating the loop; L2 remains an untuned general-purpose language model.

A state-conditioned gameplay agent interacts with each boss while the collection pipeline records gameplay video, player controls, boss skills, animation states, internal variables, and captions in alignment. Engine access is used only for data collection—not at WorldMind inference time.

144,631
L4 clips · 21 temporal latents each
279,721
non-death decision rows
14
bosses
3
games · 2D + 2.5D
  • Game A
  • Hollow Knight
  • The Binding of Isaac
BOSS-140K construction pipeline from a state-conditioned gameplay agent to frame-aligned video and state annotations Open full figure ↗
Paper figure 2 A state-conditioned agent automates collection while video, controls, NPC skills, engine variables, and action captions share one timeline.

07 / Long rollout

One-hour rollout

The loop keeps running for an hour.

Two-step distilled generator, one hour of continuous closed-loop generation, played back at 6× speed.

1 h · 6× One continuous rollout, no resets.

08 / Citation

Citation

Cite the preprint.

The preprint is available above. Code and data will be released; one of the three titles is referred to as Game A and its frames are blurred.

BibTeX

@article{deng2026worldmind,
  title   = {WorldMind: Decoupled Game World Model for State-Aware NPC Behavior},
  author  = {Deng, Zhiyang and Zhang, Boran and Chen, Danze and Jin, Yeying},
  journal = {arXiv preprint arXiv:2608.21439},
  year    = {2026}
}