ActionRoPE: Action as Coordinates for Spatially Consistent Control in Planar-Camera Game World Models

Game world models have made rapid progress toward real-time interactive generation. Yet action control is still predominantly implemented as feature conditioning, leaving the model to learn from training data how each action should move the visual world. This formulation is poorly matched to a broad class of planar-camera worlds, such as top-down and isometric games. In these worlds, player motion induces a geometrically well-defined planar camera displacement. We observe that the key to spatially consistent control in such worlds is to represent the effect of each action directly in world coordinates. To this end, we introduce ActionRoPE, a parameter-free action interface that converts action-driven camera displacement into world-coordinate rotary positions. As a result, tokens corresponding to the same world location retain the same positional address across time. These world coordinates further expose a spatial asymmetry in generation: regions covered by the initial observation are geometrically known, while newly revealed regions remain to be synthesized. We therefore propose Coordinate-Conditioned Noise Scheduling (CNS), which assigns token-wise denoising schedules according to this known–new structure. It allows observed regions to stabilize earlier, while new content is generated against an increasingly coherent world. Compared with four representative action-conditioning mechanisms on the same backbone, our approach follows the commanded motion more faithfully and reduces the revisit drift by 66% over the strongest baseline, without adding parameters to the control path. The same interface transfers across video backbones from 1.3B to 33B parameters and to out-of-distribution scenes.

Action as coordinates

ActionRoPE. In a planar-camera world every action moves the camera by a known displacement Δ_k. Instead of feeding the action as a feature, we shift the rotary position of every token of latent frame k by Δ_k/32 on the screen grid. A token that looks at the same place in the world therefore keeps the same positional address in every frame, and the first frame becomes the origin the whole clip is anchored to. Nothing is learned for the interface itself.

Known versus new. Once positions live in world coordinates, each token is either inside the footprint of the first frame (its content is known) or outside it (it has to be synthesised). This map κ is computed geometrically and is free.

CNS. Coordinate-conditioned noise scheduling gives known tokens a schedule that settles earlier while new tokens keep the standard one, so new content is generated against an increasingly coherent world. The same map is also passed to the network as a zero-initialised input channel (15K parameters, 3 ppm of the backbone) so the denoiser knows which tokens are anchors as well as when they settle.

Figure 2. Left: the action interface places each frame's tokens on the screen grid shifted by the accumulated displacement. Right: token-wise schedules for known and new regions.

Three control exams on 982 first frames

Every model receives one validation first frame and an action sequence. Gain: walk right at 0.5×–1.5× speed, regress measured against commanded displacement. Direction: walk straight, measure the angle of the background flow. Revisit: walk out and return along the same path, compare the last frame with the first.

Table 1. Action injection mechanisms on a shared Wan2.2-TI2V-5B backbone (same data, recipe and seed; 982 validation first frames). slope / R²: measured against commanded displacement over five speed multipliers; angle: median direction error in degrees; fail: videos off by more than 15° or short of half the distance; PSNR / SSIM / NCC / DINO / drift: returned frame against the first frame after a walk-out-and-return loop. †No parameters on the control path; the optional known/new mask channel adds 0.015M.

Table 2. The full model (ActionRoPE + CNS) trained with one recipe on four video DiTs spanning 1.3B–33B parameters, evaluated on the same 982 first frames. ‡MiniMax-H3 only generates 17n+5 frames: the 81-frame action plan is time-compressed to 73 frames and the output resampled to 81 frames for measurement.

Against learned action-injection baselines

Same Wan2.2-5B backbone, same data, same recipe. Prompt, additive linear, AdaLN and cross-attention injection learn a mapping from action to motion; ActionRoPE writes it into the coordinates. In every row the first frame and the command are identical; the highlighted column is ours.

Figure 3. Closed-loop revisit. Top of each row: the returned frame; bottom: absolute error to the first frame (blue low, red high). Ours returns to a frame that differs from the origin only along thin structure edges; every learned mechanism leaves the world displaced or partially re-drawn.

Unseen scenes

First frames that never appear in training: paintings, ukiyo-e, a cyberpunk street, a market. The coordinate interface does not depend on scene content, so control and the return to the origin carry over.

Figure 4. Frames f = 0 … 80 of two unseen scenes under the same return loop; the arrows give the commanded direction at each frame.

One interface, four video backbones

ActionRoPE + CNS trained with the same recipe on SkyReels-V2 (1.3B, diffusion forcing), LTX-2.5 (22B, LoRA), MiniMax-H3 (33B, LoRA) and Wan2.2 (5B, full fine-tuning). Same first frame and command in every row.

Figure 5. Frames f = 0 … 80 of one return loop on three backbones.