iWorld-Bench

Overview

Achieving Artificial General Intelligence (AGI) requires agents that learn and interact adaptively, with interactive world models providing scalable environments for perception, reasoning, and action. Yet current research still lacks large-scale datasets and unified benchmarks to evaluate their physical interaction capabilities. To address this, we propose iWorld-Bench, a comprehensive benchmark for training and testing world models on interaction-related abilities such as distance perception and memory.

We construct a diverse dataset with 330k video clips and select 2.1k high-quality samples covering varied perspectives, weather, and scenes. As existing world models differ in interaction modalities, we introduce an Action Generation Framework to unify evaluation and design six task types, generating 4.9k test samples. These tasks jointly assess model performance across visual generation, trajectory following, and memory. Evaluating 14 representative world models, we identify key limitations and provide insights for future research.

Comparison

Our benchmark focuses on interactive world modeling with comprehensive capabilities. Unlike existing benchmarks that are mostly designed for general-purpose world models or embodied world models, iWorld-Bench specifically evaluates interactive world models' responsiveness to external action sequences. It supports multiple input modalities, action control, camera control, memory ability evaluation, multi-scene coverage, multi-perspective observations, and all-weather adaptability.

Framework

We introduce a unified and comprehensive framework that supports inputs from any modality of world models, enabling the design of action tasks and guiding the generation of world models.

Action Space Definition

The generation process of a world model can be decoupled into the following representation: V_t+1 = W(I_t, C_t), where V_t+1 represents the output of the world model, W denotes the specific world model, I_t represents the current scene frame, and C_t = [D, T, R, V] is a quadruple that controls the current frame I_t.

• D represents the current action difficulty (1-6)
• T represents the translational ID (0-26)
• R represents the rotational ID (0-26)
• V represents the validity (0 or 1)

Task Design

Based on the Action Generation Framework, we designed six types of tasks to comprehensively evaluate the interaction capabilities of world models:

Evaluation Metrics (Coming Soon)

Dataset

We established a comprehensive data processing protocol to clean and standardize 12 high-quality datasets, unifying the original coordinate systems and intrinsic/extrinsic parameter formats. Additionally, we designed an automated collection and filtering pipeline to gather 100k 1080P video clips from 18 high-quality environments across 4 simulators. Combined with vision-language models, all video clips were uniformly annotated, resulting in a high-quality dataset containing 330k video clips.

Results

We selected 14 representative interactive world models for evaluation, including five text-conditioned camera control models, two one-hot-conditioned camera control models, and seven models with camera control via explicit intrinsics and extrinsics.

Action Control and Memory Ability

Camera Following Results

BibTeX

If you find this work useful, please consider citing:

@inproceedings{iworldbench2026,
    title={iWorld-Bench: A Benchmark for Interactive World Models with a Unified Action Generation Framework},
    author={Anonymous Authors},
    year={2026},
    organization={Anonymous Organization}
    }