RTFM

RTFM is an autoregressive diffusion transformer model developed by World Labs that takes one or more 2D images of a scene as input and generates new viewpoint frames in real time as a user navigates through the generated environment. Unlike traditional 3D rendering pipelines or neural radiance field approaches, RTFM does not construct an explicit 3D representation of the scene — instead, it converts input frames into neural network activations forming an implicit world representation, then renders new frames by attending to that representation as the user moves. The architecture is trained end-to-end on large-scale video data. A spatial memory mechanism positions each generated frame in 3D space, allowing the model to retrieve contextually nearby frames when rendering new views — a system World Labs calls context juggling — which enables the world to persist indefinitely without dissolving as the user navigates away from the starting point. See top alternatives. RTFM is primarily a research demonstration aimed at developers, game designers, and researchers who want to understand the current frontier of generative world models. Game developers studying AI-native world generation use it to evaluate what real-time generative environments can produce from minimal input. AI and computer vision researchers use it as a reference implementation of persistent generative navigation — a capability distinct from pre-rendered video generation or static scene reconstruction. World Labs has since integrated RTFM's technology into Marble, its commercial 3D world creation product, which extends the underlying model with text, image, and panorama inputs, AI editing, and export to Unity, Unreal, Blender, and Houdini View alternatives.