Runway AI has unveiled Gen-4, its latest artificial intelligence model capable of creating videos from natural language prompts.

Runway, a New York-based company that has raised over $230 million from Nvidia, Google, and other investors, introduced its first AI video generator, Gen-1, in February 2023. The release of Gen-4 today represents the fourth iteration in this series of algorithms.

Many video generation models are built on neural networks originally designed for image creation. This is because videos are essentially sequences of images, allowing them to be generated frame by frame. This process typically employs a technique called diffusion: the model starts with a noisy image and gradually refines it through multiple steps.

The key difference between video generators and image generators lies in maintaining visual consistency across all frames in a clip. This requires extending the core diffusion-optimized artificial neurons with additional components, increasing complexity. Even with these enhancements, ensuring frame consistency remains a significant challenge for video generators.

Runway claims its new Gen-4 model addresses this limitation. Users can upload reference images of objects they want included in their videos along with design instructions. Gen-4 ensures these objects maintain consistent appearance throughout the clip.

"Whether you're crafting scenes for long-form narrative content or generating product photography, Runway Gen-4 enables you to produce consistent results across different environments," wrote Runway's team in a blog post.

The company states that Gen-4 maintains object consistency even when users modify other details. For instance, designers can change camera angles, lighting conditions, or place objects in entirely new environments.

Gen-4 also functions as an image editing tool. Users can upload two illustrations and request the algorithm to combine them into a new image. Gen-4 generates multiple variations of each output image to minimize the need for revisions.

Initially, Runway will allow users to generate five-second and ten-second clips. The startup has released several demo videos nearly two minutes long, suggesting it may update Gen-4 in the future to enable customers to create more complex clips.

The launch of Gen-4 coincides with OpenAI temporarily disabling Sora's video generation feature for new users due to "excessive traffic." OpenAI CEO Sam Altman noted earlier today that the company gained one million new users within an hour.