A baby hippo swims in a lagoon. A fire dancer performs on a beach as waves crash into the shore. A DJ plays tunes on a turntable.
Those are Hollywood-style movie clips produced by Meta’s new AI models, which can transform text to video from a simple prompt.
The technology, called Movie Gen, uses two new foundational AI models the company highlighted in a research paper Friday that enable users to make custom movies, edit videos, and create a video using a person’s image.
“We really wanted to improve the state of the technology,” said Ahmad Al-Dahle, vice president of Meta’s generative AI division.
“They [are] exceeding our expectations on quality in terms of natural motion or aesthetics.”
Meta is not yet releasing Movie Gen to users, or providing the models open-source to developers, but it is making the evaluation data on the underlying prompts available to researchers to review.
The race to create AI models that can generate video is heating up. In February, OpenAI unveiled its own text-to-video model called Sora, which the company says can produce eye-popping videos that are up to a minute long. In May, YouTube debuted Veo, a video generation model, and Imagen 3, new text-to-image model that the company says will “support the creative process.”
Movie Gen follows similar cinematographic efforts from Meta including Make-A-Video and Emu Video, which generate less advanced videos from text.
Meta’s new model is expected to power future products that could enable creators to share more AI-generated content on its social media networks. The company has pushed out new AI tools in recent months to allow users to create their own chatbots, alter silly images of themselves, and birth imaginary scenes from text prompts.
But some critics have argued that easy AI video tools could open the floodgates to misinformation online and hurt the job market in Hollywood.
While some of the biggest studios have long relied on artificial intelligence to help create sophisticated cinematic effects, the latest advances in generative artificial intelligence have provoked fears among actors that the technology could eliminate jobs.
Al-Dahle said video generation tools will include a watermark that identifies them as generated using artificial intelligence.
Meta said Movie Gen can produce high-quality HD videos of up to 16 seconds from a text prompt and can create videos featuring a specific person, as depicted in a photograph. The model also allows people to edit existing videos with simple text prompts, such as adding thundercrack sounds or tinsel streamers to a lantern, according to the research paper.
“They have the ability to do what we call very precise editing,” Al-Dahle said. “So if you’re asking to edit, for example, a tree and remove it, we only remove the tree.”