Runway AI Inc., the generative artificial intelligence startup that builds tools for AI-generated video creation, has launched a new feature to help creators give their AI video characters more realistic facial expressions.
Called Act-One, it makes it possible for existing users to record themselves on something as simple as a smartphone camera, capture their facial expressions, and then replicate them on an AI-generated video character.
Runway said in a blog post today that the tool is being rolled out to users starting today, and can be accessed by anyone with a Runway account. That said, it’s not entirely free-to-use, as users will be required to have enough credits on their account to access the startup’s most advanced Gen-3 Alpha video generation model.
The Gen-3 Alpha model debuted earlier this year, introducing support for text-to-video, image-to-video and video-to-video modalities, meaning that users can write a description of a scene, upload an image or a video, or use a combination of those inputs as prompts. Once prompted, the model will go about creating a slick video that tries to match the user’s vision.
Although Runway’s Gen-3 Alpha model can create some impressive videos, one area where it has always been a bit weak is facial animation — particularly in creating accurate facial expressions on characters that can match the mood of the scene. In the filmmaking industry, facial animation is an intricate and expensive task that involves using sophisticated motion capture technologies, manual face rigging techniques and lots of heavy editing behind-the-scenes.
Runway is trying to make advanced facial animation more accessible with Act-One. Using the tool, creators will be able to animate their video characters in almost any way they can imagine, without needing to use pricey motion capture equipment.
Instead, Act-One makes it possible to use your own videos and facial expressions as a kind of reference, transposing them onto AI-generated characters. It’s incredibly detailed, able to replicate everything from micro-expressions to eye-lines, onto various different characters.
In a post on its official X account, Runway said Act-One can “translate the performance from a single input video across countless character designs and in many different styles.”
Although it has not yet rolled out to every Runway user, the company has already received positive feedback from creators:
I’m going to say it
This is a game changer!
— Allen T. (@Mr_AllenT) October 22, 2024
Act-One can be utilized by a range of creative professionals, including animators, video game developers and indie filmmakers, enabling them to generate more unique characters whose personality and actions can be reflected with their emotions and expressions. They’ll be able to create much more realistic, cinema-like video characters and capture them at any camera angle or focal length, Runway said, unlocking the potential for much richer, more detailed storytelling and artistic expression.
By eliminating the technical barrier associated with character animation, the company hopes to inspire a new generation of creators to better express themselves. For instance, it means an indie film producer can use a single actor to take on the role of multiple animated characters that display Hollywood-level realism, using only a consumer-grade camera.
In a post on X, Runway co-founder and Chief Executive Cristóbal Valenzuela said the filmmaking industry is becoming much more receptive to the potential of generative AI:
We are now beyond the threshold of asking ourselves if generative models can generate consistent videos. A good model is now the new baseline. Generating pixels with prompts has become a commodity. The difference lies (as it has always been) in what you do with a model – how you… https://t.co/s0wVvPga7j
— Cristóbal Valenzuela (@c_valenzuelab) October 22, 2024
Runway added that Act-One comes with a number of built-in safeguards that will prevent misuse, including guardrails that will prevent efforts to generate content featuring public figures without their express authorization. It’s also integrated with tools to verify voice usage rights, the company said. In addition, Runway will perform continuous monitoring of the tool to ensure it is used in a responsible way by creators.