AI Brings the Mona Lisa to Life: Animated Portraits with a Twist

AI Brings the Mona Lisa to Life: Animated Portraits with a Twist
AI Brings the Mona Lisa to Life: Animated Portraits with a Twist
A Microsoft sign is seen at the company's headquarters on March 19, 2023 in
Seattle, Washington.

New York CNN  — 

The Mona Lisa can now do more than smile, thanks to new artificial intelligence
technology from Microsoft.

Last week, Microsoft researchers detailed a new AI model they’ve developed that
can take a still image of a face and an audio clip of someone speaking and
automatically create a realistic looking video of that person speaking. The
videos — which can be made from photorealistic faces, as well as cartoons or
artwork — are complete with compelling lip syncing and natural face and head

In one demo video, researchers showed how they animated the Mona Lisa to recite
a comedic rap by actor Anne Hathaway.

Outputs from the AI model, called VASA-1
[], are both
entertaining and a bit jarring in their realness. Microsoft said the technology
could be used for education or “improving accessibility for individuals with
communication challenges,” or potentially to create virtual companions for
humans. But it’s also easy to see how the tool could be abused and used to
impersonate real people.

For now, Microsoft said it doesn’t plan to release the VASA-1 model to the
public immediately. The move is similar to how Microsoft partner OpenAI is
handling concerns around its AI-generated video tool
Sora: OpenAI teased Sora in February, but has so far only made it available to
some professional users and cybersecurity professors for testing purposes.

“We are opposed to any behavior to create misleading or harmful contents of real
persons,” Microsoft researchers said in a blog post. But, they added, the
company has “no plans to release” the product publicly “until we are certain
that the technology will be used responsibly and in accordance with proper

Microsoft’s new AI model was trained on numerous videos of people’s faces while
speaking, and it’s designed to recognize natural face and head movements,
including “lip motion, (non-lip) expression, eye gaze and blinking, among
others,” researchers said. The result is a more lifelike video when VASA-1
animates a still photo.

For example, in one demo video set to a clip of someone sounding agitated,
apparently while playing video games, the face speaking has furrowed brows and
pursed lips.

The AI tool can also be directed to produce a video where the subject is looking
in a certain direction or expressing a specific emotion.

When looking closely, there are still signs that the videos are
machine-generated, such as infrequent blinking and exaggerated eyebrow
movements. But Microsoft said it believes its model “significantly outperforms”
other, similar tools and “paves the way for real-time engagements with lifelike
avatars that emulate human conversational behaviors.”

newsid: iv7kmnxs4f4tlas

Related stories
2 hours ago - Microsoft will outline its vision for the future of Windows PCs on Monday, and Windows on Arm will be at the center.
3 hours ago -  The way this puppy acts at the vet is how I always feel inside when at the doctors' office. The faces made are amazingly dramatic. 
3 hours ago - Apple was caught flat-footed in the generative AI race, and the company will need to change its strategy if it wants to catch up. Also: The misguided attempt to differentiate the iPad from the Mac; get ready for a new AirTag; and Apple banks on Asia to make the Vision Pro a success.
5 hours ago - A man started a YouTube channel after collecting more than 2 million frog eggs and hatching them in his backyard pond. Previous videos on his channel show what the hatching…
5 hours ago - Grandoreiro banking trojan is back, targeting 1,500+ banks in 60+ countries. It now uses infected Outlook to spread phishing emails.
Other stories
30 minutes ago - The company is sending emails out that say its “most requested product” is coming on May 21st. Could the release be its rumored Sonos Ace wireless headphones?
30 minutes ago - An InStyle shopping editor found the seven best fashion, beauty, and home deals at Amazon this weekend starting at $6. The best Amazon deals include Hanes tees, Sam Edelman sandals, and CeraVe eye cream.
1 hour ago - Why You Can Trust CNET Our expert deal-hunting staff showcases the best price drops and discounts from reputable sellers daily. If you make a...
1 hour ago - Discover the best ways to stream March Madness 2024 on TBS without cable. Your ultimate guide for how to watch March Madness and more.
1 hour ago - Don't let inaccurate numbers discourage your wellness journey. Learn how to weigh yourself correctly.