research team has just developed a new AI that can use an existing video and a single image to make the person in the image imitate movements from the video.
Technically, the method known as video-to-video compositing takes an input video to convert it into a photorealistic video using an image.
Nvidia's research team said there are two big problems with the current range of AI models that try to achieve the same thing: First, these models need a set of target images to convert into video. Second, these models' ability to generalize the output is limited.
To overcome these obstacles, the researchers created a new model that learns to produce videos of people who weren’t in the training dataset — using just a few images of them. The team then tested various scenarios, including dance moves and conversations. You can see the AI in action in the video below:
The model can also be used on paintings or on roads to create lifelike representations. This can be really handy for creating movies and games.
AI isn't perfect, and it's hard to properly utilize all the elements in these low-resolution videos. However, it's still useful and interesting to create these types of videos .
