Google has unveiled Lumiere , its new artificial intelligence- powered video production tool , which it describes as “a space-time diffusion model for realistic video production.”
See also: Generative AI in the Video Games Industry
Beyond the technical details, Lumiere seems capable of creating videos of cute animals in unlikely situations, such as wearing roller skates, driving cars , or playing the piano.
According to Google, Lumiere uses a unique architecture to create the entire duration of a video in one process. Instead of piecing together a video from many small pieces or frames, it can create the entire video, from start to finish, in one seamless process.
Lumiere can also perform many “tricks,” which are described with examples on Google’s official presentation page. Among other things, it can convert text to video, convert still images to video, create videos in specific styles using reference images, apply consistent text-based video editing, create cinematography by animating specific areas of an image, and offer video repair capabilities (e.g., change the clothes a personality/character is wearing).
See also: NCSC: Artificial intelligence (AI) will increase ransomware attacks
Google says Lumiere creates five-second at a resolution of 1024×1024 pixels, which it describes as “low-resolution.” Despite these limitations, Google researchers say Lumiere’s results were superior to other existing AI.
Although artificial video is still in its early stages, it has made progress in its quality over the past two years. Google introduced its first image synthesis model, Imagen Video , in October 2022. Before that, Meta introduced Make-A-Video its . In June of last year, Runway ’s Gen2 allowed the creation of two-second videos from text, encouraging the creation of parody commercials. In November, we covered Stable Video Diffusion , which can create short videos from static images.
See also: Apple: Invests in artificial intelligence more than other companies
The presentation often generates reactions, as creating coherent virtual human forms is difficult, and artificial systems risk creating deviations that we easily perceive. Lumiere seems to outperform other AI video composition models, but Google is not giving up, usually keeping its AI research projects secret, without being sure if and when the general public will have the opportunity to try it.
As always, when we see video synthesis models become more capable, we can’t help but think about the future implications for our connected society, which is focused on the common sharing of multimedia. The question of video credibility will become even more important, as video synthesis tools more capable than Lumiere will make it easy to create deceptive deepfakes. Along these lines, in the “Social Implications” section of Lumiere’s paper, the researchers state: “Our goal in this project is to enable novice users to create visual content in a creative and flexible way. However, there is a risk of abuse to create false or harmful content using our technology, and we believe it is vital to develop and implement tools to detect bias and malicious use in order to ensure safe and fair use.”
Source: arstechnica
