Google Lumiere – How to use the AI video generator
Table of Contents
The search giant has unveiled its newest AI model – and it’s really good. Google Lumiere is a text-to-video diffusion model “designed for synthesizing videos that portray realistic, diverse and coherent motion”. In some respects, it’s better than Runway and Pika Labs AI video! Here’s how to use Google Lumiere.
How to use Google Lumiere AI video generator
There is no way to access or download Lumiere at this time. We predict that Lumiere will enhance the multimodal capabilities of Google Bard in the near future. Get ready to use it at release by following these steps:
Step
Google Lumiere via Bard
To use Google Lumiere, you’ll need access to Google Bard. Visit the chatbot website here.
There has been no official acknowledgement that the video model has been integrated yet. However, it’s fair to predict that Bard will be the place to use it in the near future.
Google Workspace accounts will need admin access to use Google Bard.
Step
How to download and install
If Google Lumiere becomes open-source, we will explain how to download and install it here.
What is Google Lumiere, and what can it do?
Google Lumiere is a new video diffusion model. It generates “coherent, high-quality videos using simple text prompts” and is great for stylization. It’s also multimodal, with text-to-video and image-to-video modalities. You can also use it to produce cinemagraphs and for video inpainting!
Deals season is here folks, and with it comes huge savings on some of the market's most popular hardware. Below, we be listing today's best PC hardware deals, including GPUs, CPUs, motherboards, gaming PCs, and more.
- ASUS TUF NVIDIA RTX 5080 Was $1599 Now $1349
- ASUS TUF RTX 5070 Ti Was $999 Now $849
- ASUS TUF ROG Strix XG27ACS Was $349 Now $329
- TCL 43S250R Roku TV 2023 Was $279 Now $199
- Thermaltake LCGS Gaming PC Was $1,799 Now $1,599
- Samsung Odyssey G9 (G95C) Was $1,299 Now $1,000
- Alienware AW3423DWF Was $699 Now $549
- Samsung 77-inch OLED S95F Was $4,297 Now $3,497
- ASUS ROG Strix G16 Was $1,499 Now $1,350
*Prices and savings subject to change. Click through to get the current prices.
Lumiere achieves better temporal consistency than other models due to “a Space-Time U-Net architecture that generates the entire temporal duration of the video at once, through a single pass in the model. This is in contrast to existing video models which synthesize distant keyframes followed by temporal super-resolution”.
This new AI model represents a summation of previously released Google AI tools.
Style Drop, introduced on Dember 15th, 2023, is Google’s own text-to-image generator. Its USP is that it “uses one or more style reference images that describe the style for text-to-image generation. By doing so, StyleDrop enables the generation of images in a style consistent with the reference, while effectively circumventing the burden of text prompt engineering”. As a result, StyleDrop already features the computer vision research that went into Google Lumiere.
Video Poet was the predecessor to Google Lumiere, in that it’s a large language model for zero-shot video generation. The main difference is the quality. Impressively, Video Poet was already multimodal – able to produce audio from video inputs. This is one of the least common avenues of multimodality (the most common being speech-to-text). In fact, this “autoregressive language model learns across video, image, audio, and text modalities”.
Essential AI Tools
Who made Lumiere? A Space-Time Diffusion Model for Video Generation
Google scientists published a research paper introducing Lumiere on January 23rd, 2024. The paper, titled “Lumiere: A Space-Time Diffusion Model for Video Generation” is filed under Computer Vision and Pattern Recognition – a nod to the multimodality of the model. It’s not just text models that can be multimodal! A multimodal AI video model could accept inputs of text, images, audio samples, or even other videos (perhaps a combination of two or more) to inform the generated video. As no AI chatbot currently offers much (or anything) in terms of AI video generation, Lumiere could make Google Bard the most multimodal AI assistant of 2025.
The researchers who authored the paper include Omer Bar-Tal, Hila Chefer, Omer Tov, Charles Herrmann, Roni Paiss, Shiran Zada, Ariel Ephrat, Junhwa Hur, Yuanzhen Li, Tomer Michaeli, Oliver Wang, Deqing Sun, Tali Dekel, and Inbar Mosseri.
About the Author