Home > AI > AI How To

Google Lumiere – How to use the AI video generator

Google's multimodal AI video model
Last Updated on February 15, 2024
How to use Google Lumiere AI video model with Bard.
You can trust PC Guide: Our team of experts use a combination of independent consumer research, in-depth testing where appropriate - which will be flagged as such, and market analysis when recommending products, software and services. Find out how we test here.

The search giant has unveiled its newest AI model – and it’s really good. Google Lumiere is a text-to-video diffusion model “designed for synthesizing videos that portray realistic, diverse and coherent motion”. In some respects, it’s better than Runway and Pika Labs AI video! Here’s how to use Google Lumiere.

How to use Google Lumiere AI video generator

There is no way to access or download Lumiere at this time. We predict that Lumiere will enhance the multimodal capabilities of Google Bard in the near future. Get ready to use it at release by following these steps:

Step

1

Google Lumiere via Bard

To use Google Lumiere, you’ll need access to Google Bard. Visit the chatbot website here.

There has been no official acknowledgement that the video model has been integrated yet. However, it’s fair to predict that Bard will be the place to use it in the near future.

Google Workspace accounts will need admin access to use Google Bard.

Step

2

How to download and install

If Google Lumiere becomes open-source, we will explain how to download and install it here.

What is Google Lumiere, and what can it do?

Google Lumiere is a new video diffusion model. It generates “coherent, high-quality videos using simple text prompts” and is great for stylization. It’s also multimodal, with text-to-video and image-to-video modalities. You can also use it to produce cinemagraphs and for video inpainting!

Lumiere achieves better temporal consistency than other models due to “a Space-Time U-Net architecture that generates the entire temporal duration of the video at once, through a single pass in the model. This is in contrast to existing video models which synthesize distant keyframes followed by temporal super-resolution”.

This new AI model represents a summation of previously released Google AI tools.

Style Drop, introduced on Dember 15th, 2023, is Google’s own text-to-image generator. Its USP is that it “uses one or more style reference images that describe the style for text-to-image generation. By doing so, StyleDrop enables the generation of images in a style consistent with the reference, while effectively circumventing the burden of text prompt engineering”. As a result, StyleDrop already features the computer vision research that went into Google Lumiere.

Video Poet was the predecessor to Google Lumiere, in that it’s a large language model for zero-shot video generation. The main difference is the quality. Impressively, Video Poet was already multimodal – able to produce audio from video inputs. This is one of the least common avenues of multimodality (the most common being speech-to-text). In fact, this “autoregressive language model learns across video, image, audio, and text modalities”.

Essential AI Tools

Editor’s pick
Only $0.00019 per word!

Content Guardian – AI Content Checker – One-click, Eight Checks

8 Market leading AI Content Checkers in ONE click. The only 8-in-1 AI content detector platform in the world. We integrate with leading AI content detectors to give unparalleled confidence that your content appear to be written by a human.
EXCLUSIVE DEAL 10,000 free bonus credits

Jasper AI

On-brand AI content wherever you create. 100,000+ customers creating real content with Jasper. One AI tool, all the best models.
TRY FOR FREE

WordAI

10x Your Content Output With AI. Key features – No duplicate content, full control, in built AI content checker. Free trial available.
TRY FOR FREE

Copy.ai

Experience the full power of an AI content generator that delivers premium results in seconds. 8 million users enjoy writing blogs 10x faster, effortlessly creating higher converting social media posts or writing more engaging emails. Sign up for a free trial.
TRY FOR FREE

Writesonic

Create SEO-optimized and plagiarism-free content for your blogs, ads, emails, and website 10X faster. Start for free. No credit card required.

Who made Lumiere? A Space-Time Diffusion Model for Video Generation

Google scientists published a research paper introducing Lumiere on January 23rd, 2024. The paper, titled “Lumiere: A Space-Time Diffusion Model for Video Generation” is filed under Computer Vision and Pattern Recognition – a nod to the multimodality of the model. It’s not just text models that can be multimodal! A multimodal AI video model could accept inputs of text, images, audio samples, or even other videos (perhaps a combination of two or more) to inform the generated video. As no AI chatbot currently offers much (or anything) in terms of AI video generation, Lumiere could make Google Bard the most multimodal AI assistant of 2024.

The researchers who authored the paper include Omer Bar-Tal, Hila Chefer, Omer Tov, Charles Herrmann, Roni Paiss, Shiran Zada, Ariel Ephrat, Junhwa Hur, Yuanzhen Li, Tomer Michaeli, Oliver Wang, Deqing Sun, Tali Dekel, and Inbar Mosseri.

Steve is an AI Content Writer for PC Guide, writing about all things artificial intelligence. He currently leads the AI reviews on the website.

'),b.insertAdjacentHTML(d,e));return document.getElementById(a)};c.prototype._repeatDynamicAds=function(b){var a= [],d=this._removeRecipeCapExperiment.result&&"Recipe"===b.location?99:this.locationMaxLazySequence.get(b.location),e=b.lazy?u(d,0):0;d=b.max;var f=b.lazyMax;e=Math.max(d,0===e&&b.lazy?d+f:Math.min(Math.max(e-b.sequence+1,0),d+f));for(f=0;f=d;a.push(y(y({},b),{sequence:g,lazy:h}))}return a};c.prototype._locationEnabled=function(b){b=this._clsOptions.enabledLocations.includes(b.location);var a=this._clsOptions.disableAds&&this._clsOptions.disableAds.all|| document.body.classList.contains("adthrive-disable-all"),d=!document.body.classList.contains("adthrive-disable-content")&&!this._clsOptions.disableAds.reasons.has("content_plugin");return b&&!a&&d};return c}(),Za=function(c){function b(a,d){var e=c.call(this,a,"ClsVideoInsertion")||this;e._videoConfig=a;e._clsOptions=d;e._IN_POST_SELECTOR=".adthrive-video-player";e._WRAPPER_BAR_HEIGHT=36;e._playersAddedFromPlugin=[];d.removeVideoTitleWrapper&&(e._WRAPPER_BAR_HEIGHT=0);return e}v(b,c);b.prototype.init= function(){this._initializePlayers()};b.prototype._wrapJWPlayerWithCLS=function(a,d,e){void 0===e&&(e=0);return a.parentNode?(d=this._createGenericCLSWrapper(.5625*a.offsetWidth,d,e),a.parentNode.insertBefore(d,a),d.appendChild(a),d):null};b.prototype._createGenericCLSWrapper=function(a,d,e){var f=document.createElement("div");f.id="cls-video-container-".concat(d);f.className="adthrive";f.style.minHeight="".concat(a+e,"px");return f};b.prototype._getTitleHeight=function(a){a.innerText="Title";a.style.visibility= "hidden";document.body.appendChild(a);var d=window.getComputedStyle(a),e=parseInt(d.height,10),f=parseInt(d.marginTop,10);d=parseInt(d.marginBottom,10);document.body.removeChild(a);return Math.min(e+d+f,50)};b.prototype._initializePlayers=function(){var a=document.querySelectorAll(this._IN_POST_SELECTOR);a.length&&this._initializeRelatedPlayers(a);this._shouldRunAutoplayPlayers()&&this._determineAutoplayPlayers()};b.prototype._createStationaryRelatedPlayer=function(a,d,e){var f="mobile"===this._device? [400,225]:[640,360],g=G.Video_In_Post_ClicktoPlay_SoundOn;d&&a.mediaOrPlaylistId&&(e="".concat(a.mediaOrPlaylistId,"_").concat(e),d=this._wrapJWPlayerWithCLS(d,e),this._playersAddedFromPlugin.push(a.mediaOrPlaylistId),d&&this._clsOptions.setInjectedVideoSlots({playerId:a.playerId,playerName:g,playerSize:f,element:d,type:"stationaryRelated"}))};b.prototype._createStickyRelatedPlayer=function(a,d){var e="mobile"===this._device?[400,225]:[640,360],f=G.Video_Individual_Autoplay_SOff;this._stickyRelatedOnPage= !0;this._videoConfig.mobileStickyPlayerOnPage="mobile"===this._device;if(d&&a.position&&a.mediaId){var g=document.createElement("div");d.insertAdjacentElement(a.position,g);d=document.createElement("h3");d.style.margin="10px 0";d=this._getTitleHeight(d);d=this._wrapJWPlayerWithCLS(g,a.mediaId,this._WRAPPER_BAR_HEIGHT+d);this._playersAddedFromPlugin.push(a.mediaId);d&&this._clsOptions.setInjectedVideoSlots({playlistId:a.playlistId,playerId:a.playerId,playerSize:e,playerName:f,element:g,type:"stickyRelated"})}}; b.prototype._createPlaylistPlayer=function(a,d){var e=a.playlistId,f="mobile"===this._device?G.Video_Coll_SOff_Smartphone:G.Video_Collapse_Autoplay_SoundOff,g="mobile"===this._device?[400,225]:[640,360];this._videoConfig.mobileStickyPlayerOnPage=!0;var h=document.createElement("div");d.insertAdjacentElement(a.position,h);d=this._wrapJWPlayerWithCLS(h,e,this._WRAPPER_BAR_HEIGHT);this._playersAddedFromPlugin.push("playlist-".concat(e));d&&this._clsOptions.setInjectedVideoSlots({playlistId:a.playlistId, playerId:a.playerId,playerSize:g,playerName:f,element:h,type:"stickyPlaylist"})};b.prototype._isVideoAllowedOnPage=function(){var a=this._clsOptions.disableAds;if(a&&a.video){var d="";a.reasons.has("video_tag")?d="video tag":a.reasons.has("video_plugin")?d="video plugin":a.reasons.has("video_page")&&(d="command queue");q.error(d?"ClsVideoInsertionMigrated":"ClsVideoInsertion","isVideoAllowedOnPage",Error("DBP: Disabled by publisher via ".concat(d||"other")));return!1}return this._clsOptions.videoDisabledFromPlugin? !1:!0};return b}(function(c){function b(a,d){var e=c.call(this)||this;e._videoConfig=a;e._component=d;e._stickyRelatedOnPage=!1;e._relatedMediaIds=[];a=void 0;void 0===a&&(a=navigator.userAgent);a=/Windows NT|Macintosh/i.test(a);e._device=a?"desktop":"mobile";e._potentialPlayerMap=e.setPotentialPlayersMap();return e}v(b,c);b.prototype.setPotentialPlayersMap=function(){var a=this._videoConfig.players||[],d=this._filterPlayerMap();a=a.filter(function(e){return"stationaryRelated"===e.type&&e.enabled}); d.stationaryRelated=a;return this._potentialPlayerMap=d};b.prototype._filterPlayerMap=function(){var a=this,d=this._videoConfig.players,e={stickyRelated:[],stickyPlaylist:[],stationaryRelated:[]};return d&&d.length?d.filter(function(f){var g;return null===(g=f.devices)||void 0===g?void 0:g.includes(a._device)}).reduce(function(f,g){f[g.type]||(q.event(a._component,"constructor","Unknown Video Player Type detected",g.type),f[g.type]=[]);g.enabled&&f[g.type].push(g);return f},e):e};b.prototype._checkPlayerSelectorOnPage= function(a){var d=this;a=this._potentialPlayerMap[a].map(function(e){return{player:e,playerElement:d._getPlacementElement(e)}});return a.length?a[0]:{player:null,playerElement:null}};b.prototype._getOverrideElement=function(a,d,e){a&&d?(e=document.createElement("div"),d.insertAdjacentElement(a.position,e)):(d=this._checkPlayerSelectorOnPage("stickyPlaylist"),a=d.player,d=d.playerElement,a&&d&&(e=document.createElement("div"),d.insertAdjacentElement(a.position,e)));return e};b.prototype._shouldOverrideElement= function(a){a=a.getAttribute("override-embed");return"true"===a||"false"===a?"true"===a:this._videoConfig.relatedSettings?this._videoConfig.relatedSettings.overrideEmbedLocation:!1};b.prototype._checkPageSelector=function(a,d,e){void 0===e&&(e=[]);return a&&d&&0===e.length?("/"!==window.location.pathname&&q.event("VideoUtils","getPlacementElement",Error("PSNF: ".concat(a," does not exist on the page"))),!1):!0};b.prototype._getElementSelector=function(a,d,e){if(d&&d.length>e)return d[e];q.event("VideoUtils", "getPlacementElement",Error("ESNF: ".concat(a," does not exist on the page")));return null};b.prototype._getPlacementElement=function(a){var d=a.pageSelector,e=a.elementSelector;a=a.skip;var f=da(d),g=f.valid,h=f.elements;f=S(f,["valid","elements"]);var k=ca(e),l=k.valid,p=k.elements;k=S(k,["valid","elements"]);return""===d||g?l?this._checkPageSelector(d,g,h)?this._getElementSelector(e,p,a)||null:null:(q.error("VideoUtils","getPlacementElement",Error("".concat(e," is not a valid selector")),k),null): (q.error("VideoUtils","getPlacementElement",Error("".concat(d," is not a valid selector")),f),null)};b.prototype._getEmbeddedPlayerType=function(a){(a=a.getAttribute("data-player-type"))&&"default"!==a||(a=this._videoConfig.relatedSettings?this._videoConfig.relatedSettings.defaultPlayerType:"static");this._stickyRelatedOnPage&&(a="static");return a};b.prototype._getMediaId=function(a){return(a=a.getAttribute("data-video-id"))?(this._relatedMediaIds.push(a),a):!1};b.prototype._createRelatedPlayer= function(a,d,e,f){"collapse"===d?this._createCollapsePlayer(a,e):"static"===d&&this._createStaticPlayer(a,e,f)};b.prototype._createCollapsePlayer=function(a,d){var e=this._checkPlayerSelectorOnPage("stickyRelated"),f=e.player;e=e.playerElement;var g=f?f:this._potentialPlayerMap.stationaryRelated[0];g&&g.playerId?(this._shouldOverrideElement(d)&&(d=this._getOverrideElement(f,e,d)),d=document.querySelector("#cls-video-container-".concat(a," > div"))||d,this._createStickyRelatedPlayer(y(y({},g),{mediaId:a}), d)):q.error(this._component,"_createCollapsePlayer","No video player found")};b.prototype._createStaticPlayer=function(a,d,e){this._potentialPlayerMap.stationaryRelated.length&&this._potentialPlayerMap.stationaryRelated[0].playerId?this._createStationaryRelatedPlayer(y(y({},this._potentialPlayerMap.stationaryRelated[0]),{mediaOrPlaylistId:a}),d,e):q.error(this._component,"_createStaticPlayer","No video player found")};b.prototype._shouldRunAutoplayPlayers=function(){return this._isVideoAllowedOnPage()&& (this._potentialPlayerMap.stickyRelated.length||this._potentialPlayerMap.stickyPlaylist.length)?!0:!1};b.prototype._determineAutoplayPlayers=function(){var a=this._component,d="VideoManagerComponent"===a,e=this._config;if(this._stickyRelatedOnPage)q.event(a,"stickyRelatedOnPage",d&&{device:e&&e.context.device,isDesktop:this._device}||{});else{var f=this._checkPlayerSelectorOnPage("stickyPlaylist"),g=f.player;f=f.playerElement;g&&g.playerId&&g.playlistId&&f?this._createPlaylistPlayer(g,f):q.event(a, "noStickyPlaylist",d&&{vendor:"none",device:e&&e.context.device,isDesktop:this._device}||{})}};b.prototype._initializeRelatedPlayers=function(a){for(var d=new Map,e=0;e