How do reference images keep a character consistent in Veo videos?

What is Veo, and what can it do?

Veo is a generative AI model designed to create videos from images and prompts. Specifically, Veo 3.1 is a model that generates videos up to 8 seconds long, offering resolutions of 720p, 1080p, or 4k. This model is accessible programmatically through the Gemini API and is generally available for production on Vertex AI.

Veo 3.1 builds on previous versions with stronger adherence to prompts and improved audiovisual quality when transforming images into videos. It supports variable clip lengths, allowing you to create videos of 4, 6, or 8 seconds. You can also choose between landscape (16:9) and portrait (9:16) aspect ratios for your generated content. Veo 3.1 also includes natively generated audio, which is synchronized with the video content.

How do reference images help with consistency?

Reference images are crucial for maintaining a consistent look and feel across your generated video clips. Google's Veo 3.1 allows you to provide up to three reference images to guide the content of your video. This feature, referred to as "ingredients to video," helps ensure that elements like a specific scene, character, object, or even an overall style remain consistent throughout multiple shots.

By using reference images, you can establish a visual anchor for your video. For example, if you are creating a series of clips featuring the same character or product, providing a reference image of that character or product helps Veo maintain its appearance consistently across different generated scenes. This ensures a cohesive aesthetic, which is vital for brand recognition and storytelling.

How do first and last frames create seamless transitions?

Beyond maintaining consistency within a single clip, Veo 3.1 also offers a feature to create seamless transitions between different video segments. You can generate a video by specifying both a first and a last frame. This capability allows the model to produce a natural video transition that smoothly connects a provided starting image to an ending image.

This "first and last frame" feature is particularly useful for creating a flowing narrative or connecting distinct scenes without abrupt cuts. The model generates not only the visual transition but also includes synchronized audio, ensuring that the sound complements the visual change for a more polished and immersive viewing experience.

What video formats and lengths does Veo support?

Veo 3.1 offers flexibility in the format and length of the videos it generates. You can choose from standard resolutions such as 720p, 1080p, or 4k, providing options for various display needs. However, it's important to note that 4k resolution is not available for Veo 3.1 Lite, and video extension capabilities are limited to 720p videos.

For video length, Veo 3.1 allows you to create clips of 4, 6, or 8 seconds. This range is suitable for short-form content platforms. Additionally, you can select the aspect ratio, which describes the proportional relationship between the width and height of the video. Veo supports both landscape (16:9) and portrait (9:16) orientations, catering to different viewing preferences and platform requirements.

How does Veo handle audio in generated videos?

Veo 3.1 is designed to generate videos with natively produced audio, ensuring a rich and synchronized sound experience. The model excels at creating realistic soundscapes that are guided by your prompt. This includes everything from multi-person conversations to precisely timed sound effects, all integrated seamlessly with the visual content.

The ability to generate high-quality, synchronized audio is a significant feature, as it enhances the overall impact and realism of the video. Whether you are creating a video with a natural transition between frames or a clip with specific actions, Veo 3.1 ensures that the audio elements, including dialogue and sound effects, align perfectly with the visuals, contributing to a more immersive and professional-grade output.

What is the recommended formula for optimal control?

For optimal control over the video generation process with Veo 3.1, Google suggests a five-part formula when crafting your prompts. This structured approach helps you provide clear and comprehensive instructions to the AI model, leading to more precise and desired outcomes.

The recommended formula breaks down the prompt into distinct components: [Cinematography], [Subject], [Action], [Context], and [Style & Ambiance]. By addressing each of these elements, you can guide Veo to produce videos that closely match your creative vision, from the camera work and the main focus of the video to the specific activities taking place, the surrounding environment, and the overall mood or aesthetic.

Common questions

What is Veo 3.1?

Veo 3.1 is a generative AI model from Google that creates videos from images and text prompts. It is available programmatically through the Gemini API and is stable for production on Vertex AI, offering stronger prompt adherence and improved audiovisual quality.

How many reference images can I use with Veo?

You can use up to three reference images to guide the content of your generated video. These images help maintain a consistent aesthetic across multiple shots, whether for a scene, character, object, or specific style.

What video resolutions can Veo generate?

Veo 3.1 can directly generate videos in 720p, 1080p, or 4k resolutions. However, 4k resolution is not available for Veo 3.1 Lite, and video extension is limited to 720p videos.

Can Veo generate audio for videos?

Yes, Veo 3.1 generates videos with natively produced audio. It excels at creating realistic, synchronized sound, including multi-person conversations and precisely timed sound effects, all guided by the prompt.

What are the available video lengths in Veo?

Veo 3.1 allows you to create video clips with variable lengths of 4, 6, or 8 seconds.