Connect with us

Tech

How to make lip-sync videos with Runway AI

Published

on

How to make lip-sync videos with Runway AI

With the recent announcement of their Gen-3 Alpha tools, Runway is set to take AI video creation to the next level. By the end of this guide, you’ll have a clear understanding of how to use Runway’s lip-syncing tool and be able to decide if it’s the right solution for your video creation needs.

As one of the early developers of the groundbreaking Stable Diffusion image generation platform, Runway brings a wealth of experience to all aspects of AI-powered image and video creation. One of Runway’s most accessible and popular features is its audio to video lip syncing tool. This technology allows users to create realistic talking head videos by matching lip movements to audio input. Whether you’re a content creator, educator, or business professional, this tool offers a range of applications for enhancing your video projects.

1. Log in

(Image: © Future)

Log into Runway. If it’s your first time using the AI platform, you can sign up using your Gmail account. 

2. Navigating the interface

(Image: © Future)

The video lip-sync function is under the audio menu on the Runway account dashboard alongside 3D tools, and image and video generation. Select your preferred voice from the options. 

What’s nice about the user interface is the liberal use of tutorial material at every stage, which is reassuring for beginners who are starting out on the AI media journey.

3. Input your prompt

(Image: © Future)

The lip-sync page itself is comfortingly simple and easy to use. The top left is the space for user inputs. This can be via a text prompt, an uploaded audio file or a tool for recording audio on the fly. 

If you upgrade to the Pro plan for $35 a month, and you can even record your own custom voice to use with an avatar.

4. Choose your voice

(Image: © Future)

I decided to upload an AI-generated avatar I created earlier, and combine it with a short script to test out the accuracy of the lip movements. 

There’s a great range of voices to choose from, which probably reflects Runway’s history of working closely with the movie industry, so there’s something for everyone. 

5. Generate your video

(Image: © Future)

Once you’ve done the audition and selection of voices, it’s just a matter of uploading a script (or pre-recorded audio clip) and pressing the Generate button. 

The free plan generates at a slower pace than the paid plans, so you have to wait for a few minutes in a queue before your request is actioned. The whole process takes around 3 to 4 minutes depending on the length of the video, and the server load at the time.

6. Understanding credit use

(Image: © Future)

Text to Speech uses 1 credit for every 50 text characters converted to speech, and lip sync consumes 5 credits per 1 sec of audio. 

The total video combines both costs, which in effect meant that my 38-second video used around 250 credits. Each credit is worth $0.01 USD. Standard Plan users get 625 credits each month for their $15 subscription.

The results…

Continue Reading