#6. Make 2D character images lip sync to your voice, really easily!
Turn any character image into a lip-syncing avatar that you can use in product demos, to bring your children's stories to life or create virtual gift cards!
Helloooo AI Alchemists!!! 🤖🧪
I saw something reallllly cool on LinkedIn! A senior app designer who specialises in UI/UX, Steve Collins, created this epic lip-syncing avatar of himself. Steve used it to introduce himself to his LinkedIn network, using his own voice too. SO cool!!!
Check out the actual post here, and follow Steve on LinkedIn here.
After seeing this, I dove right into learning how to make one myself.
Here is the finished result: 🥳
If you’d like to create something like this yourself, this post is a step-by-step of the process I followed.
GenAI tools used:
ChatGPT-4 for generating the avatar image. You can also use Midjourney or Stable Diffusion, there are many options.
HeyGen for making the static avatar lip-sync to an audio file I recorded of my voice on my phone. I also tried D-iD, but preferred HeyGen as it was slightly more expressive with some extra head movements.
Pika for animating the static image of a bonus dinosaur I generated. You add an image and a short text prompt of what you want it to do.
Bonus tools used:
Voice recorder on my phone for the audio recording used by HeyGen to create the lip syncing.
Mac screen recorder to record a demo screencast of one of my 100 genai projects.
Canva to remove the background from all of the videos, and to create the video itself. It was way easier than a few video editing tools I tried first.
Let’s goooo! 🚀
Step 1: Generate an avatar of yourself
Use your favourite image generating tool to create a 3D avatar of yourself. In this case, I used ChatGPT-4 to generate 30 different avatar examples based on a description of myself, then chose my favourite one.
OpenAI recently released a prompt engineering guide with 6 different strategies for improving your prompts. I used it to help me come up with the following prompt, along with a photo of myself, which led to the avatar you see in the video.
Change the [text inside of the square brackets] to create an avatar that looks like you.
You are a skilled 3D avatar creator. I want to create a 3D avatar image that looks exactly like me. Please follow these steps to create the avatar:
Step 1: Begin by replicating my facial features, including my eyes, nose, mouth, and facial shape based on the attached photo.
Step 2: Accurately recreate my hairstyle and hair colour. My hair is [describe your hair]
Step 3: Ensure that the skin tone and complexion match mine.
Step 4: Clothing-wise, I like to wear [describe your preferred clothing style].
Step 5: The background should be pure white.
Step 6: The avatar should be in a natural and relaxed pose. [Describe the exact pose you want].
Please make the avatar as realistic as possible, paying attention to details and proportions.
Now, take a moment to assess the avatar's resemblance to me. Does it accurately represent my appearance and style?
Once the 3D avatar image closely resembles me and meets the criteria, consider the task successfully completed.
You can also use Midjourney or Stable Diffusion to generate your avatar images. Soo many options.
Step 2: Animate your avatars
Besides the avatar I created to look like myself (ish), I also generated a cute little dinosaur. I tried getting the different lip-syncing tools to get the dinosaur to speak but they weren’t able to recognise a face. It won’t be long before that’s possible.
I ended up using two different tools for animating my static ai-generated images. One for lip syncing the avatar and one for making the dinosaur move.
Option A. Use Pika to add movement to your images
To add movement to the little dinosaur, I signed into Pika, uploaded the dinosaur and just wrote the single word “smile” in the prompt. You don’t have to pay anything to use this.
With a single text-based prompt you can add movement to any static image. At the moment, the animations created are limited and often create what I like to call “cursed” results, where in one case the dinosaurs head caved in on itself which was terrifying (and hilarious). 🤯 🦕 🙈
Below are some experiments I ran with a the ai-generated dinosaur, and a couple photos of crochet toys I designed. The scarf and wiggle ears of the fox makes me so happy, and has the potential to totally change the world of toy marketing down the line. Anyone will be able to bring their favourite stuffed animal or lego character to life, and possibly create their own mini-movies.
Option B. Use HeyGen to lip-sync your avatar image
Getting my avatar to lip sync to my voice couldn’t have been easier. I signed up for the free tier of HeyGen, uploaded my avatar image, along with a voice recording I took on my phone, then clicked “Submit” or “Generate Video”, and that was it.
You get 2 free credits when you sign up. The 38 second generated video I shared at the start of this post cost me 1 credit to produce.
Before going with HeyGen, I tried out D-iD which is another great option for lip syncing. I would happily use both in the future. I went with HeyGen for this one because of the slight head tilts and the mouth movements were just a tiny bit more expressive to me.
Step 3: Use your avatar in a video
There are so many possibilities for how you can use your AI-generated avatars!
Children’s bedtime stories
If you’re a parent, you can bring your kid’s bedtime stories to life by getting them to help you design a character, and even the voices for them! You can also use ai-generated voices for even more variety.
Virtual greetings cards
Imagine getting a video of an avatar who looks like your friend sitting on top of a giant cake singing you happy birthday in their actual voice! Or you know, in Morgan Freeman’s voice 😉
Product demos
You can really easily create product demos where you get your avatar to present your product for you. You don’t even need to use your own voice if you’re really strapped for time. Though at the moment I’d say it’s more natural to do so than using ai-generated voices right now.
There are SO many possibilities!
I used Canva to really easily create my ai-generated product demo. It let me drag images and videos onto a canvas and arrange them really easily. It also made it really easy to remove the background of some of the ai-generated videos. The Pika videos for example have a black background by default.
Step 4: Share your videos and ideas!
If you decide to make one of these for yourself, please share your videos with us below in the comments, or tag me in a LinkedIn post 🥰
Until next time,
Stay sparkly ✨