Automatic Muppet-style “lip-flap” or “lipsync” based on the volume/amplitude of a SoundChannel

I recently made a couple of games for a kids website where the character (let’s call him Monty) was on screen and talking to the player a lot, and we needed his mouth to flap open and shut in time with the audio. It didn’t need to be super accurate, because of the type of character he was (kids are used to seeing very rough lipsync on this guy), and plus, he had way too much to say to manually lipsync it all. So I came up with this code to make his mouth open and shut based on the audio that was playing on a sound channel.

First, make the character’s mouth. Nothing fancy, just a 20 frame movieclip, where frame 1 is all the way shut, and frame 20 is all the way open. You can shape or motion tween tween it, it doesn’t matter. Put a stop() on the first frame so he doesn’t start yapping before his audio starts.

Now let’s play a sound, on a new sound channel dedicated to the character. We don’t want to put anything else on this channel, such as sound effects, because anything on this channel will make his mouth open and shut.

var monteChannel:SoundChannel;
monteChannel = soundRef.play();

Now lets make a function that runs on enter_frame (meaning it runs 30 times per second if your SWF is a 30fps). This function will check the volume of the audio on montyChannel (using the channel’s leftPeak property), do some basic math to turn it into the frame number of the mouth, then send that number to the mouth MovieClip.

addEventListener(Event.ENTER_FRAME, checkVolume);

function checkVolume(e) {
	// number of frames in the mouth animation
	var framesInMouth = 20; 
	// set this to make the mouth open further if 
	// the volume of your audio is a little low
	var mouthMultiplier = 1.5;  
	var mouthFrame = Math.ceil(montyChannel.leftPeak * framesInMouth);
	mouthFrame = mouthFrame * mouthMultiplier;
	if (mouthFrame > framesInMouth) {mouthFrame = 20;};
	referenceToMouthMovieClip.gotoAndStop(mouthFrame);
}

Note that I have referenceToMouthMovieClip there, because the way you set up your project will determine how you need to refer to your character’s mouth.

3 thoughts on “Automatic Muppet-style “lip-flap” or “lipsync” based on the volume/amplitude of a SoundChannel”

  1. How did the lip flap turn out? Is it decent, or very crude? I’m looking to do something similar, and found your post. Can you point me to your app so I can try it?

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>