Apple #660: Green screens

Daily Apple reader Jim Bob wants to know why green screens aren't blue anymore, but green.  If you know what a green screen is, this question makes sense to you.  If you don't, this may sound like gibberish.  Or, if you are the film/video naif like I once was, the very existence of green screens may come as a total shock.  So let's begin answering Jim Bob's question by first talking about what a green screen is.

• So, you know how the TV meteorologist stands in front of a weather map and makes all sorts of gestures while the high and low pressure bars shift around and words like COLD! or HOT! suddenly appear?  The meteorologist isn't really standing in front of a screen that is dynamically doing all that behind him or her.  What's really back there is a plain screen -- which, these days, is green.

Weather lady gesturing in front of a plain green background on the left.  With the aid of technology, she winds up looking like the weather lady pointing out key features of a dynamically changing map.
(Photo from virtualstudio.tv)

• The screen behind the meteorologist might be called all sorts of things -- background screen, green screen, blue screen -- but they are all essentially the same thing.  This background is the blank into which your fancy background will eventually be placed.

• They do this same sort of thing with movies all the time, and more and more often as they're using so much digital filmmaking and special effects.  Your action super-hero isn't really dangling from a rope from a helicopter hundreds of feet above a chasm filled with fire and saber-toothed dragons.  No.  Your action super-hero is on some studio lot someplace, holding onto a rope in front of a green screen.  All the rest is added in later, and probably computer-generated besides. The saber-toothed dragons, of course, are real.

• Much of the Star Wars action scenes were filmed using green screens.  Or actually, blue screens.

• Ah, lost innocence.

For the newer Star Wars movies, they went with the blue screen.  It doesn't hurt my heart so much to look at this because the newer Star Wars movies couldn't hold a candle to the Luke Skywalker ones.
(Image posted by Voodoo Child and ComputerForums) 

• OK, so how do they swap out the green for all that dramatic scenery or the shifting weather map?  In answering this, I will answer Jim Bob's question about why the switch from blue to green.

• Back in the day, they used to use blue screens. This was because the filmmakers (TV or movie) could run the film through a red filter, which would make the blue black.  That would turn the shot of the meteorologist in front of the weather map into a black & white matte shot.

• Then they would overlay the shot of the weather map on top of the black & white shot, and voila, wherever there was black, that would get filled in with the weather map.  

• Same goes with filmmaking.  Actor is filmed in front of the background screen (keyhole image), the shot gets filtered so it becomes black & white (the mask), and the fancy background gets filled in wherever there's black (combined image).

Example showing the 3 steps of using a green screen: the keyhole shot of the person, the black & white filtered mask, and finally the combined image.
(Tiny image from Technologies for Worship)


Or maybe the process is best represented in 4 steps, as shown here.
(Image from Crystal Vision)

• For TV purposes, this was all done on the fly, live.  So sometimes you might see some fuzziness around the edges of the meteorologist, where the filter picked up some of the darker shades of the meteorologist's hair, or maybe the seams of his suit, or maybe the spots in his tie.  And since they were filming live, there wasn't much they could do about it.

• The problem wasn't so much that it was live TV, it was the color blue.  It was initially preferred because blue contrasts very well with white people's skin tones (and since white people are the ones on TV and in movies the most often, we're the most concerned with them. Ahem.)  But people tend to wear blue clothes.  Or they have blue eyes.  Even if it's not exactly the same shade of blue as the screen behind them, their blue clothes or their blue eyes could still get masked out.

• So you'd wind up with what happens in this mini-scene from Groundhog Day, in the clip below, from 4 seconds in to about 8 seconds. (Today is Groundhog Day, by the way. Don't forget your booties 'cause it's cold out there.)

• Producers used to sweat over this quite a bit, getting the actors ("the talent" ["Did he actually call himself 'the talent'?"]) or the TV guests or whoever to change out of whatever blue clothes they happened to be wearing into something not-blue.

• But then they hit on a novel solution: change the color of the screen.  Instead of making everybody change their clothes, make the background a different color.

• Now, they could have made the screen any color of the rainbow.  Especially with today's film & digital technology, the background screen could be any color at all--yellow, orange, magenta, periwinkle.  Whatever.  But they settled on neon green because people aren't very likely to be wearing that color, and it still provides a good contrast with most skin tones.

• On St. Patrick's Day, they might run into some difficulties.  But otherwise, the green screen has worked pretty well.

Life of Pi in front of the blue screen, and above a blue pit which is supposed to be the ocean. (That book was such a crock. He stole almost the entire plot from a 1981 novel, and then changed the ending so that it was all a dream.) I suppose maybe the movies stick with blue because they of course control what the people are wearing; they don't have to worry about anybody showing up unexpectedly wearing something that will match the background screen.
(Photo from SocialNewsDaily)

• If you were making your own movie, you wouldn't necessarily have to use a background that is neon green.  You could use a background that is of a different color.  But you'd better make sure your superhero's costume isn't the same color as your homemade background.

• You would also want to make sure there is no texture in your homemade background.  Say you're making a film in your high school hallway, where the walls are painted cinderblock.  That would not be a good idea because cinderblock has all sorts of pits and little holes in it, which will create shadows, and when you try to get your cinderblock background to turn black so you can swap it out, the shadows won't get picked up, so you'll wind up with your superhero standing in front of a background of the Grand Canyon with all sorts of pits and dots in it.

• Actually, an old-school green chalkboard (that's a pun but it's also accurate) works very well as a green screen.  But of course the chalkboard wouldn't go any lower than about the waist or mid-thigh, so that could present a problem.

• You also want to make sure your background and the person standing in front of it are lit separately.  Make sure you light the background evenly -- no hard spotlights that create hot white bright spots and shadows at the edges -- and try to keep your actor-person's shadow from falling on the background, which means make sure they stay far enough away from the green screen.  Shadows make that part of the background a different color, so when you try to mask it out, it will be harder to pick up.

I am certainly no expert, but based on the advice I read elsewhere, this looks to me like an example of what NOT to do. There is hard light hitting the top of the green screen, there is outdoor light mixing with the artificial light, the screen has rumples and creases in it which means it also has shadows.  The guy who worked on this set and posted the image says this set-up was "far from perfect," but acceptable given the budget and situation.
(Photo from Brian Brown, posted at DFInfo.net)


Here's a layout showing what kinds of lights should be placed where, and the formula for figuring out how far things should be placed from each other.
(Diagram and lots of helpful instructions from SteveoStudios)



This looks like a much more professional set-up. Notice, for example, how her shadow is falling forward, away from the green screen, yet her face is still brightly lit.
(Photo and more lighting tips from eyefish.tv)

• Also, if you're going to do any zooming in or out, you'd better make sure that the zooming you do of your person in front of the background matches the zooming you'll do in your fancy landscape.  Otherwise you might wind up with your person looking like they're becoming a giant against a fancy landscape that is growing, but not as fast as the person.  Generally, zooming is not recommended unless you're highly skilled because it's too dang hard to get things to match up right.

• You also want to make sure your camera while you're filming the foreground shots doesn't move.  Say you're using a handheld camera to film your friend Joe in front of the green chalkboard in the Economics classroom.  The entire shot is going to bounce and wiggle a bit, which may be OK on its own.  But once you mask out the green chalkboard and put your fancy background of computer-generated Antarctica with giant spiders behind your friend Joe, he's going to bounce and wiggle around while Antarctica stays still, and it will look obviously wrong and silly.

• That's all the mechanical stuff.  The big thing I wanted to know when I got to reading about this was, how do they get rid of the green?  Is there a magic anti-green button that they push?

• The answer is, kind of yes, but it's harder and more complicated than we'd all want it to be.  If you've ever used Photoshop and tried to swap out one background color for another, you know 1) basically how this process works; 2) it can be a real painstaking task when it seems like it should be such a simple thing to do.

• This process, by the way, is known as chrominance keying, or chroma keying for short.  Many people include using a green screen and managing the mechanics that I just described as part of the chroma keying process.  But for my money, it seems like the real chroma keying happens in post-production, when you're manipulating the images to get them to combine.

• For amateurs who want to chroma key, you could use a video editing software program like Final Cut Pro X to edit your movies and make combined shots.  Essentially, you would tell the software what color to look for -- the color of your background screen, and then you push the Matte button.  It turns that background color black and thus makes the black & white mask.

• But of course it's never so easy as that.  You might have to adjust the edges to make sure it's not ghosting -- leaving behind an outline -- or erasing parts of your actor's hair, or you might have to account for any shadows that you just couldn't get rid of when you filmed the background image, or maybe there are gradient differences in your background image and you have to deal with those one by one.

The Chroma Key Effect in Adobe Premiere.  Looks pretty simple, but that apparently simple toolbox holds a wealth of capabilities.  And it only gets more complex from here.
(Image from Media College)

In Final Cut Pro X, aligning the background shot with the foreground shot to make sure what your actor is doing makes sense in front of the fancy background you're going to give her.
(Screen shot from Apple Support)



Cleaning up an incompletely masked background in a combined image using Final Cut Pro X.
(Screen shot from Apple Support)

• So, making the black & white mask and ultimately your combined image can take a lot of work, and it's a bit of an artistic process in itself.

• That's even more true for TV & movie filming.  In those situations, their cameras are pulling in so much information, and thus generating so much data, they can't do the image combining on a PC. As powerful as our computers are today, they still can't keep up with all that data and do all the on-the-fly processing involved in chroma keying.

• So TV stations and filmmakers use hardware whose sole purpose is video mixing.  They might use a video mixer (or a vision mixer) which can do all sorts of things, like making transitions such as wipes from one shot to another, or putting together shots from different cameras to make a seamless single take, or adding effects like turning everything sepia-toned to make it look like an old movie.  Among the things a video mixer can do is combine shots & do chroma keying.

Lots of people recommend the Videonix MX-1 or MXPro video mixer for chroma keying.
(Image from Southern Advantage, whose current price is $2,500.  Amazon has some used ones for sale for $700.)



This Snell & Wilcox Kahuna HD/SD would be used by a large studio.  That word Kahuna is there for a reason.
(Image from Media College)

• For even greater and more sophisticated control of the chroma keying process, like if you're Peter Jackson and you want to make sure your Avatars look totally real in front of their fake alien background, then you would get a piece of hardware whose sole purpose is chroma keying.  You would use one of these if you have lots of other kinds of manipulations you want to do to the film, so you could do your other video mixing while you're doing the the chroma keying separately.  You would also get some really sophisticated color correction, lighting compensation, and edge correction capabilities that a video mixer or some PC software program simply wouldn't have.

The Safire 3 3G/HD/SD chroma key board. Does real-time chroma keying. It's supposedly a favorite among sports broadcasters because it uses algorithms to determine what color to mask out -- making it very good at dealing with complex background colors like grass.
(Image and product from Crystal Vision) 


Well, Jim Bob.  I think that about covers green & blue screens.  Next time you see a movie where the action hero is dangling from a rope from a helicopter, you will be more impressed, not so much with the actor, but with all the work that went into making it look like he really was doing such a damn fool thing.


Sources
Videomaker, How Does Green Screen Work?, and The ChromaKey Genie
Howstuffworks, How Blue Screens Work
Media College, Using Green Screen Footage and Vision Mixer
VirtualStudio.TV, What is Chroma Keying and how do green screens work? 
Technologies for Worship, The Basics of Video Keying
Sign Video, How Chroma Keying Works
Apple Support, Final Cut Pro X: Use chroma keys
Crystal Vision white paper, The Importance of the Chroma Keyer in a Virtual Studio System