[To return to the Focal Press e-newsletter, click here]
There’s a popular belief that a 3D movie is just a movie shot with two cameras. This would be true if making a 2D movie were just shooting it with a single camera. And what distinguishes a feature movie from a birthday party video is precisely not the camera, but all the work done before, during, and after the production on location. People who talk about two cameras are either ignorant of the moviemaking craft, or are insulting both 2D and 3D cinematography by reducing them to technical gestures when they are long and collaborative creative processes. Refusing to acknowledge the high complexity of 3D cinematography never made it simpler — it just made bad movies.
In this chapter we will see that 3D cinematography knowledge is an elusive and treacherous concept, the real thing being experience. We’ll focus on the interference of existing 2D experience, and the best way to fast-track your own acquisition of stereoscopic-imaging experience. We will then examine why the deployment of sound and color technologies offers very good analogies, helping us to cope with the current 3D evolution.
KNOWLEDGE IN 3D CINEMATOGRAPHY
Phil McNally, alias “Captain 3D” and a global stereoscopic supervisor at PDI, teaches stereoscopic 3D to hundreds of artists who have already mastered animation 3D. After many years of producing 3D movies, he is often quoted as saying: “One can teach the whole theory of stereoscopy in two hours. You can learn all about 3D moviemaking in two months. That will never give you the 10 years of experience needed to master it. Good movies are made with experience, not with knowledge.”
Another very experienced stereographer, Kommer Kleijn, says: “A good cinematographer can study 3D, and in two weeks, he’ll know how to avoid mistakes that hurt the audience, and make 3D work nicely. Nonetheless, he would use the 3D just as a technique, not as a full-fledged and compelling storytelling tool. He would produce a 3D-converted movie, not a 3D-intended movie. That would take him years to master. ”Let’s see how this impacts the crews starting a 3D project.
The treacherous 3D learning curve
One of the dangers you’ll face in learning 3D is that at first glance, it seems dead easy. Taking a pair of pictures, assembling them, and showing them in 3D is a no-brainer. You will master this in a few hours, if not in minutes. After a while, you will spend more time looking at your 3D art than it took to create it. This is the “free ride” part of learning 3D. The ratio of reward to complexity is huge, and this, by itself, explains most of the “3D curse” that made some of us 3D maniacs the moment we tried it. At first, making 3D is easy, enjoyable and highly rated as a water-cooler topic. Making basic 3D look good, or even great, is not complex at all, and within arms ’ reach of anyone who has some graphic background. After a while, you may want, or be asked, to venture into deeper waters. You will then discover that making complex 3D look good is no simple task. Failing with a single detail makes it look awful and painful, and most often instantly breaks the suspension of disbelief. Complex 3D can be incredibly hard to achieve. That’s the end of the free ride, and you are in for a couple days of cursing at the camera rig, and spending even more nights fixing shots. Eventually you’ll learn to avoid 3D’s minefields, and rebuild your productivity in this new creative universe you are exploring. You will have reached a comfortable plateau, and some colleagues will introduce you as “ the 3D guy. ”
As a result, 3D movie productions are sometimes initiated by crews who ran a couple of camera tests and feel confident they understand the basics and will figure out the details in needed time. They usually face two challenges: details are not just details but they totally break the 3D experience, and by the time you identify and understand your mistakes, it’s most likely too late to fix them. That’s not to mention the artistic creativity that just can’t surface while keeping the whole project afloat. And remember, 3D can be among the hardest things to fi x in post. As we’ll see, 3D quality is the result of end-to-end quality assessment all along the production process. If 3D was not properly handled at any given point, there’s no way it can be fixed later for a reasonable price. “ We’ll fix it in post ” is likely the single most expensive phrase ever pronounced in Hollywood (besides “ They won’t sue us ” ). In 3D, this is a dead end, and some movies would have needed to be fixed in post from end to end. 3D cinema history books are filled with stories of movies that were never released, or that performed poorly for this very reason.
The stereographers ’ paradoxes
The stereographers ’ paradoxes are an unfortunate confluence of events that prevents most existing 3D knowledge and experience from reaching the places where they are the most needed in the Hollywood system. They may be the side effect of hiring rules, 3D experts ’ behavior, the industry’s self-teaching habits, or a combination of the three.
The first stereographers’ paradox starts with the unwritten law in Hollywood that stipulates that you need to be working on medium-size movies for 10 years in order to work on a big movie. To some extent, it’s close to a paradoxical, “You need to be in the movie business to get into the movie business.” It continues with the fact that almost no 3D movies were produced until recently, and most of the recent 3D wave was produced by a handful of teams. If you don’t count the 3D movies where Peter Anderson, Robert Engle and Phil McNally were directing, shooting or supervising stereo, you are close to none. Beside these happy few big names, there are tens, if not hundreds, of talented and experienced stereographers who could bring their knowledge to the crews. They sure don’t have 10 years of experience in 3D feature movie making, and that’s for a good reason; there was no such production.
The second component of this paradoxical situation is, because of the novelty of digital 3D cinematography, most of the serious questions one may ask to a talented stereographer do not have a definite answer yet. Basically, he would know what you should not do, what has a good chance of working fine, and how to make sure that’s the case. And it’s not an easy sell to Hollywood executives to answer, “I don’t know the response to your question, but I’ll bill you to search for it.” Unless you have a proven track record of problem solving in the movie industry, which refers to the first paradox, you’re not likely to get a callback soon.
Actually, this is very good advice to use to find your 3D expert. Discard anyone that has an immediate response to your questions. Most likely he does not know that much about the complexity of stereoscopic cinematography. In some rare cases, he may actually know a lot about it, and makes the mistake of considering he has figured it all out. He will likely provide you with concrete-solid 3D expertise and prevent most big mistakes from happening, at the cost of killing all potential creativity from the rest of the crew. He will be remembered as the “3Dcan’t-do-this” guy. In 3D, one should beware of the know-it-all experts.
The usual consequence of these two paradoxes is that studios prefer to invest money in their research and professional development departments, rather than in hiring external knowledge. This trend is deeply rooted in the moviemaking community, where self-education on new techniques is a must. World class professionals are used to learn a new skill or tool on every project they join. When these persons meet a 3D expert who does not have feature movie credentials, and then read the “It’s just a pair of cameras” tale, they decide try to figure it out by themselves.
When they survive the project, they usually write papers for trade magazines, and they insist on 3D being much more than a pair of flat pictures.
EXPERIENCE AND 3D CINEMATOGRAPHY
Experience is a two-sided blade. Most of the time, it will help you. Sometimes, it will trap you in the confidence that you are doing the right thing when you are actually not. An experienced 2D professional will do his best at achieving two objectives: making the audience forget they are looking at a flat presentation of a 3D world, and cutting all possible corners to get the shots in the box, fast and cheap, without being caught by the deadline. These are the two major sources of problems that your 2D experience will smuggle into your 3D career.
You will want to build your 3D experience by working on it as if on a treadmill. You want it to get stronger, to the point where it will counterbalance your 2D experience and you’ll be able to intuitively infer parallaxes values, or instinctively detect mismatching pairs.
Watching 3D
3D experience starts with experiencing 3D. Go to see each and every 3D movie that is released. Get 3D demo reels on the internet, watch them, and critique them. You need to educate yourself and your crew about 3D for two reasons.
First, you want to build a 3D culture. There is hardly any 3D trick or effect that has not yet been used in a movie. Sometimes it was limited to one shot, sometimes it was not efficiently put together, but you’ll be surprised how much 3D cinematography has been reinventing itself, thanks to the long time span separating its reappearances on screen. And when you will watch recent 3D movies you’ll be able to catch on the fly all the references made to the old timers.
Second, you want to educate yourself and your visual system to 3D. Watching 3D is not a sport, but it involves muscular activity and reflexes. Practicing it is getting better at it. You’ll soon find yourself much quicker at fusing 3D images. This has a drawback and an advantage. The drawback is you will be tempted togo into a strong and dynamic 3D that you enjoy, but your audience may have trouble following you. The advantage is you can teach yourself to be much more efficient at catching 3D disparities and errors.
As a rule, give the artists free access to 3D displays, to 3D dailies in the theater. Just like you check a rough edit with the soundtrack, or check camera shots in color, you will want to work in 3D all throughout the production, not just at so-called “3D stations.” Not doing so is taking a chance to not catch a problem and have your project severely damaged when it needs quick fixes down the road. A sign of a team really caring about 3D is to see 3D glasses all over the place — on desks, on shelves, on monitors, in shirt pockets. Flood your facility with 3D glasses.
Forgetting 2D
We have already established that a 3D movie is much more than a pair of 2D animated pictures. It may be useful to repeat that mantra once again. It’s not the existence of these two shots that makes the 3D. It’s not their content. It’s the relationship between the two. The way they are matched makes the 3D effect. You need to master how each pixel is affected by any action in the moviemaking process and make sure it adequately affects the matched pixel on the other view. The audience’s brains will do a quality assurance pass on both pictures and consider each difference as a 3D cue. If it does not generate comfortable and enjoyable 3D, this very image disparity will be identified as a fault, and lead to visual glitches, hurt the suspension of disbelief, or even worse, give a visually induced headache.
Let’s examine the two most-often-used tools of 2D effects, known as “roto and paint” : rotoscoping is the cutout of elements, like foreground actors, and paint is the recreation of textures, say, to recreate a clean background plate. A regular process is to clone-stamp parts of the picture to remove unwanted objects like wires or markers. As seen in Chapter 8 both need a specific stereo pass otherwise they’ll break the 3D effect apart. Effects may have to be up to 10 times more accurate, as in Journey to the Center of the Earth, where the compositing artists sometimes had to shift images by a quarter of a pixel to have the actors “ stick on the ground. ”
Even in high-level artistic decisions like a shot composition, you have to think in 3D. Diagonals are a very good example. They are used to build a 2D picture, where they generate very dynamic strength lines. As soon as you see them in 3D they are just perspectives that fly away from the audience. They lose all their dynamic visual impact.
Remembering 2D
Some of your experience in 2D will be especially useful in 3D. Mostly the safeguards, protocols, and organizational tools will help you survive the production of 3D content with equipment and procedures that are not yet adapted and extended to 3D.
ASSET MANAGEMENT
The story of asset management in 3D starts with a well-known line: Your asset numbers will double. And your quality requirements will triple, for an unpaired left or right asset is a dead asset, with another dead asset somewhere in your data tree or tape vault, and its share of lost time and resources. Furthermore, this matching is not taken care of by default by most asset management tools that are not 3D-aware. To prevent this from happening, some teams stack the pictures together, as a side-by-side or over-under format in the digital realm. In our physical world this is done by color-coding the physical supports, say red for left and blue or green for right, as on a pair of anaglyph glasses. Usually, all the equipment, especially the cables, are likewise color coded. Regarding the cables, in order to avoid mismatching with analog RGB cabling, it may be wise to use other colors for left and right equipment, like orange and purple, with yellow being sometimes reserved for digital links.
RESOURCES MANAGEMENT
Making a 3D movie is much more effort than making a 2D movie, and the workload increase on your pipeline is pretty close to a factor of two. Basically, you will handle twice the amount of data, and this applies to storage, manipulation, and archiving. At this point you should be aware of your network bandwidth if you run a big facility. You can double the number of CPUs, hard drives and workstation screens. It’s costly, but, at least, it’s a known cost. You can’t double the network bandwidth, for it’s a fixed figure based on your network topography, most likely using gigabit Ethernet. File copying and moving will inherently tend to be twice as long. Then you will be handling twice as much rendering, and you will want renderings to match. The quality checks need to be in 3D, and will likely generate more re-renders for technical reasons. While a missing effect or inadequate render parameter on a preview pass would be acceptable in 2D, it will not go forth in 3D, because it makes this asset uneven with its matching asset and prevents adequate stereoscopic checking of the pair.
TRANSITIONING TO 3D CINEMA
In order to explain or assess the impact of the 3D renaissance on the entertainment industry, experts frequently use the deployment of sound or color technologies as a comparison. Lenny Lipton, the leading expert on 3D cinema, published a paper entitled “ The Last Great Innovation: The Stereoscopic Cinema ” in the November 2007 issue of Motion Imaging Journal, published by the Society of Motion Picture and Television Engineers (SMPTE). His article explains how the cinema extended his medium’s palette from black-and-white silent movies, to widescreen color movies with soundtracks, and, now, depth. This comparison is so useful, we will study it in this chapter, and we will bring it back throughout this book. It may even help you spontaneously find answers to questions regarding your 3D project. When you are facing a dilemma, or trying to evaluate the impact of 3D on your work or organization, ask yourself, “What would be the response if this was a black-and-white versus color, or a silence versus sound question?”
An interesting contribution from Phil McNally, alias “ Captain 3D, ” global stereoscopic supervisor at PDI, is to consider that all technical progress in the cinema industry brought us closer to the ultimate entertainment experience: the dream. We dream in color, with sound, in an incoherent world with no time reference. The cinema offers us a chance to dream awake for an hour. And because we dream in 3D, we ultimately want the cinema to be a 3D experience, not a flat one.
Comparison with color, a technical evolution
Conversion to color and conversion to 3D present many technical similarities. Earlier color systems relied on hand-painting the film, frame by frame, in a way that preceded the current labor-intensive 2D/3D conversion system based on roto and paint. Duplicating the camera, or more exactly, the film, was the first color cinema process, as the first generations of color camera used a beam splitter to illuminate two or three films in the red, green, and blue domains. Some dual projection systems were tried, but eventually, dye transfers were used to generate single-strip color films for exhibition.
The same evolution happened to the widescreen systems. Cinerama was the most famous, with its three cameras and three projectors set in high-end venues using a special screen. The early widescreen theaters were cashing in on the very same business model that the current large format 3D theaters like the IMAX 3D have been using since the 1980s and 1990s. They used multiple camera and multiple projectors, an improved sound system, and presented movies loaded with scenes shot from an airplane.
FIGURE 3.2The 70 mm conversion of a Cinerama triptych, showing vertical lines and vignetting artifacts where the three strips of film are sewn together.
|
FIGURE 3.1
Three cameras (C1, C2, and C3) were assembled in a rig (A) in the Cinerama process. The camera and projection system had the same geometry and synchronism issues as 3D ones.
|
Soon, the various multicamera systems were killed by the improvements of film stocks. Increased finesse and sensitivity opened the path to light-efficient color layering and the compression of a wide picture inside an academic aperture. The invention by Eastman Kodak of the three-layer Kodachrome film eventually made the multicamera color rigs useless and turned any camera into a color camera. The anamorphic lenses brought wide aspect ratios to any camera and projector. Eventually, even multiple soundtracks found their way to the fi lm strip too, using various forms of visual encoding.
There have been many attempts to apply the fi lm-gain leverage to stereoscopic production. The 1970s and 1980s 3D revival was mostly based on the single-strip format, using over-and-under, side-by-side, or even in-camera anaglyph processes. The camera bulkiness, postproduction complexity, and exhibition requirements were taken care of once and for all at the camera lens. Why could it not bring 3D to full fruition? Three reasons: one design flaw and two missing links.
First, you need the camera to be bulky to artistically accommodate any scene inside a 3D movie. Whether it be a super-wide interocular distance for hyper stereo, or a beam-splitter configuration with a giant mirror and short wide angle lens, good 3D does not come from a small camera. And then, you need some sort of real-time control, for directors and cinematographers are graphic artists. They do not want to create visual mediums by blindly relying on trigonometric formulas. And eventually, because producing perfect 3D is artistically a challenge and mechanically a chimera, you need to be able to fi x images at very low cost. These functions were not provided by the analog postproduction processes. To some extent, the digitization of the production pipeline allows for virtually putting the two eyes inside a single support, a single digital fi le, something the physical film stock could not do to the extent needed for perfect 3D quality. And there’s no 3D quality but perfection, digital perfection.
That transition, where the 3D movie is no longer a pair of 2D movies, has yet to reach the camera. Some companies are working on such an approach, with mechanical, optoelectronic, digital, and software integration into a single digital 3D camera, not a 3D rig. This was, as of 2009, the most challenging and promising area of development for 3D cinematography.
Despite all technical progress, 3D production will never come as cheap as 2D. After almost a century, producing a black-and-white or silent movie would still be cheaper. Three-D is initiated by additional visual stimuli and generates more emotions. There’s no such a thing as a free lunch, and 3D can’t and won’t come for free.
In the case of color, the release of Kodachrome single-strip color stock made all the existing equipment color-capable. Cameras, Moviolas, projectors — everything was manipulating color in an overnight makeover. The same is not likely to happen in 3D, even if some links in the production chain were to enjoy such an unexpected retrofit, most notably the digital projectors made 3D-capable by the simple addition of filters and glasses. Some companies have developed 3D encoding systems that are 2D compatible, and allows the processing of 3D footage through, say, a 2D OB van and a 2D satellite link. We can foresee the conversion of oversized 1080 60p equipment retrofitted into HD-3D television at 30p.
As we have seen, the underlying technical reasons of the success of color or widescreen cinema deployment gives us some clue as to whether the 3D may be passing hype, or stay for good. It mostly underlines how much the pipeline digitization is the key element that will free the 3D from its drawbacks.
Comparison with sound, a storytelling revolution
It is close to impossible to tell a story without sound or speech. When the movies were silent, directors found a way to put some of the soundtrack into the picture with billboards stating the actors’ lines. In the very same way, we are loading our 2D movies with monoscopic depth cues. We feel the urge to present the set depth even on a 2D medium. There is not such a compelling need to put the color in a movie, and we have watched color movies on black-and-white TV sets for many years. This is a strong common point between sound and depth.
In the fi rst years of the sound era, two technologies fought for dominance: sound on a disk and sound on the fi lm. Sound on disk was simpler to put together, but relied on mechanical synchronization of the record player with the fi lm projector. And this came with the same issues as the dual projector 3D systems. If there is some chance for failure, time, statistics, and luck will make sure it happens. Because it was much more foolproof, the sound-on-film system eventually caught up for large deployment in small town theaters. It took eight years, from 1919 to 1927, from Lee De Forest’s first patents to the agreement between the big five studios on a common sound-on-film system. The impact on how the distribution and exhibition businesses were conducted was minimal, but it had a huge effect on the production world.
The addition of sound changed the movie industry from end to end. Writing, acting, directing, editing — each and every step in the creation process of the movies was impacted. People and equipment had to remain silent on stage, directors used close-up to emphasis on the lip-syncing effect, and music was integrated into the creativity process. Cinematography was actually affected to the point that many consider cinema as having a silent age before it entered talking’s reign. You don’t make such a distinction with the introduction of color.
It is not yet known if 3D cinema will be referred to as a new era for movies. Nonetheless, 3D will deeply impact the movie production processes in two ways. Inside each department, adjustment will be needed to accommodate new constraints and exploit new opportunities. Across departments, new coherency tools and synchronization procedures will have to be implemented to coordinate the work on 3D, to maintain the pristine quality it requires.
MOVIE PROJECTS AND 3D CINEMATOGRAPHY
If you are reading this book, you are most likely doing so with a movie project on your desk or on your bedside table. That project can be 3D, and you are evaluating it, or 2D and you are considering adapting it to 3D. Your decision to go 3D will depend on many factors, and one of the purposes of this book is to help you understand if, why, and how you should, or should not, do this movie in 3D. Not knowing the project you are working on, we will likely have hard time answering the “if” part. You’ll nonetheless find clues to answer by yourself the “why and how” dilemma. If you are not actually working on a project, pick one from your classic movies collection and start planning to remake it in 3D.
Movies, projects, and productions come in every size, from home hobby to super-productions costing hundreds of millions of dollars. Yet all 3D movie projects are the same, and throughout this book you will fi nd examples based on three typical projects: amateur, low budget, and big budget. Why would we bother look at low-budget productions when it comes to high-end entertainment industry techniques? Because it’s important to understand that, just like 2D movies, if 3D movies benefit from high-end tools and big crews, the real quality comes from actual medium mastering from the director. This book will provide you with the tools and blueprints that allow you to make a nice 3D shot for free. If you can’t do that, no millions of dollars will bring it to you, unless you are clever enough to spend a chunk of a million hiring experienced stereographers and have them help you expedite the cumbersome, and sometime painful, 3D trial-and-error phase.
These are the three typical main roads to enter into the world of stereoscopic cinema: As a lone voyager freely wandering around, as a small group with limited resources and higher expectations, or as an army of professionals with the highest quality objectives. Before this book addresses the nuts and bolts used by each of them, let’s define our typical journeyer.
Level one: Amateur movie and self-teaching 3D
If you want to have a personal journey into 3D, traveling does not have to be expensive. We’ll see how to reuse existing equipment and how to reach the last mile for free.
You want to make 3D pictures, show them to friends, or publish them on the internet. You want to make short 3D animation, and maybe a couple 3D video shoots using a low-end 3D rig. The workshops in this book will show you how to make great 3D pictures, and some 3D cinematography for free, or almost free. All you’ll need is a personal computer, a digital still camera and the 3D glasses and the DVD provided in this book. You will be using free software, and get references for cheap equipment you may consider buying, like pairs of webcams or shareware, for a few dollars. You may be interested in going one step further and try real 3D shooting, editing, and posting. You’ll then enter Level Two.
Level two: Student movies and low-budget movies
You can be leading a small exploration team mapping its corner of the 3D jungle. This 3D project can be a student movie, or a hands-on professional training, a warm-up skill acquisition for your production crew, or even a small budget commercial. Say that you want to produce a short that would look nice on your 60-inch 3D television as well as on the 30-foot screen at the digital cinema down the street. You own or rent some 2D movie gear, have access to workstations loaded with serious production software, and plan to spend a few hundred dollars in 3D shopping or rental.
The advanced workshops will show you how to use generic 2D equipment to produce a 3D movie. The Level-One proficiency will be needed to really benefit from them. They will cover the hardware side, like pairing an HD video camera into a 3D rig, and the software side, like using Adobe Premiere and After Effects, or Sony Vegas to post some 3D content. You will be presented a few items recommended for acquisition or rental, like 3D displays and accessories, camera synchronization electronics, and 3D plug-ins for 2D software.
It would be advisable, for a Level-Two project, to look for a “3D maniac” in your neighborhood. They are plenty and you are most likely to have one in your family or circle of friends. In a company related in any way to the cinema industry, the likelihood of finding hidden 3D experience among the employees is extremely high.
Level three: Feature 3D movies
Level Three projects are feature movies, or any short that aims at a feature-like quality level. We are talking about large crews, highly skilled and specialized. We are talking millions of dollars and weeks of production. If you are in operations management, you will need to build, buy, or rent one or more 3D rigs, check all your equipment against 3D compatibility, upgrade your screening room to 3D, and prepare your pipeline to gulp and spit at least twice as much data as it used to. This book will provide you with a road map for running a 3D studio and a good overview of the pitfalls. If you are a producer or a director, you will have to ask your teams for additional work, with a whole new technical and artistic vocabulary. You’ll need to improve your quality assurance tools, and tighten the bonds between remote teams.
No project of that scope can succeed without hiring a 3D expert from the very beginning. Producing a Level Two project to get the technical team together and up to speed with stereo is a must.
■ The need for 3D professionals in the movie industry is ramping up.
1. 3D is making a strong comeback with 17 movies slated for release in 2009.
2. Even if all cinema is not going 3D, you surely will do some 3D in the next few years.
■ Current experience in 3D moviemaking in the studios is scarce.
1. Since the 1950s, very few 3D movies where produced until this 3D renaissance.
2. This renaissance is a side effect of the recent glass-to-glass digitization and relies on cutting-edge production techniques.
3. The recruitment rules work against the hiring of 3D experience in a “stereographers’ paradox.”
■ Acquiring 3D experience requires a lot more practice than you would suspect.
1. The amount of knowledge is not that huge; it may even be possible to put most of it in a 250-page book. Knowledge is not that important in 3D; experience is the key.
2. Nobody can give you experience but yourself, by practicing.
3. You’ll need to build or acquire some 3D tools, and use them a lot.
4. This is the only way you’ll experience, understand, and overcome the “3D learning curve.”
This chapter is brought to you by...
3D Movie Making by Bernard Mendiburu
[To return to the Focal Press e-newsletter, click here]