VFXhack

Just finished this breakdown for the VFX from the “Fringe” pilot. Enjoy!

The Journey Begins

The pilot for J.J. Abrams new series “Fringe” takes the viewer on a mysterious adventure through the world of fringe science. The idea being, that everything that happens in the show is just outside the realm of what we currently understand to be possible. To create this illusion the show required a series of “invisible” visual effects. Anything that happens in the course of the show has to be convincing because the dramatic source of the effects isn’t magic or some alien world but the logical extension of what we know to be true and possible. VFX Kevin Blank, took this seriously from the beginning of the show mandating to all the creative people involved that realism was the order of the day. To that end innovative workflows and techniques were developed to ensure the seamless marriage of live action and CG elements.

Digital Prosthetics

The Airplane

The show begins with a virus that infects unsuspecting airline passengers and later takes down one of the main characters. This concept posed an interesting VFX challenge. The natural methodology for this was a make-up application that created the illusion of dripping skin. However, in the climax of the sequence the writers wanted to take this idea to the very limit; thereby having a co-pilot’s melting face, accelerate to the point that his jaw actually unhinged and fell to the ground.

The bases of the effect was the done in practical make-up, with the added challenge that his particular vfx shot, of the jaw dropping itself was actually conceived after the footage on the plane was shot. Which meant that there were no tracking markers available for the movement of the face. A process dubbed “Performance Capture” was created for this shot which involved taking many 2d tracks from a high contrast version of the plate and transferring the 2d tracks from After Effects to a 3d mesh in Maya. The 3d mesh in this case being a optimized scan of one of the other actors in the show.

Once the camera track and face match move were complete dynamic motion was added to the shot. A simple facial rig was created to drop the jaw. An nCloth simulation was performed in Maya to generate correct motion for the drooping folds of flesh. Maya hair curves were used to create the motion for the sticky strands of mucus that extend from the face to hand. Texturing of jaw and mouth interior began with a projection of the plate photography that was then taken into Photoshop where detail was added. The shot was rendered in passes separating the diffuse, reflection, specular and ambient occlusion passes for reconstitution in composite. Compositing and integration was an enormous challenge for this shot. Not only did the CG elements have to blend seamlessly with the complex make-up work on the face, but the flashes of lightning illuminating the cockpit added an addition level of complexity. Extensive roto also needed to be done on the pilot’s shirt to create previously obscured background when the jaw falls away from the rest of the face. Along with all the render passes supplied by CG, a entirely additional set of renders was needed for this shot. The CG was rendered all the way through with the lightning “on” and again “off”. With two complete renders of each set-up the compositor was able to control the timing and intensity of the lightning without having to send the shot back to CG.

The Invisible Man

Later in the show, the virus that ravaged the airplane infects an FBI investigator. Luckily, he is taken to high-tech facility where the degrading of his flesh is slowed to keep him alive. As his disease progresses, the character’s skin, bones and organs get more and more transparent to the point where the view can see through his body to reveal his inner workings. Many methodologies were considered before settling on the final approach. The base of the effect was a complex make-up application by Gordon Smith. The first layer of transparence was created practically by the application of 3d tattoos, images of a medically accurate 3d model printed on silicon and glued onto the surface of the actors skin. Then veins were painted on the body followed by a translucent layer of gel to create the look of the top layer of skin. All of the internal body systems were created in 3d and integrated in composite with the actor in makeup.

The 3d effects for these shots started out with a medically accurate model of all system of the human body. This model was purchased from a vendor specializing in medical illustration and broken out into its component parts (skin, skeleton, organs, veins etc.) and re-textured for greater realism. The internal systems then needed to be deformed to conform to the skin of the model derived from the actor that was in the scene. After the model and texture was complete it needed to be matched to the moving photography with a great degree of accuracy. This was accomplished using a combination of 3d and 2d techniques. 3d camera moves were created in syntheses and refinements were made using 2d tracking points to warp the 3d image into place. The model with then lit and reflections were added using extremely high resolution HDRI panoramas of the set and all of its lighting elements. Again the 3d elements were rendered in passes, each body system (complete with animating heart, lungs and blood flow) rendered separately along with its component parts. Areas of detail were lifted from the make-up application in compositing and laid back over the CG to enhance integration and realism.

A Mechanical Arm

Another effect in the show that called for an extreme amount of interaction between CG and live action was the mechanical prosthetic arm used by the show’s villain. The script called for the actress to peel back her skin like a glove to reveal the metal and plastic underneath. The skin was a latex appliance that could be peeled off on camera and was color corrected to match the skin tone of the actress. The tracking and match moving on these shots was a particular challenge as the camera was in motion for most shots and movement of the arm needed to exactly match the motion of the actress on set. To help with the match moving, green make-up was applied to the actresses arm and tracking markers were painted onto the skin so they would not interfere with the motion of removing the skin glove.

The CG arm was designed and modeled prior to shooting making sure to reference the current level of robotic technology. The reflective surface of the arm was achieved using on set reference and by once again rendering out the 3d object split out into passes to adjust the level of secularity and reflection on a shot by shot basis.

The Dreamscape

In the story of Fringe the only way for one of the hero agents to contact her ailing partner is to ender his dreams by was of a sensory deprivation tank. This launched her into a surreal environment populated by her dreams and memories. This dreamscape was completely created in post and all of the performance footage was shot on green screen. In order to create the dreamy feel of the sequence large sweeping techocrane moves were used in combination with the actors on a turntable. This created many complex tracking issues solved mostly by BouJou however hand tracking was also used. The environment themselves were created using a combination of 3d elements and panoramic photography. Foreground and mid-ground elements were modeled and render in Maya while distant imagery was created by mapping 360 degree panoramas onto large CG spheres. The green screen plates and CG elements were color corrected together for the final integration.

Dramatic Conclusions

It took a lot of planning flawless execution to create the effects seen in the Fringe pilot. With the need for a suspension of disbelief, the science of Fringe had to come off to the viewer as real science just pushed a bit further. Using the advanced techniques available to the VFX industry today the Fringe team took the audience along on a journey that has only just begun.