The Visual FX Production Pipeline

 

The Visual FX pipeline is somewhat flexible in its structure and can vary in different studios.

The Pre-production phase consists of research and development, storyboarding, basic tests and modelling. This is usually roughly done to begin with, so the team will be able to pre-visualise how the visual fx will work once the final models have been completed.

The Production phase is when the final high resolution modelling begins and once completed they're rigged to be animated. When the video shoot begins, the selected sequences of the scenes to include visual fx will have numerous reference photos taken. This is useful for the modelling, texturing and lighting.

The Post-Production phase is when the shoot has finished and the editing begins, the selected film is scanned and motion tracking begins. The camera's lens measurements would have been taken on set so they could be applied to the CG camera in order to get the correct focal length. Once an accurate camera track exists, any match moving or body tracking can be done. A body tracking is an animated CG character or object which perfectly mimics what the on set equivalent did. This would be used for things such as placing an CG element onto a on-set character.

 

The last process in the pipeline is compositing. This is where all the CG elements and the scanned images are brought together to create a seamless finished image. Once the compositing is approved, it is supplied back to the client in the same form as the scan was received, usually a series of digital image files. The client can then use these finished composited frames for creative grading and eventually to be shot out onto film or DCP for release in cinemas.

Fill, Back & Background Lighting

In every environment there is the Key Light. This is the main light shown within a scene. For example, if it was outside it would be the sun light or it would be the light within a studio.

 

The Purpose of the fill light is to partially fill in shadows created from horizontal and vertical angles of the key light. The fill light in an ideal world should be 90 degrees away from the key light and about 45 degrees from the camera. By lighting a full 90 degree afea, an important margin of safety is created in case subjects move unexpected during the production, and prevents the need for having to have the camera angles changed and creating a possible costly delay. A fill light should be softer than the key as it is then able to subtly fill in some of the key's shadows without creating a second catchlight in the eyes.

 

When outside and the sun is being used as a key, a reflector board may be used and positioned at about 90 degrees from the sun to reflect the light into the shadow areas. Large white styrofoam or foam core boards are oftern used during closeups.

 

The back light or sometimes reffered to as a hair light is placed directly behind the subject in relation to the close-up camera. They purpose is to light up the back of subjects. Back lights alone can be used to create a silhouette effect.

 

The background light illuminates the front of a background area and adds a depth and separation between scene elements.

 

Any type of light can be used as a background light as long as it provides fairly even illumination across the background, does not hit the central subject matter, and is at the appropriate intensity.

If the background has detail or texture, you would want to put the background light on the same side as the key. This keeps the dominant light consistent in the scene.

White & Colour Balancing

In white balancing, a sensor on the camera averages the light in the scene and automatically adjusts the camera's internal colour circuitry to zero out any generalised colour bias. Although this works reasonably well under the proper conditions, automatic circuitry can't be relied on to always produce accurate colour. So skilled camera operators adjust the white balance themselves. White card or paper is useful to test with. With the camera zoomed in full frame on a pure white card, the operator pushes a white balance button and the camera's chroma channels will automatically adjust to produce pure white. When the dominant light source in a scene changes in any way, you must again white balance your camera.

Going from sunlight to shadow will necessitate white balancing the camera again, as will moving from outside to inside. When shooting outside, even the passing of a few hours will result in a slight colour shift in illumination.

If you do not white or colour balance your camera, you risk scene-to-scene colour changes. This is especially noticeable with skin tones in multiple-camera productions.

 

Colour balancing is much like White balancing and is easy to to control, however can be difficult when using more than one camera for a scene. Some studio and field cameras are designed to use a "smart card." About the size of a credit card, it records all the parameters on the first camera you (carefully) set up. Then when you insert this card into successive cameras, they will adjust them to conform to the first camera's parameters. Studio engineers use a central camera control unit to adjust all studio cameras at a central location.

Colour reproduction is subjective since the human eye can be fooled quite easily to not be able to detect the colour variations between cameras. Studies have shown that people prefer the colours on their TV sets to be more saturated than in real life. This preference does not mean that the hues are inaccurate but the colours are more stronger to the human eye.

Camera Types

High Definition

 

In 2000, the DVCPRO HD codec was launched by Panasonic. They were used as full size DVCPRO cassettes. This format was designed to be used in professional camcorders and later on in 2003, HDV was released as a much more affordable method to capturing HD video due to the use of inexpensive MiniDV cassettes. This started being implemented by companies such as Sony, Canon & Sharp with their equipment and is a popular format used today.

Tapeless/File Formating

 

The first tapeless video format was produced in 2003 by Sony using PFDs. In 2004, DVCPRO-HD began using memory cards as is recording medium by Panasonic. Later in 2006 companies introduced AVCHD as as inexpensive consumer-grade HD video format which was tapless. This format is the most common used by non-proffesionals. In 2007 Sony released XDCAM EX which records onto SxS memory cards.

3D Cameras

3D cameras became available to the general market in 2010 by Panasonic, this was due to the sudden raise of interest in 3D by films such as Avatar. This form of camera uses two lenses to capture both sides of the video and then outputs the two sides together to create the 3D video which can be seen on screens that are able to produce the 2 signals and wearing 3D glasses.

Final Video Recording

After planning how my video would be recorded, I now recorded my final version which will be used with MatchMover to add the Visual FX. In this video, I was concious of keeping the white balance stable and used a few lights to even out the light in the room. The more light, the better.

 

I decided to record this in a different room as I wouldnt be disturbed and had more flexibility of movement. I used the camera on a tri-pod for some sections and placed it on a office chair with wheels to move it across the room, allowing me to create the tracking shots.

 

The shot footage turned out rather well. The movement was as smooth as I could make it. The light is nice and vibrant (apart from a bit of shadow towards the end) and there's no sign of blurred vision. This will work well with MatchMover's tracking points.

Camera Tests

 

In order to make sure I knew exactly how the video would need to be filmed. I tried some demo shots to work out if these would be possible to be tracked efficiently, when the video is imported into Match Mover.

 

This first video I recorded was a quick video shot from my Camera Phone to work out the directions of movement, the scene changes. Timing wasn't considered in this video.

 

 

This next video was recorded with a professional Camera recorder with the most accuracy I was able to achieve to the final video. Taking into account of camera movements, scene changes and timing. Although the timing is more accurate, the camera movements are too shakey and need to be more still to keep the video from becoming blurly as this will make it more difficult to add efficient tracks to the video.

 

 

 

Chosen Visual FX Idea

 

When I came towards thinking of my idea for what I would like to film and include an 3D object within a real environment. I kept thinking about using an Aircraft land in an unlikely place such as a public park, so I created a storyboard. The idea was relatively simple, but didn't excite me at all...

 

 

So I decided to try again and this time I thought about making a simple game character enter the real world and then I could imagine our Animation/Computer room being ideal to film this in. The first and most simple non deforming game character I could think about was Pacman. This would work well as he could come out of the computer screen and continue to collect the pellets around the room, whilst a ghost chases him. So I worked on a storyboard to develop this idea... Trying to keep it simple, yet as visually interesting as possible.

 

 

 

 

After deciding this to be my chosen idea for my Visual FX piece. I then created an Animatic of the storyboard to display more clearly what I had in mind. I also point out what type of camera shots I will require for each section of the scene.

Practice Scene

To begin learning how to do visual effects. I first went outside and captured about 2 seconds worth of footage. In order for the Match Mover software to pick out points for the 3D space overlay, it's important to have some markers. These are known as Trackers... To effectively use these, it's best to find points which meet in the middle and keep the camera nice and steady, so it doesn't shake and cause distortion to the video. This would make it more difficult for the software to pick up the tracks.

 

I created 25 different track points which providing the tracking are majorly green the camera is able to pin point the 3D space within the 2D video.

Animation Test

 

 

To get a rough idea of how I will produce the animation to be overlaid on the video. I created a quick test in Maya. I modeled Pacman & the ghost from 2 spheres. The final versions will be much more smoother and true to the original designs. Plus will be textured with their faces.

 

 

This is roughly where the movement of the characters will be doing, camera angles will be changed to make this more visually appealing.

Types of Camera Movement

When working with Cameras, proper terms are used for the majority of movements:

Pan:

The Camera swivels smoothly on a tripod.

Tilt:

The camera is tilted up or down.

Dolly:

The camera and its support is moved along a form of track to follow an action.

Tracking:

The camera follows the characters as either handheld or using a sling or steadicam.

Crane:

The camera is moved through the air using a crane, jib or aerial track.

Push in / Pull back:

The camera is pushed towards the action or pulled back. This is oftern used to subtly suggest an increasing affinity or dislike of a character as a clip or scene progresses.

Zoom in / Zoom Out:

The camera lens zooms in or out to change our spatial and emotional relationship to the scene and characters

Focus:

Filmmakers oftern keep areas in an image deliberately in or out of focus to help control the audience's relative interest in different areas of the field of view:

• Deep Focus is where much of the field of view is in focus, this is oftern used to help underline the characters relationship to his or her surroundings.
• Shallow Focus is where only a small area of the image is in focus at any one time, this is used to focus the audiences attention on one character, a detail in the field of view or to suggest a character is caught up in their own thoughts.

Pull Focus:

Here the cinematographer adjusts focus between two or sometimes three characters or areas of interest within the field of vision to literally refocus the audience's attention on different aspects of the action, or to suggest changing audience sympathies with the characters shown.

What are Visual FX?

A visual effect is the integration of live-action footage and generated imagery to create environments which realistic, but would be dangerous, costly or simply impossible to capture on film. Visual effects using CGI has become increasingly common in big-budget films and have also recently become accessible to amateur filmmakers with affordable animation and compositing software being available. A Visual effect is basically digitally manipulated live action footage. The use of computers to add or remove anything that was not originally in the shot would be classed as an visual effect.

There are various softwares which specialise in visual effects such as Auto Desk Match Mover and Adobe After Effects. 3D modelling programs are also needed if using an animated character or object within a scene.

There are different techniques used in visual effects aswell including things such as Motion Tracking, which will be the method I shall be using to create my animated piece. This allows the insertation of computer graphics into live action footage with correct position, scale and orientation by using tracks on points of the film.

Green screens is another method of creating visual effects by compositing two images together, the chroma keying is set to pick up a colour and remove it, creating a transparancey. For example if you wanted to film a person and give them a different background, you would film them infront of the green screen and then make the green transparent, ready to place the remaining image onto a new background.

Physical simulators are also useful in the Visual FX industry as they're softwares which allow simulation of the way bodies of many types are affected by a variety of physical stimuli. They are also used to create dynamical simulations without having to know anything about physics. Physic engines are oftern used in high budget films such as 2012 which involve natural elements like flooding water.