Compositing

 

After rendering my animation files with multiple passes. I then opened up Autodesk Composite and imported the Open EXR files. I could then choose on the options what pass channel I would like to use and blend them together with settings so I can control the intensity of these mixes.

 

First off, I selected the Diffuse channel, this being my animations major image. I then used 'blend and combine' with the Ambient Occlusion channel on the front and back gain. I reduced the opacity of the AO to 0.26000 to just give the diffuse a bit of tone.

 

 

Next I continued to use blend and combine with my animation's Specular, this is the spot where the light bounces off the object. I then added grain, blur and cc basics to the composition. This allows me to make the animation look like its part of the scene more where I can even out the pixelation and colour temperature.

I imported the original video frames and used the resize tool to scale the video down to 1024x576 and used CC Basics again to control the colour temperature of the video. But to make my animation appear in front of the background video, I had to use an alpha channel.

 

 

To achieve this, I had to go back into Maya & under the diffuse Material pass add 4 channels (4^th being an alpha) and I also changed the plane textures to lambert, so they no longer blended in with the background so I could extract the shadow correctly from these and rendered another set of EXR files. I then imported the alpha channel and used control edge to remove some of the dark outline from my animation by making the outer distance -1.00 and inner 1.00 with a inner softness at 1.50. I added this alpha channel to the masking tab, allowing the previous channels to overlay on the background video.
 

 

Lastly, I used blend and composition to add the new shadow channel I created and the reflection channel to complete the composition. I rendered the completed video into a PNG image sequence, then used Adobe After Effects to merge the frames together and output the final AVI video.

Animating The Scenes

 

After finally creating workable tracked scenes. I then opened these in Maya and began to add my animation objects.

 

 

 

Pacman's mouth flap is animated using an additional copy of the object with the mouth open. To make the flaps animate I used the blend shape tool which would animate the difference between these 2 objects. I only had one of the objects on display, whilst the other was invisible.

 

 

Shot 1 was fairly easy to animate. To make Pacman & the Ghost appear as if they're coming out of the computer screen, I simply animated the objects to move through the plane which projects the video. The only problem I had with the tracking was that the camera would move sideways as it moved back. I felt the best way to save time was to move the characters forward at an angle would make characters to appear to move in a straight line.

 

 

Shot 2 was much more complicated as the tracking left some wobbly tracking. I decided to manually re-position the camera back into place when it went off track. This was quite a time taking process. The pellets were useful as a guide for where the camera needed to be placed in the incorrect frames. The ghost in this scene has another copy which is invisible until pacman eats the power pellet, making him bigger and the ghost turning blue making the current red version invisible on the same position.

 

 

Shot 3 was very simular to shot 1 but just pacman moving back into the screen. The tracking worked better on this scene, so Pacman was able to move in a straight line easier.

 

 

 

Lastly to correctly light up the objects to appear as though they belong to the scene. I placed spot lights in the same positions that I did from originally filming the scenes. Then I added area lights above for the ceiling lights. To also give a reflective light from the surface, I used another area light on the path with a slight tint of the counter's colour. To make the shadows visible I inserted planes within the scene, textured with 'use background' which makes these objects invisible, whilst allowing shadow.

 

Here's how the animated scenes will look before being composited and blended into the video.

Tracking the Footage and Testing

After importing the video frames into match mover, I used the auto tracking to see how well it would work with positioning the camera within the scenes.

 I wanted to test how these would work with characters in Maya as they were:

Shot 1 (Test 1)

As you can see in this first test, the camera is quite unstable and causes pacman to move sideways.

Shot 2 (Test 1)

This test shows multiple issues with the translation and rotation. It's also far too shakey.

Shot 3 (Test 1)



The tracks in this test appear to be quite stable, until the camera moves towards the computer monitor and the gameplay footage causes the tracks to lose their place.

I decided to attempt on fixing the camera issues manually where the screen would shake majorly, I repositioned the translate and rotations. I found this to be very useful, furthermore time saving than having to keep going back and repetively try to re-track.

Shot 1 (Test 2)

As you can see, although the initial part is still shakey, this is irrelevant since nothing is physically in the shot, I didnt begin to align the scene until Pacman began to leave the screen. Fortunately this shot didnt need too many adjustments to achieve a result I was happy with.

Shot 2 (Test 2)

Although this appeared much more improved that the previous test and it moved across fairly smoothly. I had alot of rotation issues remaining. I decided that I couldn't waste anymore time trying to tweak this scene. So I went back to matchmover and added an even higher number of auto-tracks.

Shot 3 (Test 2)

I was happy with the result of this test and felt it was ready for the final animation.

So I decided to leave Shot 1 & 3 as they were since I was satisfied but I felt a need to add more tracks to Shot 2 to fix the rotation errors, so i added the increased number of auto-tracks, this finally got the camera's translation & rotation solved better, I still had to manually fix some frames in translation, but this was a fairly simple task compared to before.

Shot 2 (Test 3)

After achieving a better result, it was time to add the proper animated elements to the scene and manually fix camera errors.

The Visual FX Production Pipeline

 

The Visual FX pipeline is somewhat flexible in its structure and can vary in different studios.

The Pre-production phase consists of research and development, storyboarding, basic tests and modelling. This is usually roughly done to begin with, so the team will be able to pre-visualise how the visual fx will work once the final models have been completed.

The Production phase is when the final high resolution modelling begins and once completed they're rigged to be animated. When the video shoot begins, the selected sequences of the scenes to include visual fx will have numerous reference photos taken. This is useful for the modelling, texturing and lighting.

The Post-Production phase is when the shoot has finished and the editing begins, the selected film is scanned and motion tracking begins. The camera's lens measurements would have been taken on set so they could be applied to the CG camera in order to get the correct focal length. Once an accurate camera track exists, any match moving or body tracking can be done. A body tracking is an animated CG character or object which perfectly mimics what the on set equivalent did. This would be used for things such as placing an CG element onto a on-set character.

 

The last process in the pipeline is compositing. This is where all the CG elements and the scanned images are brought together to create a seamless finished image. Once the compositing is approved, it is supplied back to the client in the same form as the scan was received, usually a series of digital image files. The client can then use these finished composited frames for creative grading and eventually to be shot out onto film or DCP for release in cinemas.

Fill, Back & Background Lighting

In every environment there is the Key Light. This is the main light shown within a scene. For example, if it was outside it would be the sun light or it would be the light within a studio.

 

The Purpose of the fill light is to partially fill in shadows created from horizontal and vertical angles of the key light. The fill light in an ideal world should be 90 degrees away from the key light and about 45 degrees from the camera. By lighting a full 90 degree afea, an important margin of safety is created in case subjects move unexpected during the production, and prevents the need for having to have the camera angles changed and creating a possible costly delay. A fill light should be softer than the key as it is then able to subtly fill in some of the key's shadows without creating a second catchlight in the eyes.

 

When outside and the sun is being used as a key, a reflector board may be used and positioned at about 90 degrees from the sun to reflect the light into the shadow areas. Large white styrofoam or foam core boards are oftern used during closeups.

 

The back light or sometimes reffered to as a hair light is placed directly behind the subject in relation to the close-up camera. They purpose is to light up the back of subjects. Back lights alone can be used to create a silhouette effect.

 

The background light illuminates the front of a background area and adds a depth and separation between scene elements.

 

Any type of light can be used as a background light as long as it provides fairly even illumination across the background, does not hit the central subject matter, and is at the appropriate intensity.

If the background has detail or texture, you would want to put the background light on the same side as the key. This keeps the dominant light consistent in the scene.

White & Colour Balancing

In white balancing, a sensor on the camera averages the light in the scene and automatically adjusts the camera's internal colour circuitry to zero out any generalised colour bias. Although this works reasonably well under the proper conditions, automatic circuitry can't be relied on to always produce accurate colour. So skilled camera operators adjust the white balance themselves. White card or paper is useful to test with. With the camera zoomed in full frame on a pure white card, the operator pushes a white balance button and the camera's chroma channels will automatically adjust to produce pure white. When the dominant light source in a scene changes in any way, you must again white balance your camera.

Going from sunlight to shadow will necessitate white balancing the camera again, as will moving from outside to inside. When shooting outside, even the passing of a few hours will result in a slight colour shift in illumination.

If you do not white or colour balance your camera, you risk scene-to-scene colour changes. This is especially noticeable with skin tones in multiple-camera productions.

 

Colour balancing is much like White balancing and is easy to to control, however can be difficult when using more than one camera for a scene. Some studio and field cameras are designed to use a "smart card." About the size of a credit card, it records all the parameters on the first camera you (carefully) set up. Then when you insert this card into successive cameras, they will adjust them to conform to the first camera's parameters. Studio engineers use a central camera control unit to adjust all studio cameras at a central location.

Colour reproduction is subjective since the human eye can be fooled quite easily to not be able to detect the colour variations between cameras. Studies have shown that people prefer the colours on their TV sets to be more saturated than in real life. This preference does not mean that the hues are inaccurate but the colours are more stronger to the human eye.

Camera Types

High Definition

 

In 2000, the DVCPRO HD codec was launched by Panasonic. They were used as full size DVCPRO cassettes. This format was designed to be used in professional camcorders and later on in 2003, HDV was released as a much more affordable method to capturing HD video due to the use of inexpensive MiniDV cassettes. This started being implemented by companies such as Sony, Canon & Sharp with their equipment and is a popular format used today.

Tapeless/File Formating

 

The first tapeless video format was produced in 2003 by Sony using PFDs. In 2004, DVCPRO-HD began using memory cards as is recording medium by Panasonic. Later in 2006 companies introduced AVCHD as as inexpensive consumer-grade HD video format which was tapless. This format is the most common used by non-proffesionals. In 2007 Sony released XDCAM EX which records onto SxS memory cards.

3D Cameras

3D cameras became available to the general market in 2010 by Panasonic, this was due to the sudden raise of interest in 3D by films such as Avatar. This form of camera uses two lenses to capture both sides of the video and then outputs the two sides together to create the 3D video which can be seen on screens that are able to produce the 2 signals and wearing 3D glasses.