Compositing

 

After rendering my animation files with multiple passes. I then opened up Autodesk Composite and imported the Open EXR files. I could then choose on the options what pass channel I would like to use and blend them together with settings so I can control the intensity of these mixes.

 

First off, I selected the Diffuse channel, this being my animations major image. I then used 'blend and combine' with the Ambient Occlusion channel on the front and back gain. I reduced the opacity of the AO to 0.26000 to just give the diffuse a bit of tone.

 

 

Next I continued to use blend and combine with my animation's Specular, this is the spot where the light bounces off the object. I then added grain, blur and cc basics to the composition. This allows me to make the animation look like its part of the scene more where I can even out the pixelation and colour temperature.

I imported the original video frames and used the resize tool to scale the video down to 1024x576 and used CC Basics again to control the colour temperature of the video. But to make my animation appear in front of the background video, I had to use an alpha channel.

 

 

To achieve this, I had to go back into Maya & under the diffuse Material pass add 4 channels (4^th being an alpha) and I also changed the plane textures to lambert, so they no longer blended in with the background so I could extract the shadow correctly from these and rendered another set of EXR files. I then imported the alpha channel and used control edge to remove some of the dark outline from my animation by making the outer distance -1.00 and inner 1.00 with a inner softness at 1.50. I added this alpha channel to the masking tab, allowing the previous channels to overlay on the background video.
 

 

Lastly, I used blend and composition to add the new shadow channel I created and the reflection channel to complete the composition. I rendered the completed video into a PNG image sequence, then used Adobe After Effects to merge the frames together and output the final AVI video.

Animating The Scenes

 

After finally creating workable tracked scenes. I then opened these in Maya and began to add my animation objects.

 

 

 

Pacman's mouth flap is animated using an additional copy of the object with the mouth open. To make the flaps animate I used the blend shape tool which would animate the difference between these 2 objects. I only had one of the objects on display, whilst the other was invisible.

 

 

Shot 1 was fairly easy to animate. To make Pacman & the Ghost appear as if they're coming out of the computer screen, I simply animated the objects to move through the plane which projects the video. The only problem I had with the tracking was that the camera would move sideways as it moved back. I felt the best way to save time was to move the characters forward at an angle would make characters to appear to move in a straight line.

 

 

Shot 2 was much more complicated as the tracking left some wobbly tracking. I decided to manually re-position the camera back into place when it went off track. This was quite a time taking process. The pellets were useful as a guide for where the camera needed to be placed in the incorrect frames. The ghost in this scene has another copy which is invisible until pacman eats the power pellet, making him bigger and the ghost turning blue making the current red version invisible on the same position.

 

 

Shot 3 was very simular to shot 1 but just pacman moving back into the screen. The tracking worked better on this scene, so Pacman was able to move in a straight line easier.

 

 

 

Lastly to correctly light up the objects to appear as though they belong to the scene. I placed spot lights in the same positions that I did from originally filming the scenes. Then I added area lights above for the ceiling lights. To also give a reflective light from the surface, I used another area light on the path with a slight tint of the counter's colour. To make the shadows visible I inserted planes within the scene, textured with 'use background' which makes these objects invisible, whilst allowing shadow.

 

Here's how the animated scenes will look before being composited and blended into the video.

Tracking the Footage and Testing

After importing the video frames into match mover, I used the auto tracking to see how well it would work with positioning the camera within the scenes.

 I wanted to test how these would work with characters in Maya as they were:

Shot 1 (Test 1)

As you can see in this first test, the camera is quite unstable and causes pacman to move sideways.

Shot 2 (Test 1)

This test shows multiple issues with the translation and rotation. It's also far too shakey.

Shot 3 (Test 1)



The tracks in this test appear to be quite stable, until the camera moves towards the computer monitor and the gameplay footage causes the tracks to lose their place.

I decided to attempt on fixing the camera issues manually where the screen would shake majorly, I repositioned the translate and rotations. I found this to be very useful, furthermore time saving than having to keep going back and repetively try to re-track.

Shot 1 (Test 2)

As you can see, although the initial part is still shakey, this is irrelevant since nothing is physically in the shot, I didnt begin to align the scene until Pacman began to leave the screen. Fortunately this shot didnt need too many adjustments to achieve a result I was happy with.

Shot 2 (Test 2)

Although this appeared much more improved that the previous test and it moved across fairly smoothly. I had alot of rotation issues remaining. I decided that I couldn't waste anymore time trying to tweak this scene. So I went back to matchmover and added an even higher number of auto-tracks.

Shot 3 (Test 2)

I was happy with the result of this test and felt it was ready for the final animation.

So I decided to leave Shot 1 & 3 as they were since I was satisfied but I felt a need to add more tracks to Shot 2 to fix the rotation errors, so i added the increased number of auto-tracks, this finally got the camera's translation & rotation solved better, I still had to manually fix some frames in translation, but this was a fairly simple task compared to before.

Shot 2 (Test 3)

After achieving a better result, it was time to add the proper animated elements to the scene and manually fix camera errors.