IBK

IBK (Image-Based Keyer) is a non-native node to nuke, it was developed in a studio.

Increasing to a large size in IBKColour node to clean the plate may cause atrfacts, instead, we adjust darks to correct the edges and use stacked IBKColour nodes to shrink the objects. Then connect to IBKGizmo node to show the objects.

We can create two green constants, one from the bright region, one from the dark region, to even the green screen.

Keylight

We need to cleanup the plate for CG compositing.

This week we use 3D projection to clean the plate(remove unwanted assets).

I just use the 3D tracking piece from week 4 to do the projections. And I choose to remove the toys on the ground.

For the object between different planes, we can create patches for different part in planes separately. We just rotopaint in good frame as normal and create the projection regions, then hold the chosen frame.

In 3D projections, we need the same camera as before, but this time we call it the projector(Blue one). We need to hold the projector in the chosen frame as well.

We can build the cards directly using the modelbuilder node, position the required cards in correct region for the patch that we created. This time I just adjust the cards in 3D tracking and import them into 3D projections. Remember the cards should be renderd in texture.

Tips: We can use cornerPin nodes to deal with the sides between the planes. The second cornerPin is the same parameters as the first one, but inverted.

Lens Distortion

• 

• 

• 

We use lensDistortion node to undistort the lens grid.

Also, we can use STmap and shuffle node to make the estimate to share the same distortion information.

PointsTo3D

Apart from using pointcloud, we can also use PointsTo3D node, starting from the camera, to place an asset in 3D place.

We generate the tracking axis from PointsTo3D node and create a geometry with the same route as the axis. Here I used a sphere.

We put pointA at the starting frame, B in the middle, C at the end.

We can also bake selected vertices to mesh when necessary.

Create camera

This week we learn to create camera and import into maya. First thing is to denoise the plate. I use the lake plate here.

With the information from the original plate we can change the settings for the project and the camera.

• 

• 

• 

Before exporting the camera, we need to undistort the lens, because virtual camera has no distortion. We can also keep using this node later to redistort the CG asset before compositing.

However, this week, we do not need to worry too much about the LensDistortion node, just use the node to deal with the Lens Grid card to see the difference.

If using LensDistortion, we may need transform node to cut out the black regions.

Then we just track the camera as normal, put it in the scene and scanlineRender it.

If we know the position where the asset should be put, we can create an axis on the tracking point. (Reference for the asset in the node graph)

• 

• 

We then use WriteGeo node to export .fbx file including camera. We can use ReadGeo node to inport the .fbx file as well for CG compositing in nuke.

Compositing the Beauty AOV

The RGBA beauty AOV can be split into smaller AOVs where each contains part of the lighting. In compositing, these AOVs can then be individually modified and added together to get the full beauty AOV.

More AOVs give more control in compositing, but also extra work to handle, and they take up more memory and disk space, especially combined with light groups.

We can create material ID pass and turn it into alpha when necessary.

We use shuffle node to swap between RGBA channels and grade node to manipulate each pass.

In this node graph, Diffuse is used as the base, highlight stuffs on the right, others on the left.

We are recommended to start with the diffuse direct and then indirect; when we merge shadows or ambient occlution, we use multiply operation; when merging hightlights, we use screen operation.

Then we merge them and write out.

We will talk about Distortion and z-depth next week. Finally, apply the noise to match the original plate and merge on it.

First we change the settings for the project based on information of the original plate.

• 

• 

• 

It is a more complicated footage, so it is better to divide into two parts(inside room and outside room) to track.

The mask created here is used in many cases, it will be set to inverted for tracking in different zones when necessary.

We need to recreate the camera and match the camera movement for 3D tracking.

We denoise the plate firstly and use grade node to make it easier for nuke to track every point in the plate. We may need to roto out some elements which are affected by other factors apart from camera movement.

Use camera tracker to track the plate. We need to track first and solve next. Adjust some parameters can reject the points with larger errors.

Remember to set ground points beforehand in CameraTracker node.

Use point cloud to accurately position the asset in the 3D scene, for example, the cards.

Position the cones and cards in the plate. Select particular tracked point in the plate and left click to choose creating the axis, we can link these axis with the suitable geometries such as cones.

We need a FrameHold node to prevent camera moving automatically for perspective projection adjustments.

Overscan in ScanlineRender node is helpful for rendering extra space around the format range, avoid any artefacts.

Smart vector is a powerful and expensive node in nuke for difficult tracking of face.

Sometimes we may use a combined method to do the tracking with the help of swith node.

Mocha is also a good plugin for tracking.

Smart vectors are usually used in beauty work:

Beauty work in reference frame

We do all the beauty work in a reference frame.

The first step is always denosing the plate. In this case we use denoise node as well as median node, they are both heavy-working nodes and median node is a kind of filter node.

We clean up the plate afterwards. With rotopaint node, we clone for large flaws, remember to set the output in RGBA-alpha channel.

Also we have a excellent gizmo node called VanityFix, it works perfectly with tiny and high-intesity flaws.

We union those alpha outputs and premult them.

• 

• 

• 

Tracking with smart vectors

SmartVector node may take some time to run, depending on machines. The best extension for vectors file is .exr.

Then we will use VectorDistort node to track the face directly, and regrain to match the original plate.

The final step will be merge the beauty work with the original plate, we can merge it twice, the last one is used to recover some imperfection to make the final plate more believable.