Cleanup is an important but also time-expensive process in VFX pipeline. We need to remove the unwanted elements from the shot and make sure the affected region in the original plate as less as possible. Roto and tracking techniques are used to help cleaning up.
Rotopaint node is just an older version compared with roto node, the former contains more actions than the latter, such as clone. Clone is mostly used in cleanup.
We may need to denoise the plate firstly to help tracker work in a better performance. On the other hand, since the noise is generated randomly, we need to create a cleaned part based on denoised background, otherwise the cleaned part will contain fixed noise. Finally, we can use regrain to generate a similar and dynamic noise subject to the original plate.
Clean up
When doing cleanup, there are loads of factors to consider about, such as the change on the environment due to the removed elements. In this shot, the shadow behind the cup should be removed along with the cup itself.
I deformed the overall shape firstly in mudbox. For sculpting with better quality, I subdivided the origial mesh into a higher level and then sent the lower level with less polygons back to maya2020 directly in this step without any painting layers. If using maya 2018 or earlier version, we will have to export the model and load it in maya.
Before sending to mudbox for detailed sculpting and painting, I needed to unwrap the UV maps appropriately. It is important to assign new materials to each object and rename them to preventing from sharing one texture mapping afterwards.
I tried to generate tessellation meshs this time, however, I found the triangles did not work well with my selected vertical pattern.
Tesselation meshes
Therefore, I still used quadrilateral mesh to work with and it did not take too many faces compared with tessellation mesh in this case. My highest subdivison level was 3 this time and I started to painted it by creating a painter layer. It is noticeable that simple blinn has no paintable channels so we need to assign to mudbox defalt material at first.
Shots without spatial or perspective movement can use 2D tracking, such as parallax movement.
Final Plate
Original PlateNode Graph
Quality Control using Laplacian node:
The region outside of the replacement feature is all black, which means it is good.
Planar Tracker (For Perspective Motion)
Shots without spatial movement can use planar tracker, such as perspective movement or parallax movement.
Planar Tracking
Node graph
We have two ways to create a planar tracker node, by searching for planar tracker or converting a roto node to a planar tracker, as it is a kind of roto node literally.
roto to planar tracker (1)
roto to planar tracker (2)
We have some choices in tracking setting when exporting a roto node, tracker is an option.
With relative cornerpin2D, the asset image stays the same, following the tracking information to move.
With absolute cornerpin2D, the asset image will be altered to fit the selected tracking plane when following the tracking path.
In the case that we need to adjust the image to proper shape and position, we need another cornerpin node to do a transfer.
Before tracking the image sequences, we always denoise the original plates, sometimes people use median node as well. Grade node is another helpful tool to increase the contrast. However, denoise and median nodes usually cause crash, so remember to export the sequences and reload it again as the new denoised image sequences to carry on the project in Nuke.
Here is an example that I did for 2D tracking:
Final plate
Original plate
I used the denoised image sequences to complete the tracking process, and then apply it to the original image sequences.
To start with, I downloaded a 3D printable human head model(.obj). In maya, I reduced the number of polygons and made it live to retopologize manually. Then I was able to send it to mudbox to deform and add the details.
Here is my reference mask:
To complete the whole mask, I needed to create the assets such as horns, eyes and teeth. I just created simple topologies in maya and would send to mudbox for further editing as well.
Attack on Titan is a Japanese dark fantasy anime television series based on the manga of the same name by Hajime Isayama. Produced by Wit Studio and directed by Tetsurō Araki, a first season aired in 2013. The story revolves around humanity’s battle against the sudden appearance of bizarre, man-eating giants called Titans.
Attack on Titan Logo Design
Concept and Story Design
Since the story of TV-series is based on the manga of the same name, it has been given a strong reference before the production. However, TV-series is coloured wheras the manga is black-and -white, hence it is crucial to determine the tone colour adapted to the theme.
Illustration Art of Attack on Titan
The combination of black and red, the two most sinister colours, conveys a sense of power and strike, addressing the destruction, darkness and death brought by the war. What’s more, the composition of the elements in this illustartion drawing, the titan, the man, the roaring fire, the broken wall and tenements, clearly and directly show the concept of ‘attack on titan’.
The villages surrounded by the wall before the attack mainly use the yellow tone, which represents a sence of peace and warmth. However, it additionally give the impression of sunset due to the fading light, which is symbolic of the forces of darkness.
Environment recreation by Instagram damdam_arts
Environmental design
Still image from TV season 1
In anime series, the eyecatch between the commercial messages is designed to show the Current Publicly Available Information which gives an introduction to the known settings such as the props and the story background. This intriguing design helps the audience get to know the world step by step together with the main characters in the story.
Current Publicly Available Information in manga
Current Publicly Available Information in anime
Environment in anime
Human regions protected by three walls
There are three main walls protecting the human kingdom, the Wall Maria, Wall Rose and Wall Sina. The Wall Maria is the outermost wall with a height of 50 meters and a distance from Wall Rose that is 100 km. Five years ago it was breached by the Colossal Titan whose height is over 50 m, then the gate was open for the titans to enter, and the story began here.
About the Walls
A History of Battles Against the Titans
The oldest available records say the Titans appeared and devoured most of humanity from 107 years ago with unknown reasons.
Humanity retaliated with cannons and developed the wall-mounted artillery, but they were not useful enough in the face of the Titans.
Wall-Mounted Artillery and Cannons Development
Flying using 3D maneuver gear
In order to attack against the titans, the soilder needs to maneuver in 3D space, and it is necessary to conduct relevant trainings.
The ODM Gear’s Body Weight Transfer Equipment
Furthermore, the suspense and attraction of the story is not only arising from battleing with the titans, but deeply involving in the complex relationship regarding to politics of citizenship as well as human nature.
Survey Corp, Military Police Brigade, Garrison, Training Corps (left to right)
Corpses are divided into four with different duties, and new commanders have different reasons to join one of them. The main characters joined the Survey Corps. The Survey Corp is the one who travels outside the walls to collect information, fights the titans and explores the truth of the world. They are the most highly tactical and most in danger compared with others. Despite having little success, they still symbolize the “hope of mankind” with their insignia being known as the “Wings of Freedom.”
Generally, the story has numerous details and clues. They are linked closely to create such a completely realistic and cruel world, leaving the audience thinking.
Character Design
The story is not only about one person. Although all the characters are in the same uniform, it shapes diverse personalities in a team, addressing the uniqueness and irreplaceability of everyone through the entire story successfully. some are calm, the other are impulsive; some are brave, the other are coward. Whatever their characteristics is, the story witnesses each one’s growth. However, it is common to see tragic and horrible endings happening on some lovely characters, subverting expectations again and again.
The 104th Training Corpsa
Mikasa
Young Mikasa
Misaka is one of the deuteragonists. She appeared to be a cheerful, outgoing, and gentle child before witnessing her partents’ death. Growing up, Mikasa has become emotionally withdrawn and noticeably dangerous. She is stoic, level headed and considered the best soldier among the 104th Training Corpsa. Her appearance is designed to be a fairly tall, physically fit, and well-toned woman with muscles lined throughout the arms, legs, and abdomen. She is of partial Asian heritage, with pale skin, gray eyes, and shaggy black hair that was long until she cut it to chin-length when joining the military.
Painting Precautions
The titans all have a human-like appearance but the type varys including the height, the hair style and the intelligence. The corresponding design is matched with the setting, laying the foundation of the following plot.
Visual Development and Storyboard
The scene describes that the protagonist, Eren, witnesses his mother’s death, which is the saddest and the most important part in the story plot. Focusing on the development of the apperance of the characters in different angles of view with loads of closeups, the flashback of warm memories with Erens family, as well as the brutality of the titans, it engages the audience to suffer from the same tragic experience with Eren.
Overall, this is an entirely captivating, enthralling and thrilling series. A large number of secrets and mysteries along the plot line with a very clear narrative vision catch the audiences’ eyes and help them follow the story. It is a masterpiece.
Attack on Titan is incredibly detailed. Even the side stories twist and turn, the plot seeming to change with every episode. There is just no watching them fast enough and with each episode running for about 20 minutes, it’s incredibly easy to binge. Overall, this is an entirely captivating, enthralling and thrilling series.
There are three term to consider with colour correction: Hightlights, Midtones, Shadows.
Grade
Grade node is one of the most powerful nodes in Nuke, it can be used to white-balance the image sequences by sampling the whitepoint and blackpoint; change the darkest(shadows) and lightest area(highlights) by adjusting lift and gain. Gamma can be used to grade midtones. We can also choke the image by adjusting gamma through alpha channel.
Grade node
We always grade the unpremulted images, so we need to unpremult the image firstly if necessary, then premult it after grading. We need the premulted images for assets added on the background before compositing.
ColorCorrect is a more precise node compared to grade node, it can be used to modify the shadows, midtones and highlights. Here offset is used to grade shadows, similar as lift in grade node.
ColorCorrect node
We can modify the region curves to change the particular regions that we want to control.
In order to create compositing more naturally, it is necessary to control the colour settings for the elements. I chose 2 still images to try this task as follows:
White-balancing is the first step of the colour matching, by choosing both the darkest and brightest points when viewing in Luminance mode (shortkey y), as well as adjusting gamma parameter until I find no difference before and after grading the images.
Then I used Grade node to match the colour of the two images, by adjusting lift and gain parameters to match the shadow and highlight respectively.
Review:
The two original images are not matched so properly due to other factors such as lighting and angle of shot, which should be taken into consideration before compositing.
Entry requirements: Computer Graphics related degree, programming skills, proficiency in using C language
In computer graphics, a shader is a type of computer program originally used for shading in 3D scenes to produce light, darkness, and color in a rendered image. Nowadays shaders can also handle other special effects as well.
Most shaders are coded for and run on a graphics processing unit (GPU), though this is not a strict requirement. There are a bunch of shading languages out there for different platforms, but since they’re all adapted to run on the GPU, they’re all very similar. The official OpenGL shading language is OpenGL Shading Language, also known as GLSL, and the official Direct3D shading language is High Level Shader Language, also known as HLSL. Apple released its own shading language called Metal Shading Language as part of the Metal framework.
Unity and Unreal Engine include node-based editors that can create shaders without the need for actual code, which lower the threshold of shaders for artists.
Shader development TDs are focusing on writing the algorithm to alter and create different shading effects to match the need of the project.
Shader development TDs may have related experience in lookdev TD or lighting artists. They are familiar with CG lighting, and good at programming at the same time. In some companies, the roles of lookdev TD and shader dev TD are combined.
Short intro to shaders
Shaders in UE4
Creature TD
Alias: Character TD, Creature effects (FX) TD, Groom TD
Technical department
Entry requirements: sufficient experience in VFX industry, programming and coding skills(e.g. Python), proficiency in using 3D software packages such as Houdini, Maya, Ziva Dynamics
Creature TD is a more senior role in VFX companies. They develop and program the software that enables VFX artists to create realistic fur, feathers and skin on their creature models, such as dinosaurs, animals or magnificent beasts. Creature creation is one of the most difficlut parts in the project and creation TDs need to keep up to date with the most cutting-edge technology to overcome the obstacles and find solutions.
Creature TDs need to have thorough knowledge of an area of creature specialisation and a broad understanding of the other areas, which include rigging and enveloping in Maya, cloth, hair, flesh, rigid body destruction, crowd simulations etc.
Creature TDs are also required to have strong visual and artistic skills to reach the aesthetic requirements for the project. They always work with other modelling artists and animators. Also, they are involved in the development of departmental tools and techniques, so they need to report to pipeline TDs on time.
The scope of the creature TD role varies depending on the size of the studio. In some studios, the role includes rigging, the skeletons and muscles that they created will be passed to animators to work on as a basis. In larger studios, the role of creature TD is focussed on creating the fur, hair, feathers, skin. Though the task is challenging, they need to be comfortable with the more technical aspects of creature work where appropriate.
Creature TD demo reel
Furry creature workflow
Pipeline TD
Alias: Pipeline technician
Technical department
Entry requirements: knowledge of scripting languages such as Python and C++ and software such as Maya or Houdini, experience as an assistant technical director role in VFX industry, proficiency in Linux
VFX industry is an arts field involving in cutting-edge technology, especially with computer graphics. There are so many digital tools in the industry and various instructions are applied across the field. For the same software package, the version is upgrading all the time, so this cause art workers to face loads of problems during the work.
Additionally, the problems that they may encounter are always across multi-departments, sometimes, experts in one field may not be familiar with another strategy. In this case, recruiting a pipeline engineer who has good problem-solving skills with strong math and computer knowledge combined with a flair for the arts will be a solution.
VFX pipeline
A Pipeline TD will work with a range of people, including 3D Animators, Riggers, Modellers, Lighters, and Compositors. They communicate with VFX Artists across the team to understand their needs. If the project that they are working on isn’t running smoothly, then it is the Pipeline TD’s responsibility to identify what kind of tools need to be developed to get the issue fixed. Issues that arise might be technical ones to do with digital art, or productivity issues. Their job involves writing or modifying code to solve such problems. They also provide face-to-face technical assistance. Pipeline TDs will work closely with research and development teams, who design and test any new software.
Pipeline development is a complex and ongoing process. Ideal technologists develop coding tools and APIs(application programming interface) to simplify the workflow of artists and TDs.
Gradually these processes become the standard in the industry and can be reused and improved rather than starting from scratch on each new project, which significantly enhances the efficiency.
Pipeline TD reel
Pipeline TD reel
Python skills for VFX
R&D Programmer
Alias: Software developer, Research and development programmer
Technical department
Entry requirements: strong coding skills, Python and C++ languages, high familarity with VFX pipeline, a degree in computer science or related fields
R&D Programmer have a great understanding of computer graphics and strong programming skills to actively participate in the development and integration of creative tools into an overall production pipeline. They not only research, design, implement, test, provide technical documentation and support CG software systems and tools for the studio, but also need to ensure that new tools fit within the existing software systems at the same time, which enables the efficient passing of assets from one VFX process to the next.
Software developers are responsible for identifying and optimizing bottlenecks in algorithms and workflows as well as finding optimized solutions to support, and collaborate with other VFX artists.
They are always staying up-to-date with industry trends and new technologies, being accountable for further improvement of software tools and technology development, and finding innovate ways to enable the artists within the pipeline to complete their work as fast and as well as possible.
Denoise first before tracking. Blue channel contains most noises. Write it out then read the denoised plate for the following editing to prevent from crash.
Alias: Line producer, VFX line producer, VFX production manager, Scheduler
Production management department
Entry requirements: experience in runners, production coordinators in studio, relevant study in business, film, media.
Production managers ensure that VFX prouction run reliably and efficiently. They follow the decisions that have been made by the VFX producer, plann and organise project schedules, assess project and resource requirements and look after the budget.
They supervise and oversee the work of the production coordinator, re-negotiate timescales or schedules as necessary. They might also be involved in casting or hiring people for different departments and drafting contracts.
Production managers are responsible for liaising with the VFX artists and technical directors (TDs) from all parts of the VFX pipeline to make sure that work is completed on time. They are also important in communicating with the producer of company shooting the live-action footage and producing the film or TV programme.
VFX Producer
Alias: Executive VFX producer, Show producer, Bidding producer
Production management department
Entry requirements: sufficient experience in VFX studio such as runners, production coordinators, production manager; relevant study in business, film, media.
VFX producers manage the whole process of creating the VFX for film or TV, they lead their VFX teams and ensure that the projects gets completed on time and on budget. Also, they work closely with the live-action production crew and the post-production editor to make sure the project reach the expectation because their goal is to make the client come back for next project. In this case, it requires them to have great communication and management skills to be a producer.
Cinesite is hiring a VFX producer
VFX producers write the bid; the document through which they persuade the film or TV series’ producer to take their VFX studio on to do VFX work on a project. For example, if any requests come in that are out of the original scope awarded, VFX producer will provide client production with a bid for what the additional work would cost.
VFX producers put together the team of VFX artists and other technical staff. They are the overall decision makers. They set the schedules for the work and they manage the budget. Their job plan is an ongoing process throughout the whole project, due to client changes or studio needs changing.
In larger companies, they hand over the day-to-day management of the prject to a manager. In smaller companies, they do the project management themselves.
A day in the life of a VFX producer
Let us watch how a VFX producer introduces their team’s project:
Animator
Alias: CG animator, 3D animatormanager
Computer-generated (CG) department
Entry requirements: proficiency in using VFX software, particularly Maya, V-Ray and 3ds Max.
In VFX industry, animation refers to moving images in computer-generated imagery. There are two types of computer animation: 2D and 3D. However, 3D computer graphics has been widely used to generate two-dimensional animation in recent years. 3D animation allows the creator to add on and modify the movable elements which are not available for shotting in live-action site, it can be a dancing character or a spinning electric fan. Their motion will be displayed frame by frame on computer monitors, usually 24, 25, or 30 frames per second.
For 2D figure animations, separate objects and separate transparent layers are used with or without that virtual skeleton. Then different parts of the figure are moved by the animator on key frames. The deformation are calculated by the computer in a process known as tweening or morphing. Finally, the animation is rendered.
For 3D animations, models are built and textured in 3D software packages such as Maya and are rigged with a virtual skeleton. Different parts of the figure are linked with associated bones and joints, stimulating the relationship between skin and skeleton in reality. Animator can make the body move by controlling the skeleton.
However, making a realistic motion is not an easy task, as people’s eyes are sensitive to such human-like movement. In VFX industry, people are using motion capture techniques to record actors’ movement by setting loads of motion trackers on their skin-tight suits, so that the movement and expression of their performance can be captured digitally and translated into a different-looking character animation model.
There is one of the most challenging fields in CGI which is called facial animation, which requires a lot of data to process in computer. Since facial expression refers to a type of detailed art, especially on realistic characters, any artifacts will be exaggerated on the screen.
Animators in the VFX industry usually produce work to be integrated into the live-action footage of a film or TV programme. They animate 3D objects as dictated by background film plates, which means that there is footage and a set camera position that they must work to. They need to match their animation to the reference footage.
Important skills for animators
Effects technical director (FX TD)
Alias: Technical producer
Technical department
Entry requirements: Knowledge of scripting languages such as Python and C++ and software such Maya or Houdini, some years’ experience working in VFX
The creation of effects like explosions, billowing smoke and rushing water are fully supported by effects technical directors. All TDs are problem-solvers. They helped the visual effect artists to put expected effects in shots, they are also stay up-to-date with the latest research and techniques and develop software technological boundaries.
It is the FX TD’s responsibility to make sure the physics-based digital effects that they create look believable and are also consistent with the style of animation, to ensure that they blend in seamlessly with the live action components of the film or TV programme. They aim to meet the Director’s vision in terms of digital effects, using the resources available to them.
FX TD is both an artistic and technical role in VFX studio. As a senior role, a FX TD is responsible for providing guidance to more junior Artists, giving face-to-face technical help to all Animation and Lighting Artists, as well as keeping communicating with other departments.
