The cooperation project did not run smoothly in general, but it is very valuable for learning.

I will summarize here some of the problems encountered in the cooperation process.

Video Specs

Through communication with another team, I found that our team did not emphasize the use of the same and lossless video codec in the cooperation for the final editing. Another group use Apple ProRes 422 for output during process and H.264 for the final movie.

At the same time, some team members used the wrong frame rate and size format when editing the project file, resulting in the lack of detail quality.

The most important thing is that we did not use high-quality original footage for compositing, which caused a serious decline in video quality and increased the difficulty of rotoscoping, tracking and compositing.

Live shooting

Another serious mistake was that the green screen was not used for shooting, which made rotoscoping difficult in post-production. When shooting, we did not consider the lighting of the scene thoroughly, which caused some avoidable problems and had to be fixed in post-production.

For example, the hard light that needs to be removed in the scene, we did not block it, resulting in unwanted light effects, and greatly affected the faces of the characters.

Coherence in design

Although we prepared storyboards, script and mood boards for the pre-production, and conducted on-site inspections of various shooting scenes, in actual shooting, we did not go exactly as planned. Looking back at the preliminary preparations, we seem to have done a lot but in fact did not make a thorough arrangement. At the same time, we did not develop a coherent style during post-production. The styles of the footage produced by each compositor are independent of each other.

Lack of quality management, unclear roles and responsibilities

At the beginning, we designated everyone’s work but did not clearly divide the work, and did not conduct inspections at each stage of the work, resulting in the handover not going smoothly. For example, a member of our group did the wrong cleaning plate but her progress was not checked, which caused the compositor to do it all over again; another example, the work done by roto artists and 3D tracker did not reach the standards, leading to a delay in progress. It is incovenient to fix the roto data given by others because we can not see the roto key frames in our own nuke, in this case I need to check each frame to ensure if the key frame causes the problem. It may take less time to roto all over again. So I think the roto artists should check the quality of the work done and fix the issues before uploading it, however, the coordinator thought it was part of composer’s job.

From my understanding, good collaboration means smooth workflow, but everyone in our group just texted messages in the chat, but no one checked the quality at each stage. This leads us to waste a lot of time doing repetitive work, especially when the project is reaching a deadline, it took time to deal with work that should have been completed at the beginning. At the same time, it also leads to conflicts and disputes in cooperation.

But finally we reached agreement and finished the work. Disagreements are inevitable in teamwork, because everyone is making efforts to achieve better results in their own ways. It is truly valuable to learn how to be more effective and professional through this collab project also in the future.

Design

Since our story belongs to the category of magic and suspense, according to the mood board we made before, I referenced some concept maps of science fiction games and simply composited them with the original scenes. Due to the lighting problem of the original footage(strong hard light and outdoor light), I finally decided to change the shooting room into a semi-open scene to weaken it.

Since setting the scene on the ground is more restrictive, the style of the room will affect the composite background to a greater extent, and the shooting angle limits the field of view, so I finally decided to make the shooting room float in the air.

I tried the HDRI that I can find on the Internet as the background, but the style is not ideal. Because the specular reflection in the scene can be ignored, I decided to use the pictures with buildings as the distant view.

• 

• 

• 

For the last shot, referring to the script, I designed it as two characters walking towards a secret room.

Add a CG light source

Since the strong light source was not removed during shooting, the result of roto was not satisfactory. After discussion, our teammate thought that adding a light source will help. The choice of the light source need to make sense in such environment. It is better to use a static CG glowing stuff, because the moving light source will affect the shadows in the original scene. Due to the complexity of the original scene light and shadow and the architectural structure, it will take time to render the shadows with moving light sources. However, this stationary light source also need to satisfy the layout of the scene so that it is not abrupt.

The use of CG light source is to fix artifacts that cannot be solved in rotoscoping for both shot 8 and shot 9. When choosing a static light, I found it can not be placed at correct positions. Finally I chose to use a sci-fi robot.

• 

• 

Because it would take much time to render shadow, I decided to create it in nuke using original robot sequence with some transform nodes, grade and blur alpha channel, keying at certain frames to match the movement of the robot.

I also used flare node at the end to match the lighting effect from the CG light.

3D Tracking

The tracking information given by other people was useless because all the points look correct in 2D viewport but make no sense in 3D viewport. I tried many times by using grade nodes and adjusting settings in camera tracker but no help at all. However, the camera rotation path looked ok. So I finally placed the cards with a camera to match the scene.

Animation and Rendering

The following CG assets are downloaded from sketchfab,

Door: https://sketchfab.com/3d-models/rr-door-aed85da0529f419d89dd724cdd883fb5

Floating stones: https://sketchfab.com/3d-models/european-and-american-game-scencemagic-portal-31182bdc05dd40ddbc7f284297e8f225

Sci-fi camera: https://sketchfab.com/3d-models/sci-fi-camera-drone-3dbf07385d1b471a96dce6de6cba97d6

I just imported the cards and camera from nuke to maya so that I could adjust the position of the assets in the scene. I also made animation or did adjustments to the existing animation to match the movement in the shot.

I used a spherical light source with image background to illuminate close-up objects in maya.

In order to improve the rendering speed, I set the mesh to blocker to block the invisible part of the CG after compositing. At the same time, use arealights and blocker to create corresponding shadow effects for indoor CG objects.

After testing, arnold GPU rendering and CPU rendering under the same settings, although it will be several times faster, but in this scene will cause fireflies, so I tried, and finally found that enable adaptive sampling and set max to 10 is more efficient In practice, when the default max is 20, the speed is almost the same as the CPU rendering.

Compositing

Given roto data on characters from my group mates, I started compositing for shot 8 and 9.

Image background

Because the image background is a still image, it works in shot 8 but need to be rotating in shot 9 as the camera rotates. So I projected the mirrored image to a cylinder to scanrender.

Roto on the archs

For shot 8 it is easy because the camera does not move, so I just simply roto out the shape.

For shot 9 it is difficult to make it stick stably so I tried to roto at single frame and project them using camera on the cards with the help of shuffle nodes, because alpha channel cannot be scanrendered. However, it does not fit very perfectly all the time and there is always a shift, no matter how I adjusted the cards, it could not be solved. I tried to use transform and cornerpin nodes to adjust these but it became more complicated.

In this case, I just used them for certain frames, and for frames before 105, I rotoed it frame by frame. It was difficult to get a perfect result because of blurred pixels on the low-contrast edges.

Portal

The portal we initially used has some problems to match the scene, in this case I tried to use a more transparent one to replace.

• 

• 

The new portal is a downloaded resource from Doctor Strange, I graded and transformed it to match the scene.

Merge

Because CG assets did not match the arch very perfectly, so I need to use rotopaint to fix some flaws.

We went to Graffiti tunnel and Lincoln’s Inn (Chancery Lane) to shoot the footage. I was recording the details.

• 

• 

Mindmap

Moodboard

Script

• 

• 

Production Elements

Storyboard

I created the storyboard for scene 2 as a schedule. But it was not used in shooting because we decided to make less shots for each scene.