All posts by Xuran Ni

Scene 5 CG Design and Compositing

Design

Since our story belongs to the category of magic and suspense, according to the mood board we made before, I referenced some concept maps of science fiction games and simply composited them with the original scenes. Due to the lighting problem of the original footage(strong hard light and outdoor light), I finally decided to change the shooting room into a semi-open scene to weaken it.

Since setting the scene on the ground is more restrictive, the style of the room will affect the composite background to a greater extent, and the shooting angle limits the field of view, so I finally decided to make the shooting room float in the air.

I tried the HDRI that I can find on the Internet as the background, but the style is not ideal. Because the specular reflection in the scene can be ignored, I decided to use the pictures with buildings as the distant view.

Background Image

For the last shot, referring to the script, I designed it as two characters walking towards a secret room.

Add a CG light source

Since the strong light source was not removed during shooting, the result of roto was not satisfactory. After discussion, our teammate thought that adding a light source will help. The choice of the light source need to make sense in such environment. It is better to use a static CG glowing stuff, because the moving light source will affect the shadows in the original scene. Due to the complexity of the original scene light and shadow and the architectural structure, it will take time to render the shadows with moving light sources. However, this stationary light source also need to satisfy the layout of the scene so that it is not abrupt.

The use of CG light source is to fix artifacts that cannot be solved in rotoscoping for both shot 8 and shot 9. When choosing a static light, I found it can not be placed at correct positions. Finally I chose to use a sci-fi robot.

Because it would take much time to render shadow, I decided to create it in nuke using original robot sequence with some transform nodes, grade and blur alpha channel, keying at certain frames to match the movement of the robot.

CG robot light shadow

I also used flare node at the end to match the lighting effect from the CG light.

3D Tracking

The tracking information given by other people was useless because all the points look correct in 2D viewport but make no sense in 3D viewport. I tried many times by using grade nodes and adjusting settings in camera tracker but no help at all. However, the camera rotation path looked ok. So I finally placed the cards with a camera to match the scene.

shot 9 – 3D tracking

Animation and Rendering

The following CG assets are downloaded from sketchfab,

Door: https://sketchfab.com/3d-models/rr-door-aed85da0529f419d89dd724cdd883fb5

Floating stones: https://sketchfab.com/3d-models/european-and-american-game-scencemagic-portal-31182bdc05dd40ddbc7f284297e8f225

Sci-fi camera: https://sketchfab.com/3d-models/sci-fi-camera-drone-3dbf07385d1b471a96dce6de6cba97d6

I just imported the cards and camera from nuke to maya so that I could adjust the position of the assets in the scene. I also made animation or did adjustments to the existing animation to match the movement in the shot.

I used a spherical light source with image background to illuminate close-up objects in maya.

In order to improve the rendering speed, I set the mesh to blocker to block the invisible part of the CG after compositing. At the same time, use arealights and blocker to create corresponding shadow effects for indoor CG objects.

shot8
shot9

After testing, arnold GPU rendering and CPU rendering under the same settings, although it will be several times faster, but in this scene will cause fireflies, so I tried, and finally found that enable adaptive sampling and set max to 10 is more efficient In practice, when the default max is 20, the speed is almost the same as the CPU rendering.

Compositing

Given roto data on characters from my group mates, I started compositing for shot 8 and 9.

Image background

Because the image background is a still image, it works in shot 8 but need to be rotating in shot 9 as the camera rotates. So I projected the mirrored image to a cylinder to scanrender.

Roto on the archs

For shot 8 it is easy because the camera does not move, so I just simply roto out the shape.

For shot 9 it is difficult to make it stick stably so I tried to roto at single frame and project them using camera on the cards with the help of shuffle nodes, because alpha channel cannot be scanrendered. However, it does not fit very perfectly all the time and there is always a shift, no matter how I adjusted the cards, it could not be solved. I tried to use transform and cornerpin nodes to adjust these but it became more complicated.

In this case, I just used them for certain frames, and for frames before 105, I rotoed it frame by frame. It was difficult to get a perfect result because of blurred pixels on the low-contrast edges.

Portal

The portal we initially used has some problems to match the scene, in this case I tried to use a more transparent one to replace.

The new portal is a downloaded resource from Doctor Strange, I graded and transformed it to match the scene.

Merge

Because CG assets did not match the arch very perfectly, so I need to use rotopaint to fix some flaws.

Character creation

Design

For this project I want to design a cute character, so I used chibi’s body shape (2 heads).

Color palette

For the colorof this character, I want to keep saturation and value(brightness) at lower level. I choose purple pink and yellow, which give peaceful, tender and innocent feelings.

Modelling and Texturing

Since I couldn’t find suitable resources (most of the body proportions were incorrect), I referred to the three views in 2D to model the character.

The lighting effects under default setting looks horrible, it need to be set the light used and shadows to observe the desired result. Also, depth map shadows should be enabled to see the shadows.

I unwraped UV for face and painted it in Sai2(digital painting software).

Rigging and skinning

I used Mixamo to rig the character so that I can use the animation from mixamo directly. However, the imported character should be cleaned and without strange shapes. Because my character is too small, the result was not as expected, I need to paint it again to fix bugs in Maya.

When skinning is completed, it is best to export it for future use, or save in another script to copy from.

Blendshape and face animation

I used blendshape here to make face animation, it also can be applied in game engine.

When making eye animation, I grouped(ctrl+G) eyes, eyelashes and face to create blend shape.

When making mouth animation, I grouped face,teeth and tongue to create blend shapes.

They can be moved out of the group (or moved into another group)after finishing current morph deformation.

However, the base mesh(group and its hierarchy) is fixed for each blend shape.

When creating blendshape, I set to world because local will ignore transformation, I need translation infomation here so I keep it.

The blend shape target can be rebuild so the mesh could be created in case we need it again.

The deformation order should be pre-deformation (before skinning),because it is has binded skin.

I drawed some images with alpha channel to create 2D appearances here.

It is better to separate two meshes with the same number of vertices (eyelashes) when making blend shapes, because it will exchange the their targets sometimes.

In order to animate eyes, I rigged a bone to control both eyes, the joint should be on the intersection of their vertical bisectors so that the eyes can rotate in the correct region. I also limited the translation here.

face animation test

Non-photorealistic rendering research

Shading

Blender Tutorial - Physically Based Rendering (PBR) and the Principled Node  - YouTube
PBR is photorealistic rendering

In traditional painting, there are six shaded regions.

Outlines are the one of the most important features in cel painting. And shading is simplified to 2 types: halftone and shadow. Hence the boundary between the light and shadow is another feature of cel painting. Nowadays, highlight is always used in cel animation through adding special effects.

Animation Cel Painting by Christopher DeStefano - YouTube

In digital(modern) paintings, different techniques for shading are used together for artists to create diverse art styles.

阴阳师:CP吸屁2018年设计稿曝光,这么好看的御馔津可惜了
原神:1.3版本實裝,魈的魅力太大,全球玩家心甘情願為他買單- 街拍漂亮小姐姐
Image

For non-photorealistic rendering, the performance of lighting is the most important feature, and it is also inseparable from the topology of the 3D mesh.


Another thing worth noting is that the 2D animation will show omitted topology details (such as the highlights on the hair) through light and shadow, but the highlights rendered by toon shading renderer will expose the real structure of the 3D topology. When applied to real-time toon shading, artists often fix the highlight on the material by texturing. This method solves the problem to some extent, but it seems strange when the light changes in the scene.

Painting media

Some details should be reduced or altered depending on strokes and techniques.

Watercolor

Josepth Zbukvic
https://www.bilibili.com/video/BV1C4411S78E


Watercolor gradients are mostly used in the creation of shadows.

Gouache(opaque watercolor)

Gouache is similar to watercolor in that it can be re-wetted and dries to a matte finish, and the paint can become infused into its paper support. It is similar to acrylic or oil paints in that it is normally used in an opaque painting style and it can form a superficial layer.

Watermedia Mystery: Really, What is Gouache? - Creative Catalyst Productions

Opaque watercolors look thick(strong coverage), wheras watercolors are more transparent(fluidity).

MY NEIGHBOR TOTORO  1988. 50th Street Filmscourtesy Everett Collection

Watercolor is more suitable for illustrations or still scene(background). Opaque watercolor is more suitable for animated character and front scene. Opaque watercolor and watercolor are usually mixed, where watercolor used as background and opaque watercolor for the details.


Alcohol-based markers

Alcohol markers are different because the ink is permanent. However, you can still layer the colors to create striking overlays. For example, gray overlays will dull colors down while yellow will brighten them up.

https://www.bilibili.com/video/BV1mJ411k7qT?from=search&seid=12035595433229581903

To me, markers style reminds me of childhood, and traditional watercolors look more romantic.

Crayon

https://blendernpr.org/crayon-charcoal-bge-test-train-face-boy/

Cel shading(toon shading)

Cel shading, as a 3D animation and rendering techniques, have regularly been used by Ghibli since the production of Princess Mononoke (1997) – the first feature to benefit from the distribution deal – to achieve complex effects that would be impossible using traditional pen-and-ink methods. Those CG shots were seamlessly blended with the hand-drawn images in production.

Some paintings of other styles that usually appear in the form of still frames can also be made into animation, but the cost would be high, such as this fully painted feature film which took six years to produce: Loving Vincent.

Motion design and frame rate

I think one of the problems when using non-photorealistic rendering is that the animation looks too fluid and realistic than it should be, especailly for the use in limited animation. Fully animated films are animated at 24 frames per second, wheras limited animation uses 8-12 drawings per second, thereby limiting the fluidity of the animation.

For those who are used to watching limited 2D animation, watching the 3D animation of toon shading will give them some strange feelings, because the movement of the characters is too realistic, leading to the movement of non-photorealistic shadow look too realistic . The human eye is usually very sensitive, such a realistic movement reveals the topology used in 3D animation.

2D hair fabric motion design
3D
https://www.youtube.com/watch?v=qzJ4yzuIrRI
3D

Scripts and storyboard (Version.2)

The script and storyline have been redesigned, the theme is about an elf’s adventure in the dessert theme park.

Scripts

Various fruit decorations slowly move forward on the food shaft, being picked up one by one by the food pincers. The elf (nibbing fruit) appeared on the food reel, and then was picked up by the food pincers, and came to the top of the oven. After the oven was turned on, the tart came out of the oven, and the elf was placed on the tart. The elves and egg tarts were placed on Tilt-A-Whirl and then rotated and swiped down. The elf jumped off the egg tarts and came to the Log Rides made of cream, riding on the donuts, avoiding the puffs in the cream torrent. Suddenly the elf was involved in the cream whirlpool, rushed out of the other end of the whirlpool after spinning many times, landed on the cream waterfall ski resort, and rushed all the way down. In the end the elf fell on the big cake, and standing beside him was a fondant elf.

These images are not the storyboard, just the illustration for storyline. I plan to use one-shot for this short movie.

Props reference

watercolor style

Character design

Introduction

Summary

This semester I am going to try to use non-photorealistic rendering to make a music video for my song.

The music duration is 1:13, and the video should be a bit longer than this, which will be divided into two parts by the verse(~0:41) and the chorus (0:41~). The verse part will be the story of the girl’s adventure at home, and the chorus part will be the story of the girl entering the wonderland.

Animation characters

The girl

The toys

The background

Moodboard, art direction

The verse

The chorus

flying magic object1
flying magic object2
flying magic object3

cloud
slide

The end

Script, storyboard

0:00-0:12 Early in the morning, the little girl is still sleeping in bed, the hour hand is already pointing to ten o’clock.

0:12-0:14 Suddenly, the doorbell rang and the little girl was awakened.

0:14-0:24 The little girl tiptoed her head out to see what happened, she looked at the door thoughtfully, remembering that mother said there was a big bad wolf outside.

0:24-0:35 The little girl moved the table and stool and wanted to see what happened through the cat’s eye.

0:35-0:41 The little girl was stunned and found that the house was flying in the air.

0:41-0:56 {the first part of the chorus}

The little girl opened the door and the house was flying in the air, and the little girl was looking around in excitement. Below is the road connected by stars. In the distance is a rainbow, reflected in the girl’s eyes

0:56-1:07 {the second part of the chorus}

The little girl is running on the rainbow, jumped up and slid down, then fell on a large cloud. The little girl lies on the clouds with her eyes closed.

1:07-1:13 At the end, the little girl fell on the “cotton”, the camera flipped 90° then zoom out, the little girl is sleeping soundly on the bed in the room.

Pre-production

Mindmap

by Diana Chinalieva

Moodboard

by Diana Chinalieva

Script

Production Elements

by Madeleine Spivey

Storyboard

Scene1 by Evanna Cao
Scene 2

I created the storyboard for scene 2 as a schedule. But it was not used in shooting because we decided to make less shots for each scene.