Shengze Zhang

Download literature review&technical document.

Human Skin Rendering, or Character Rendering, is a promising next-generation computer graphics and game techonology.

As a new researcher, I choose this topic, not only my own great interest but also one of future development of game engine, and start my "fresh" research under the supervision of Professor Xubo Yang, Digital Art Laboratory, Shanghai Jiao Tong University.

• Skin Color Determination
  • Monte Carlo Method is used in light transport in multi-layered tissues.
  • Three-layered skin model, including sebum, epidermis and dermis, is adopted.
  • Absorption and Scattering is the two main simulation sections.
  • An adjustable parameter model is applied, including melanin volume fraction and eumelanin fraction to pheomelanin in epidermis, and hemoglobin fration in dermis.
  • The difference of linear absorption(faster) and exponential absorption(more accurate) is analyzed.
  • Result for Spectral Power Distribution(SPD) and skin color:

    Change in skin color as melanin volume fraction f.mel from 1.3% to 43%. The eumelanin fraction is fixed at 0.7, hemoglobin fraction is fixed at 0.5%.

    #	fraction.melanin = 1.3%	fraction.melanin = 6.3%	fraction.melanin = 16%	fraction.melanin = 43%
    Linear absorption
    Exp absorption

    Change in skin color as hemoglobin fraction f.h from 0.1% to 10%. The melanin fraction is fixed at 1.3%, with eumelanin fraction at 0.5.

    #	fraction.hemoglobin = 0.1%	fraction.hemoglobin = 0.5%	fraction.hemoglobin = 2%	fraction.hemoglobin = 10%
    Linear absorption
    Exp absorption

    Change in skin color as eumelanin fraction f.emel from 0 to 1. The melanin fraction is fixed at 43%. The hemoglobin fraction is constant at 0.1%.

    #	fraction.eumelanin = 0	fraction.eumelanin = 0.33	fraction.eumelanin = 0.67	fraction.eumelanin = 1
    Linear absorption
    Exp absorption

• Subsurface Scattering Simulation
  • A naive BSSRDF model is applied.
  • Coordinate system is converted into local normal-binormal-tangent system when a ray hits the human mesh. Therefore light transport is simulated in the normalized local system and the result is converted back to the world system.
  • Result, or outgoing point mainly, is projected into the real 3-D mesh, fulfilling the task of BSSRDF model.
  • This method is used primarily for eliminating the diffuculty of light transport in real and arbitrary 3-D mesh.
  • An assumption that skin is semi-infinite is made to prove the correctness of this BSSRDF model, and the result in highly curve position(nose, ear) is largely biased as a result. Transparence is reduced as well under this model.
  • Result with high resolution and for different human races:

    500-Sampled result.

    Skins for different human races are simulated with our model. For Caucasian, f.m = 1.3%, f.em = 0.7 and f.h = 0.5%. For Asian, f.m = 16%, f.em = 0 and f.h = 1%. For African, f.m = 43%, f.em = 0.7 and f.h = 1%.

    Caucasian	Asian	African

• Considering the non-chromatic factors(wrinkle, pore) is the most prior improvements, which can enhance the render quality greatly.
• BSSRDF model needs to be totally changed. Two ways:
  • Using pratical model
  • Simulating light transport in real 3-D mesh
• Despite the acclerate techniques of K-D Tree, multi-thread and cache, the render time is terrible largely due to Monte Carlo Method. Also two ways:
  • Speeding up Monte Carlo Method itself
  • Building a real BSSRDF model for looking up the in-out radiation distribution

Final Project of Computer Graphics

Ray Tracing is a basic render method other than rasterization-based render method(GPU architecture).
Theoretically, it needs more time to render but can creat more realistic result than rasterization does.

The most powerful ability of Ray Tracing is that as long as a basic tracing framework is established, any small change can make a direct improvement in the render result.
Therefore, let me show my work in the order from simple to complex to demonstrate the power of Ray Tracing. Some scenes in the following render result is the same as Jacco Bikker's Article

• Phong Shading
• Reflection
• Refraction
• Hard Shadow
• Antialiasing
• Soft Shadow
• Real Reflection
• Texture&Bump Map&Model

• Phong Shading is a classical light model.
• Ambient, Diffuse and Specular parts are combined to generate reflection in this model.
• An acceptable result.

• Reflection is added.
• Result is greatly enhanced by a just 12-line patch.

• Normal
• Beer¨CLambert law

• Refraction is added.
• Result is awesome, and some really advanced effects such as magnifier-glass can be directly got.

• The Beer¨CLambert law generalizes the realtion between the absorption of light and properties of the material through which the light is traveling.
• The direct visual effect of Beer's Law is that the transparent part(two big balls) is a little dimmed compared to result without Beer's Law.

• Shadow is added.
• The render result becomes more and more realistic.
• The word "Hard" means the sharp division between shadowed part and non-shadowed part, which is very different from real shadow(Such shadow is called soft shadow and the simulation of it can be seen in the following tab).

• Antialiasing is added.
• Serration effect is largely eliminated.(Please compare it to the previous result(Hard Shadow Tab).)

• Hard Shadow(For Comparison)
• Soft Shadow(Direct Implement)
• Soft Shadow(Monte Carlo Method)

• This result is rendered with a point light and just for comparison with the following two results.
• ...But it's a new scene!

• The reason why the shadow is "hard" is the point light source, which means only one sample on the light source.
• Soft shadow can be implemented by turning the point light source to area light source and multi-sampling on the area.
• The result is great, but despite the relatively high sample rate of 8 * 8 = 64, the banding effect(Please see the small result above) is still detectable.

• Monte Carlo Sampling is used on the area light source.
• With the same sample rate of 64, the banding effect is completely eliminated.

• Soft Shadow(For Comparison)
• Real Reflection

• Real reflection, which means that due to the roughness of the material, the reflected ray can redirect to arbitrary directions in a cone area, rather than a mirror reflection on a smooth surface, is added.
• Monte Carlo Sampling is used as well.
• The reflected parts are blurred. The left ball is rougher than the right ball, therefore the left ball is more blurry than the right one.
• As a type of super-sampling, real reflection also accomplishs part function of antialiasing.

• Texutre and model are added.
• Bump map is added(Please see the wrinkles in the small result).

• Multi-thread accleration.
• K-D tree accleation.

The two accleration technologies speed up the render process by more than 500%.

One interesting further improvement called incremental rendering is raised by me and Yuetao Xu, my classmate, during our discussion.
The main idea of incremental rendering is to change the tracing sequence of rays.

Use tree traversal as example. Common ray tracing, my method as well, uses depth first traversal: one primary ray is traced; if any, reflected secondary ray is traced; if any, reflected ray of the reflected ray is traced; ...; ...; if any, refracted ray is traced, ...; one primary ray is completely traced, trun to the next primary ray.

Inremental ray tracing uses a concept of priority first traversal: every primary rays are traced first to maintain every pixel has color, and generated secondary rays are not traced temporarily and are put in an ordered container by their father ray's result weight.
Then the secondary ray in the head of the container is traced and added to the result. If any new secondary rays are generated, the traced ray's result weight is multiplied by it's father ray's result weight as the new result weight of the new generated rays, and these new rays are dynamically added to the container by their weight.
The new head of the container is traced then. Tracing will conitnue until the number of traced rays reaches a threshold or the result is good enough.

The motivation of increamental rendering is our discovery of huge difference of the contribution to render result among rays.
The common method usually record the trace depth of a ray and set a max-depth value. One ray traveling in the material with high reflectivity, high refractivity and low absorption will trace the same times as another ray traveling in the material with low reflectivity, low refractivity and high absorption, and their contributions to the result is definitely different.

Incremental ray tracing can largely eliminate this phenomenon and give those "powerful" rays more chances to "express themselves".
Therefore the final result will be exquisite in those bright and complex areas such as a mirror group, and be mediocre in those glommy and simple areas such as a plane or an opaque object.

This online benchmark evaluates the performance of WebGL on the specific browser you are using.

Color, Move, Light and Shadow, Texture, the four most important features in GL are evaluated.

I cooperate with Shengsi Tong, Xiaoning Liu and Jianyu Gu to develop the benchmark, and I take charge of Light&Shadow Section, integration of four sections and front end GUI.

Color Section

Move Section

Light and Shadow Section

Texture Section

Why not have a try, see how your broswer supports WebGl.

Final Project of Game Programming

Lock(Powered by Unity3D) tells a story about a young man who loses his memory, wakes up in a strange house and finally gets his memory back as the progress of the game goes on.

It is hard to classify the game type of Lock for elements of different game types are combined, ACT, AVG, PUZZLE, RPG.
Generally, Lock is an Adventure Game like Tomb Raider(2013), but classical puzzle elements like those in Machinarium(2009) are added.

I lead a group of three and cooperate with Jingjie Yang(Scene Designer) and Yunjia Ge(Play Designer) and act as programmer simultaneously.

Main play points of Lock include, but are not limited to:

Mysterious adventure story The story play is revised for several times and the final version describe a both mysterious and compelling story. All facts will be clear in the grand finale. Player will experience a journey of recovering memory.
Well-designed decryption clues All clues are closely linked and demand player to carefully search every corner of the scene and make his own reasoning on the already-found clues. Design of some clues is ingenious, which is a big point of Lock. Some clues is well combined with the combat system.
Good balance between puzzle and plot Player will neither be stuck in the puzzle part, forgetting why he should figure out this puzzle, nor read a large number of plot texts, thinking he is watching a TV play rather than a computer game. The pace of puzzle and plot is well planned, making player immerse himself in the game.
Simple but interesting combat system Monsters have AI, and player needs to find clues behind the action pattern. An important but hidden element subtly combining with the main plot is in the out-door combat system. A little ACT element is added, and a skillful player can finish the fight without a damage.
Enhanced visual effects of self-developed shaders Camera effect of old photo is added to build a glommy and mysterious atmosphere. The whole screen will gradually regain it's color with the progress of the game. Camera effect of shading is added to express the pain of character when recalling his lost memory. Aim for these effects is build a more immersive feeling. Object highlight is also implemented by shader.
Beautiful environment Particle systems of butterfly and Willow catkin Water surface and bubbles Sun light and halo

Final Project of Digital Image Processing(DIP)

This course project is a researh project of DIP's lecturer Kai Xiao. The primary goal of this assignment is to find the potential methods. The research objective is precisely segmenting plant photos taken by a fixed HD camera.

• ANN is trained by Backpropagation(BPNN).
• Color space and position information are adopted as eigenvector.
• Mean-Shift algorithm is used as preprocessing to blur the image and generate data block for later training and segmenting.
• Target image sample and Segmenting result(6 parts):
  • Trunk of trees
  • Upper left small tree leaves
  • Upper right narrow tree leaves
  • Bottom left orchid leaves
  • Dead leaves on the ground
  • Small green plant on the ground
  • Trunk of trees
  • Upper left small tree leaves
  • Upper right narrow tree leaves
  • Bottom left orchid leaves
  • Dead leaves on the ground
  • Small green plant on the ground
• The NN training algorithm is stable and with high accuracy of segmenting. One further improvement is adding texture information, such as co-occurrence matrices, into current eigenvector system.

Qing Ying is an Interactive Multimedia Creating and Sharing Platform. The word "Interactive Multimedia" refers to the unique, animation-style form of works in Qing Ying.

Qing Ying is a HTML5-based platform, therefore creating, watching, sharing and all the functions can be done online with only one raw broswer supporting HTML5.
No Flash. No plug-in.

The slogan of Qing Ying is "Simple, Share, Cooperate, Interact". Qing Ying intends to enable everyone to create a piece of multimedia work without any diffculties.

I cooperate with Yuetao Xu, Cheng Gu and Xiaoning Liu to design and develop Qing Ying, and Qing Ying win 2nd Prize in the 5th Intel Cup National Collegiate Software Innovation Contest and participate in the 6th National Undergraduate Innovation Program as well.

Then two main functions of Qing Ying is watching, or playing, and creating, or edting:

• Qing Ying Player

• Qing Ying Advanced Editor

Download a dated video to have a direct feeling about Qing Ying.

Qing Ying is an online platform with complete functions, and an English User Manual which can be downloaded or read online is available.

Any showy literal introduction is no better than a just 5-minute try. So having read so many texts, let's go to Qing Ying and have your first impression.

Source code is available here or you can checkout it by: svn checkout http://light-shadow.googlecode.com/svn/trunk/ light-shadow-read-only