About me

My name is Deheng Zhang, I am currently an Msc CS student at ETH Zürich. I obtained my Bachelor’s degree in CityU of Hong Kong, I was supervised by Prof. Dr. Jing Liao for my Bachelor’s thesis. Now I am doing my master thesis in VLG, supervised by Dr. Sergey Prokudin and Shaofei Wang, overseen by Prof. Dr. Siyu Tang. Previously, I also work on my semester thesis in Disney Research (Studio) Zürich about 3D style transfer, supervised by Dr. Clara Fernández Labrador and overseen by Prof. Dr. Markus Gross.

My current research interest is an intersection between computer vision and computer graphics, and I would like to explore more possibilities to combine deep learning with traditional rendering or 3D vision algorithms.

My hobbies include Rendering, Photography, Video Game, Finger-style guitar, Table Tennis, Skiing, and Hiking.

News

• 2023.10: 🎉🎉 My first first-author paper CoARF: Controllable 3D Artistic Style Transfer for Radiance Fields has been accepted by 3DV 2024! Thanks to my supervisor Dr. Clara Fernández Labrador for the help and tons of useful advice for this project.

Education

• 2021 - Present, Master of Science, Computer Science, ETH Zürich, Switzerland.
• 2017 - 2021, Bachelor of Science with First Class Honours, Computer Science (AI Stream), City University of Hong Kong, Hong Kong SAR.
• 2014-2017, Senior High School Diploma, Shandong Experimental High School, China.

Publications

CoARF: Controllable 3D Artistic Style Transfer for Radiance Fields Deheng Zhang, Clara Fernández Labrador, Christopher Schroers, 3DV 2024 Paper | Project website In this paper, we introduce Controllable Artistic Radiance Fields (CoARF), a novel algorithm for controllable 3D scene stylization. CoARF enables style transfer for specified objects, compositional 3D style transfer and semantic-aware style transfer. We achieve controllability using segmentation masks with different label-dependent loss functions. We also propose a semantic-aware nearest neighbor matching algorithm to improve the style transfer quality.

Selected Projects

Point-Based Radiance Fields for Controllable Human Motion Synthesis Deheng Zhang*, Haitao Yu*, Peiyuan Xie*, Tianyi Zhang* (* means equal contribution) Course project of Digital Human in ETH Zürich Paper | Project website | Github Repo In this project, we proposed a new method for learning an animable human avatar model with point-based primitives. Specifically, our method exploits the explicit point cloud to train the static 3D scene based on Point-NeRF and apply the deformation by encoding the point cloud translation using a deformation MLP. We also guarantee rendering consistency by performing rotation-only ray-bending. The final animating avatar is comparable to other state-of-the-art animable human models.

NICE-SLAM with Adaptive Feature Grids Deheng Zhang*, Ganlin Zhang*, Feichi Lu*, Anqi Li (* means equal contribution) Course project of 3D Vision 2022 in ETH Zürich Paper | Github Repo In this project, we present a sparse version of NICE-SLAM, which is a SLAM system incorporating the idea of Voxel Hashing into NICE-SLAM framework. Instead of initializing feature grids in the whole space, voxel features near the surface are adaptively added and optimized.

Holo-Spot: Accessible Robot Control in Mixed Reality Deheng Zhang*, Ganlin Zhang*, Longteng Duan*, Guo Han* (* means equal contribution) Course project of Mixed Realiy 2022 in ETH Zürich Paper | Github Repo | Project website In this project, we design, implement and deploy a mixed-reality-based method with HoloLens 2 that enables users to control the Boston Dynamics Spot robot.

Kombu: Physically-based Renderer based on Nori in C++11 Deheng Zhang*, Ganlin Zhang* (* means equal contribution) Course project of Computer Graphics 2022 in ETH Zürich Github Repo | Project website In this project, I implemented part of the Kombu physical-based renderer. The function implemented by me includes volumetric rendering with heterogeneous participating media (ray marching, delta/ratio tracking as the transmittance estimation method), bilateral filter denoising, directional light, and object instancing. Finally, we produced an image about Christmas on the Moon for the rendering competition.

SAVA: Style-Attention-Void-Aware Style Transfer Deheng Zhang Bachelor Thesis in CityU HK Paper | Github Repo In this project, I propose a novel self-attention mechanism with specific mathematical meaning and a novel style transfer mechanism to learn the blank-leaving information in the style image. I also implement the code for training and testing, with a web-based GUI.

Optimization by Particle Swarm Using Surrogates via Bunch-Kaufman Pivoting and Standard Optimization Deheng Zhang*, Ganlin Zhang*, Junpeng Gao*, Yu Hong* (* means equal contribution) Course project of Advanced System Lab 2022 in ETH Zürich Paper | Github Repo Focus on speeding up black-box optimization algorithm OPUS from paper Particle Swarm with Radial Basis Function Surrogates for Expensive Black-box Optimization by Rommel G. Regis. Besides, we implement the speed-up C++ version of Bunch-Kaufman Pivoting.