About me

My name is Deheng Zhang, I am currently Ph.D. student at the University of Tübingen supervised by Prof. Hendrik Lensch. Previously, I finished my MSc at ETH Zürich where I worked on 3D Vision/Graphics research projects in Disney Research (Studio) Zürich overseen by Prof. Dr. Markus Gross and VLG overseen by Prof. Dr. Siyu Tang. Before that, I finished Bachelor’s degree at CityU of Hong Kong.

My current research interest is an intersection between computer vision and computer graphics, and I would like to explore more possibilities to combine deep learning with traditional rendering or 3D vision algorithms.

My hobbies include Rendering, Photography, Video Game, Finger-style guitar, Table Tennis, Skiing, and Hiking.

News

• 2024.09: I have joined IMPR-IS University of Tübingen as a PhD student, supervised by Prof. Hendrik Lensch!
• 2023.10: 🎉🎉 My first first-author paper CoARF: Controllable 3D Artistic Style Transfer for Radiance Fields has been accepted by 3DV 2024! Thanks to my supervisor Dr. Clara Fernández Labrador for the help and tons of useful advice for this project.

Education

• 2024 - Present, Doctoral Student, Computer Graphics, University of Tübingen, Germany.
• 2021 - 2024, Master of Science, Computer Science, ETH Zürich, Switzerland.
• 2017 - 2021, Bachelor of Science with First Class Honours, Computer Science (AI Stream), City University of Hong Kong, Hong Kong SAR.
• 2014-2017, Senior High School Diploma, Shandong Experimental High School, China.

Publications

RISE-SDF: a Relightable Information-Shared Signed Distance Field for Glossy Object Inverse Rendering Deheng Zhang*, Jingyu Wang*, Shaofei Wang, Marko Mihajlovic, Sergey Prokudin, Hendrik P.A. Lensch, Siyu Tang 3DV 2025 Paper | Project website | Dataset In this paper, we propose a novel end-to-end relightable neural inverse rendering system that achieves high-quality reconstruction of geometry and material properties, thus enabling high-quality relighting. The cornerstone of our method is a two-stage approach for learning a better factorization of scene parameters.

CoARF: Controllable 3D Artistic Style Transfer for Radiance Fields Deheng Zhang, Clara Fernández Labrador, Christopher Schroers, 3DV 2024 Paper | Project website In this paper, we introduce Controllable Artistic Radiance Fields (CoARF), a novel algorithm for controllable 3D scene stylization. CoARF enables style transfer for specified objects, compositional 3D style transfer and semantic-aware style transfer. We achieve controllability using segmentation masks with different label-dependent loss functions. We also propose a semantic-aware nearest neighbor matching algorithm to improve the style transfer quality.

Selected Projects

Point-Based Radiance Fields for Controllable Human Motion Synthesis Deheng Zhang*, Haitao Yu*, Peiyuan Xie*, Tianyi Zhang* (* means equal contribution) Course project of Digital Human in ETH Zürich Paper | Project website | Github Repo In this project, we proposed a new method for learning an animable human avatar model with point-based primitives. Specifically, our method exploits the explicit point cloud to train the static 3D scene based on Point-NeRF and apply the deformation by encoding the point cloud translation using a deformation MLP. We also guarantee rendering consistency by performing rotation-only ray-bending. The final animating avatar is comparable to other state-of-the-art animable human models.

NICE-SLAM with Adaptive Feature Grids Deheng Zhang*, Ganlin Zhang*, Feichi Lu*, Anqi Li (* means equal contribution) Course project of 3D Vision 2022 in ETH Zürich Paper | Github Repo In this project, we present a sparse version of NICE-SLAM, which is a SLAM system incorporating the idea of Voxel Hashing into NICE-SLAM framework. Instead of initializing feature grids in the whole space, voxel features near the surface are adaptively added and optimized.

Holo-Spot: Accessible Robot Control in Mixed Reality Deheng Zhang*, Ganlin Zhang*, Longteng Duan*, Guo Han* (* means equal contribution) Course project of Mixed Realiy 2022 in ETH Zürich Paper | Github Repo | Project website In this project, we design, implement and deploy a mixed-reality-based method with HoloLens 2 that enables users to control the Boston Dynamics Spot robot.

Kombu: Physically-based Renderer based on Nori in C++11 Deheng Zhang*, Ganlin Zhang* (* means equal contribution) Course project of Computer Graphics 2022 in ETH Zürich Github Repo | Project website In this project, I implemented part of the Kombu physical-based renderer. The function implemented by me includes volumetric rendering with heterogeneous participating media (ray marching, delta/ratio tracking as the transmittance estimation method), bilateral filter denoising, directional light, and object instancing. Finally, we produced an image about Christmas on the Moon for the rendering competition.

SAVA: Style-Attention-Void-Aware Style Transfer Deheng Zhang Bachelor Thesis in CityU HK Paper | Github Repo In this project, I propose a novel self-attention mechanism with specific mathematical meaning and a novel style transfer mechanism to learn the blank-leaving information in the style image. I also implement the code for training and testing, with a web-based GUI.

Optimization by Particle Swarm Using Surrogates via Bunch-Kaufman Pivoting and Standard Optimization Deheng Zhang*, Ganlin Zhang*, Junpeng Gao*, Yu Hong* (* means equal contribution) Course project of Advanced System Lab 2022 in ETH Zürich Paper | Github Repo Focus on speeding up black-box optimization algorithm OPUS from paper Particle Swarm with Radial Basis Function Surrogates for Expensive Black-box Optimization by Rommel G. Regis. Besides, we implement the speed-up C++ version of Bunch-Kaufman Pivoting.