01 · about
I am a Member of Technical Staff at Kimi (Moonshot AI), working on agentic coding post-training. Previously, I was a research engineer at Zhipu.AI, where I worked on improving the performance of VLMs on video, alignment, reasoning, etc. My research interests lie in visual understanding and generation, language modeling and world modeling.
I received my Master’s degree from the School of Computer Science at Peking University, at CILab & AIIC, advised by Prof. Boxin Shi and Mr. Ming Lei. I received my Bachelor of Engineering degree in Automation and Bachelor of Arts degree in Economics from Tsinghua University in 2021. My academic research topic involves 3D modeling with neural implicit representations, computational photography, and image quality enhancement.
Deep learning is a useful tool (arguably more useful than most people think) for real-world applications. I enjoy tackling various tasks (e.g., molecule design, quantitative trading, interior design, recommendation system, image quality enhancement) with AI techniques and seeing the real-world impact of my work. I have worked (mostly interned) at wonderful AI start-ups (Collov, QuanMol), corporate companies (Alibaba, ByteDance), and a quantitative investment firm (Definitive Capital Management). Along the way, I have met many mentors and peers in the field of intelligence.
02 · news
-
We released Kimi K3, a 2.8T-parameter open MoE model with native multimodality and 1M-token context, built for long-horizon agentic coding.
-
Our paper “MotionBench: Benchmarking and Improving Fine-grained Video Motion Understanding for Vision Language Models” is accepted by CVPR 2025.
-
I will join Zhipu.AI as a Research Engineer.
-
Our paper “Colorizing Monochromatic Radiance Fields” is accepted by AAAI 2024 and selected for oral presentation.
-
Our paper “SPLiT: Single Portrait Lighting Estimation Via a Tetrad of Face Intrinsics” is accepted by T-PAMI.
03 · selected publications
-
Arxiv · 2025
GLM-4.5V and GLM-4.1V-Thinking: Towards Versatile Multimodal Reasoning with Scalable Reinforcement Learning
abstract
We present GLM-4.1V-Thinking and GLM-4.5V, a family of vision-language models (VLMs) designed to advance general-purpose multimodal understanding and reasoning. In this report, we share our key findings in the development of the reasoning-centric training framework. We first develop a capable vision foundation model with significant potential through large-scale pre-training, which arguably sets the upper bound for the final performance. We then propose Reinforcement Learning with Curriculum Sampling (RLCS) to unlock the full potential of the model, leading to comprehensive capability enhancement across a diverse range of tasks, including STEM problem solving, video understanding, content recognition, coding, grounding, GUI-based agents, and long document interpretation. In a comprehensive evaluation across 42 public benchmarks, GLM-4.5V achieves state-of-the-art performance on nearly all tasks among open-source models of similar size, and demonstrates competitive or even superior results compared to closed-source models such as Gemini-2.5-Flash on challenging tasks including Coding and GUI Agents. Meanwhile, the smaller GLM-4.1V-9B-Thinking remains highly competitive—achieving superior results to the much larger Qwen2.5-VL-72B on 29 benchmarks. We open-source both GLM-4.1V-9B-Thinking and GLM-4.5V.
-
Arxiv · 2025
GLM-4.5: Agentic, Reasoning, and Coding (ARC) Foundation Models
abstract
We present GLM-4.5, an open-source Mixture-of-Experts (MoE) large language model with 355B total parameters and 32B activated parameters, featuring a hybrid reasoning method that supports both thinking and direct response modes. Through multi-stage training on 23T tokens and comprehensive post-training with expert model iteration and reinforcement learning, GLM-4.5 achieves strong performance across agentic, reasoning, and coding (ARC) tasks, scoring 70.1% on TAU-Bench, 91.0% on AIME 24, and 64.2% on SWE-bench Verified. With much fewer parameters than several competitors, GLM-4.5 ranks 3rd overall among all evaluated models and 2nd on agentic benchmarks. We release both GLM-4.5 (355B parameters) and a compact version, GLM-4.5-Air (106B parameters), to advance research in reasoning and agentic AI systems.
-
CVPR · 2025
MotionBench: Benchmarking and Improving Fine-grained Video Motion Understanding for Vision Language Models
abstract
In recent years, vision language models (VLMs) have made significant advancements in video understanding. However, a crucial capability—fine-grained motion comprehension—remains under-explored in current benchmarks. To address this gap, we propose MotionBench, a comprehensive evaluation benchmark designed to assess the fine-grained motion comprehension of video understanding models. MotionBench evaluates models’ motion-level perception through six primary categories of motion-oriented question types and includes data collected from diverse sources, ensuring a broad representation of real-world video content. Experimental results reveal that existing VLMs perform poorly in understanding fine-grained motions. To enhance VLM’s ability to perceive fine-grained motion within a limited sequence length of LLM, we conduct extensive experiments reviewing VLM architectures optimized for video feature compression and propose a novel and efficient Through-Encoder (TE) Fusion method. Experiments show that higher frame rate inputs and TE Fusion yield improvements in motion understanding, yet there is still substantial room for enhancement. Our benchmark aims to guide and motivate the development of more capable video understanding models, emphasizing the importance of fine-grained motion comprehension.
-
ICLR · 2025
CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer
abstract
We present CogVideoX, a large-scale text-to-video generation model based on diffusion transformer, which can generate 10-second continuous videos that align seamlessly with text prompts, with a frame rate of 16 fps and resolution of 768 x 1360 pixels. Previous video generation models often struggled with limited motion and short durations. It is especially difficult to generate videos with coherent narratives based on text. We propose several designs to address these issues. First, we introduce a 3D Variational Autoencoder (VAE) to compress videos across spatial and temporal dimensions, enhancing both the compression rate and video fidelity. Second, to improve text-video alignment, we propose an expert transformer with expert adaptive LayerNorm to facilitate the deep fusion between the two modalities. Third, by employing progressive training and multi-resolution frame packing, CogVideoX excels at generating coherent, long-duration videos with diverse shapes and dynamic movements. In addition, we develop an effective pipeline that includes various pre-processing strategies for text and video data. Our innovative video captioning model significantly improves generation quality and semantic alignment. Results show that CogVideoX achieves state-of-the-art performance in both automated benchmarks and human evaluation.
-
Tech Report · 2024
CogVLM2: Visual Language Models for Image and Video Understanding
abstract
Beginning with VisualGLM and CogVLM, we are continuously exploring VLMs in pursuit of enhanced vision-language fusion, efficient higher-resolution architecture, and broader modalities and applications. Here we propose the CogVLM2 family, a new generation of visual language models for image and video understanding including CogVLM2, CogVLM2-Video and GLM-4V. As an image understanding model, CogVLM2 inherits the visual expert architecture with improved training recipes in both pre-training and post-training stages, supporting input resolution up to 1344×1344 pixels. As a video understanding model, CogVLM2-Video integrates multi-frame input with timestamps and proposes automated temporal grounding data construction. Notably, CogVLM2 family has achieved state-of-the-art results on benchmarks like MMBench, MM-Vet, TextVQA, MVBench and VCGBench.
-
Arxiv · 2024
DreamPolish: Domain Score Distillation With Progressive Geometry Generation
abstract
We introduce DreamPolish, a text-to-3D generation model that excels in producing refined geometry and high-quality textures. In the geometry construction phase, our approach leverages multiple neural representations to enhance the stability of the synthesis process. Instead of relying solely on a view-conditioned diffusion prior in the novel sampled views, which often leads to undesired artifacts in the geometric surface, we incorporate an additional normal estimator to polish the geometry details, conditioned on viewpoints with varying field-of-views. We propose to add a surface polishing stage with only a few training steps, which can effectively refine the artifacts attributed to limited guidance from previous stages and produce 3D objects with more desirable geometry. The key topic of texture generation using pretrained text-to-image models is to find a suitable domain in the vast latent distribution of these models that contains photorealistic and consistent renderings. In the texture generation phase, we introduce a novel score distillation objective, namely domain score distillation (DSD), to guide neural representations toward such a domain. We draw inspiration from the classifier-free guidance (CFG) in text-conditioned image generation tasks and show that CFG and variational distribution guidance represent distinct aspects in gradient guidance and are both imperative domains for the enhancement of texture quality. Extensive experiments show our proposed model can produce 3D assets with polished surfaces and photorealistic textures, outperforming existing state-of-the-art methods.
-
AAAI · 2024
[Oral] Colorizing Monochromatic Radiance Fields
abstract
Though Neural Radiance Fields (NeRF) can produce colorful 3D representations of the world by using a set of 2D images, such ability becomes non-existent when only monochromatic images are provided. Since color is necessary in representing the world, reproducing color from monochromatic radiance fields becomes crucial. To achieve this goal, instead of manipulating the monochromatic radiance fields directly, we consider it as a representation-prediction task in the Lab color space. By first constructing the luminance and density representation using monochromatic images, our prediction stage can recreate color representation on the basis of an image colorization module. We then reproduce a colorful implicit model through the representation of luminance, density, and color. Extensive experiments have been conducted to validate the effectiveness of our approaches.
-
IEEE T-PAMI · 2023
SPLiT: Single Portrait Lighting Estimation Via a Tetrad of Face Intrinsics
abstract
This paper proposes a novel pipeline to estimate a non-parametric environment map with high dynamic range from a single human face image. Lighting-independent and -dependent intrinsic images of the face are first estimated separately in a cascaded network. The influence of face geometry on the two lighting-dependent intrinsics, diffuse shading and specular reflection, are further eliminated by distributing the intrinsics pixel-wise onto spherical representations using the surface normal as indices. This results in two representations simulating images of a diffuse sphere and a glossy sphere under the input scene lighting. Taking into account the distinctive nature of light sources and ambient terms, we further introduce a two-stage lighting estimator to predict both accurate and realistic lighting from these two representations. Our model is trained supervisedly on a large-scale and high-quality synthetic face image dataset. We demonstrate that our method allows accurate and detailed lighting estimation and intrinsic decomposition, outperforming state-of-the-art methods both qualitatively and quantitatively on real face images.
-
IEEE TASE · 2023
Fault Diagnosis of Energy Networks Based on Improved Spatial–Temporal Graph Neural Network With Massive Missing Data
abstract
In order to ensure the safe and reliable operation of the energy system, real-time fault diagnosis technology is indispensable. Energy systems are typically complex systems consisting of multiple subsystems that are coupled with each other. Before and after the occurrence of a fault, the system is generally in an abnormal or even harsh environment, which may cause a large number of randomly missing measurement data and make the application of fault diagnosis technology extremely difficult. In this paper, the graph attention network (GAT) is improved by a Gaussian mixture model (GMM) for incomplete-data representation. The iteratively updated expectation of the GMM serves as the characterization of missing data, which significantly improves the ability to fill in missing data. The GAT fuses multi-source data according to the topology structure so as to comprehensively exploit the spatial information. The gated recurrent units (GRU) extract dynamic fault information from embedded spatial features and classify the time series into various fault types. Moreover, we propose a loss function in the form of weighted focal loss so that the fault-class imbalance issue brought by the data deficiency can be solved. The proposed uniform spatial-temporal graph neural network classification framework together with the GMM (GM-STGNN) can effectively improve fault diagnosis performance and is applied on an experimental platform of an authentic industrial estate.
-
IEEE TIM · 2022
Fault Diagnosis of Energy Networks: A Graph Embedding Learning Approach
abstract
For industrial parks containing energy systems, fault diagnosis technology is of great significance for their safe operation. In recent years, the topology of energy systems has become more complex due to the use of technologies such as cogeneration, leading to multienergy coupling. Critical equipment and user nodes in these complex energy networks are vulnerable to a lack of sensor data or non-idealities in the measurement environment. There is an urgent need for a unified and robust fault-diagnosis framework for the overall system to identify faults even under non-ideal data conditions. In this article, to address the problem of fault identification and state prediction, a novel deep learning model is constructed based on graph-embedded recurrent neural networks (RNNs) with self-attentional layers. Unstructured data are put into the graph neural network to extract common spatial features. An additive attention mechanism is implemented in the graph attention network (GAT) to integrate multiscale node information. The graph operator is computed within a gated recurrent unit (GRU) that captures the full range of temporal features. In addition, loss functions are introduced for fault identification and state prediction. Data from an industrial park experiment platform are used for fault identification experiments. The advantages of the proposed approach are illustrated by comparative experiments with different levels of missing data.
-
CVPR · 2020
Structure-Preserving Super Resolution With Gradient Guidance
abstract
Structures matter in single image super resolution (SISR). Recent studies benefiting from generative adversarial network (GAN) have promoted the development of SISR by recovering photo-realistic images. However, there are always undesired structural distortions in the recovered images. In this paper, we propose a structure-preserving super resolution method to alleviate the above issue while maintaining the merits of GAN-based methods to generate perceptual-pleasant details. Specifically, we exploit gradient maps of images to guide the recovery in two aspects. On the one hand, we restore high-resolution gradient maps by a gradient branch to provide additional structure priors for the SR process. On the other hand, we propose a gradient loss which imposes a second-order restriction on the super-resolved images. Along with the previous image-space loss functions, the gradient-space objectives help generative networks concentrate more on geometric structures. Moreover, our method is model-agnostic, which can be potentially used for off-the-shelf SR networks. Experimental results show that we achieve the best PI and LPIPS performance and meanwhile comparable PSNR and SSIM compared with state-of-the-art perceptual-driven SR methods. Visual results demonstrate our superiority in restoring structures while generating natural SR images.
04 · contact
email· scholar· github· linkedin
yean_cheng[at]126.com — Haidian, Beijing, China
