美团智播——数字人直播技术创新与实践

introduction

With the rapid development of large language models, generative AI and multi-modal interaction technology, digital human live broadcast has broken through the early bottleneck of rigid image and single interaction, showing huge business potential in local life, e-commerce, entertainment and other fields. Meituan Zhibo is an AI digital human live broadcast solution for local life scenes. It integrates large models, digital humans and multi-modal interaction technology to provide a rich set of industry-specific digital human images. It supports 1:1 replicas of real people and customization of real-life stores. It can generate live broadcast materials in 30 seconds, complete broadcast configuration in 3 minutes, and be broadcast stably 24 hours a day, 7 days a week. In the past year, relying on generative AI technology, the monthly and daily average GTV of digital human live broadcasts increased by 82.12% year-on-year, the average monthly and daily viewers increased by 163.44% year-on-year, and the number of broadcasts increased by 11 times, fully verifying the actual value of AI digital human live broadcasts in the closed loop of local life and business.

From July 8 to 10, 2026, Meituan Zhibo made its industry debut at the China Internet Conference, debuting in the technology interactive exhibition area with the theme of “real-time interactive digital people + live broadcast digital people with goods”. In the exhibition area, real-time interactive digital people naturally communicated with the audience through second-second voice questions and answers, and the digital people live broadcast room made participants sigh that “they can’t be seen as digital people at all” – this appearance marks that Meituan Zhibo has officially moved from internal polish to the industry stage.

How to make digital human live broadcasts achieve real-life performance and meet the requirements for large-scale commercialization? This article will focus on the core capabilities of Meituan Zhibo and systematically introduce the key technological breakthroughs that support the product – covering high-fidelity image generation, natural motion drive, voice and gesture coordination, efficient reasoning deployment and system architecture design. These technologies together constitute a complete technical closed loop for digital human live broadcasting from “usable” to “easy to use” and then to “large-scale mass production”. With these technological accumulations, Meituan Zhibo won the 2026 “China Multimedia Enterprise Innovation Technology Award”. This award is guided by the China Computer Federation (CCF) and the China Society of Image and Graphics (CSIG), and is established and awarded by the China Multimedia Conference (ChinaMM). It is industry recognition of Meituan’s independent innovation capabilities in the field of digital human live broadcasting.

1. Background and challenges of digital human live broadcast

1.1 Business background: The “cost-efficiency-experience” dilemma faced by millions of merchants

Meituan connects millions of merchants with hundreds of millions of consumers, and digital live broadcasts are accelerating their penetration into merchants’ daily operations. However, merchant live broadcasts face four practical difficulties:

1. High barriers to entry: Professional digital human image customization often costs tens of thousands of yuan, and the labor cost of a 7×24-hour live anchor team is even more unbearable for small and medium-sized businesses;
2. Strict timeliness requirements: Local life scenarios such as limited-time flash sales, sudden hot spots, sudden weather changes, etc. require “minute-level” response to live content. The traditional schedule and content production cycle of live anchors cannot meet the real-time marketing rhythm.
3. Homogenization of image and content: A large number of businesses rely on generic digital human templates, and the image of anchors is the same. They lack a sense of exclusivity that matches the brand tone, and it is difficult to establish differentiated recognition.
4. Large-scale deployment costs are high: Tens of thousands of merchants start broadcasting at the same time. If each channel monopolizes a large number of GPUs, the inference cost will expand linearly, and it is impossible to realize the inclusive implementation of digital human live broadcast.

The deeper problem is that most existing digital human solutions are generally “fake at first glance” – stiff faces, repetitive movements, disconnected gestures and voices, and the user’s stay time is significantly lower than that of real anchors. How to make digital live broadcasts comparable to real live anchors is the core direction of technological breakthroughs.

1.2 Technical Challenges: Multidimensional Problems of High-Quality Digital Human Live Broadcasting

Digital human live broadcast provides new ideas for the above problems, but to truly achieve large-scale commercial use, it is far from a simple video generation task and faces four intertwined technical challenges.

Challenge 1: High-fidelity image, from “fake at first glance” to “comparable to real people”

Any facial stiffness or imbalance of facial features will be seen by the audience at a glance. The higher requirements are: merchants need to reproduce the real person 1:1, and maintain identity consistency when changing clothes and backgrounds. How to ensure high fidelity and strong consistency in the entire generation and editing chain is the core problem of image generation technology.

Challenge 2: Natural and diverse movements, from “puppet show” to “real person feeling”

7×24-hour live broadcast requires extremely high diversity of actions. Loop playback of a limited action library leads to audience fatigue and reduced retention rates; dynamic generation solutions also face problems such as unstable quality and joint distortion. How to achieve high-quality and high-diversity action generation in a single model is a major challenge for long-term live broadcasts.

Challenge 3: Coordination of pronunciation and semantics, from “reading machine” to “consistent sound and picture”

The gestures of excellent anchors are highly coordinated with semantics. Existing technology can generate spontaneous movements (nodding, waving) that are synchronized with the rhythm, but cannot generate coordinated gestures that are accurately bound to semantics. How to bridge the “semantic gap” between voice and action is a key bottleneck in upgrading interactive experience.

Challenge 4: Scaled inference efficiency, from “expensive for one channel” to “concurrency of 10,000 channels”

The digital human live broadcast system is a multi-agent collaboration platform. How to efficiently collaborate with multiple sub-agents and significantly reduce single-channel reasoning overhead while ensuring quality and real-time performance is the ultimate threshold for digital human live broadcast to become inclusive.

1.3 Overview of technical solutions

Faced with the above challenges, Meituan Zhibo has developed key technologies by itself, focusing on the “digital human anchor”Looks real → moves accurately → acts vividly → sells well“Form a complete technology closed loop and achieve a full-link leap.

• Looks real: High-fidelity digital human image generation and editing technology to collaboratively overcome high-fidelity and diversified image generation problems;
• Move accurately: Text-driven high-quality motion generation technology generates highly controllable, natural and smooth body movements, and generates highly expressive digital human videos through motion transfer;
• Acting well: Speech-gesture coordination generation technology bridges the semantic gap between speech and action, allowing gestures and explanations to form semantic-level coordination;
• Sell ​​well: Token compression and inference acceleration technology for large-scale deployment, enabling efficient collaboration of multiple agents and breaking through thousands of concurrency bottlenecks.

2. Make AI anchors “real-looking”—high-fidelity digital human image generation and editing

2.1 Product pain points and needs

Merchants have three progressive demands for digital human images: quick availability (customized broadcast in 3 minutes), highly personalized (1:1 reproduction of real people), and flexible and editable (keeping the identity consistent when changing clothes and backgrounds). Customized image generation methods in the past faced core issues such as identity and posture coupling, time-consuming fine-tuning, and editing that destroys identity features.

2.2 Structural decoupling of identity personalization and self-reward precise editing

In response to the dual challenges of identity fidelity and flexible editing in digital human generation, the Meituan Zhibo R&D team proposed structural decoupling identity personalization (SDIP) and self-reward precise editing (SREdit) technical solutions.

2.2.1 Structural decoupling of identity personalization

The core idea of ​​SDIP (Structure-Decoupled Identity Personalization) is to structurally decouple the three dimensions of posture, identity, and background, and includes two core modules:

• Accurate separation module of identity and posture: In the Diffusion-Based backbone network, identity features are integrated in the form of residuals through structural residual injection to avoid pose space pollution; spatial decoupling regularization loss, focusing on the face, explicitly constrains the spatial distribution of identity features; regional dynamic masking isolates the flow of features in different semantic areas.
• Detail Fidelity Enhancement Module: In order to solve the problem of insufficient detail fidelity, the high-frequency details of the reference image are injected into the generation through the visual reference enhancer to improve the texture restoration; combined with the mask perceptual quality correction strategy, the occlusion and boundary areas are refined and corrected.

2.2.2 Self-reward precise editing

In daily operations, businesses often need to make partial edits to digital humans—changing seasonal clothing, matching event backgrounds, and adjusting makeup. The core constraint is: the identity characteristics before and after editing must be consistent, and “you cannot recognize the person after changing his clothes.”

In response to this demand, the team proposed SREdit (Self-Rewarding Editing), which was included in ACM MM 2026. Its technological innovation is reflected in three levels:

1. SIA command atomization: In the MLLM-DiT architecture, complex editing instructions are decomposed into atomic-level operating units. Each atom corresponds to a clear spatial area and editing semantics to avoid accidental modification of identity information due to semantic ambiguity.
2. Space reward weighting: Dynamically adjust the reward weight according to the spatial distribution of the editing area – the editing area is given a higher editing quality weight, and the non-editing area is given a higher fidelity weight, guiding the model to find the optimal balance between “bold editing” and “cautious maintenance”.
3. cognitive closure architecture: The same model serves as both editor and evaluator, eliminating reward hacking (the editor learns to deceive the evaluator) caused by the separation of the editor and evaluator in the traditional solution. When both share the same set of visual understanding capabilities, optimization goals and actual quality goals are intrinsically consistent.

Experimental results show that SREdit’s overall performance improves by 2.5% to 3.8% on the public data sets GEdit-Bench, ImgEdit-Bench, and KRIS-Bench. In terms of visual effects, it achieves industry-leading editing accuracy while maintaining identity consistency.

Examples of digital human image generation and editing effects:

3. Let AI anchors “move accurately” – text-driven high-quality actions

3.1 Product pain points and needs

When digital humans introduce products via 7×24 live broadcast, different products require different styles of action coordination, which poses a double challenge to the action generation system: ensuring the quality of a single action (smooth, natural, and physically reasonable) while ensuring the diversity and controllability of long sequences.

3.2 MoTiGA: Multi-level causal LLM action generation

The MoTiGA (Motion generation via multi-level causal Transformers with LLM-Guided Alignment) method proposed by the Meituan Zhibo team was included in CVPR 2026. MoTiGA introduces the autoregressive generation paradigm of large language models into human action generation, achieving both quality and efficiency improvements through multi-level causal structure and preference optimization.

• Causal RVQ-VAE: Traditional RVQ-VAE uses bidirectional convolution. The model can “see the future” during training, but can only be generated autoregressively during inference, resulting in inconsistency between training and inference. MoTiGA replaces all convolutions with causal convolutions, fundamentally eliminating behavioral differences. The reconstructed FID dropped from 0.114 to 0.031, and the quality improved by more than 72%.
• Time-lagged Causal Prediction: Standard autoregression generates token-by-token serialization, which is inefficient. MoTiGA observed that there is a time-lag causal dependence between different quantization layers of RVQ – the coarse-grained layer can directly guide the prediction of the fine-grained layer. Based on this observation, a multi-level causal prediction architecture is designed. Different quantization layer tokens can be generated in parallel, and the reasoning efficiency is improved by K times while maintaining the correctness of causal dependencies.
• MHPO（Motion Human Preference Optimization）: The industry’s first work to introduce human preference optimization into action generation. The team constructed 101,490 sets of preference pairs on HumanML3D and used a DPO-like strategy to directly optimize the model on the preference data, making the generated results more consistent with human subjective judgments of “natural actions”.

Experimental results show that MoTiGA has an FID of 0.041 on the HumanML3D data set (82.3% lower than the baseline) and 0.180 on the KIT-ML data set (64.7% lower than the baseline). The generated action sequence is highly consistent with the real human action distribution, achieving a naturalness and fluency close to that of a real person.

4. Let AI anchors “act lively” – coordinated generation of voice and gestures

4.1 Product pain points and needs

When the live anchor says, “This pizza has a lot of ingredients,” he will open his hands and gesture with large amounts, and when he says, “Please see the link on the right,” his fingers will point to the right. This coordination of voice and gestures is an important clue for the audience to judge whether the digital person looks like a real person. The existing technology mainly focuses on “spontaneous gestures” (nodding, waving), lacks deep binding with semantic content, and cannot accurately generate coordinated gestures such as “pointing to the product” and “gesturing”. In terms of model design, the traditional solution uses a global denoising architecture, which requires waiting for the entire speech to end and generates all actions at once, which cannot meet the real-time and unlimited duration requirements of live broadcast scenarios.

4.2 StreamingTalk: Streaming common language action generation

The core innovation of StreamingTalk is to reconstruct the global denoising process into a streaming causal generation architecture, which realizes the streaming generation of non-spontaneous coordinated actions for the first time, and supports the real-time, continuous, and controllable generation of synchronous gesture actions when digital people speak. The paper has been submitted to AAAI 2027. Its core technical innovations include:

• Streaming causal generation architecture: Full sequence denoising is converted into chunk-by-chunk progressive output through lower triangular time step scheduling. Each chunk is submitted independently, and the inference delay has nothing to do with the total duration; bidirectional attention is used to refine action details within the active window, and previously submitted frames are used as anchor points to constrain continuity. The calculation amount is constant and does not increase with the sequence, achieving real-time performance of “talking and moving” in live broadcast scenarios.
• Long sequence stability guarantee: The Self-Forced Clean Anchoring strategy is used in the training phase to replace the real anchor points with the model’s own predictions to simulate the error accumulation during inference and bridge the distribution difference between training and inference; at the same time, tail bias sampling is introduced to increase the training exposure of the end frame of the window and compensate for the insufficient tail coverage of the sliding window. The two work together to ensure that the quality of gestures generated over a long period of time does not decay or drift.
• time-varying action control mechanism: Each chunk can be bound with independent action tags to dynamically switch performance styles at the boundary; voice features ensure beat synchronization through cross-attention injection, and action tags serve as semantic conditions to control gesture types. The two-level condition synergy allows gestures to follow the rhythm of the voice and dynamically switch according to the content – displaying both hands when introducing products, waving excitedly when talking about discounts, holding one’s chin up to think when answering questions, and the whole process transitions smoothly in a continuous flow.

StreamingTalk has verified for the first time that streaming Flow Matching can generate infinitely long high-quality gesture streams under constant delay and constant memory, providing core technical support for digital humans to upgrade from “active but rigid” to “real-time performance, long-lasting performance, and flexible performance” with high expressiveness.

5. Let AI anchors “sell well” – efficient reasoning supports thousands of concurrent connections

5.1 Product pain points and needs

Meituan Zhibo’s live broadcast scenario has significant characteristics of scale: tens of thousands of merchants need to start broadcasting at the same time, and each live broadcast requires real-time visual understanding, intelligent interaction and screen rendering. According to the traditional multi-modal large model inference method, each live broadcast needs to process a large number of visual tokens, and a single GPU consumes a lot of money. When the concurrency reaches the level of 10,000 channels, the cost of large model inference will become the fundamental bottleneck for product universalization. Therefore, how to significantly reduce visual reasoning overhead while ensuring generation quality is the key to making the product universally accessible.

5.2 Glance2Gaze: Glance Fusion and Gaze Compression

The Glance2Gaze method proposed by the Meituan Zhibo team was included in NeurIPS 2025. Through an innovative visual token compression strategy, it achieved a token compression rate of 75% and an inference acceleration of 2.5 times without sacrificing model performance. Its naming is inspired by the “saccade-gaze” mechanism of the human visual system: the human eye first obtains global information of the scene through rapid glances (glance), and then focuses on key areas to obtain detailed information through gaze (gaze).

• Glance Fusion: Traditional visual encoders (such as ViT) usually only use the output of the last layer as visual representation, discarding the rich information of the middle layers. Glance Fusion uses a multi-layer feature fusion mechanism to aggregate features from different depth layers of ViT to obtain richer visual representations without increasing the number of output tokens. This is equivalent to obtaining more comprehensive scene information during the “scan” stage, providing a better information redundancy basis for subsequent compression.
• Gaze Compression: Different from the traditional independent compression scheme after the visual encoder output, Gaze Compression embeds the compression module directly into the decoder layer of LLM, achieving progressive compression of visual tokens – gradually compressing from 576 tokens to 288 tokens, and then further compressing them to 144 tokens.

This “embedded progressive compression” design has three key advantages: the compression process can take advantage of LLM’s semantic understanding capabilities to more intelligently determine which visual information is redundant and which is critical; progressive compression avoids the cliff-like loss of information caused by one-time large compression; it is fully compatible with FlashAttention-2 and can be directly deployed to existing inference infrastructure without additional engineering adaptation.

Experimental results show: Glance2Gaze achieves 75% visual token compression (576 → 144). The performance loss of the model in various visual understanding tasks is controlled within 2%, the inference speed is increased by 2.5 times, and the concurrency cost is reduced by more than 60%.

6. System architecture: dual-engine driven smart broadcast technology center

Breakthroughs in a single technology are only the foundation. How to integrate multiple technologies into an efficient and collaborative system is the key to product implementation. Meituan Zhibo has built a “dual-engine-driven” technical mid-end architecture at the system level to achieve the dual capabilities of real-time interaction and content mass production.

6.1 Real-time interaction engine

The real-time interaction engine is the core component of Meituan Zhibo to cope with the “real-time user interaction” scenario. When viewers ask questions or trigger interactions in the live broadcast room, the system needs to complete the full-link response of “semantic understanding → answer generation → action synthesis → picture rendering” in a very short time. Meituan Zhibo’s real-time interaction engine realizes full-duplex communication, and the system parallelly schedules core capabilities such as images, text generation, and digital human driving. Through streaming connections, part of the output of the previous stage triggers the next stage of calculation, compressing the overall delay to the maximum extent.

6.2 Content mass production engine

For “planned content” such as product explanations and event introductions in daily live broadcasts, Meituan Zhibo has built a content mass production engine for multi-agent collaboration. The engine adopts a five-stage collaborative architecture of “creativity-retrieval-thinking-generation-evaluation”:

• creative agent: Based on product characteristics and live broadcast style requirements, generate creative explanation plans;
• Retrieve agent: Retrieve relevant product information, selling points and successful cases from the knowledge base;
• Thinking Agent: Integrate creative solutions and search results, and plan complete explanation logic and action scripts;
• Generate agents: Scheduling image generation, video generation, digital human driver and other technical modules to generate complete video content;
• Evaluation agent: Evaluate the quality of the generated content. If it is not up to standard, it will be fed back to the upstream for regeneration.

This multi-agent collaborative architecture enables content production to achieve an “industrial assembly line” model. The generation time of a single product explanation video is shortened from hours of manual production to minutes, and the quality is stable and controllable.

6.3 Knowledge base support system, “image-action-scene” decoupling and reuse architecture

Supporting the efficient operation of the dual engines are the three major knowledge bases carefully constructed by Meituan Zhibo:

• Outfit aesthetic library: Precipitate industry matching rules and color aesthetic knowledge to guide the clothing selection and style matching of digital human images;
• Live broadcast expert knowledge base: Gathering the explanation skills and interactive experience of outstanding live anchors to provide professional reference for content generation by AI anchors;
• AI operation brain: Build an intelligent decision-making system based on massive live broadcast operation data, analyze live broadcast room indicators in real time, and dynamically adjust strategies such as product switching, discount triggering, and rhythm control to form a data-driven continuous optimization closed loop.

Meituan Zhibo follows the core principle of three-dimensional decoupling of “image-action-scene” in system design. The three dimensions of digital human image, body movements, and live broadcast scenes are completely independently managed and can be freely combined and reused. This decoupled architecture allows the same image to be matched with different action styles, the same set of actions can be applied to different images, and the same scene can be quickly switched between different anchors. Combined reuse greatly reduces the marginal cost of content production, making personalized live broadcasts possible for “thousands of people and thousands of faces”.

7. Summary and future prospects

Meituan Zhibo has achieved large-scale implementation in multiple business lines, and its core indicators are outstanding: it only takes 3 hours from demand to online for customized digital models, which is more than 10 times more efficient than traditional customized shooting; merchants’ free time transaction volume increased by 7.3% on average, and 7×24 hours of non-stop broadcasting effectively fills the traffic gap during the rest periods of real anchors; the overall broadcasting efficiency increased by 60%, and AI automatically generates words, decorates live broadcast rooms, and triggers marketing actions, freeing merchants from tedious live broadcast preparations.

Looking forward to the future, Meituan Zhibo will continue to evolve in the following directions:

• Emotional expression and personalized interaction: In the future, digital human anchors will have emotional computing capabilities and give appropriate emotional responses to different feedback from users.
• Adaptive style transfer: Based on the differentiated needs of the industry, brand, and time period, the digital anchor will automatically adjust the expression style.
• Highly expressive real-time interaction: Continuously optimize the generation and rendering pipeline, moving towards a highly expressive real-time interactive product form.

References：

• [1] Chen X, Bao Q, Liu X, et al. Multi-level Causal LLM-based Text-to-Motion Generation with Human Alignment. CVPR 2026.
• [2] Chen J, Liu H, Ao Y, et al. Glance2Gaze: Efficient Vision-Language Models from Glance Fusion to Gaze Compression. NeurIPS 2025.
• [3] Wang Y, Bao Q, Ao Y, et al. SREdit: A Self-Rewarding Framework with Intrinsically Aligned Critic for Image Editing. ACM MM 2026.
• [4] Zhang Z, Liu K, Chen Z, et al. InfVSR: Toward Consistency-Driven Streaming Generative Video Super-Resolution. ICML 2026.
• [5] Fang F, Yang S, Yang W. CoordSpeaker: Exploiting Gesture Captioning for Coordinated Caption-Empowered Co-Speech Gesture Generation. CVPR 2026.
• [6] Chen X, Liu W, Bao Q, et al. Motion Capture from Inertial and Vision Sensors [J]. IEEE Transactions on Multimedia, 2026.
• [7] Liu H, Bao Q, Chen X, et al. Stable Layout Image Diffusion for Content-Aware Layout Generation. ICASSP 2026.

Source link