Meituan officially released LongCat-2.5which is a native multi-modal model for long-range agent tasks. The total parameters of the model are 1.6 trillioneach token activates approximately 48 billionParameters, based on LongCat-2.0, it further strengthens the native visual understanding and agent programming capabilities.
The new model has significantly improved performance in long-range agent tasks. This is due to the fact that we have added more long-range trajectory data to long-sequence training, and through large-scale long-range reinforcement learning, we have improved the model’s end-to-end engineering capabilities in tasks such as complex code base development and complete project construction. We continue to expand this training link around data synthesis, environment construction and cross-Harness hybrid training. In addition, we also transform the failure trajectories of real Agents into reproducible and verifiable training tasks, and continue to supplement training data to address shortcomings in model capabilities.
At the same time, LongCat-2.5 has native visual understanding capabilities, which can represent visual signals as multi-level discrete tokens, allowing the backbone model to process visual and text input in the form of unified tokens, perform well in Visual Coding, and support tasks such as creative design and 3D.
Behind these upgrades is the continuously evolving infrastructure: we have optimized long-context training and W4A4 inference on domestic computing power, and supported the efficient operation of large-scale long-range RL through training and push value consistency and long-context rollout optimization.
indicator board
In order to visually present the current capability level of LongCat-2.5, we select GPT-5.5, GPT-5.6-Sol, Claude-Opus-4.8 and Claude-Opus-5 for comparison in the figure below. Detailed evaluation results covering more models are shown in the table below.
LongCat-2.5 represents visual signals as multi-level discrete tokens, allowing the backbone model to process visual and text input in the form of unified tokens. This design combines native resolution visual discretization, offline visual preprocessing, and visual embedding jointly learned with the backbone model, which not only supports native multi-modal fusion, but also simplifies visual computing and load management during training.
Discrete native resolution visual transformer: To enable visual primitive modeling to have token-level abstraction similar to language modeling, we represent visual signals as multi-level vocabulary tokens. Follow LongCat-Next[1]This method applies Residual Vector Quantization (RVQ) to the native resolution features generated by the Semantic-and-Aligned Encoder (SAE) to solve the information bottleneck of visual discretization.
Offline visual token preprocessing: To reduce online vision computation and workload heterogeneity, the fixed tokenizer allows us to precompute and cache vision token IDs offline. The backbone model therefore directly processes pre-segmented sequences, simplifying the packaging and segmentation of long sequences, and improving load balancing for distributed training.
native visual representation: Joint learning of visual codebook embedding and backbone model. On a smaller-scale MoE model with an average activation parameter of about 3B, preliminary verification shows that visual and text embeddings are intertwined in t-SNE visualization, indicating the integration of the two in a shared semantic space.
RL scaling is a key step to improve model intelligence. We have built a comprehensive Agentic RL training link for LongCat-2.5 to conduct large-scale expansion of long-range tasks from the aspects of data, environment, algorithm and Harness.
Large-scale data synthesis and environment construction: We build a scalable data synthesis pipeline to automatically generate training tasks, interactive environments and verifiable test cases. The data sources include code tasks from GitHub public warehouses and internal real-life scenarios. For the filtered PRs and associated issues, the Agent is used to automatically mark and synthesize the environment, and perform fail-to-pass (F2P) and pass-to-pass (P2P) test verification. In RL training, real-time rollout trajectories and pass rates assist us in secondary data marking and screening, and we continue to revise and expand F2P test cases based on false negative and false positive analysis to improve the coverage of task function descriptions and verify the accuracy of signals.
Asynchronous reinforcement learning and stability optimization: We use the partial rollout asynchronous training strategy to alleviate the long-tail efficiency problem: each RL step maintains a certain oversampling rate, interrupts after gathering the batch, saves the unfinished trajectory, and resumes in the next sampling stage. The oversampling rate is adjusted based on the task pass rate, and zero-gradient sample groups that are all right or all wrong are filtered to ensure effective gradients. In partial rollout, tokens of the same trajectory may come from different historical checkpoints, causing sample staleness. To ensure training stability, we trim the difference between logprob during sampling and current training, and adjust the gradient contribution of each token based on the historical cumulative difference to limit the gradient bias caused by staleness; in addition, we provide an optimization solution for training and inference consistency from an infra perspective.
Reward hacking and output behavior optimization: We limit the speculative behavior of the model by interrupting the network in the task environment, clearing the git history and file cache, and introducing LLM judge supervision tracks. In terms of output behavior optimization, we impose additional penalties on repeated thinking and repeated tool calls; problems with overthinking and excessive length growth are suppressed by redistributing Reward in positive samples.
Mixed training and generalization: The model can autonomously learn and adapt approximation tasks and Harness in RL. The pre-cold start phase only needs to cover a small number of adapted scaffold use cases. In the RL phase, hybrid training across multiple tasks and across scaffolds can significantly improve model generalization capabilities and promote performance improvements on different tasks.
In order to integrate capabilities in various fields into the mainline model, we continue to use the expert group mechanism of LongCat-2.0 to optimize Agent capabilities, reasoning capabilities and interactive experience respectively, covering multi-round task execution, single-round reasoning, fine-grained instruction compliance, and security defense without sacrificing usefulness. In MOPD, the student model samples online in tasks in various fields, and domain experts use the full vocabulary KL divergence on the sampling results.[2]Train students.
The sample should not only reflect students’ actual task behavior, but also allow teachers to give credible learning signals. To limit the distribution shift, previous work split the trajectories seen by the teacher in the SFT or RL stage into per-turn prefixes as context for student sampling. We further split each round of teacher responses into per-cut prefixes of fixed length, and limited the maximum length of student self-sampling to this value to handle distribution shifts in long responses in a single round.
In order to control computing and storage costs, the prefixes of the same teacher trajectory are grouped into a group and sent to the same inference engine, sharing prefill calculation and KV cache. During training, a tree is constructed using teacher trajectories as the trunk and student samples as branches, combined with Tree Attention[3]complete the OPD training of all samples in the same group through a single forward pass and back pass, avoiding redundant calculation and storage on the teacher’s trajectory.
LongCat-2.5 uses long sequence training in both the medium training and supervised fine-tuning phases. In view of the limitation that domestic computing power has less than 80 GB of video memory on a single card, the training system focuses on optimizing the video memory overhead and designing parallel strategies around the super-node architecture.
LongCat-2.5 configures the parallel group according to the supernode topology. The FSDP and EP parallelism are consistent with the number of cards of a single supernode, and the CP parallelism does not exceed the number of cards, so that the parameters AllGather, expert token AllToAll and KV AllGather are all completed in the high-bandwidth interconnection domain within the supernode.
The hybrid word list parallel slices the basic word embedding layer, LM head and N-gram embedding layer in the joint parallel domain of DP and CP to further reduce the memory usage.
To address the load imbalance problem in large-scale expert parallelism, LongCat-2.5 introduces a threshold-triggered dynamic expert replication mechanism based on MoonEP. Replication is enabled only when the EP load imbalance reaches the preset threshold to avoid additional communication overhead exceeding the load balancing benefits, and to prevent routing tokens from causing the receiver’s memory to overflow, and communication operations to cause the sender’s memory to overflow.
Generating image tokens in advance eliminates the need for visual encoders in the training calculation graph, simplifying parallel implementation. Using the known number of image tokens, the system can estimate the load proportion of each mode when data is packed and broken up, so that the proportion of modes on each data parallel rank is close to each other.
LongCat-2.5 introduces dual sharding FSDP. Model parameters are sharded in the FSDP domain within the supernode. FP32 master weight and optimizer status are further sharded across FSDP domains through HSDP copies, which reduces resident video memory while maintaining parameter communication efficiency.
Parameters, gradients, and optimizer status are unified into continuous flattened buffers, supporting non-uniform sharding and Muon optimizer sharding. At the same time, by directly writing weight gradients and reusing temporary buffers, intermediate copies are reduced and peak video memory usage is reduced.
The system uses zero-copy AllGather/ReduceScatter to eliminate intermediate copies introduced by the default communication algorithm. EP packages and transmits tokens and routing probabilities, and adopts AIV-based communication implementation to alleviate the AICPU task deployment bottleneck. Parameter prefetching enables AllGather to overlap execution with calculations while controlling resource competition with CP; ScMoE enables EP communication to overlap execution with dense branch calculations.
The system reduces host-side overhead by executing gradient operations in batches, merging index reduction, CPU binding, and NUMA-aware scheduling.
Based on the above optimization, the single-card throughput of this FSDP configuration in long sequence SFT tasks is improved compared with the DDP baseline. 86%。
Based on IsoExec[4]、GLM-5[5] and XoRL[6]As part of our work, we have built a production-level training and push consistency solution so that training, prefill, and decode can independently select parallel configurations. While the performance is close to lossless, the bitwise consistency of the token logprob is maintained. Both ends of training and push adopt efficient batch-invariant operators and align the framework calculation logic to complete backward support. This solution has been enabled by default in experimental and production tasks, providing a unified numerical regression benchmark for operator and framework iteration. In order to solve the problem of inconsistent training and push due to different parallel methods, we have further expanded the training and push consistency support of TP and DCP based on the community’s solution.
TP: Use complete K-dimensional calculation instead of piecewise summation, and the performance is close to lossless. The training side retains weight sharding, and independently completes the complete K-dimensional GEMM through weight All-Gather and input AlltoAll, without the need for Reduce-Scatter summation of local results. Prefill follows this scheme; Decode changes o_proj/fc2 to column-parallel and completes the calculation by inputting All-Gather, complete K-dimensional GEMM and outputting AlltoAll, which avoids aggregation of complete weights at each step and avoids cross-card numerical summation.
DCP: Reproduce the cumulative path of Decode in training and Prefill. We access the matching sparse attention operator in training forward and inference prefill to reproduce the KV block and reduction sequence of Decode under the specified DCP configuration, aligning the calculation accuracy of the three.
For load-controllable long-context multi-round rollout, we use PD separation to select appropriate parallel strategies for Prefill and Decode respectively, and collaboratively optimize from three aspects: cache storage, cross-layer offloading and request scheduling:
- Expand effective cache capacity: Prefill uses QCP to allocate and calculate by token, and KVP to disperse storage by page to avoid KV redundancy caused by TP segmentation under MLA. After KV fetches the video memory, HiCache releases main memory copies that are not referenced by other reads, reducing the duplication of two-level caches. At the same time, based on Mooncake’s reconstructed weight management and Actor offload scheme, more memory is released for HiCache.
- Hide handling overhead through cross-layer prefetching: Utilize LSA[7] The odd-numbered layer reuses the characteristics of the top-k index of the previous even-numbered layer, offloads the complete KV of the odd-numbered layer to the main memory, and only retains a small-capacity working cache in the video memory. After the index is generated for the even-numbered layer, the KV required for the next odd-numbered layer is loaded in advance and overlaps with the subsequent calculation of the current layer, nearly doubling the Decode KV cache capacity without loss of TPOT.
- Avoid recalculation avalanches with capacity-aware scheduling: Router controls the release of requests based on the L1+L2 cache capacity and the total amount of request tokens, and prioritizes Groups with higher completion levels to gather training batches and release caches as early as possible to avoid repeatedly eliminating each other’s prefixes when multiple long trajectories are advanced alternately, causing large-scale recalculation.
For the 1.6T MoE model, we combined the model structure and the hardware characteristics of domestic AI accelerators to carry out collaborative optimization around quantification accuracy, underlying operators and inference pipelines to build a W4A4 efficient inference solution.
This solution quantizes expert weights and activations to 4 bits at the same time, and performs corresponding matrix calculations through the native INT4 matrix multiplication unit.
In terms of accuracy, based on FlatQuant’s learnable equivalent transformation, the quantization accuracy of the MoE model is further optimized, maintaining performance comparable to the pre-quantization model on various downstream tasks that have been evaluated.
In terms of operators and data paths, through input transformation shared by experts, quantification is completed before token copying and distribution, so that the same INT4 representation runs through expert communication and calculations. Integrate transformation, scale calculation, quantization and packaging into dedicated operators, and combine weight layout and transposed access optimization to reduce data handling and conversion overhead.
At the system level, based on the execution characteristics of prefill and decode, and based on the data dependencies and hardware resource constraints of the model, the pipeline overlap of calculation and communication is optimized.
In the online stress test, compared to the W8A8 INT8 baseline, prefill and decode throughput increased by 43.3% and 42.77% respectively, while TTFT and TPOT decreased by 15.91% and 19.72% respectively. At the operator level, GMM achieves an average acceleration of 1.73× relative to W8A8.
Demonstration of capabilities
For details, please refer to the official website technology blog:
