Inside DeepLink's four-layer bet on mixed-chip inference

Shanghai AI Laboratory and its partners presented DeepLink's cross-region mixed-training system at WAIC on July 26, 2025. The documentary presentation photograph captures the supply-chain premise: interoperability has to be maintained by institutions, framework teams, and chip vendors together.[7]

At the 2025 World Artificial Intelligence Conference, representatives from Shanghai AI Laboratory, chip companies, and research institutions lined up beneath a blue launch screen. The system they were presenting, DeepLink, followed an earlier Shanghai prototype that had kept a hundred-billion-parameter model training for 20 days across four kinds of accelerator, according to partner Biren Technology. Biren reported 90% of the efficiency of a homogeneous-accelerator cluster for that prototype. At WAIC, the partners presented a further cross-region system intended to connect computing centers separated by more than a thousand kilometers.[7] The group photograph was ceremonial; the engineering problem behind it was not.

On March 9, 2026, Shanghai AI Laboratory announced that DeepLink had moved from mixed training to mixed inference. Its new system was said to coordinate Huawei Ascend, MetaX, T-Head, and Biren hardware, with different chips assigned to different parts of a request.[1] This is a more consequential claim than broad “domestic chip support.” It proposes that unlike accelerators can work inside one serving path rather than merely hosting separate copies of the same model.

The design is easiest to understand as four contracts. DLInfer adapts models and serving frameworks to hardware. DLSlime moves data between devices and services. DLRouter sends requests toward an appropriate backend. DLSolver is supposed to choose a prefill/decode placement from hardware measurements, model configuration, and a service objective.[1] As of July 14, 2026, those layers have sharply different levels of public evidence. That asymmetry—not the headline percentage gain—is the most useful signal in DeepLink's update.

The request is being split before the cluster is pooled

Large-model inference contains two unlike phases. Prefill processes the prompt and builds the initial key-value cache; decode repeatedly reads that state while producing tokens. If both phases share one undifferentiated pool, a long prompt can contend with active decodes and make first-token latency or inter-token pacing less predictable. Prefill/decode disaggregation gives an operator the option to place and scale those phases separately.[1]

DeepLink extends that idea across accelerator families. Its announcement describes a common inference middleware, low-latency communication, intelligent traffic routing, and a policy solver. Rather than asking every chip to behave identically, the solver is intended to match the compute- and memory-heavy parts of serving to the hardware best suited to each role.[1] In theory, that can turn stranded cards in separate clusters into a usable fleet.

But “heterogeneous” can hide several different achievements. A model may run on four chip families independently. Two families may communicate inside one job. A router may send whole requests to separate homogeneous pools. Or prefill on one architecture may hand live cache state to decode on another. Only the last case proves the strongest form of mixed-chip inference. DeepLink's public description points toward that final architecture, but each handoff needs its own inspectable contract.

DLInfer exposes the compatibility contract

DLInfer sits closest to the model. It defines fused inference operators between upper-layer serving frameworks and vendor kernels, with an eager path that dispatches operations directly and a graph path that lowers coarser operations into a vendor compiler. The project currently integrates through LMDeploy and distinguishes tested, unsupported, and untested combinations rather than presenting compatibility as one yes-or-no badge.[3]

That matrix is unusually useful. The July 12 repository snapshot lists specific model families, precisions, execution modes, and hardware: Huawei Atlas A2, A3, and 300I Duo systems; MetaX C500; and Cambricon cloud accelerators. Coverage differs by row. Some Qwen and InternLM configurations span several columns, while graph mode, quantization, multimodal models, and multi-card operation retain explicit gaps. The documentation also exposes an unresolved contradiction: its installation section says A3 supports only Qwen models in eager mode, while the matrix marks A3 support for several Llama and InternLM rows. Non-Qwen A3 coverage is therefore not a dependable public promise yet. Only certain Ascend systems have a pip path, and Cambricon multi-card startup currently requires a manual Ray step.[3]

This is what a real portability layer looks like before marketing smooths the edges: a collection of bounded promises. It also reveals a gap between the system announcement and the public adapter record. The March launch names Ascend, MetaX, T-Head, and Biren as the mixed-inference set; the cited DLInfer snapshot visibly documents Ascend, MetaX, and Cambricon, not that exact four-vendor combination.[1][3] The two claims may come from different branches, internal integrations, or deployment-specific adapters. Until the released support matrix converges with the demonstration, outsiders cannot reproduce the full hardware mix from the public repository alone.

DeepLink's older documentation shows how much standardization work precedes this inference layer. The cited source snapshot describes at least 300 standard operator interfaces, at least 11,000 consistency-test cases, and training support for at least seven chip products.[2] Those are project-published counts, but they establish the method: portability is maintained through interfaces and conformance tests, not obtained merely by renaming a device target.

DLSlime gives the transport a visible shape

Once prefill and decode separate, state has to move. DLSlime is the communication runtime in this chain. Its public design centers on a PeerAgent for data movement, NanoCtrl for discovery and coordination metadata, and endpoint backends for RDMA, TCP, NVLink, and Ascend Direct. Applications can use direct endpoints, add peer discovery, or build services such as a shared cache and RPC layer without forcing all traffic through one transport.[4]

That modularity matters because “one cluster” may contain very different links. A local pair of devices might use NVLink; two hosts may have RDMA; an Ascend deployment can use its direct path; a less capable route may fall back to TCP. The fallback does not magically inherit every RDMA feature: the repository explicitly marks immediate-data operations as RDMA-only. Build flags for NVLink and Ascend Direct are off by default, while MetaX support for torch-backend builds must be enabled separately.[4] The communication contract is therefore portable at the API level while performance remains topology-specific.

Shanghai AI Laboratory reports that DLSlime exceeded 97% bandwidth utilization in “core scenarios.” Its benchmark chart shows two request profiles—8K input/2K output and 30K input/2K output—and plots both time to first token and throughput for each. The accompanying account separately says that, across multimodal-generation and high-concurrency intelligent-service scenarios, time to first token improved by as much as 34.5% against a homogeneous-accelerator solution. For long-input, short-output scientific-paper processing, it reports a 32% throughput increase against a homogeneous-accelerator prefill/decode baseline. The chart frames both comparisons as mixed-chip prefill/decode disaggregation versus homogeneous-chip-type prefill/decode disaggregation on what the laboratory calls a thousand-card-scale inference cluster.[1]

These figures are useful as engineering leads, not general performance estimates. The release does not disclose the model, exact accelerator count and mix, exact concurrency level or request distribution, precision, network topology, runtime versions, raw measurements, repetition count, variance, or tail results; the chart's vertical axes also carry no numeric values. “Maximum” can describe the best cell in a larger test matrix. The percentages show that a laboratory-run system produced a promising result; they do not tell an operator what to expect under another workload or hardware mix.

Routing is public; placement policy is still mostly described

DLRouter supplies an OpenAI-compatible gateway in front of LMDeploy, vLLM, SGLang, or DLEngine backends. Its public strategies include round-robin, weighted-random, consistent-hash, latency-aware, and prefix-cache-aware routing. It can register and health-check nodes and delegate prefill/decode flows to the selected serving backend.[5]

The limitations are as informative as the feature list. One DLRouter process is configured for one backend type at startup; SGLang's disaggregated path uses static discovery; and LMDeploy's prefill/decode features require the corresponding runtime dependencies. At the cited June 11 snapshot, package metadata declared version 0.1.0, but the repository had no Git tag or formal GitHub Release.[5] This is credible early infrastructure, not yet a versioned compatibility promise across every backend and topology.

DLSolver is the decisive fourth layer because it is meant to convert measurements into placement. The launch material says it ingests accelerator benchmarks, model configuration, and user service-level goals, then selects a prefill/decode arrangement.[1] Yet the cited public material describes the solver at product level rather than disclosing its objective function, measurement schema, rebalance cadence, or failure policy. Those details determine whether “best” means lowest first-token latency, highest throughput, lowest cost, most available capacity, or a compromise that changes with the queue.

This evidence split sets DeepLink apart from a simple open-source release. The adapter, communication runtime, and router can be inspected, with varying maturity. The component that decides how the heterogeneous fleet should be used is still the least legible. That may be an intentional boundary between an open compatibility ecosystem and an operated platform. It also means the complete scheduling claim cannot yet be audited from code alone.

The supply-chain value is optionality, not sameness

China's accelerator ecosystem does not need every chip to expose identical performance. It needs enough common contracts that model and serving teams do not rebuild the whole path for each vendor. DeepLink's strongest idea is to preserve hardware differences and make them schedulable. A fleet could absorb available cards, place phases according to their measured behavior, and change the mix without replacing the northbound application API.

The cost is a wider failure surface. Model correctness can drift by backend. Operator coverage can break at a new shape or precision. Cache transfer can erase the gain from phase separation. Router health can lag real device health. Compiler, driver, framework, model, and container versions can move on independent clocks. A policy optimized for steady traffic can make the wrong choice during bursts. Each abstraction reduces one integration burden while creating another contract that must be versioned, observed, and tested.

There is at least an institutional route for that work. In October 2024, Shanghai AI Laboratory and the China Academy of Information and Communications Technology launched joint AISHPerf–DeepLink testing around chip applications, large-model clusters, and heterogeneous training pools. The laboratory said DeepLink evaluations had already covered more than ten domestic chip vendors through eight rounds and over seventy reports, with quarterly hardware reports planned.[6] The useful next step is to apply that discipline to the mixed-inference path end to end, not only to individual devices.

The ambition is widening. Twenty days after the inference announcement, Xinhua reported that Shanghai AI Laboratory had launched a DeepLink platform intended to coordinate conventional, supercomputing, and AI-computing resources for scientific research, with links to major computing centers around China.[8] That institutional expansion makes a precise public contract more important. The more workloads and centers share the name, the less a single launch benchmark can establish.

Four disclosures would turn the March result into durable infrastructure evidence:

A matched end-to-end benchmark naming the model, request distribution, precisions, accelerator counts, network, runtime versions, routing policy, baseline, and tail latencies—not only peak percentage gains.
One versioned support matrix connecting the four announced chip families to released adapters, transport paths, serving backends, tested models, and known failures.
A solver contract that states the optimization target, input measurements, constraints, fallback behavior, and the conditions that trigger re-placement.
Fault-injection results showing what happens to queued requests and transferred cache state when a device pool, link, router, or control-plane service disappears.

The falsifier is practical. If the advantage vanishes after cache movement, tail latency, failures, and the full software matrix are counted, DeepLink will remain an impressive heterogeneous-compute demonstration. If those four disclosures become repeatable release artifacts, it will represent something rarer: a serving system in which China's fragmented accelerator supply becomes an operating choice rather than an application rewrite.

cronfeed.work