The most consequential new file in Huawei’s large-model training stack may be an unglamorous one: train_fsdp2.py.

MindSpeed-LLM is Ascend’s distributed training suite, sitting between model weights and the lower-level NPU software stack. Its familiar route is Mcore, the project’s name for its recommended Megatron Core backend. On February 10, 2026, however, the repository announced a prototype FSDP2 backend for Qwen3-Next. The current tree now includes an FSDP2 launcher, configuration directories, a detailed Qwen3 tutorial, and entries for much larger models.[1][2]

As of 2026-08-02T16:38:15Z UTC, this is not evidence that Ascend training has become hardware-neutral. It is evidence of something narrower and strategically useful: part of the porting boundary is moving toward an upstream PyTorch interface. MindSpeed-LLM can now offer two distinct PyTorch training lanes—Megatron’s explicit model-parallel machinery and FSDP2’s per-parameter sharding contract—without pretending that they have reached the same maturity.

The project’s own support table makes that last distinction unusually clear. Mcore rows repeatedly carry Pass certification. The newer FSDP2 rows still carry Test. The second lane is real enough to run, but not yet broad enough to treat as an interchangeable road.

A second backend changes the handoff

FSDP2 is not simply a different command for launching the same distributed job. PyTorch’s fully_shard API converts parameters into DTensor objects, gathers the full parameters around forward and backward computation, then returns them to their sharded form. Its design uses per-parameter sharding and eager-mode hooks rather than asking an application to own every distributed transformation itself.[5]

That contract matters on Ascend because it gives framework users a recognizable control surface. MindSpeed-LLM’s Qwen3-8B tutorial starts from Hugging Face or ModelScope weights, launches with torchrun, and expresses sharded modules, expert parallelism, recomputation, optimizer settings, and data choices in a YAML file. The example sets fsdp_size equal to the eight-NPU world size and names transformer layers, embeddings, and the language-model head as sharding targets.[3]

The hardware has not disappeared. The tutorial limits the supported path to Ascend 950 products and Atlas A3 or A2 training systems with at least 64 GB of on-chip memory per NPU. Its sample uses eight NPUs, and the document warns that using fewer may cause an out-of-memory failure. The displayed logs show that several pretraining and fine-tuning iterations complete; they are a runnable smoke test, not a throughput comparison or a multi-node reliability study.[3]

Megatron offers a different kind of control. Its core design combines data, tensor, pipeline, context, and expert parallelism, allowing operators to partition batches, layers, long sequences, and mixture-of-experts models along different axes. That makes it a powerful large-cluster substrate, but also a specialized one with its own configuration and checkpoint assumptions.[6] MindSpeed-LLM’s Mcore lane adds Ascend-specific adaptations and tools around that machinery. The FSDP2 lane instead asks how much of the training workflow can remain legible in standard PyTorch terms.

Neither route is universally superior. Megatron’s attraction is fine-grained scaling control; FSDP2’s attraction is a smaller conceptual jump for teams already working in PyTorch. The supply-chain question is whether Ascend can support both without turning every model, checkpoint, kernel, and upgrade into a separate port.

The model table is more useful than a launch slogan

The clearest maturity signal is not the number of model names in the repository. It is the way MindSpeed-LLM labels the two backends under comparable recipes.

For Qwen3-32B at a 4,000-token sequence length, the table lists an Mcore recipe on two nodes of eight NPUs as Pass, while the FSDP2 recipe uses one node of 16 and remains Test. For the Qwen3-235B-A22B mixture-of-experts model, both backends list a 16-by-16-NPU configuration; Mcore is Pass and FSDP2 is Test. The Qwen3-Next 80B-A3B rows similarly place both backends on four nodes of 16 NPUs, again with Pass for Mcore and Test for FSDP2.[2]

Those rows are not benchmarks. A matching card count does not establish matching speed, memory use, numerical behavior, fault recovery, or cost. What they do establish is more modest: the FSDP2 work has advanced beyond an eight-card tutorial into model-specific scripts for dense and sparse architectures at multi-node scales. The table also keeps the evidence boundary visible by declining to award the new route the same certification label.

That distinction should survive any retelling of the project. “Supported” can mean that a script exists, that a first step completes, that an internal test ran, or that a maintained release passed a defined qualification suite. MindSpeed-LLM’s table separates at least two of those states. A credible next release would explain the exact criteria behind Test and Pass, then publish reproducible logs that let an outside operator see the transition.

The standard API still sits on a versioned machine

FSDP2 can reduce framework-specific work while leaving the platform stack tightly coupled. MindSpeed-LLM’s 26.0.0 release notes define a reference combination of MindSpeed Core core_v0.12.1, PyTorch 2.7.1, Ascend Extension for PyTorch 26.0.0, CANN 9.0.0, Triton-Ascend 3.2.1, and Python 3.10. Separate compatibility tables also mark that MindSpeed-LLM release as compatible with older Ascend Extension and CANN versions. Even so, the document says Triton-Ascend is strongly bound to CANN and should be matched one-to-one, and it warns that a software upgrade interrupts the running workload.[4]

This is the less visible AI supply chain: not accelerators in boxes, but a sequence of drivers, firmware, CANN libraries, torch_npu, compiler components, distributed frameworks, model adapters, and checkpoints that must agree. A standard sharding API is valuable precisely because it can stabilize one seam in that chain. It cannot by itself make an unsupported operator appear, preserve a checkpoint across incompatible layouts, or prove that collective communication behaves well when the job expands from one machine to hundreds.

The current release matrix therefore turns “PyTorch compatibility” from a slogan into a set of testable questions. Can the same model definition enter through the FSDP2 lane without an Ascend-only fork? Can a checkpoint resume after a minor runtime upgrade? Are optimizer state and expert partitions portable between recipes? Do public multi-node runs survive worker loss and still converge to the same result? Do the listed compatibility ranges work as complete, tested combinations, or do operators still need to keep most of the reference stack in lockstep?

MindSpeed-LLM does not yet answer all of them. It does make the questions easier to locate.

Layered decoupling becomes concrete

At HUAWEI CONNECT in September 2025, Huawei described Ascend’s software direction as “layered decoupling” and named PyTorch, Triton, vLLM, and verl among the upstream communities with which it wanted deeper collaboration.[7] That statement was broad. The FSDP2 lane is a smaller, inspectable implementation of the idea: keep hardware-specific work below an interface that more training code already understands.

The decisive evidence will be operational rather than rhetorical. First, FSDP2 rows should graduate from Test to Pass under published criteria. Second, the project should show checkpoint round trips and resumed training, not only clean starts. Third, multi-node reference runs should report utilization, memory, convergence, and failure recovery under a pinned version matrix. Finally, compatibility should survive ordinary upgrades; a portable API that requires a full-stack rebuild for every point release remains an expensive form of portability.

For now, the Mcore lane is the mature road and FSDP2 is the bridge being load-tested beside it. That is still meaningful progress. China’s accelerator challenge is no longer only to manufacture compute or make one celebrated model run. It is to let engineers choose familiar training abstractions while the platform earns its performance underneath them.

train_fsdp2.py does not dissolve the Ascend stack. It relocates one of its boundaries—and, helpfully, the support table tells readers exactly where the new lane still stops.

Sources

1. Ascend, “MindSpeed-LLM” (official repository; project scope, repository structure, February 2026 FSDP2 announcement, and backend navigation).
2. Ascend, “PyTorch framework model support list” (official MindSpeed-LLM table distinguishing Legacy, Mcore, and FSDP2 recipes, cluster sizes, and Pass/Test status).
3. Ascend, “Quick start: Qwen3-8B training with the FSDP2 backend” (official hardware boundary, eight-NPU configuration, YAML controls, and sample run logs).
4. Ascend, “MindSpeed-LLM release notes” (official 26.0.0 component matrix, compatibility notes, and upgrade impact).
5. PyTorch, “PyTorch FSDP2 (fully_shard)” (official API contract for per-parameter sharding, DTensor, and gather/reshard hooks).
6. NVIDIA, “Megatron Core Parallelism Strategies Guide” (official overview of data, tensor, pipeline, context, expert, and fully sharded parallelism).
7. Huawei, “Ascend: Open for All to Build a Vibrant Ecosystem” (September 20, 2025; official account and conference photograph covering layered decoupling and upstream collaboration).