744B / 40B
Base scale
The released Base checkpoint has 744B total and 40B activated parameters across 78 decoder layers.
Sparse + linear attention · visualized
Inside GLM 5.3
A source-grounded map of two very different checkpoints: the 744B all-DSA Base model and the 320B multimodal Flash model that interleaves KDA, sparse attention, four-stream mHC, KPool, and sparse experts.
744Btotal parameters
40Bactivated parameters
78decoder layers
1Mtoken context
System symbols
DSA
KDA
KPool / routing
mHC
GLM MoE
MTP
The GLM 5.3 family at a glance
Base and Flash share DSA, latent KV attention, sparse experts, and NextN, but Flash introduces a separate hybrid multimodal architecture.
320B / 18B
Flash scale
Flash reduces active compute while adding native image and video inputs.
2048
Sparse memory budget
Every DSA query selects 2048 historical tokens through a 32-head lightning indexer.
3 : 1
Flash hybrid schedule
Three recurrent KDA layers alternate with one full DSA layer, ending on KDA.
4 tokens
KPool groups
Flash DSA pools four index keys to shortlist groups before expanding selected groups to token indices.
8 + 1
Activated experts
Eight routed experts run beside one always-on shared expert after the first three dense FFNs.
4 streams
Flash mHC width
Dynamic A, B, and C maps wrap both attention and FFN sublayers; Base uses a standard residual path.
FP8 E4M3
Released precision
Both published configs use 128 × 128 block-quantized FP8 weights.
Primary sources
Architecture claims are reconciled against Z.ai’s technical report, released configs, model cards, and TokenSpeed implementations.
DSA, model scale, MoE, MTP, and long-context system design.
Open report ↗ Base configExact widths, DSA index schedule, expert counts, and released precision.
Open Base config ↗ Flash configKDA/DSA schedule, KPool, mHC, vision tower, and expert topology.
Open Flash config ↗