inference-optimization/GLM-4.5-Air-0.72B-MTP
Text Generation • 0.7B • Updated
llm-c mtp test models
Toy-trained BF16/FP8 fixtures for PR #3225 on Transformers 5.17. Model cards document DSA and Inkling limitations.
Note BF16 source; two-GPU quantization and vLLM MTP execution passed.
inference-optimization/GLM-4.5-Air-0.72B-MTP-FP8-Dynamic
Text Generation • 0.7B • UpdatedNote FP8_DYNAMIC; two-GPU quantization and vLLM MTP execution passed.
inference-optimization/GLM-4.7-Flash-0.82B-MTP
Text Generation • 0.8B • UpdatedNote BF16 source; two-GPU quantization and vLLM MTP execution passed.
inference-optimization/GLM-4.7-Flash-0.82B-MTP-FP8-Dynamic
Text Generation • 0.8B • UpdatedNote FP8_DYNAMIC; two-GPU quantization and vLLM MTP execution passed.
inference-optimization/DeepSeek-V3-0.86B-MTP
Text Generation • 0.9B • UpdatedNote BF16 source; two-GPU quantization and vLLM MTP execution passed.
inference-optimization/DeepSeek-V3-0.86B-MTP-FP8-Dynamic
Text Generation • 0.9B • UpdatedNote FP8_DYNAMIC; two-GPU quantization and vLLM MTP execution passed.
inference-optimization/GLM-5-0.88B-MTP
Text Generation • 0.9B • UpdatedNote BF16 source; backbone validated, MTP execution unverified.
inference-optimization/GLM-5-0.88B-MTP-FP8-Dynamic-MFPTQ
Text Generation • 0.9B • UpdatedNote MFPTQ FP8_DYNAMIC; backbone validated, MTP execution unverified.
inference-optimization/Inkling-0.92B-MTP
Image-Text-to-Text • 0.9B • UpdatedNote BF16 multimodal fixture; text backbone validated. Two-rank pipeline and MTP serving are not validated.