SandLogicTechnologies/minicpm5-2b-gguf

MiniCPM5-2B

MiniCPM5-2B is a compact dense language model from the MiniCPM5 series, developed by OpenBMB for local assistants, coding agents, tool-use workflows, reasoning tasks, and resource-constrained deployment. The model is designed to provide strong capabilities within a relatively small parameter and memory footprint.

The model contains approximately 2.52 billion parameters and uses a standard LlamaForCausalLM architecture with Grouped Query Attention. It supports a native context length of up to 131,072 tokens, making it suitable for applications that require long-context processing while maintaining a compact model size.

MiniCPM5-2B is the final post-trained release in the MiniCPM5-2B model line. Its training pipeline includes supervised fine-tuning, reinforcement learning, and On-Policy Distillation (OPD), with dedicated training aimed at reasoning, coding, tool use, and agentic capabilities.

---

Model Overview

• Model Name: MiniCPM5-2B
• Base Model: openbmb/MiniCPM5-2B
• Architecture: LlamaForCausalLM
• Model Type: Dense Causal Language Model
• Parameter Count: 2,516,756,480
• Context Window: 131,072 tokens
• Modalities: Text
• Languages: English, Chinese
• Developer: OpenBMB
• License: Apache 2.0

---

Quantization Formats

This repository provides GGUF-converted versions of MiniCPM5-2B for efficient local inference. The supplied conversion includes an F16 reference file together with IQ3_M, IQ4_NL, IQ4_XS, and Q6_K variants.

The original F16 conversion file was deleted after conversion, while the four quantized variants were retained.

IQ3_M

• Size reduction of approx 75.69% (1.14 GB) compared to 16-bit (4.69 GB)
• Aggressive 3-bit quantization intended to substantially reduce the memory and storage requirements of the 2B model
• Suitable for compact local deployments, edge-oriented applications, and systems where available memory is limited
• Provides a practical low-footprint option for conversational, reasoning, coding, and tool-use workloads
• The lower numerical precision can introduce greater changes in generation behavior on demanding reasoning and long-context tasks compared with higher-precision variants

IQ4_NL

• Size reduction of approx 70.15% (1.40 GB) compared to 16-bit (4.69 GB)
• 4-bit non-linear quantization designed to retain more parameter information than the 3-bit representation
• Provides a balanced option for applications requiring both reduced memory consumption and dependable generation quality
• Suitable for coding, reasoning, conversational generation, tool-use workflows, and long-context inference
• Requires a larger model footprint than IQ3_M while providing a higher-precision representation

IQ4_XS

• Size reduction of approx 71.64% (1.33 GB) compared to 16-bit (4.69 GB)
• Compact 4-bit quantization designed for efficient local deployment
• Offers a practical middle ground between aggressive compression and higher-precision representations
• Suitable for local assistants, coding workflows, tool-calling applications, and long-context tasks
• Provides a slightly smaller footprint than IQ4_NL while remaining within the same 4-bit quantization class

Q6_K

• Size reduction of approx 58.85% (1.93 GB) compared to 16-bit (4.69 GB)
• Higher-precision 6-bit K-Quant representation designed to retain more parameter information than the lower-bit variants
• Suitable for workloads where generation fidelity is prioritized while still benefiting from a substantially smaller model footprint than F16
• Provides a larger representation than IQ3_M, IQ4_NL, and IQ4_XS while remaining considerably smaller than the F16 conversion
• The higher bit depth is expected to reduce quantization-related information loss compared with the lower-bit variants, although comparative benchmarking is required to establish the actual quality difference

---

Training Overview

MiniCPM5-2B follows a multi-stage training process covering base training, mid-training, and post-training. The training approach is part of OpenBMB's UltraData-based training pipeline.

Base and Mid-Training

The early stages focus on developing core language capabilities, training stability, and adaptation to target data distributions.

Training data released alongside the model includes:

• Ultra-FineWeb
• Ultra-FineWeb-L3
• UltraX
• UltraData-Code
• UltraData-Math

These datasets support general language learning as well as specialized improvements in coding and mathematical capabilities.

Post-Training

The final MiniCPM5-2B release uses a three-stage post-training process:

• Supervised Fine-Tuning (SFT)
• Reinforcement Learning (RL)
• On-Policy Distillation (OPD)

The SFT stage uses deep-thinking data to develop reasoning and general conversational capabilities. Subsequent RL stages train specialized teachers for mathematics, coding, agentic tasks, writing, and related domains. OPD then combines capabilities from multiple RL-trained expert models into the final release model.

The model card reports that the RL + OPD process improves reasoning and general capabilities as well as agentic capabilities across the evaluated benchmarks.

---

Core Capabilities

• Reasoning Designed for analytical and multi-step reasoning tasks, including mathematical and technical problem solving.

• Coding Provides capabilities for code generation, code reasoning, debugging, and software-development workflows.

• Long-Context Processing Supports a 131,072-token context window for processing long documents, codebases, conversations, and extended task histories.

• Tool Use Designed for workflows where the model interacts with external tools and structured tool interfaces.

• Agentic Workflows Supports coding-agent and general agent tasks requiring multiple steps, tool interaction, and task execution.

• Instruction Following Post-training improves the model's ability to follow user instructions across conversational and structured tasks.

• Mathematical Reasoning Includes specialized post-training for mathematical reasoning and reports strong results across mathematical evaluation tasks.

• Conversational Generation Supports general conversational interactions while maintaining a compact deployment footprint.

---

Example Usage

llama.cpp

./llama-cli \
  -m SandLogicTechnologies/MiniCPM5-2B_IQ4_NL.gguf \
  -p "Explain the architectural differences between dense and Mixture-of-Experts language models."

Recommended Use Cases

• Local AI assistants
• Coding-agent workflows
• Software-development applications
• Tool-calling systems
• Agentic AI applications
• Mathematical reasoning systems
• Long-context document processing
• Conversational applications
• Structured generation workflows
• Research involving compact language models and agentic systems

---

Acknowledgments

These quantized models are based on the original work by the OpenBMB development team.

Special thanks to:

• The OpenBMB team for developing and releasing MiniCPM5-2B.

• Georgi Gerganov and the llama.cpp open-source community for enabling efficient GGUF-based local inference.

---

Contact

For any inquiries or support, please contact us at support@sandlogic.com or visit our Website.