Update README.md

README.md

@@ -7,7 +7,7 @@ license: other
 license_name: stepfun-community-license
 license_link: https://huggingface.co/SHSLab/Step-5-Preview-BF16/blob/main/LICENSE
 library_name: transformers
-pipeline_tag: text-generation
+pipeline_tag: image-text-to-text
 tags:
 - stepfun
 - step-5
@@ -19,6 +19,7 @@ tags:
 - long-context
 - 1m-context
 - multimodal
+- image-text-to-text
 - text-generation
 - image
 - video
@@ -48,11 +49,11 @@ tags:
 
 </div>
 
-> **🔥 Step-5-Preview is now available.**
+> **🔥 Step-5-Preview is now available!**
 >
-> Step-5-Preview is a 600B-parameter sparse Mixture-of-Experts model with 27B active parameters, a 1M-token context window, and native support for text, image, and video inputs.
+> Step-5-Preview is our flagship foundation model for real-world agentic work — a 600B-parameter sparse Mixture-of-Experts model with 27B active parameters, a 1M-token context window, and native support for text, image, and video inputs.
 >
-> Weights are available on Hugging Face (`SHSLab/Step-5-Preview-BF16`). Available via Step API, or self-hosted with vLLM / SGLang.
+> Weights are available on Hugging Face (`SHSLab/Step-5-Preview-BF16`). Try it via our API, or deploy locally with vLLM / SGLang.
 
 ---
 
@@ -60,30 +61,39 @@ tags:
 
 - [Introduction](#-introduction)
 - [Key Features](#-key-features)
+- [Model Architecture](#-model-architecture)
 - [Model Specifications](#-model-specifications)
+- [Training Data](#-training-data)
 - [Benchmark Results](#-benchmark-results)
+- [Agentic Capabilities](#-agentic-capabilities)
+- [Real-World Use Cases](#-real-world-use-cases)
 - [Quickstart](#-quickstart)
 - [Deployment](#-deployment)
+- [Evaluation](#-evaluation)
+- [Limitations](#-limitations)
+- [Ethical Considerations](#-ethical-considerations)
+- [Hardware Requirements](#-hardware-requirements)
+- [Performance Metrics](#-performance-metrics)
+- [Citation](#-citation)
 - [License](#-license)
 - [Contact](#-contact)
-- [More details](#-more-details) *(architecture, training, agentic demos, evaluation, limitations)*
 
 ---
 
 ## 🚀 Introduction
 
-**Step-5-Preview** is StepFun's flagship foundation model, designed for **real-world agentic tasks**. It targets professional domains such as **AI coding, software engineering, professional knowledge work, and financial analysis**.
+**Step-5-Preview** is StepFun's flagship foundation model, designed from the ground up for **real-world agentic tasks**. It targets professional domains such as **AI coding, software engineering, professional knowledge work, and financial analysis**.
 
-StepFun's core philosophy for Step 5 is the **"Pareto Frontier"** — balancing intelligence against cost. While earlier scaling efforts traded more compute for stronger intelligence, the next phase focuses on improving the **efficiency of converting compute into intelligence**.
+StepFun's core philosophy for Step 5 is the **"Pareto Frontier"** — achieving the optimal balance between intelligence and cost. While previous scaling efforts focused on trading more compute for stronger intelligence, the next phase requires improving the **efficiency of converting compute into intelligence**.
 
-> **Why Step 5 Preview?**
+> **💡 Why Step 5 Preview?**
 >
-> - 600B total parameters, 27B active per token — near-frontier performance at a fraction of the compute.
-> - 1M-token context window without proportional cost increases.
-> - Competitive benchmark scores against models with 3–5× more parameters.
-> - Built for agents: long-horizon reasoning, tool use, and autonomous execution.
+> • **600B total parameters, only 27B active** — near-frontier performance at a fraction of the compute.
+> • **1M-token context window** without proportional cost increases.
+> • **Competitive benchmark scores** against models with 3–5× more parameters.
+> • **Built for agents** — long-horizon reasoning, tool use, and autonomous execution.
 
-Step-5-Preview skips the entire Step 4.x line, going directly from Step-3.7-Flash to Step 5.
+Step-5-Preview represents a generational leap, with StepFun **skipping the entire Step 4.x line** entirely, going directly from Step-3.7-Flash to Step 5. This decision reflects the magnitude of improvement achieved in this release.
 
 ---
 
@@ -100,7 +110,31 @@ Step-5-Preview skips the entire Step 4.x line, going directly from Step-3.7-Flas
 - **Parallel Tool Calling:** Natively supported for agentic workflows.
 - **Strict JSON Schema Output:** Reliable integration into structured systems.
 - **OpenAI-Compatible API:** Available via Step API and third-party gateways.
-- **Open Weights:** BF16 checkpoint available under `SHSLab/Step-5-Preview-BF16`.
+- **Open Weights:** BF16 checkpoint available now under `SHSLab/Step-5-Preview-BF16`.
+
+---
+
+## 🏗️ Model Architecture
+
+<div align="center">
+  <img src="./Step-5/architecture.png" alt="Step 5 Architecture" width="85%">
+</div>
+
+### 92-Layer "Narrow but Deep" Design
+
+Step-5-Preview uses a **92-layer Transformer** with a narrow-deep configuration. This design is specifically intended to create **longer information propagation paths** for implicit multi-hop reasoning during long prefill operations.
+
+### Sparse Grouped-Query Attention (GQA) with Block-Wise Token Merging
+
+To handle the 1M-token context window efficiently, Step-5-Preview introduces **Sparse GQA with block-wise token merging**. This mechanism uses sparse indexing to select only historical information relevant to the current task, reducing the number of tokens that actually enter attention computation. StepFun states this cuts indexer and top-k selection costs to approximately **one-eighth** of a denser baseline.
+
+> **⚡ Efficiency-First Scaling**
+>
+> Step 5 Preview achieves near-frontier performance with **600B total parameters** but only **27B active per token**. This is the core of StepFun's efficiency-first philosophy.
+
+### Multimodal Encoder
+
+The model incorporates a unified multimodal encoder that processes text, images, and video frames into a shared latent space. Video is sampled at adaptive frame rates and encoded with temporal attention, allowing the model to understand motion and long-range dependencies in screen recordings, demonstrations, and real-world footage.
 
 ---
 
@@ -122,12 +156,27 @@ Step-5-Preview skips the entire Step 4.x line, going directly from Step-3.7-Flas
 | **Reasoning Effort** | `low` / `medium` / `high` (`xhigh`) |
 | **Tool Calling** | Parallel, strict JSON schema |
 | **Intelligence Index** | 44 (Artificial Analysis v4.3.2) |
-| **Open Weights** | BF16 checkpoint available |
+| **Open Weights** | BF16 checkpoint available now |
 | **API Availability** | Immediate (OpenAI-compatible) |
 | **License** | StepFun Community License |
 
 ---
 
+## 📚 Training Data
+
+Step-5-Preview was trained on a massive, carefully curated corpus spanning:
+
+- **Code repositories** from multiple languages (Python, C++, Rust, JavaScript, Go, etc.)
+- **Technical documentation**, API references, and software engineering forums
+- **Scientific papers** in computer science, mathematics, physics, and finance
+- **Financial reports**, earnings calls, and market analyses
+- **Multimodal data** including screenshots, UI mockups, video tutorials, and screen recordings
+- **Agentic trajectories** from simulated and real tool-use environments
+
+The data mixture was optimized for long-horizon reasoning and tool use, with a strong emphasis on real-world professional tasks. All data was filtered for quality, safety, and license compliance. The training process used a combination of next-token prediction and reinforcement learning from human feedback (RLHF) with a focus on agentic objectives.
+
+---
+
 ## 📊 Benchmark Results
 
 <div align="center">
@@ -140,16 +189,30 @@ Step-5-Preview skips the entire Step 4.x line, going directly from Step-3.7-Flas
 
 The index covers 10 evaluations including AA-Briefcase, GDPval-AA v2, Terminal-Bench 4.0, SciCode, and Humanity's Last Exam.
 
-> **How to read the tables below**
->
-> - **Bold** marks the best score in the row across all six models.
-> - **🥇** flags the leader for that benchmark.
-> - **—** indicates the model was not evaluated or did not report a score.
-> - Step-5-Preview results use the `high` reasoning-effort setting unless noted.
->
-> **Comparison set:** Step-5-Preview · GPT-6 Astra (Max) · Fable 5.1 · Claude Opus 5 (Max) · Kimi K3 (Max) · GLM-5.3 (Max)
->
-> **Availability:** Open-weight — Step-5-Preview, Kimi K3, Qwen3.8 Max, GLM-5.3. Closed-source — GPT-6 Astra, Claude Opus 5. Fable 5.1 — not publicly stated.
+<div style="border: 1px solid #8884; border-radius: 12px; padding: 20px 24px; margin: 24px 0; background: transparent;">
+
+<h4 style="margin-top:0;">📐 How to read these tables</h4>
+
+- <strong>Step-5-Preview</strong> is always the first data column, for fast scanning.
+- <strong>Bold</strong> marks the <strong>best score in the row</strong> across all six models.
+- <strong>🥇</strong> flags the category leader for that benchmark.
+- <strong>—</strong> indicates the model was not evaluated or did not report a score.
+- All Step-5-Preview results use the <code>high</code> reasoning-effort setting unless noted.
+
+<strong>Comparison set:</strong> Step-5-Preview · GPT-6 Astra (Max) · Fable 5.1 · Claude Opus 5 (Max) · Kimi K3 (Max) · GLM-5.3 (Max)
+
+<details>
+<summary><strong>🏷️ Open-weight vs. closed-source in this comparison</strong></summary>
+
+**Open-weight:** Step-5-Preview · Kimi K3 (Max) · Qwen3.8 Max · GLM-5.3 (Max)
+
+**Closed-source:** GPT-6 Astra (Max) · Claude Opus 5 (Max)
+
+**Not disclosed:** Fable 5.1
+
+</details>
+
+</div>
 
 ---
 
@@ -162,6 +225,18 @@ The index covers 10 evaluations including AA-Briefcase, GDPval-AA v2, Terminal-B
 | **AA-LCR v1.1** | 88.3% | 80.7% | 85.3% | 79.3% | **88.7%** 🥇 | 79.7% |
 | **CritPt** | 20.9% | **31.7%** 🥇 | 29.7% | 29.1% | 23.4% | 19.1% |
 
+On **AA-LCR v1.1** — long-context reasoning — Step-5-Preview scores **88.3%**, effectively tied with Kimi K3 (88.7%) and ahead of GPT-6 Astra by **+7.6** and Claude Opus 5 by **+9.0**, despite those models being far larger.
+
+<details>
+<summary><strong>📝 Benchmark descriptions</strong></summary>
+
+- **GPQA Diamond** — 198 PhD-level science MCQs (biology, physics, chemistry). Human expert average: 81%.
+- **HLE (Humanity's Last Exam)** — Frontier academic knowledge benchmark; HLE w/ tools reported separately under Agents.
+- **AA-LCR v1.1** — Artificial Analysis Long-Context Reasoning.
+- **CritPt** — 71 unpublished research-level physics challenges simulating full-scale junior-PhD research projects.
+
+</details>
+
 ---
 
 ### 💻 Coding & Software Engineering
@@ -183,6 +258,27 @@ The index covers 10 evaluations including AA-Briefcase, GDPval-AA v2, Terminal-B
 | **StepCode-Bench-Daily** | 64.9% | — | — | **77.6%** 🥇 | 57.7% | 69.1% |
 | **StepCode-Bench-General** | 65.0% | 64.3% | — | **68.3%** 🥇 | 65.2% | 62.0% |
 
+Step-5-Preview leads on **DeepSWE v1.1**, **StepCodeBench**, **ProgramBench**, and **SWE-Atlas-QnA** among the four open-weight models in the set, and posts the best **CyberGym** score overall (**84.7%**). On **Terminal-Bench 4** it reaches **33.3%** — **2.6× Kimi K3** (12.6%).
+
+<details>
+<summary><strong>📝 Benchmark descriptions</strong></summary>
+
+- **DeepSWE v1.1** — SWE-agent harness, `temperature=1.0`, `top_p=0.95`.
+- **Terminal-Bench 2.1** — Verified refresh of TB 2.0; 89 curated terminal tasks across SWE, sysadmin, data processing.
+- **Terminal-Bench 4** — Next-generation terminal agent benchmark.
+- **CyberGym** — Security-focused agent benchmark.
+- **SciCode** — Scientific coding benchmark.
+- **RoadmapBench** — 115 long-horizon coding tasks across 17 repos, 5 languages; median ~3,700 LOC changed.
+- **ProgramBench** — Rebuild programs from binary + docs; 200 tasks, 248K+ behavioral tests.
+- **SWE-Marathon v1.1** — Ultra-long-horizon SWE; 20 realistic multi-hour tasks with hidden/adversarial tests.
+- **MLS-Bench-Lite** — 30-task subset of MLS-Bench; tests inventing generalizable ML methods across 12 research domains.
+- **SWE-Atlas-QnA** — 124 tasks on deep code comprehension across 11 production repos (Go, Python, C, TypeScript).
+- **SWE-Atlas-Test-writing** — 90 tasks on writing production-grade unit, integration, and acceptance tests.
+- **StepCodeBench** — StepFun internal coding benchmark; **49.0% avg@4**.
+- **StepCode-Bench-Daily / General** — StepFun internal; 553 repos, 9 task types, 20 domains, 33 languages.
+
+</details>
+
 ---
 
 ### 🤖 Agents, Tool Use & Automation
@@ -203,6 +299,26 @@ The index covers 10 evaluations including AA-Briefcase, GDPval-AA v2, Terminal-B
 | **BrowseComp** | 88.7% | **91.5%** 🥇 | — | 90.2% | 91.2% | — |
 | **HLE w/ tools** | 59.4% | 57.2% | **65.0%** 🥇 | 63.6% | 56.0% | 62.5% |
 
+Step-5-Preview outperforms GPT-6 Astra on **τ³-Banking** (42.5% vs. 41.4%) and **Draco** (83.3% vs. 76.8%), and edges Kimi K3 on **MCP-Atlas** (85.6% vs. 85.3%) and **Draco** (83.3% vs. 78.5%). With tools, HLE rises from **46.5% → 59.4%** (+12.9 points).
+
+<details>
+<summary><strong>📝 Benchmark descriptions</strong></summary>
+
+- **GDPval-AA v2** — Artificial Analysis, Sep 19, 2026.
+- **τ³-Banking** — Multi-turn banking agent benchmark.
+- **AutomationBench-AA / Public** — Agentic automation tasks, private and public splits.
+- **AA-Briefcase** — Elo score; agentic knowledge work across data science, product, banking, heavy industry.
+- **Toolathlon-Verified** — 108 expert-authored tasks; multi-app workflows averaging ~20 turns.
+- **MCP-Atlas** — 1,000 tasks across 36 real MCP servers and 220 tools; 3–6 tool calls per task.
+- **PresentBench** — 238 slide-generation instances; avg 54.1 rubric checklist items per instance.
+- **JobBench** — 130 tasks across 35 occupations; avg 35.6 binary rubric criteria per task.
+- **Apex-Agents** — Long-horizon tasks in investment banking, consulting, and corporate law.
+- **DRACO** — Cross-domain deep research benchmark; LLM-as-judge with binary rubric verdicts.
+- **BrowseComp** — 1,266 hard-to-find web information retrieval questions.
+- **HLE w/ tools** — Humanity's Last Exam with tool augmentation.
+
+</details>
+
 ---
 
 ### 💰 Finance & Professional Work
@@ -214,9 +330,24 @@ The index covers 10 evaluations including AA-Briefcase, GDPval-AA v2, Terminal-B
 | **FinStepBench-FinanceDR** | 55.8% | 45.0% | — | **59.1%** 🥇 | 48.9% | 53.3% |
 | **FrontierFinance** | 66.4% | 55.0% | — | **69.7%** 🥇 | 62.6% | 64.1% |
 | **OfficeQA Pro** | 60.3% | **67.7%** 🥇 | — | 64.7% | 62.6% | 59.1% |
-| **SpeadSheet v2** | 29.4% | 31.4% | — | **32.8%** 🥇 | 31.9% | 30.5% |
+| **Spreadsheet v2** | 29.4% | 31.4% | — | **32.8%** 🥇 | 31.9% | 30.5% |
 | **GDP.pdf** | 14.8% | **31.0%** 🥇 | 26.2% | 21.6% | 22.0% | 11.2% |
 
+On **FrontierFinance** Step-5-Preview scores **66.4%**, ahead of GPT-6 Astra by **+11.4**, Kimi K3 by **+3.8**, and GLM-5.3 by **+2.3**. On **FinStepBench-FinanceDR** it reaches **55.8%**, ahead of GPT-6 Astra (+10.8), Kimi K3 (+6.9), and GLM-5.3 (+2.5).
+
+<details>
+<summary><strong>📝 Benchmark descriptions</strong></summary>
+
+- **FinStepBench-LiveSearch** — StepFun internal; live web search for financial research.
+- **FinStepBench-CorporateValuation** — StepFun internal; corporate valuation tasks.
+- **FinStepBench-FinanceDR** — StepFun internal; deep-research finance workflows with source traceability, reproducible assumptions, auditable reports.
+- **FrontierFinance** — Finance reasoning benchmark.
+- **OfficeQA Pro** — Databricks; 90 questions over large enterprise financial document collections.
+- **Spreadsheet v2** — SpreadsheetBench 2; end-to-end business spreadsheet workflows. Best model ≈34.89%.
+- **GDP.pdf** — Document-heavy visual QA over PDFs.
+
+</details>
+
 ---
 
 ### 👁️ Multimodal
@@ -226,16 +357,41 @@ The index covers 10 evaluations including AA-Briefcase, GDPval-AA v2, Terminal-B
 | **MMMU-Pro** | 76.0% | **87.0%** 🥇 | — | 85.0% | 81.0% | — |
 | **GDP.pdf** | 14.8% | **31.0%** 🥇 | 26.2% | 21.6% | 22.0% | 11.2% |
 
+Multimodal reasoning is the primary improvement target for the next release. MMMU-Pro at **76.0%** trails GPT-6 Astra (87.0%), Claude Opus 5 (85.0%), and Kimi K3 (81.0%).
+
+<details>
+<summary><strong>📝 Benchmark descriptions</strong></summary>
+
+- **MMMU-Pro** — Robust multimodal benchmark; filtered text-only questions, augmented candidates, vision-only settings. Significantly harder than MMMU.
+- **GDP.pdf** — Document-heavy visual QA over PDFs.
+
+</details>
+
+---
+
+### 🏅 Category Standing
+
+| Domain | Step-5-Preview Standing | Highlight |
+|:---|:---|:---|
+| **Long-Context Reasoning** | 🥈 Near-tied for #1 | AA-LCR v1.1 **88.3%** |
+| **Coding / SWE** | 🥇 #1 open-weight | DeepSWE v1.1 **67.7%**, StepCodeBench **49.0%**, ProgramBench **80.5%** |
+| **Security / Cyber** | 🥇 #1 in set | CyberGym **84.7%** |
+| **Finance** | 🥈 #2 overall, #1 open-weight | FrontierFinance **66.4%**, FinanceDR **55.8%** |
+| **Terminal Agents** | 🥈 #2 open-weight | Terminal-Bench 4 **33.3%** |
+| **Tool Use** | 🥉 Competitive | MCP-Atlas **85.6%**, HLE w/ tools **59.4%** |
+| **General Knowledge** | Top tier | GPQA Diamond **93.5%**, HLE **46.5%** |
+| **Multimodal** | Trailing | MMMU-Pro **76.0%** |
+
 <details>
 <summary><strong>📝 Benchmark methodology notes</strong></summary>
 
 - **DeepSWE v1.1** was evaluated using the SWE-agent harness with `temperature=1.0` and `top_p=0.95`.
 - **StepCodeBench** achieved **49.0% avg@4**.
 - **GDPval-AA v2** results are from Artificial Analysis as of September 19, 2026.
-- **AA-LCR v1.1**: Step-5-Preview scored **88.3%**, statistically tied with Kimi K3 (88.7%).
-- **Terminal-Bench 4**: Step-5-Preview **33.3%** vs. Kimi K3 ~**12.6%** (2.6×) and DeepSeek V4.1 Flash **26.8%** (1.24×).
-- **SciCode**: Step-5-Preview **58.9%**, above GPT-6 Astra (56.5%) and Claude Opus 5 (56.4%).
-- **Multimodal**: MMMU-Pro and GDP.pdf were run with the unified multimodal encoder at default resolution.
+- **AA-LCR v1.1**: statistically tied with Kimi K3 (88.7%).
+- **Terminal-Bench 4**: **33.3%** vs. Kimi K3 **~12.6%** (2.6×) and DeepSeek V4.1 Flash **26.8%** (1.24×).
+- **SciCode**: **58.9%**, above GPT-6 Astra (56.5%) and Claude Opus 5 (56.4%).
+- **Multimodal**: MMMU-Pro and GDP.pdf run with the unified multimodal encoder at default resolution.
 - **Output Speed**: 99.8 tokens/sec (GLM-5.3: 72.1 tokens/sec).
 - **Time to First Token**: 2.96 seconds (GLM-5.3: 2.99s; Claude Opus 5: 56.84s at max effort).
 - Missing entries (**—**) reflect benchmarks that were not publicly reported for that model at the time of writing.
@@ -244,6 +400,52 @@ The index covers 10 evaluations including AA-Briefcase, GDPval-AA v2, Terminal-B
 
 ---
 
+## 🤖 Agentic Capabilities
+
+<div align="center">
+  <img src="./Step-5/agentic_workflow.png" alt="Agentic Workflow" width="90%">
+</div>
+
+### 24-Hour Autonomous GPU Kernel Optimization
+
+In a landmark demonstration of sustained agentic execution, Step-5-Preview was tasked with **autonomously optimizing an H100 GPU kernel for up to 24 consecutive hours**. The model:
+
+- Independently modified code
+- Ran tests and compared results
+- Iterated based on performance outcomes
+- **Reached 508 TFLOPS after approximately 22 hours**
+
+For comparison, **Claude Opus 5 achieved 493 TFLOPS** in the same experiment. This demonstrates Step-5-Preview's ability to sustain productive work over extended periods without human intervention.
+
+### Automated Post-Training Experiments
+
+In another 24-hour experiment, Step-5-Preview autonomously improved the accuracy of **Qwen3-30B-A3B on AIME24 from 53.3% to 60%** through automated post-training experiments. This showcases the model's capacity for self-directed research and optimization.
+
+### Long-Horizon Agent Workflows
+
+The model is specifically optimized for agent workflows that require:
+
+- Searching and information retrieval
+- Running code and processing tool returns
+- Multi-turn tool calls with sustained execution
+- Iterative refinement based on intermediate results
+- Self-correction and error recovery over thousands of steps
+
+---
+
+## 💼 Real-World Use Cases
+
+StepFun demonstrated the model's capabilities across several complex, real-world projects:
+
+- **ESP32 Development Board Modifications:** Executed development tasks for over 3 hours, demonstrating hardware programming capabilities.
+- **Front-End Design with 3D Asset Generation:** Full-stack development workflows including visual design.
+- **Full-Process Financial Research:** End-to-end investment research workflows, from data gathering to report generation.
+- **Software Engineering:** Comprehensive coding tasks beyond traditional code generation, including front-end, visual development, and programmable hardware scenarios.
+- **Autonomous Research Assistant:** Capable of reading papers, running experiments, and summarizing findings.
+- **Customer Support Automation:** Handles multi-turn conversations with tool calls to internal systems.
+
+---
+
 ## ⚡ Quickstart
 
 ### Installation
@@ -396,216 +598,89 @@ response = client.chat.completions.create(
 print(response.choices[0].message.content)
 ```
 
-> **Recommended deployment configurations**
+> **📦 Recommended Deployment Configurations**
 >
-> - **BF16:** 8× H100 80GB (tensor parallel)
-> - **FP8:** 4× H100 80GB (coming soon)
-> - **Context length:** Up to 1M tokens
-> - **Reasoning parser:** Use `stepfun` for vLLM / SGLang
+> • **BF16:** 8× H100 80GB (tensor parallel)
+> • **FP8:** 4× H100 80GB (coming soon)
+> • **Context length:** Up to 1M tokens
+> • **Reasoning parser:** Use `stepfun` for vLLM / SGLang
 
 ---
 
-## 📜 License
-
-Step-5-Preview is released under the **StepFun Community License**. See the [LICENSE](https://huggingface.co/SHSLab/Step-5-Preview-BF16/blob/main/LICENSE) file for full terms.
+## 📈 Evaluation
 
-> **Usage restrictions**
->
-> - Commercial use is permitted under the StepFun Community License.
-> - Redistribution must include the license and attribution.
-> - See LICENSE for full details.
-
----
-
-## 📬 Contact
-
-- **Hugging Face:** [SHSLab](https://huggingface.co/SHSLab)
-- **GitHub:** [github.com/stepfun-ai](https://github.com/stepfun-ai)
-- **Discord:** [Join our Discord](https://discord.gg/stepfun)
-- **Email:** [opensource@stepfun.com](mailto:opensource@stepfun.com)
-- **Website:** [stepfun.com](https://stepfun.com)
-
----
-
-## 📚 More details
-
-<details>
-<summary><strong>🏗️ Model Architecture</strong></summary>
-
-<div align="center">
-  <img src="./Step-5/architecture.png" alt="Step 5 Architecture" width="85%">
-</div>
-
-### 92-Layer "Narrow but Deep" Design
-
-Step-5-Preview uses a **92-layer Transformer** with a narrow-deep configuration. This design creates longer information propagation paths for implicit multi-hop reasoning during long prefill operations.
-
-### Sparse Grouped-Query Attention (GQA) with Block-Wise Token Merging
-
-To handle the 1M-token context window efficiently, Step-5-Preview uses **Sparse GQA with block-wise token merging**. This mechanism uses sparse indexing to select only historical information relevant to the current task, reducing the number of tokens that enter attention computation. This cuts indexer and top-k selection costs to approximately one-eighth of a denser baseline.
-
-### Multimodal Encoder
-
-A unified multimodal encoder processes text, images, and video frames into a shared latent space. Video is sampled at adaptive frame rates and encoded with temporal attention, allowing the model to understand motion and long-range dependencies in screen recordings, demonstrations, and real-world footage.
-
-</details>
-
-<details>
-<summary><strong>📚 Training Data</strong></summary>
-
-Step-5-Preview was trained on a carefully curated corpus spanning:
-
-- **Code repositories** from multiple languages (Python, C++, Rust, JavaScript, Go, etc.)
-- **Technical documentation**, API references, and software engineering forums
-- **Scientific papers** in computer science, mathematics, physics, and finance
-- **Financial reports**, earnings calls, and market analyses
-- **Multimodal data** including screenshots, UI mockups, video tutorials, and screen recordings
-- **Agentic trajectories** from simulated and real tool-use environments
-
-The data mixture was optimized for long-horizon reasoning and tool use, with a strong emphasis on real-world professional tasks. All data was filtered for quality, safety, and license compliance. Training combined next-token prediction with reinforcement learning from human feedback (RLHF) focused on agentic objectives.
-
-</details>
-
-<details>
-<summary><strong>🤖 Agentic Capabilities</strong></summary>
-
-<div align="center">
-  <img src="./Step-5/agentic_workflow.png" alt="Agentic Workflow" width="90%">
-</div>
-
-### 24-Hour Autonomous GPU Kernel Optimization
-
-Step-5-Preview was tasked with autonomously optimizing an H100 GPU kernel for up to 24 consecutive hours. The model:
-
-- Independently modified code
-- Ran tests and compared results
-- Iterated based on performance outcomes
-- **Reached 508 TFLOPS after approximately 22 hours**
-
-For comparison, Claude Opus 5 achieved 493 TFLOPS in the same experiment.
-
-### Automated Post-Training Experiments
-
-In another 24-hour experiment, Step-5-Preview autonomously improved the accuracy of Qwen3-30B-A3B on AIME24 from 53.3% to 60% through automated post-training experiments.
-
-### Long-Horizon Agent Workflows
-
-Optimized for workflows that require:
-
-- Searching and information retrieval
-- Running code and processing tool returns
-- Multi-turn tool calls with sustained execution
-- Iterative refinement based on intermediate results
-- Self-correction and error recovery over thousands of steps
-
-</details>
-
-<details>
-<summary><strong>💼 Real-World Use Cases</strong></summary>
-
-- **ESP32 Development Board Modifications:** Executed development tasks for over 3 hours.
-- **Front-End Design with 3D Asset Generation:** Full-stack development workflows including visual design.
-- **Full-Process Financial Research:** End-to-end investment research, from data gathering to report generation.
-- **Software Engineering:** Comprehensive coding tasks including front-end, visual development, and programmable hardware.
-- **Autonomous Research Assistant:** Reading papers, running experiments, and summarizing findings.
-- **Customer Support Automation:** Multi-turn conversations with tool calls to internal systems.
-
-</details>
-
-<details>
-<summary><strong>📈 Full Evaluation Table</strong></summary>
-
-All evaluations used the `high` reasoning effort setting unless otherwise noted.
+Step-5-Preview was evaluated on a comprehensive suite of public and internal benchmarks. All evaluations used the model's `high` reasoning effort setting unless otherwise noted.
 
 | Benchmark | Score | Notes |
 |:---|:---|:---|
 | **DeepSWE v1.1** | 67.7% | SWE-agent harness, temp=1.0, top_p=0.95 |
 | **StepCodeBench** | 49.0% | avg@4 |
 | **ProgramBench** | 80.5% | Rebuild programs from binary + docs; 200 tasks, 248K+ behavioral tests |
-| **Terminal-Bench 2.1** | 85.0% | Verified refresh of TB 2.0; 89 curated terminal tasks across SWE, sysadmin, data processing |
+| **Terminal-Bench 2.1** | 85.0% | Verified refresh of TB 2.0; 89 curated terminal tasks |
 | **Terminal-Bench 4** | 33.3% | 2.6× Kimi K3 |
 | **CyberGym** | 84.7% | Best in comparison set |
 | **SciCode** | 58.9% | Above GPT-6 Astra (56.5%) and Opus 5 (56.4%) |
-| **RoadmapBench** | 54.3% | 115 long-horizon coding tasks across 17 repos, 5 languages; median ~3,700 LOC changed |
-| **SWE-Marathon v1.1** | 72.7% | Ultra-long-horizon SWE; 20 realistic multi-hour tasks with hidden/adversarial tests |
-| **MLS-Bench-Lite** | 40.5% | 30-task subset of MLS-Bench; tests inventing generalizable ML methods across 12 research domains |
-| **SWE-Atlas-QnA** | 63.6% | 124 tasks on deep code comprehension — tracing execution paths, explaining architecture across production repos |
-| **SWE-Atlas-Test-writing** | 50.8% | 90 tasks on writing production-grade unit, integration, and acceptance tests for real repositories |
-| **StepCode-Bench-Daily** | 64.9% | StepFun internal; 553 repos, 9 task types, 20 domains, 33 languages; daily-difficulty slice |
-| **StepCode-Bench-General** | 65.0% | StepFun internal; same corpus as StepCodeBench; general-difficulty slice |
-| **Agents' Last Exam (ALE-CLI)** | 29.5% | Linux-only CLI subset of ALE; 40 industry subfields; best agent pass rate ~25.2% |
+| **RoadmapBench** | 54.3% | 115 long-horizon coding tasks; median ~3,700 LOC changed |
+| **SWE-Marathon v1.1** | 72.7% | Ultra-long-horizon SWE; 20 realistic multi-hour tasks |
+| **MLS-Bench-Lite** | 40.5% | 30-task subset; inventing generalizable ML methods |
+| **SWE-Atlas-QnA** | 63.6% | 124 deep code-comprehension tasks |
+| **SWE-Atlas-Test-writing** | 50.8% | 90 test-writing tasks across production repos |
+| **StepCode-Bench-Daily** | 64.9% | StepFun internal; 553 repos, 9 task types, 33 languages |
+| **StepCode-Bench-General** | 65.0% | StepFun internal; general-difficulty slice |
+| **Agents' Last Exam (ALE-CLI)** | 29.5% | Linux-only CLI subset; 40 industry subfields |
 | **GDPval-AA v2** | 1571 | Artificial Analysis, Sep 19, 2026 |
-| **AA-Briefcase** | 1417 | Elo; agentic knowledge work across data science, product, banking, heavy industry; private held-out test set |
-| **Toolathlon-Verified** | 74.1% | 108 expert-authored tasks; multi-app workflows averaging ~20 turns; strictly verifiable via scripts |
-| **MCP-Atlas** | 85.6% | 1,000 tasks (500 public + 500 private); 36 real MCP servers, 220 tools; 3–6 tool calls/task |
-| **PresentBench** | 76.8% | 238 slide-generation instances; avg 54.1 rubric checklist items per instance |
-| **JobBench** | 59.0% | 130 tasks across 35 occupations; avg 35.6 binary rubric criteria per task |
-| **Apex-Agents** | 37.8% | Long-horizon tasks in investment banking, consulting, corporate law |
-| **DRACO** | 83.3% | Cross-domain deep research; accuracy, completeness, objectivity, citation quality |
-| **BrowseComp** | 88.7% | 1,266 hard-to-find web information retrieval questions |
+| **AA-Briefcase** | 1417 | Elo; agentic knowledge work across industries |
+| **Toolathlon-Verified** | 74.1% | 108 expert-authored tasks; ~20 turns average |
+| **MCP-Atlas** | 85.6% | 1,000 tasks across 36 MCP servers, 220 tools |
+| **PresentBench** | 76.8% | 238 slide-generation instances; avg 54.1 rubric items |
+| **JobBench** | 59.0% | 130 tasks across 35 occupations |
+| **Apex-Agents** | 37.8% | Long-horizon IB, consulting, corporate law tasks |
+| **DRACO** | 83.3% | Cross-domain deep research benchmark |
+| **BrowseComp** | 88.7% | 1,266 hard web information retrieval questions |
 | **HLE w/ tools** | 59.4% | +12.9 pts over no-tools HLE |
 | **FrontierFinance** | 66.4% | +11.4 pts over GPT-6 Astra |
 | **FinStepBench-LiveSearch** | 74.5% | Tied with GPT-6 Astra |
 | **FinStepBench-CorporateValuation** | 60.6% | Tied with Kimi K3 |
-| **FinStepBench-FinanceDR** | 55.8% | StepFun internal; full deep-research finance workflows |
-| **OfficeQA Pro** | 60.3% | Databricks; 90 questions over large enterprise financial document collections |
-| **SpeadSheet v2** | 29.4% | SpreadsheetBench 2; end-to-end business spreadsheet workflows; best model ≈34.89% |
+| **FinStepBench-FinanceDR** | 55.8% | Deep-research finance workflows |
+| **OfficeQA Pro** | 60.3% | 90 questions over enterprise financial documents |
+| **Spreadsheet v2** | 29.4% | SpreadsheetBench 2; best model ≈34.89% |
 | **GDP.pdf** | 14.8% | Known weak spot |
 | **GPQA Diamond** | 93.5% | 198 PhD-level science MCQs; human expert avg 81% |
 | **HLE** | 46.5% | 59.4% with tools |
 | **AA-LCR v1.1** | 88.3% | Tied with Kimi K3 (88.7%) |
 | **CritPt** | 20.9% | 71 unpublished research-level physics challenges |
-| **MMMU-Pro** | 76.0% | Robust multimodal benchmark; significantly harder than MMMU |
+| **MMMU-Pro** | 76.0% | Robust multimodal benchmark; harder than MMMU |
 | **Output Speed** | 99.8 tokens/sec | GLM-5.3: 72.1 tokens/sec |
 | **Time to First Token** | 2.96s | GLM-5.3: 2.99s; Claude Opus 5: 56.84s (max effort) |
 
-</details>
-
-<details>
-<summary><strong>🏅 Category summary</strong></summary>
-
-| Domain | Standing | Highlight |
-|:---|:---|:---|
-| **Long-Context Reasoning** | Near-tied for #1 | AA-LCR v1.1 **88.3%** (+7.6 over GPT-6 Astra, +9.0 over Opus 5) |
-| **Coding / SWE** | #1 open-weight | DeepSWE v1.1 **67.7%**, StepCodeBench **49.0%**, ProgramBench **80.5%** |
-| **Security / Cyber** | #1 in comparison set | CyberGym **84.7%** |
-| **Finance** | #2 overall, #1 open-weight | FrontierFinance **66.4%**, FinanceDR **55.8%** |
-| **Terminal Agents** | #2 open-weight | Terminal-Bench 4 **33.3%** (2.6× Kimi K3) |
-| **Tool Use** | Top-3 open-weight | MCP-Atlas **85.6%**, HLE w/ tools **59.4%** |
-| **General Knowledge** | Top tier | GPQA Diamond **93.5%**, HLE **46.5%** |
-| **Multimodal** | Trailing | MMMU-Pro **76.0%** — targeted for improvement |
+---
 
-</details>
+## ⚠️ Limitations
 
-<details>
-<summary><strong>⚠️ Limitations</strong></summary>
-
-- **Knowledge Cutoff:** Knowledge is current up to mid-2026. May not be aware of later events.
-- **Hallucination:** Can generate plausible but incorrect information, especially in domains with sparse training data.
-- **Long Context Degradation:** Performance may degrade for extremely long contexts beyond 500K tokens in certain tasks.
-- **Tool Use Reliability:** Tool calling is capable but not infallible. Complex multi-tool workflows may occasionally fail or require human intervention.
+- **Knowledge Cutoff:** The model's knowledge is current up to mid-2026. It may not be aware of events after that date.
+- **Hallucination:** Like all large language models, Step-5-Preview can generate plausible but incorrect information, especially in domains with sparse training data.
+- **Long Context Degradation:** While the model supports 1M tokens, performance may degrade for extremely long contexts beyond 500K tokens in certain tasks.
+- **Tool Use Reliability:** Tool calling is highly capable but not infallible. Complex multi-tool workflows may occasionally fail or require human intervention.
 - **Multimodal Limitations:** Video understanding is limited to clips under 5 minutes and 128 MB. Extremely high-resolution images may be downscaled.
-- **Language Coverage:** Primarily optimized for English and Chinese. Performance in other languages may vary.
+- **Language Coverage:** While multilingual, the model is primarily optimized for English and Chinese. Performance in other languages may vary.
 
-</details>
+---
 
-<details>
-<summary><strong>⚖️ Ethical Considerations</strong></summary>
+## ⚖️ Ethical Considerations
 
-StepFun is committed to the responsible development and deployment of AI. Measures taken:
+StepFun is committed to the responsible development and deployment of AI. We have taken the following measures:
 
-- **Safety Alignment:** Fine-tuned with RLHF to refuse harmful requests and promote helpful, honest, and harmless behavior.
-- **Bias Mitigation:** Training data was filtered to reduce harmful stereotypes and biases. Residual biases may exist.
-- **Transparency:** Detailed model cards and benchmark results are provided to enable informed use.
+- **Safety Alignment:** The model was fine-tuned with RLHF to refuse harmful requests and promote helpful, honest, and harmless behavior.
+- **Bias Mitigation:** Training data was filtered to reduce harmful stereotypes and biases. However, residual biases may exist.
+- **Transparency:** We provide detailed model cards and benchmark results to enable informed use.
 - **License Restrictions:** The StepFun Community License prohibits certain high-risk uses, including autonomous weapons, surveillance, and malicious cyber activities.
-- **Content Provenance:** Users are encouraged to clearly label AI-generated content.
+- **Content Provenance:** We encourage users to clearly label AI-generated content and to use the model ethically.
 
-All users should consider the ethical implications of their applications and implement appropriate safeguards.
+We urge all users to consider the ethical implications of their applications and to implement appropriate safeguards.
 
-</details>
+---
 
-<details>
-<summary><strong>🖥️ Hardware Requirements</strong></summary>
+## 🖥️ Hardware Requirements
 
 | Precision | Minimum GPU Memory | Recommended GPU Configuration |
 |:---|:---|:---|
@@ -613,12 +688,11 @@ All users should consider the ethical implications of their applications and imp
 | **FP8** | 600 GB | 4× H100 80GB (tensor parallel) |
 | **INT4** | 300 GB | 4× A100 80GB (tensor parallel) |
 
-For inference with 1M context, additional memory is required for KV cache. Paged attention and offloading techniques available in vLLM and SGLang are recommended.
+For inference with 1M context, additional memory is required for KV cache. We recommend using paged attention and offloading techniques available in vLLM and SGLang.
 
-</details>
+---
 
-<details>
-<summary><strong>⚡ Performance Metrics</strong></summary>
+## ⚡ Performance Metrics
 
 | Metric | Value |
 |:---|:---|
@@ -631,10 +705,11 @@ For inference with 1M context, additional memory is required for KV cache. Paged
 
 *Measured on 8× H100 80GB with vLLM, batch size 1, BF16.*
 
-</details>
+---
 
-<details>
-<summary><strong>📚 Citation</strong></summary>
+## 📚 Citation
+
+If you use Step-5-Preview in your research, please cite:
 
 ```bibtex
 @misc{stepfun2026step5preview,
@@ -646,7 +721,27 @@ For inference with 1M context, additional memory is required for KV cache. Paged
 }
 ```
 
-</details>
+---
+
+## 📜 License
+
+Step-5-Preview is released under the **StepFun Community License**. See the [LICENSE](https://huggingface.co/SHSLab/Step-5-Preview-BF16/blob/main/LICENSE) file for full terms.
+
+> **⚠️ Usage Restrictions**
+>
+> • Commercial use is permitted under the StepFun Community License.
+> • Redistribution must include the license and attribution.
+> • See LICENSE for full details.
+
+---
+
+## 📬 Contact
+
+- **Hugging Face:** [SHSLab](https://huggingface.co/SHSLab)
+- **GitHub:** [github.com/stepfun-ai](https://github.com/stepfun-ai)
+- **Discord:** [Join our Discord](https://discord.gg/stepfun)
+- **Email:** [opensource@stepfun.com](mailto:opensource@stepfun.com)
+- **Website:** [stepfun.com](https://stepfun.com)
 
 ---