KRAFTON/A.X-K2-Raon-Speech-21B-A3B
Any-to-Any • 21B • Updated • 19.1k • 102
Geo Maciolek PRO
GeoMaciolek
AI & ML interests
None yet
Recent Activity
liked a model 2 days ago
KRAFTON/A.X-K2-Raon-Speech-21B-A3B liked a Space 5 days ago
pollen-robotics/microduck-simulator liked a model 7 days ago
superwhisper/s1-miniOrganizations
None yet
liked a
model 2 days ago
liked a
Space 5 days ago
liked a
model 7 days ago
Mostly into it! But- a bit heavy on the emoji (for *my* taste)
3
#1 opened 8 days ago
by GeoMaciolek
liked 2
models 8 days ago
updated a
model 8 days ago
published a
model 8 days ago
liked 2
models 8 days ago
liked a
model 9 days ago
How are people using this to generate subtitles?
#7 opened 9 days ago
by GeoMaciolek
Seems to work decently with a heretic LoRA too!
2
#2 opened 11 days ago
by GeoMaciolek
replied to salma-remyx's post 12 days ago
These are preliminary results reproducing the paper’s findings, so of course YMMV across workloads. n=50 and the 1.40× ceiling are stated explicitly, and we’re already asking maintainers whether it should be in-tree or a plugin. BWAP also doesn’t discard weights. The mask is periodically refreshed.
I don't make these accusations lightly, but, it seems from looking at their profile, AdrienneNoctis is probably a bot, largely or entirely LLM driven.
I'm not sure what reason people have for setting these things loose with instructions that seem to be "denigrate content and argue with people on huggingface" but that seems to be their modus operandi. This thread has several.
And, @adriennenoctis - please feel free to respond saying otherwise.
reacted to salma-remyx's post with 🔥🔥 12 days ago
Post
3530
Inspired by the methods described in "Batch-wise Adaptive Pruning" (arxiv 2608.14003, COLM '26), we implemented a training-free FFN-neuron-pruning knob for SGLang.
Authors were motivated by the reality that decode is HBM-bandwidth-bound; the gated MLP is the bulk of weights read per step. Threshold methods (TEAL/CATS) collapse under batching; BWAP's periodic top-k over a max-aggregated score keeps the shared batch mask stable.
The method is complementary to KV-sparsity (FFN-weight bandwidth vs KV read, context-length-independent).
Our implementation uses an adaptive mask under a captured graph (topology static = k-wide GEMM; mask change = between-replay buffer update via version-gated post_fill; prune steps replay, explore steps eager).
Preliminary Results GSM8K n=50, ±6pp: 7B dense 92% → ρ=0.5 84% (−8pp) at up to 1.40× (probe ceiling; ~10% realistic under the adaptive schedule; smaller models need lower ρ as accuracy scales with size).
Read more in the upstream issue: https://github.com/sgl-project/sglang/issues/35987
Authors were motivated by the reality that decode is HBM-bandwidth-bound; the gated MLP is the bulk of weights read per step. Threshold methods (TEAL/CATS) collapse under batching; BWAP's periodic top-k over a max-aggregated score keeps the shared batch mask stable.
The method is complementary to KV-sparsity (FFN-weight bandwidth vs KV read, context-length-independent).
Our implementation uses an adaptive mask under a captured graph (topology static = k-wide GEMM; mask change = between-replay buffer update via version-gated post_fill; prune steps replay, explore steps eager).
Preliminary Results GSM8K n=50, ±6pp: 7B dense 92% → ρ=0.5 84% (−8pp) at up to 1.40× (probe ceiling; ~10% realistic under the adaptive schedule; smaller models need lower ρ as accuracy scales with size).
Read more in the upstream issue: https://github.com/sgl-project/sglang/issues/35987
- 5 replies
liked 2
models 13 days ago
Notebook to replicate the training?
#2 opened 17 days ago
by GeoMaciolek