Hmm. For now, there seem to be quite a few reports of successful runs with similar setups:
The available evidence suggests that a 35B-class hybrid MoE with a 100k+ context is feasible on a single RTX 5090 in some configurations. I would not conclude that the GPU is categorically too small.
For this exact AEON-7/Ornith-1.0-35B-AEON-Ultimate-Uncensored-NVFP4 checkpoint, however, I could not find a fully reproducible public report that includes all four of the following:
- an RTX 5090,
- vLLM with the original
compressed-tensors Safetensors checkpoint,
- the complete pinned environment and launch command,
- an actual 100k-token input completing successfully.
The closest exact-checkpoint report I found is a community test of the 23.7 GB AEON model on an RTX 5090 with vLLM/NVFP4, reporting about 190 tok/s. That is useful evidence that the model can at least load and generate on this GPU, but the article does not give the exact vLLM build, full launch command, selected kernels, or longest prompt tested, so I would not treat it as a 100k reproduction.
The route I would use first
I would avoid starting from the full DGX Spark deployment recipe. Instead:
- Pin one vLLM image or commit and one model revision.
- Start with plain vLLM serving:
- text requests only,
- one sequence,
- 16k or 32k context,
- no speculative decoding,
- no prefix caching,
- no forced experimental FP4/MoE backend,
- no multimodal request.
- Confirm that it:
- loads all weights,
- finishes profiling and warm-up,
- reaches
server ready,
- returns a short, coherent response.
- Record which linear, MoE, attention, and GatedDeltaNet/Mamba backends vLLM actually selected.
- Increase the real prompt length, for example 32k → 64k → 100k, without changing several other options at the same time.
- Only then compare memory-saving or performance branches such as FP8 KV cache, Marlin, prefix caching, vision, or speculative decoding.
A compatibility-probe command would therefore be closer to this shape than to the full Spark command:
export VLLM_LOGGING_LEVEL=DEBUG
vllm serve <MODEL_PATH_OR_ID> \
--quantization compressed-tensors \
--max-model-len 32768 \
--max-num-seqs 1 \
--max-num-batched-tokens 4096 \
--gpu-memory-utilization 0.90 \
--trust-remote-code
This is deliberately not presented as a known-good final command. Some flags and defaults have changed between recent vLLM builds, and the correct next branch depends on where the current run fails.
The three most useful pieces of information are the complete command, the exact vLLM/container version or digest, and the first exception together with the preceding backend-selection lines.
| Where it fails |
First place I would look |
| Before or during weight loading |
Checkpoint metadata, compressed-tensors loader compatibility, model revision, missing/skipped parameter names |
| During compilation, profiling, or warm-up |
VRAM headroom, CUDA graphs, FP4/MoE workspace allocation, selected backend, or simply a very long cold start |
| Server starts, but the first request fails |
Forward kernel, GatedDeltaNet state initialization, speculative decoding, attention backend, or a correctness regression |
| Short prompts work, but long prompts fail |
Actual prefill memory, temporary workspace, KV/recurrent-state allocation, chunking, and reserved output length |
| It works, but is unexpectedly slow |
Native SM120 FP4 path versus Marlin or another fallback; dense and MoE layers may use different paths |
Comparable reports, and what each one actually demonstrates
Closest exact-checkpoint report
A community benchmark of the exact AEON Ornith NVFP4 checkpoint says that it fits on an RTX 5090 and generates at approximately 190 tok/s under vLLM.
That is quite close to the hardware/checkpoint/runtime combination in this thread. Its limitations are that it does not publish:
- the exact vLLM commit or container digest,
- the complete launch command,
- the selected linear and MoE kernels,
- the configured context length,
- the longest input actually processed.
I would interpret it as evidence against “this checkpoint can never run on a 5090,” but not as evidence that this exact artifact has already been reproduced at 100k.
Same AEON trunk, different runtime
The GGUF/MTP conversion of the AEON Ornith checkpoint reports validation on an RTX 5090 with recent llama.cpp, including initialization at a 262k context and BLACKWELL_NATIVE_FP4 = 1.
This is a strong control for:
- the RTX 5090 hardware,
- the general model family,
- fitting the model and a long context in 32 GB under a lean runtime.
It is not a control for:
- the original Safetensors loader,
compressed-tensors,- vLLM’s hybrid cache manager,
- vLLM’s NVFP4 or fused-MoE backend selection.
The GGUF artifact also includes a compatible grafted MTP block, so it is not byte-for-byte the same deployable artifact.
Same GPU and model scale, different quantization
A Hugging Face discussion documents a Qwen3.5-35B-A3B GPTQ-Marlin configuration on an RTX 5090 using:
- vLLM,
- GPTQ-Marlin,
- FP8 KV cache,
--max-model-len 131072,- approximately 31.3 GB VRAM.
This is useful evidence that a 35B-A3B hybrid model and a nominal 131k context can fit in a 5090 vLLM deployment.
It does not establish compatibility of the AEON compressed-tensors NVFP4 checkpoint. The same discussion also contains another user who could load the model but encountered an OOM on the first request even after lowering max-model-len, which is a good reminder that “server initialized” and “request completed” are separate milestones.
Similar uncensored 35B compressed-tensors checkpoint
The model card for lyf/Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive-NVFP4 is especially relevant because it documents:
- an RTX 5090,
- vLLM,
compressed-tensors,- Marlin NVFP4 GEMM,
- FP8 KV cache,
- a text-only
--max-model-len 100000 command,
--max-num-seqs 1.
Its explicitly described real-device smoke test uses a 4096 context, while the 100k section is a proposed long-context serving configuration. I could not find an accompanying actual 100k input log, TTFT measurement, or independent reproduction, so I would treat it as a promising configuration branch rather than conclusive evidence.
It is also not the same checkpoint. Its quantization recipe, excluded modules, source conversion pipeline, and retained BF16 components differ from the AEON artifact.
Another Ornith NVFP4 vLLM route
ressl/Ornith-1.0-35B-NVFP4 publishes a vLLM route using ModelOpt, FlashInfer attention, and Marlin MoE with a 262k configured context. That shows that the Ornith/Qwen hybrid architecture is not inherently outside vLLM’s scope.
However, it is a different NVFP4 export format, and its documented validation hardware is not the same 32 GB RTX 5090 environment. It is therefore more useful as an architecture/backend reference than as a direct reproduction.
Evidence hierarchy
For future comparisons, I would rank reports approximately like this:
- Same artifact, GPU, runtime, pinned environment, and real 100k input log.
- Same artifact, GPU, and runtime with verified short generation.
- Same architecture, GPU, runtime, and quantization format.
- Same GPU and model size with another quantization.
- Same architecture and GPU under another runtime.
- A command that configures 100k but does not show a real 100k request.
- A model card stating a maximum context without runtime logs.
Most current reports are useful, but they occupy different levels in that hierarchy.
Why the DGX Spark recipe is not a neutral RTX 5090 baseline
The AEON model card describes this checkpoint as a 23.7 GB, MLP-only, weight-only W4A16 NVFP4 quantization:
- MoE and shared-expert MLP weights: NVFP4,
- full-attention projections: BF16,
- GatedDeltaNet/SSM path: BF16,
- vision tower: BF16,
- routers, embeddings,
lm_head, norms: BF16.
Therefore, 23.7 GB is the checkpoint footprint, not the complete peak VRAM requirement of vLLM.
The model-card quickstart enables several things together:
262144 maximum model length
float32 Mamba/GatedDeltaNet cache
multimodal image/video support
FlashAttention
chunked prefill
prefix caching
reasoning and tool parsers
The more detailed DGX Spark QuickStart explicitly says that it is built and tuned for a DGX Spark with 128 GB unified memory. It also uses GB10-specific settings and an AEON container, including:
TORCH_CUDA_ARCH_LIST=12.1a
CUTE_DSL_ARCH=sm_121a
The guide itself says that other GPUs may need different values. Its --gpu-memory-utilization, concurrency cap, DFlash buffers, BF16 KV requirement, vision configuration, and 262k context were chosen as one integrated Spark deployment profile.
That recipe is valuable as documentation of a validated Spark setup, but it introduces too many variables to be the first compatibility test for a 32 GB RTX 5090.
There is also a useful historical example in the AEON repository: an earlier DFlash combination failed during engine initialization with unify_kv_cache_spec_page_size. The current QuickStart now recommends an all-full-attention drafter or a hotfix for the older sliding-window drafter.
That does not imply that the target model itself is broken. It shows that on this hybrid architecture, adding a drafter changes the cache-page contract and can create an engine-initialization failure unrelated to the basic ability to load the model.
For a first test on the 5090, I would therefore leave out:
- DFlash or another speculative drafter,
- prefix caching,
- vision requests,
- high concurrency,
- forced Spark architecture variables,
- experimental backend environment variables.
Those features can be added later after a plain short-context request works.
Four separate compatibility questions are hiding behind “NVFP4 support”
1. Checkpoint/exporter compatibility
NVFP4 is not a complete runtime contract.
Nearby artifacts use several different paths:
compressed-tensors,- NVIDIA ModelOpt,
- GGUF,
- GPTQ-Marlin,
- W4A16 weight-only NVFP4,
- W4A4 weight-and-activation NVFP4,
- expert-only quantization,
- broader quantization with explicit exclusion lists.
The AEON checkpoint was exported with llm-compressor to the compressed-tensors nvfp4-pack-quantized format. A ModelOpt NVFP4 checkpoint or a GGUF labeled NVFP4 can use different parameter names, scales, packing, and runtime loaders.
If failure occurs during weight loading, the most informative clues are usually:
- the exact model revision,
quantization_config from config.json,- the vLLM and
compressed-tensors versions,
- the first missing, unexpected, or skipped parameter name,
- whether all shards downloaded successfully.
A successful ModelOpt or GGUF run should not be treated as proof that the compressed-tensors loader path is healthy.
2. Kernel/backend selection
The current vLLM linear-kernel reference lists multiple NVFP4 execution paths, including:
- vLLM CUTLASS,
- FlashInfer b12x for SM120+,
- other FlashInfer wrappers,
- Marlin W4A16,
- software emulation.
Dense linear layers and fused MoE layers may also select different backends. “FlashInfer enabled” or “NVFP4 detected” is therefore not enough information by itself.
A current open issue shows an RTX 5090 ModelOpt mixed-NVFP4 checkpoint routing W4A16 layers through Marlin, while logging a warning that sounds as though the GPU itself lacks native FP4 support. The issue reporter’s analysis is that this may be a path-selection or messaging problem rather than an actual RTX 5090 hardware limitation.
So I would read startup logs literally:
Detected quantization format
Selected linear backend
Selected fused-MoE backend
Selected attention backend
Selected GDN/Mamba implementation
Any fallback warning
If a Marlin path works, that is still a useful functional baseline. Native FP4 and Marlin can be compared later for speed and memory rather than making the native path a prerequisite for the first successful request.
3. Runtime combination
For SM120 and rapidly changing Qwen/Ornith support, “latest” or “nightly” is not reproducible enough.
The practical runtime contract includes at least:
vLLM commit or image digest
PyTorch build
CUDA build
FlashInfer version
Transformers version
driver version
Linux versus WSL2
model revision
A new build can fix a loader problem while introducing a backend or workspace regression. Conversely, an older stable release may predate architecture support.
I would pin one known environment for each comparison and change only one layer at a time.
4. Memory and long-context execution
The runtime needs more than checkpoint weights:
- unquantized BF16 modules,
- KV cache for full-attention layers,
- GatedDeltaNet/Mamba recurrent and convolution state,
- activations,
- compilation and CUDA-graph allocations,
- MoE workspaces,
- long-prefill temporary buffers,
- multimodal encoder buffers,
- speculative draft/verify buffers.
vLLM has a dedicated Hybrid KV Cache Manager because models mixing full attention with Mamba-like layers need multiple cache groups and page-layout rules. The documentation also warns that the feature is still evolving.
This is why a simple calculation such as:
32 GB - 23.7 GB checkpoint = 8.3 GB for context
is not sufficient.
A recent, version-specific vLLM issue illustrates the point: flashinfer_b12x MoE static workspace allocation reportedly combined about 21.94 GiB of model weights with roughly 8.5 GiB of static workspace on an RTX 5090, causing startup OOM. That does not prove the same backend is causing the failure here, but it demonstrates that backend workspace can consume almost all apparent post-weight headroom.
A conservative validation ladder
Stage 1: Establish a short-context baseline
The first target is not 100k. It is:
all weights loaded
profiling completed
server ready
one short request completed
output looked coherent
For this stage:
- use one sequence,
- use 16k or 32k,
- send text only,
- omit speculative decoding,
- omit prefix caching,
- leave backend selection on automatic unless the logs identify a known bad path,
- avoid adding FP8 KV and several backend overrides simultaneously.
If a graph-related crash points near CUDAGraph replay, the vLLM troubleshooting guide recommends --enforce-eager as a diagnostic way to disable CUDAGraph and expose the underlying CUDA operation. I would use it only as an A/B test, not necessarily as the final performance configuration.
Stage 2: Distinguish a slow warm-up from a crash
A recent open vLLM report for a 35B NVFP4 model on an RTX 5090 measured approximately:
- 18 seconds for weight loading,
- 43 seconds for first-run compilation,
- 210 seconds for KV profiling and warm-up,
- 4.6 minutes total cold start.
That timing is not a standard for AEON and may be tied to that particular build. It does show that a few minutes with no ready endpoint can be warm-up rather than a dead process.
Useful distinctions are:
Process still consuming GPU/CPU and logging progress
versus
Process exited with a traceback
versus
Process alive but no resource activity for a long period
Stage 3: Probe cache capacity
Current vLLM supports:
--max-model-len auto
or:
--max-model-len -1
to search for the largest configured model length that fits the available cache memory. See the vllm serve reference and the auto-fit implementation.
This is useful as a capacity probe, but it does not prove that a real prompt at that length can finish. Auto-fit reasons about the cache allocation model; a real long prefill can still need temporary workspace or hit a different kernel shape.
Stage 4: Send real inputs of increasing length
I would distinguish the following claims:
| Claim |
Evidence needed |
| “100k is configured” |
Startup says max_model_len=100000 |
| “100k cache fits” |
Engine reports sufficient cache capacity |
| “100k prefill works” |
An actually tokenized ~100k prompt completes |
| “100k plus output works” |
The prompt completes with the required output-token reservation |
| “It is stable” |
The test succeeds repeatedly, not once |
| “The model uses 100k effectively” |
Retrieval/reasoning checks at long distances |
A practical sequence is:
2k → 16k → 32k → 64k → 100k
For each point, record:
actual tokenizer count
requested output tokens
TTFT
generation speed
peak VRAM
backend-selection lines
success or first exception
Remember that max_model_len covers prompt plus generated output. A 100k prompt with a several-thousand-token answer needs more than a nominal 100k combined limit.
Stage 5: Try a text-only FP8 KV branch if needed
The official vLLM quantized-KV documentation explains that FP8 KV can substantially reduce cache memory and allow more tokens, but also documents different scaling/calibration modes and backend-specific behavior.
For this exact model, the AEON deployment documentation says that vision and DFlash use BF16 KV because of their non-causal attention path. Therefore I would treat FP8 KV as a separate text-only experiment, not append it blindly to the full multimodal/DFlash command.
A useful comparison would be:
default KV, 32k
default KV, maximum stable length
FP8 KV, 32k correctness control
FP8 KV, maximum stable length
If FP8 KV extends the limit, verify output quality with a few fixed prompts rather than testing only whether the server remains alive.
Stage 6: Add features back one at a time
After 100k text-only succeeds, possible additions are:
- prefix caching,
- tool/reasoning parsers,
- multimodal inputs,
- speculative decoding,
- higher concurrency.
Each changes a different part of memory or execution. Adding all five at once would make a new failure hard to attribute.
How I would interpret common failure shapes
Weight-loader errors
Examples include:
missing parameter
unexpected parameter
skipped weights
input_scale not found
shape mismatch
Likely branches:
- wrong or partially updated model revision,
- vLLM loader regression,
compressed-tensors metadata/version mismatch,- a parameter-renaming change in Transformers or the model implementation,
- a checkpoint exported for a different quantization loader.
Useful comparison:
- same environment, another known
compressed-tensors NVFP4 checkpoint;
- same target checkpoint, another pinned vLLM build.
If another compressed-tensors model also fails, the environment becomes more suspect. If only AEON fails, the checkpoint/loader contract becomes more suspect.
OOM during profiling or warm-up
This does not necessarily mean the weights themselves do not fit.
Possible consumers include:
- activation profiling,
- CUDA graph capture,
- FP4/MoE static workspace,
- cache initialization,
- other processes already using VRAM.
Useful branches:
Try with max-num-seqs=1
Lower max-model-len
Compare --enforce-eager
Check the selected MoE backend
Check nvidia-smi before launch
Do not enable speculative decoding
Changing gpu-memory-utilization upward is not always a solution. Some backend or speculative allocations are not fully represented by the KV-cache budget, so leaving no headroom can move the OOM from startup to the first request.
unify_kv_cache_spec_page_size or hybrid-cache assertions
This points away from a simple checkpoint-size explanation.
The AEON DFlash issue is a concrete example where the target and a sliding-window drafter produced incompatible cache-page requirements. The revised guide changes the drafter rather than reducing context.
For the first baseline, remove the speculative configuration and prefix caching. If plain serving works, reintroduce the drafter only with the currently recommended pairing.
Server ready, then first-request OOM or CUDA error
Possible causes include:
- forward-time temporary allocations,
- a kernel path that was not exercised during loading,
- GatedDeltaNet state initialization,
- multimodal encoder work,
- speculative verification buffers,
- CUDA graph replay.
For a device-side assert or illegal memory access, the official troubleshooting guide suggests:
export CUDA_LAUNCH_BLOCKING=1
to identify the CUDA kernel more accurately.
It also documents:
export VLLM_LOGGING_LEVEL=DEBUG
for additional logging.
VLLM_TRACE_FUNCTION=1 is available as a last resort, but the documentation warns that it can slow generation by more than 100Ă—, so it is not a first-line diagnostic.
Short inputs work, long prefill fails
That pattern makes a basic loader failure less likely.
Likely branches:
- cache capacity estimate versus real prefill allocation,
- chunked/continuation-prefill workspace,
- output-token reservation,
- GDN/Mamba state layout,
- a kernel shape used only at larger token counts.
Reduce the problem to the first reproducible failing length. A stable boundary such as “64k completes, 80k fails during prefill” is more actionable than “100k does not work.”
It generates, but output is empty, corrupted, or unstable
Do not use “the server returned HTTP 200” as the only correctness check.
A small control set can catch silent kernel or scale problems:
- a deterministic copy task,
- a short arithmetic or known-answer task,
- the same prompt repeated with low-temperature sampling,
- the same prompt under another backend,
- comparison with a GGUF or GPTQ version.
Native SM120 FP4, Marlin, and another runtime can then be compared for both output and speed.
It works, but Marlin warnings appear
The current vLLM code contains multiple NVFP4 kernels, and Marlin is a legitimate W4A16 implementation. The open SM120/Marlin fallback issue shows that a warning can describe the selected path poorly enough to sound like the RTX 5090 itself lacks FP4.
I would separate:
Functional success
Correct output
Memory use
Prefill speed
Decode speed
Native versus fallback backend
A working Marlin baseline is more useful than blocking all progress while searching for a native path. The native backend can be evaluated afterward.
Useful controls and alternative routes
These are controls for isolating layers, not claims that one model or runtime is universally better.
| Control |
What it tests |
What it does not test |
Exact AEON GGUF under llama.cpp |
GPU, model family, long-context feasibility under a lean runtime |
Safetensors loader, compressed-tensors, vLLM kernels/cache |
| Qwen3.5 35B GPTQ-Marlin under vLLM |
5090 capacity, vLLM hybrid serving, FP8 KV, 131k configuration |
NVFP4 loader and kernels |
lyf uncensored 35B compressed-tensors model |
5090 + compressed-tensors + Marlin + FP8 KV branch |
Exact AEON export and verified real 100k input |
| Another recent compressed-tensors NVFP4 checkpoint |
General environment compatibility |
Ornith-specific checkpoint behavior |
| Same AEON checkpoint on another pinned vLLM build |
Version regression |
Hardware-independent checkpoint validity |
| Same target with speculation disabled |
Base target path |
DFlash/MTP performance path |
A sensible three-step comparison could be:
- Start a known recent
compressed-tensors NVFP4 model at 16k.
- Start AEON Ornith in the same environment at 16k with no extra features.
- Increase only AEON’s real prompt length.
Possible interpretations:
Neither model loads
→ environment, driver, vLLM, CUDA, or FlashInfer branch
Control loads, AEON fails while loading weights
→ AEON checkpoint/revision/loader branch
Both generate at 16k, AEON fails only at long input
→ memory, hybrid cache, or long-prefill branch
AEON works under GGUF but not Safetensors/vLLM
→ vLLM/checkpoint/backend path, not a general GPU impossibility
A compact reproducibility template
Even a partial result in this format would make comparisons with other reports much easier:
GPU:
VRAM:
OS / WSL:
Driver:
Container image and digest:
vLLM version or commit:
PyTorch build:
CUDA build:
FlashInfer version:
Transformers version:
Model ID:
Model revision / commit:
Download date:
Complete launch command:
Resolved architecture:
Detected quantization:
Selected linear backend:
Selected MoE backend:
Selected attention backend:
Selected GDN/Mamba backend:
Checkpoint weight memory:
Non-Torch memory:
Activation/profile memory:
KV cache capacity:
Cold-start duration:
Text-only or multimodal:
Speculative decoding:
Prefix caching:
KV cache dtype:
Mamba cache dtype:
max_model_len:
max_num_seqs:
max_num_batched_tokens:
Actual prompt token count:
Requested output tokens:
Result:
Peak VRAM:
First failing prompt length:
First exception:
The key is to record the first meaningful failure, not every later process-shutdown traceback. Once the engine core fails, API-server and multiprocessing errors often follow and can hide the original cause.
Caveats for interpreting the reports
A few distinctions seem important:
- RTX 5090 hardware FP4 support does not guarantee that a particular checkpoint/runtime combination selects a native SM120 kernel.
- A Marlin fallback warning does not necessarily mean that the GPU lacks FP4 hardware.
- A GGUF success does not prove that a Safetensors
compressed-tensors checkpoint will load.
- A GPTQ success does not prove NVFP4 loader or kernel compatibility.
- A 100k
max_model_len setting does not prove that a real 100k prompt completed.
- A successful cache allocation does not prove that long prefill has enough temporary workspace.
- A server reaching
ready does not prove that the first request will fit.
- A short response does not prove long-context stability or correctness.
- A model-card command is useful evidence, but weaker than a complete log with an actual token count.
- A DGX Spark/GB10 recipe should not be copied with its architecture environment variables into an RTX 5090 setup without checking what the pinned build expects.
- “Use the newest nightly” is not a reproducible fix; an image digest or commit is much more useful.
- Native FP4 is not automatically the most stable or lowest-memory route for every current vLLM build.
Finally, there is a difference between allocatable context and effective context. After an actual 100k prompt runs reliably, a small retrieval/control test can verify that the model still uses information near the beginning of the context. Projects such as NVIDIA RULER were created specifically because claimed context length, successful execution length, and effective usable context are not always the same.
So, based on the public evidence, I would not give up on the RTX 5090. The most conservative interpretation is:
The hardware appears capable of closely related 35B/100k configurations, and there is at least one short-generation report for this exact AEON checkpoint on a 5090. What is still missing is a pinned, reproducible vLLM configuration for the exact checkpoint with an actual 100k input.
The shortest path forward is therefore not another large matrix of random flags. It is one minimal 16k/32k baseline, the first real exception and selected-backend lines, followed by one-variable-at-a-time expansion toward 100k.