fp8 version or custom quants?
#3
by froilo - opened
would be nice to have some quants to run it on consumer HW like 16GB VRAM
there are plenty of T5 quants on HF
also flux quants
also qwen2-vl
just no TF skills and time here ;(
Look on CivitAI I have posted a version that is q4 and q8 as well as one that works in the diffusion models and checkpoints folder in ComfyUi
Look on CivitAI I have posted a version that is q4 and q8 as well as one that works in the diffusion models and checkpoints folder in ComfyUi
i went to ur civitai page and you have no models shared,
so what did u mean?
i found this quant though https://civitai.com/models/982213/qwen-2vl-flux
yeah that's the one, I should've linked it sorry but yeah that one has q4 and q8 available