Qwen3.8-27B GSQ-RCO GGUF Quantizations
GGUF quantizations of Qwen3.8-27B at four sizes, with a vision projector for multimodal use.
Verified
Core facts
Read by a script from the official API, not rewritten by a model.
- License
- apache-2.0
- Maintainer
- ISTA-DASLab
- Verified
What it is
This repository provides GGUF quantizations of Qwen3.8-27B at four sizes. It also provides the model's vision projector for multimodal use. In contrast to uniform quantization, which applies a single quantization type to all weight tensors, each model here assigns a separate quantization type to every tensor. The resulting files are standard GGUF and run unmodified in llama.cpp, Ollama, and LM Studio.
Main features
The assignment is obtained by a gradient-based search that allocates precision according to per-tensor sensitivity, subject to a total size budget.
GSQ is post-training scalar quantization that jointly learns the per-coordinate grid assignments and the per-group scales via a Gumbel-Softmax relaxation.
RCO assigns one of K quantization types to each of N tensors under a total size budget.
RCO reformulates the budget constraint as a smooth Riemannian manifold in logit space, which permits gradient-based optimization directly on the task loss while enforcing the budget exactly, without constraint-specific hyperparameter tuning.
Each quantization also ships an optional mtp build, about 0.35 GB larger, that carries the model's Multi-Token Prediction head for speculative decoding in llama.cpp.
Getting started
The projector was converted directly from the base checkpoint and verified against these quantizations.
The IQ3_XXS file is 3.00 bpw and 10.1 GB.
# download (requires: pip install -U "huggingface_hub[cli]")
hf download ISTA-DASLab/Qwen3.8-27B-GSQ-RCO-GGUF Qwen3.8-27B-GSQ-RCO-IQ3_XXS.gguf --local-dir .
llama-cli -m Qwen3.8-27B-GSQ-RCO-IQ3_XXS.gguf -p "Explain mixed-precision quantization." -ngl 99hf download ISTA-DASLab/Qwen3.8-27B-GSQ-RCO-GGUF mmproj-Qwen3.8-27B-BF16.gguf --local-dir .
llama-mtmd-cli -m Qwen3.8-27B-GSQ-RCO-IQ3_XXS.gguf \
--mmproj mmproj-Qwen3.8-27B-BF16.gguf \
--image photo.jpg -p "Describe this image."ollama run hf.co/ISTA-DASLab/Qwen3.8-27B-GSQ-RCO-GGUF # pick the file matching your memory budgetWho it is for
All models are evaluated against the BF16 base model and the Unsloth Dynamic (UD) quantizations of the same base model. The results report perplexity on wikitext2, C4, and FineWeb-Edu, the average over five zero-shot tasks, recovery (zero-shot average relative to BF16), and three reasoning and generation benchmarks: AIME25, GPQA-Diamond, and LiveCodeBench v6. The publisher reports that GSQ-RCO IQ3_S (3.50 bpw) leads UD-IQ3_S by 3.33 points on AIME25 and 1.71 on LiveCodeBench while being 0.2 GB smaller, though UD holds GPQA-Diamond by 0.51.
The publisher reports that the IQ3_XXS file (3.00 bpw, 10.1 GB) already matches the base model on AIME25, and that at matched file size (8.4 GB) IQ2_XS leads UD-IQ2_S by 10.00 points on AIME25, 8.59 on GPQA-Diamond, and 4.57 on LiveCodeBench v6. The mtp builds have their own allocation dumps, which list the same per-tensor assignment as the base model plus the 15 tensors of the MTP head. These quantized weights inherit the license of the base model.
The GSQ-RCO tooling is released by the Deep Algorithms and Systems Lab under its repository license.
Related tools
Related tools listed on this site.
updream
Described on its site as an AI video creation product developed in-house by Bilibili for its uploaders, stressing a lightweight and intelligent creation experience.
抠抠图
Koukoutu is an image-processing tool platform. Its official site describes batch and interactive background removal, image generation/editing, product images, upscaling and restoration, plus image translation, layer separation and vectorization.
Recraft AI
Recraft is an image-model and design platform for generating and editing images, vectors, custom styles and mockup assets.
QuiverAI
QuiverAI generates editable SVG vector graphics, describing text/image-to-SVG, editing and animation with platform and developer entry points.
腾讯混元大模型
An official Tencent Hunyuan project entry. This source covers Hunyuan-Large MoE models, pretrained and instruction variants, and inference and fine-tuning guidance.
讯飞星辰MaaS
The public primary description positions iFLYTEK Xingchen MaaS as a developer-facing model fine-tuning service, covering data processing, model optimization, evaluation and deployment/hosting. This review did not run those console workflows.
CleverSee
Alibaba Cloud’s enterprise AI search and question-answering platform, describing web retrieval, private multimodal data and configurable applications.
01Agent
AI creation workspace for marketing teams. Its site describes articles, visual posts, posters, presentations, reports, e-commerce assets and videos from briefs, with further editing on an infinite canvas and reusable skill templates.
魔珐星云
Xingyun is an open digital-human and embodied-interaction platform describing perception, knowledge, speech/action expression, device-side rendering and real-time SDKs.
Vivix
Real-time interactive multimodal platform describing responsive characters, scenarios and worlds with model, research and developer entry points.
LightX2V Studio
LightX2V Studio’s public page positions it as an AI digital-human video and multimodal-content platform, starting from character assets, narration scripts and AI voices and organizing work through projects, Canvas and reusable workflows.
MiniMax
Multimodal foundation-model company whose site introduces text, video, speech and music models plus products such as MiniMax Code, Design, Audio and Talkie, and an API entry.
Sources
This page is written from the official sources below. Open them to check the original text.
- Hugging Face model cardhttps://huggingface.co/ISTA-DASLab/Qwen3.8-27B-GSQ-RCO-GGUFFetched
Verified ·
This page is written by AI from the official sources above and checked automatically. This site has not tested it. Found a mistake? Tell us.