Underdog compresses Qwen 3.8-27B to 7.89GB, and its self-tested tool calling performance outperforms the original model.

Beating AI Express (Dongcha) News: a16z-backed local AI assistant Underdog has launched Saluki 27B, a model built on Alibaba's Qwen3.8-27B. The model uses approximately 2-bit quantization, cutting its size from ~54GB (full-precision version) to 7.89GB. According to officials, it can run on laptops with 16GB of RAM and supports deployment via existing tools like llama.cpp. The model weights are open-sourced under the Apache 2.0 license. Saluki leverages quantization work from ISTA-DASLab (Austrian Institute of Science and Technology). The lab previously compressed Qwen3.8-27B to ~8.4GB; Underdog further reduced the model size while focusing on optimizing tool call performance. In 120 tool call tasks selected from BFCL, Saluki answered 88 correctly, the ISTA-quantized version got 76 right, and the 54GB original version scored 84. In 50 real GitHub issue-fixing tasks, Saluki resolved 30, while the original version fixed 33. Officials noted that Saluki retains an average of 96% of the original model's performance across 9 benchmarks, though performance varies significantly across different tasks. Its scores on math competitions and complex reasoning tasks only reach ~83% and 85% of the original, respectively.