// benchmarks · qwen3.8-27b · single rtx 3090 · vllm 0.33.0
Qwen3.8-27B on a Single RTX 3090 — 78 Days of Measured Telemetry.
A castor stats ledger captured every
turn over 78 days (Sep 4 – Nov 21, 2026) on one RTX 3090
running Qwen3.8-27B-W4A16-AutoRound-fast
under vLLM 0.33.0 with MTP speculative decoding — 2.77 billion
tokens, with 93.8% of the 2.71B prompt tokens
served from VRAM via multi-turn session KV-cache reuse, and
$10.12 of electricity against a
$6,044.48 cloud-API counterfactual ($6,034.36 net saved).
Total tokens
2.77B
2,770,000,000 over 78 days
KV cache hit
93.8%
2.53B / 2.71B · session reuse
Prefill (effective)
3,247
tok/s effective prefill (warm + cold)
Net savings
99.83%
$10.12 power vs. $6,044.48 cf
The Complete Measured Ledger
Every figure below is a direct output of
castor stats aggregated over the
2026-09-04 → 2026-11-21 window.
Window
78 days
2026-09-04 → 2026-11-21
Prompt tokens
2,710,000,000
2.71B
Session-cached tokens
2,530,000,000
2.53B · 93.8% of prompt
Completion tokens
62,150,000
62.15M
Reasoning tokens
52,580,000
52.58M · 84.6% of completion
Autonomous turns
49,414
1,187 sessions · 228 tasks
Sandboxed tool calls
67,506
97% success
Surgical edit ratio
3.67×
surgical edit_file vs full write_file rewrites
Sustained decode
70.4 tok/s
78-day mean
Concurrent decode
260 tok/s
aggregate across n=4 concurrent · warm KV prefix
Effective prefill
3,247 tok/s
effective prefill (warm + cold)
GPU compute time
224h 56m
160h 30m session wall-clock
Electricity cost
$10.12
56.2 kWh @ $0.18/kWh · 262.5 W locked limit (350 W stock)
Cloud counterfactual
$6,044.48
Sonnet-tier $2/$10 API pricing
Net savings
$6,034.36
99.83% of cloud cost
$10.12 electricity (56.2 kWh) vs. $6,044.48 Sonnet-tier counterfactual — full 78-day window
vLLM 0.33.0 /metrics &
nvidia-smi
Single-day vLLM /metrics snapshot
(2026-11-21): 70.5% MTP draft
acceptance with a 3.45 mean acceptance
length, alongside the serving stack's
nvidia-smi readings from the same day.
- GPU
- NVIDIA GeForce RTX 3090
- VRAM used
- 23,869 / 24,576 MiB
- Power draw
- 244.6 W / 262.5 W limit
- Temperature
- 56 °C
- Driver
- 616.92
Max model len
150K
KV cache hit
93.8%
MTP acceptance
70.5%
mean accept len 3.45
Avg TTFT
3.22 s
VRAM Tiers: Same Kernel, Different Ceiling
The 24 GB+ row is the configuration this page benchmarks — vLLM 0.33.0 with Qwen3.8-27B-W4A16-AutoRound-fast and MTP speculative decode against a 150K KV pool. Smaller VRAM tiers run smaller (7B–14B) or more aggressively quantized models, not the full 27B W4A16 build.
| VRAM | Engine profile | Context | Decode throughput |
|---|---|---|---|
| 8–12 GB GPUs | llama.cpp / Ollama | 7B–14B coding models (W4A16 / Q4_K_M) | ~22–53 tok/s depending on model & quant |
| 16 GB GPUs | llama.cpp / Ollama | 14B–22B local models or heavily quantized 27B (Ternary-Bonsai-2-27B-PQ2_0) | ~38–53 tok/s |
| 24 GB+ Desktop GPUs (RTX 3090, RTX 4090, RTX A5000 Desktop, RTX 5090) | vLLM 0.33.0 + Qwen3.8-27B-W4A16 | 150K KV pool | 70.4 tok/s sustained n=1 · 260 tok/s n=4 concurrent |
Qwen3.8-27B — Vendor-Published Benchmark Scores
The model card (released 2026-08-14,
Apache-2.0 license) publishes the scores below.
Castor serves the W4A16-AutoRound-fast
quantization — ~¼ the weight footprint of BF16 (~14 GB vs. ~54 GB
BF16) on this 24 GB GPU.
SWE-bench Pro
61.7
LiveCodeBench v6
90.3
Terminal-Bench 2.1
73
OSWorld-Verified
84.3
How These Numbers Were Measured
Hardware Rig
Single NVIDIA GeForce RTX 3090 — 24 GB VRAM, 350 W stock TDP (power limit locked to 262.5 W for this run), 936 GB/s memory bandwidth, driver 616.92.
Serving Engine
vLLM 0.33.0 serving Qwen3.8-27B-W4A16-AutoRound-fast with MTP speculative decoding against a 150K-token KV pool.
78-Day Workload
Full castor stats ledger (Sep 4 – Nov 21,
2026): 49,414 autonomous turns,
67,506 sandboxed tool calls,
2.77B tokens, every entry from the local audit log.
Cloud Cost Baseline
Counterfactual priced at Sonnet-tier $2/$10 per 1M tokens with
$0.18/kWh residential electricity — reproduced from the
open-source castor stats command.
Run it Yourself. Read the Ledger.
Install Castor as an MCP server on your machine, point it at your
local engine, and run castor stats to
watch your own local token and cost ledger build from day one.