// benchmarks · qwen3.8-27b · single rtx 3090 · vllm 0.33.0

Qwen3.8-27B on a Single RTX 3090 — 78 Days of Measured Telemetry.

A castor stats ledger captured every turn over 78 days (Sep 4 – Nov 21, 2026) on one RTX 3090 running Qwen3.8-27B-W4A16-AutoRound-fast under vLLM 0.33.0 with MTP speculative decoding — 2.77 billion tokens, with 93.8% of the 2.71B prompt tokens served from VRAM via multi-turn session KV-cache reuse, and $10.12 of electricity against a $6,044.48 cloud-API counterfactual ($6,034.36 net saved).

Total tokens

2.77B

2,770,000,000 over 78 days

KV cache hit

93.8%

2.53B / 2.71B · session reuse

Prefill (effective)

3,247

tok/s effective prefill (warm + cold)

Net savings

99.83%

$10.12 power vs. $6,044.48 cf

the 78-day ledger · castor stats 01 / 05

The Complete Measured Ledger

Every figure below is a direct output of castor stats aggregated over the 2026-09-04 → 2026-11-21 window.

Window

78 days

2026-09-04 → 2026-11-21

Prompt tokens

2,710,000,000

2.71B

Session-cached tokens

2,530,000,000

2.53B · 93.8% of prompt

Completion tokens

62,150,000

62.15M

Reasoning tokens

52,580,000

52.58M · 84.6% of completion

Autonomous turns

49,414

1,187 sessions · 228 tasks

Sandboxed tool calls

67,506

97% success

Surgical edit ratio

3.67×

surgical edit_file vs full write_file rewrites

Sustained decode

70.4 tok/s

78-day mean

Concurrent decode

260 tok/s

aggregate across n=4 concurrent · warm KV prefix

Effective prefill

3,247 tok/s

effective prefill (warm + cold)

GPU compute time

224h 56m

160h 30m session wall-clock

Electricity cost

$10.12

56.2 kWh @ $0.18/kWh · 262.5 W locked limit (350 W stock)

Cloud counterfactual

$6,044.48

Sonnet-tier $2/$10 API pricing

Net savings

$6,034.36

99.83% of cloud cost

$10.12 electricity (56.2 kWh) vs. $6,044.48 Sonnet-tier counterfactual — full 78-day window

hardware snapshot · 2026-11-21 02 / 05

vLLM 0.33.0 /metrics & nvidia-smi

Single-day vLLM /metrics snapshot (2026-11-21): 70.5% MTP draft acceptance with a 3.45 mean acceptance length, alongside the serving stack's nvidia-smi readings from the same day.

nvidia-smi · snapshot 2026-11-21
GPU
NVIDIA GeForce RTX 3090
VRAM used
23,869 / 24,576 MiB
Power draw
244.6 W / 262.5 W limit
Temperature
56 °C
Driver
616.92
vLLM 0.33.0 · GET /metrics Qwen3.8-27B-W4A16-AutoRound-fast

Max model len

150K

KV cache hit

93.8%

MTP acceptance

70.5%

mean accept len 3.45

Avg TTFT

3.22 s

vram tier matrix 03 / 05

VRAM Tiers: Same Kernel, Different Ceiling

The 24 GB+ row is the configuration this page benchmarks — vLLM 0.33.0 with Qwen3.8-27B-W4A16-AutoRound-fast and MTP speculative decode against a 150K KV pool. Smaller VRAM tiers run smaller (7B–14B) or more aggressively quantized models, not the full 27B W4A16 build.

VRAM Engine profile Context Decode throughput
8–12 GB GPUs llama.cpp / Ollama 7B–14B coding models (W4A16 / Q4_K_M) ~22–53 tok/s depending on model & quant
16 GB GPUs llama.cpp / Ollama 14B–22B local models or heavily quantized 27B (Ternary-Bonsai-2-27B-PQ2_0) ~38–53 tok/s
24 GB+ Desktop GPUs (RTX 3090, RTX 4090, RTX A5000 Desktop, RTX 5090) vLLM 0.33.0 + Qwen3.8-27B-W4A16 150K KV pool 70.4 tok/s sustained n=1 · 260 tok/s n=4 concurrent
model card · 2026-08-14 · Apache-2.0 04 / 05

Qwen3.8-27B — Vendor-Published Benchmark Scores

The model card (released 2026-08-14, Apache-2.0 license) publishes the scores below. Castor serves the W4A16-AutoRound-fast quantization — ~¼ the weight footprint of BF16 (~14 GB vs. ~54 GB BF16) on this 24 GB GPU.

SWE-bench Pro

61.7

LiveCodeBench v6

90.3

Terminal-Bench 2.1

73

OSWorld-Verified

84.3

methodology 05 / 05

How These Numbers Were Measured

Hardware Rig

Single NVIDIA GeForce RTX 3090 — 24 GB VRAM, 350 W stock TDP (power limit locked to 262.5 W for this run), 936 GB/s memory bandwidth, driver 616.92.

Serving Engine

vLLM 0.33.0 serving Qwen3.8-27B-W4A16-AutoRound-fast with MTP speculative decoding against a 150K-token KV pool.

78-Day Workload

Full castor stats ledger (Sep 4 – Nov 21, 2026): 49,414 autonomous turns, 67,506 sandboxed tool calls, 2.77B tokens, every entry from the local audit log.

Cloud Cost Baseline

Counterfactual priced at Sonnet-tier $2/$10 per 1M tokens with $0.18/kWh residential electricity — reproduced from the open-source castor stats command.

Run it Yourself. Read the Ledger.

Install Castor as an MCP server on your machine, point it at your local engine, and run castor stats to watch your own local token and cost ledger build from day one.

// quickstart · sixty seconds from zero to telemetry

Run it. Watch it. Verify it.

bash · castor-cli
  1. # 1. Install & register MCP tools with Claude Code / Antigravity

    $ npx -y mcp-castor install --client all
  2. # 2. Verify your local serving engine (vLLM / llama.cpp / Ollama)

    $ castor server status
  3. # 3. Inspect live token throughput, KV cache hit rate, and savings

    $ castor stats

CLAUDE CODE · ANTIGRAVITY · CURSOR · ANY MCP HOST | vLLM · llama.cpp · SGLANG · OLLAMA · LM STUDIO