Menu

← Leaderboard

Qwen3.6-35B-A3B with Hermes Agent

NVFP4 (ModelOpt, safetensors) on vLLM 0.32.0. Tested

Task completion
100% (30/30)
Tool-call success
75.2%
Decode tok/s
73.7
Prefill tok/s
6631.8
Server memory
47.3 GB
Avg turns
3.7

Setup

Model
Qwen3.6-35B-A3B
Family / maker
Qwen · Alibaba Qwen
Parameters
35B (3B active)
Architecture
MoE
Weights
nvidia/Qwen3.6-35B-A3B-NVFP4
Quant
NVFP4 (ModelOpt, safetensors)
Engine
vLLM 0.32.0
Hermes Agent
v0.22.5
Tested
2026-10-24
Context
192K for all 10 tasks
Memory tier
≤128 GB (measured)

NVIDIA's official NVFP4 checkpoint of Qwen3.6-35B-A3B, served with vLLM. Replaces the earlier 'Qwen3.8 35B-A3B' placeholder, which has no official release.

Per-task results

  • t01: 3/3
  • t02: 3/3
  • t03: 3/3
  • t04: 3/3
  • t05: 3/3
  • t06: 3/3
  • t07: 3/3
  • t08: 3/3
  • t09: 3/3
  • t10: 3/3

Copy-paste config

1. Start the model server (vLLM)

Exact command used in our test. vLLM loads the official checkpoint nvidia/Qwen3.6-35B-A3B-NVFP4 (safetensors, no --trust-remote-code); we served a local snapshot of it.

vllm serve nvidia/Qwen3.6-35B-A3B-NVFP4 --served-model-name qwen3-6-35b-a3b-nvfp4 --host 127.0.0.1 --port 18081 --max-model-len 196608 --gpu-memory-utilization 0.40 --max-num-seqs 4 --kv-cache-dtype fp8 --enable-auto-tool-choice --tool-call-parser qwen3_xml --reasoning-parser qwen3 --quantization modelopt --attention-backend flashinfer --moe-backend marlin --enable-prefix-caching --limit-mm-per-prompt {"image":0,"video":0}

2. Point Hermes Agent at it

hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:18081/v1
hermes config set model.default qwen3-6-35b-a3b-nvfp4

Other Qwen setups