Menu

← Leaderboard

Gemma 4 26B-A4B with Hermes Agent

UD-Q4_K_M (GGUF) on llama.cpp commit a37cc46. Tested

Task completion
100% (30/30)
Tool-call success
66.2%
Decode tok/s
49.8
Prefill tok/s
3096.6
Server memory
17.7 GB
Avg turns
5.3

Other quants of this model

QuantScoreDecode tok/sPrefill tok/sServer memoryHermes
Q8_0 (GGUF) 30/30 47 2575.7 26.9 GB v0.22.5
UD-Q4_K_M (GGUF) (this page) 30/30 49.8 3096.6 17.7 GB v0.22.5

Same engine, same 10 tasks × 3 and the same context rules. Memory is the llama-server process at 64K context. Differences of a few points are within noise.

Setup

Model
Gemma 4 26B-A4B
Family / maker
Gemma · Google
Parameters
26B (4B active)
Architecture
MoE
Weights
unsloth/gemma-4-26B-A4B-it-GGUF
Quant
UD-Q4_K_M (GGUF)
Engine
llama.cpp commit a37cc46
Hermes Agent
v0.22.5
Tested
2026-10-24
Context
192K for all 10 tasks
Memory tier
≤24 GB (measured)

Per-task results

  • t01: 3/3
  • t02: 3/3
  • t03: 3/3
  • t04: 3/3
  • t05: 3/3
  • t06: 3/3
  • t07: 3/3
  • t08: 3/3
  • t09: 3/3
  • t10: 3/3

Copy-paste config

1. Start the model server (llama.cpp)

Exact command used in our test. Download the GGUF from unsloth/gemma-4-26B-A4B-it-GGUF into ./models/.

llama-server --model ./models/gemma-4-26B-A4B-it-UD-Q4_K_M.gguf --alias gemma-4-26b-a4b-q4 --host 127.0.0.1 --port 18080 --ctx-size 196608 --parallel 1 --n-gpu-layers 999 --flash-attn on --jinja

2. Point Hermes Agent at it

hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:18080/v1
hermes config set model.default gemma-4-26b-a4b-q4

Other Gemma setups