VeloBenchmark

What it does

An instrument, not a scoreboard

Every number is computed server-side from real streaming token timing and snapped to the provider's authoritative usage counts when reported. The UI is a window onto the same data the reports store. Token timings are stamped as bytes arrive from the engine, so a slow browser tab delays what you see — never what is measured.

Live instruments

Rolling decode tok/s, TTFT, inter-token latency, min/median/max and stability — streamed per token while the answer generates, with PNG/PDF export of the panel.

AI-assisted regime analysis

A helper LLM classifies the generated output as it streams — prose, code, math, JSON, reasoning — and every chart splits by those regimes, so a model slowing down on math and speeding up on code is visible inside a single answer.

Repeatable suites

A visual builder with five step types — sections, prompts, exact context fills, fixed-shape bench requests and vision steps — plus built-in reference suites and a JSON editor.

Concurrent runner

N workers walk the same test behind a step barrier. Per-worker snapshots live, and the report gains a Requests + Σ decode timeline: your effective concurrency throughput.

Reports & comparisons

Per-session analytics — throughput, latency distributions, token composition, speculation diagnostics — and persistent side-by-side comparisons of any sessions, exportable to PNG/PDF or copyable as a plain-text report.

Telemetry receiver

A built-in OTLP/HTTP-JSON receiver turns a serving engine's OpenTelemetry stream into live per-stream panels — and one click records a rolling window into a permanent session report.

AI-assisted classification

Regimes, labelled as they stream

A helper model tags the generated output — prose, code, math, JSON, reasoning — token group by token group. The decode timeline tints by regime, sessions carry their regime mix, and every report splits throughput and latency per regime, so "fast" stops being a single misleading average.

Measurement vocabulary

The metrics the field actually uses

VeloBenchmark speaks the ecosystem's language: the same definitions as vLLM, AIPerf and Artificial Analysis, computed from the stream alone where possible and labelled estimate where they are inferred. Speculative-decoding behaviour surfaces as an acceptance-rate estimate and a speculation-depth distribution — from bimodal inter-token latency, without server counters.

TTFT / TTST / TTFO decode tok/s rolling + final TPOT decode-normalised ITL p50 / p90 / p99 ICL jitter prefill t/s = ISL / TTFT acceptance α estimate spec depth distribution goodput vs SLO regimes AI-classified per token group

How each metric is computed →

Live stats panel in motion: decode rate, responsiveness, progress, timeline and histograms updating during a stream

The live stats deck — identical on the chat page and on every telemetry panel.

Two products, one lineage

Measure it, then run it

The benchmarking and live-stats console. Chat with live instruments, build test suites, scale them to concurrency, compare sessions, and front your serving engine's OpenTelemetry stream with the built-in OTLP receiver. One binary; the whole UI embedded; settings and sessions stored server-side.

Rust + Angular AGPL-3.0 Linux · macOS

Documentation Repository

veloGB10

A from-scratch Rust + CUDA inference engine for NVIDIA GB10 systems (DGX Spark & compatible), documented on its own site at veloengine.ai.

Rust + CUDA Apache-2.0 sm_121

veloengine.ai Repository

Inside the console

Every screen is a report in the making

Open source is the product. Services make it fit.

VeloBenchmark and veloGB10 are free forever under AGPL-3.0 and Apache-2.0. When you need it integrated into your pipeline, extended for your workload, or tuned on your hardware, that work is available under contract — scoped, quoted, and delivered with acceptance criteria.

Services & licensing