Benchmark numbers for LLM serving are usually a single average tokens-per-second figure, measured by whoever is selling something, with the denominator left undefined. These projects exist because measuring properly — per token, per regime, per request, under load — turned out to be worth the effort of building the instrument first.
VeloBenchmark is a single-binary benchmarking and live-stats console for any OpenAI-compatible endpoint. It began as the question “what is this server actually doing per token?” and grew into chat instrumentation, repeatable test suites, a concurrent runner, session reports, comparisons, and an OTLP telemetry receiver that can sit in front of a serving engine's own OpenTelemetry stream.
veloGB10 is a from-scratch Rust + CUDA inference engine for NVIDIA GB10 systems (DGX Spark and compatible OEM boxes), where ~255 GB/s of memory bandwidth is the scarce resource and every design decision is subordinate to spending it well. It is documented at veloengine.ai, and it emits the OpenTelemetry stream that VeloBenchmark's receiver was built to consume — the two projects measure and serve the same reality.
VeloBenchmark is AGPL-3.0: free to use, modify, and deploy, including commercially; network deployments of modified versions share their source. That is a deliberate choice for a measurement tool — the methodology stays inspectable by the people whose decisions it informs. veloGB10 is Apache-2.0. Paid work (integration, extensions, tuning, support, and commercial licences where the AGPL does not fit) is described on the services page; none of it gates the software itself.
Both projects are developed with strong assistance from open-source LLMs, under experienced human technical direction — architecture decisions, ideas, testing and extensive debugging included — and this is stated openly because it shaped how they were built. The people who developed these projects and directed the AI have extensive hands-on development experience as software architects, with decades of practice designing, building and shipping systems of this kind: the direction is informed by that work, not delegated to a tool. Every performance figure is gated or measured on physical hardware; where a number is an estimate, the docs say so. The acknowledgement matters in both directions: none of this exists without the kernels, formats, and hard-won engineering knowledge of the wider open ecosystem.
Both projects are written and maintained by Stav Katsoulis (sf-stav) — kernels, scheduler, transport, gates, console, docs and releases. Bug reports and well-formed issues are always free and welcome; contracted work is quoted per request. The shortest path to a human is stav@scalarflow.co.uk.