bench-my-llm

Simple CLI to benchmark LLM latency, throughput, cost, and quality across providers

PlatformWebfreeglobal

bench-my-llm is a CLI that benchmarks any OpenAI-compatible LLM API for latency, throughput, cost, and quality, producing a side-by-side terminal report instead of leaving model choice to guesswork. It supports OpenAI, Anthropic, Ollama, vLLM, and other compatible providers, with built-in and custom prompt suites, exportable reports, and historical trend tracking per model.

Website

Full match profile

Behind the summary, Matchbox keeps a richer profile of bench-my-llm - the signals our matcher actually reads to decide when to surface it. It stays private; claim the listing to see and control it.

Problem & pain-point mapping
Who we surface it to (audience fit)
What it's a strong alternative to
Trust & credibility signals

Claim bench-my-llm to unlock →

Something wrong with this listing — dead link, not a real product, wrong info?

Try Matchbox with your own problem

Describe what is not working - we’ll show you whether bench-my-llm (or something else) actually fits.

Try a problem