Eight languages, one machine

Four small programs, written once per language and checked to print the same bytes, timed on the same idle machine — with the controls that say which of the differences are real.

The four programs

They were chosen for four different costs. The most useful result on this page is that they do not agree on an ordering.

Linux · aarch64 — 6-core aarch64 guest under QEMU, 7 GB

Compute time

Every figure is the minimum of the runs minus that runtime's own hello-world time, so a JVM's boot is not charged to its arithmetic. The minimum rather than the median: a machine under interference produces a long right tail and no left one.

fib(35) — 9.2M recursive calls

loops — 100M iterations

objects — 10M allocations

lists — 1M elements through map / filter / fold

How solid each figure is

How far the median sits above the minimum, as a percentage. Below about 10% a figure is settled; well above it, two languages a few percent apart are not ranked by this data, and the page does not pretend otherwise.

Toolchains

macOS · Apple M4 — 10-core Apple M4 (4 performance + 6 efficiency), arm64, bare metal, 24 GB — the only machine here that is not a guest

Compute time

Every figure is the minimum of the runs minus that runtime's own hello-world time, so a JVM's boot is not charged to its arithmetic. The minimum rather than the median: a machine under interference produces a long right tail and no left one.

fib(35) — 9.2M recursive calls

loops — 100M iterations

objects — 10M allocations

lists — 1M elements through map / filter / fold

How solid each figure is

How far the median sits above the minimum, as a percentage. Below about 10% a figure is settled; well above it, two languages a few percent apart are not ranked by this data, and the page does not pretend otherwise.

Toolchains

Windows · x86_64 — 4-core / 8-thread AMD EPYC 9354 guest on a Shadow cloud PC, 16 GB; a slice of a 32-core part, and the neighbours are outside our control

Compute time

Every figure is the minimum of the runs minus that runtime's own hello-world time, so a JVM's boot is not charged to its arithmetic. The minimum rather than the median: a machine under interference produces a long right tail and no left one.

fib(35) — 9.2M recursive calls

loops — 100M iterations

objects — 10M allocations

lists — 1M elements through map / filter / fold

How solid each figure is

How far the median sits above the minimum, as a percentage. Below about 10% a figure is settled; well above it, two languages a few percent apart are not ranked by this data, and the page does not pretend otherwise.

Toolchains

The same ratios, machine by machine

Where two machines disagree about a ratio, the disagreement is the result — the number was a property of one box rather than of the language. Read the denominator first: several rows moving together and in the same direction is the signature of the C baseline having changed, not of five languages changing at once.

fib(35)

loops

objects

lists

What was checked before any of this was believed

A benchmark is an instrument, and an instrument nobody calibrated reports its own defects as findings. Each of these could have come back negative.

What this does not establish

lists is not like-for-like. Keal's List<Int>, C++'s vector and Rust's Vec hold unboxed 64-bit integers; Java and Kotlin box every element into a Long. That difference is most of what the JVM pays in that column, and it is a property of the libraries rather than of the compilers.

Each timing is a whole process, run once. No warm-up loop inside the program, no steady-state measurement. This is what running the program costs, which flatters ahead-of-time compilers and understates what a JIT does in a long-lived server.

Four programs are four programs. They were chosen for four different costs, but no set this small predicts a real workload, and none of them touches I/O, strings or concurrency.

The ratio to C carries the C compiler with it. A ratio divides by whatever the local C compiler produced, so it absorbs how fast the hardware is but not the baseline's own optimisation decisions. fib is where that shows. On the Linux machine, gcc at -O2 inlines the recursion several levels deep — 244 instructions in the function body against 30 with inlining turned off — and the same source, same compiler and same flag then runs in 11.8 ms against 20.3 ms. That 1.7× swing is about the size of the gap between the machines' own C baselines on that program. objects is the second such column and it points the other way: against the clang machine, six of seven rows move together by about 1.6× because that C baseline is the fast one there. Between the two gcc machines the same column stays inside the pack, which is what says the effect belongs to the compiler rather than to the hardware. So those two ratios compare within a machine and not across C compilers, and any row that moves between machines has to be read against its denominator before it is read as a fact about the language.

Subtracting startup weighs most on the rows that have the most of it. Each figure has its own machine's hello-world time taken off it, which is what keeps a JVM's boot out of its arithmetic. But a JVM's boot is tens of milliseconds where a native binary's is under two, so the JVM rows carry by far the largest correction — and it moves: on the macOS machine, aligning the JDK changed Java's startup by 8 ms, which is 8 ms on all four of its figures and a tenth of its lists. Comparing a JVM row across machines compares two startup corrections as well as two runtimes.

The machines do not share a toolchain. A ratio to C absorbs how fast the hardware is. It does not absorb a different gcc, a different Go, or a different JVM, and each machine reports whatever it had. Where the runtime is itself the thing being measured — the Java and Kotlin rows — the machines align on a JDK major version, because otherwise a gap between two machines would confound the operating system with the virtual one. Everywhere else the version is declared rather than forced, and the toolchain table under each machine is that disclosure.

A ratio is a property of a machine until a second machine agrees. The first machine here failed to reproduce a ratio this project had published, by a factor of two, for exactly that reason. Until the table below has more than one column, read every number as measured rather than as true.

Running it yourself

The programs are in bench/ports/ and the harness that times them is bench/ports/run.py. It builds what it finds a toolchain for, names what it skips, and prints an entry ready to append to the machine list. Every language uses 64-bit signed integers and, where it has the choice, the optimisation level keal build itself passes to cc.