A benchmark is only as useful as the questions it refuses to answer. The Phase 7 performance report measures eight workloads at O0 through O3 against optimised Rust, C and Python references, then spends as much space on methodology and limitations as it does on the table.
Eight workloads, four levels
The suite covers integer and float loops, function calls, recursion, nested loops, array multiplication, array reduction and array copies. Each Chuchu workload runs at all four levels. Every timed execution validates a checksum, and the full run records seven execution samples per variant with a warmup. Compilation and RSS sampling happen outside the execution timer.
export LLVM_SYS_231_PREFIX="$(brew --prefix llvm)"
cargo build --release
python3 benchmarks/run.py \
--cheeku ./target/release/cheeku \
--output benchmarks/results/phase7-full.jsonMedians, not best runs
Results are reported as median and interquartile range in milliseconds per process. That includes process startup and output capture, because that is what a user experiences. The report is explicit that these are process-level comparisons, not isolated kernel throughput or a universal language ranking.
Ratios are process-level comparisons, not isolated kernel throughput or universal language rankings.
What the numbers do not say
- Python uses arbitrary-precision integers and lists, so its costs are not directly comparable.
- The native references use different compilers and flags, documented alongside the results.
- A vectorisation probe is included, but it cannot isolate a SIMD speedup from other LLVM transformations.
- O3 is not assumed to be faster than O2. The report shows where it is and where it is not.
The performance page explains how to rerun the suite on your own machine. If a result looks surprising, the raw samples, compiler stages, flags and hashes are all in the report.