Setting Up KotlinX For Benchmark

Writing benchmarks for JVM is not necessarily easy. JVM is good at optimizing code, so maybe the code you’re executing won’t be what you think it is. Also, coming from the world of physics it would be ideal if somehow the code that is tested could be separated from the code that is measuring. Now fortunately for us there are tools there to help you.

Enter JMH (Java Microbenchmark Harness). I first encountered it recently and was curious. I then wanted to implement a benchmark for some Kotlin code that I had written and I found that there is a kotlin wrapper you can use kotlinx-benchmark.

The following is a quick guide on how to set it up, because I had some small issues and couldn’t find any good guidance online. You can find the code here: github

Code of the Day

Setup

Install gradle. I used version 6.8.3 for this example. Using sdkman is a great way of installing gradle, using your package manager if you’re on linux might result in a quite old version.
Run gradle init. Since we want to go minimum choose basic application with kotlin DSL.

Gradle

repositories{
    jcenter()
}

plugins {
    // simply for using kotlin
    kotlin("jvm") version "1.4.10"
    // this does your gradle configuration for you.
    id("org.jetbrains.kotlinx.benchmark") version "0.3.0"
}

dependencies {
    // you need the runtime, otherwise it won't work
    implementation("org.jetbrains.kotlinx:kotlinx-benchmark-runtime:0.3.0")
}

// you can do loads of configurations here, you don't really need all of this though.
benchmark {
    configurations {
        named("main") {
            iterationTime = 5
            iterationTimeUnit = "sec"
        }
    }
    targets {
        register("main") {
            this as kotlinx.benchmark.gradle.JvmBenchmarkTarget
            jmhVersion = "1.21"
        }
    }
}

Benchmark Code

@State(Scope.Benchmark)
@Warmup(iterations = 1)
@Measurement(iterations = 3, time = 1, timeUnit = TimeUnit.SECONDS)
open class ExampleBenchmark {

    var num: Int = 0

    @Setup
    fun setUp() {
        num = 3
    }

    @Benchmark
    fun exampleBenchmark(blackhole: Blackhole) {
        val sum = 5 + 2 * num
        // use blackhole here to prevent JVM optimizations. There are loads of gotchas, read it.
        blackhole.consume(sum)
    }

}