Profiling JVM inside Kubernetes
Have you ever needed to profile a problem that is occurring inside your kubernetes cluster? There are a couple of commercial tools available for this, but for the puritan there’s also a way to do this without having to have a license (at least as long as you’re not doing it on your production Cluster). You can use Java Flight Recorder. Please read the details about the licensing before using it though.
So how would you do it?
You have likely packaged your application as a docker image. You then also likely have some java -jar
command that launches your application. Add the following flags to this command:
java \
...
-XX:+ExitOnOutOfMemoryError \
-XX:+UnlockDiagnosticVMOptions \
-XX:+DebugNonSafepoints \
-XX:+FlightRecorder \
-XX:StartFlightRecording=delay=230s,duration=1m,name=Default,filename=recording.jfr,settings=default \
-Xlog:jfr=info \
...
-jar $JAR_FILE
This will now start recording your performance metrics after 230 s
and record it for 1 minute
. The results are available in the pod as /recording.jfr
After the recording has been done you can retrieve it by running
kubectl cp POD_NAME:recording.jfr .
And voila. You now have your jfr file. This can be analyzed with for example intellij.
Don’t know when you want to perform your recording? Then you likely need to involve JMX
or something similar to trigger a flight recording. That’s outside of the scope for this post though.