Ah, I have to take that back, since benchmarks run in the order of seconds and they use sockets to start and stop the timer, which likely means compilation time is not included.