I read it as an estimate of worst-case latency, which will be a sum of many different latencies at different layers of the service: application, network, database, disk, memory, synchronization, CPU contention, VM contention - the whole lot.
Doesn't this depend on how "tuned" your subcomponents are? If I had a system that's supposedly heavily optimized, I would simply start seeing the worst-case latency on a higher percentile, 99.9th and so forth. Gil Tene has an excellent talk[1] on this.
