Glossary

Latency budget

A latency budget is the total time allocated for a complete user-facing request, distributed across every architectural hop: DNS resolution, TLS handshake, load balancer, application servers, database queries, cache reads, and downstream RPC calls. If the sum of all hop latencies exceeds the budget, the request misses its SLA. A budget makes latency a first-class engineering constraint rather than an afterthought discovered in production.

Formula: total_latency = sum(sequential_hops) + max(parallel_group). Keep measured p50 at least 20% below your SLA ceiling to absorb GC pauses and traffic spikes.

Last updated: March 2026