Slow SQL queries degrade user experience, cause cascading failures, and turn simple operations into production incidents. But more telemetry means more things to look at, not necessarily more understanding. Instead of treating traces as a data stream we might analyze someday, we should be opinionated about what matters at the moment of decision. As we argued in The Signal in the Storm , raw telemetry only becomes useful when we extract meaningful patterns.
In this guide, you'll build a repeatable workflow that turns OpenTelemetry database spans into span-derived metrics you can dashboard and alert on—so you can identify what's slow, what matters most, and what just regressed. We'll make this concrete with slow SQL queries, serving two use cases: We'll build a lab where your app emits OpenTelemetry traces, and we distill those into actionable metrics, starting with simple slow query detection, then adding traffic-weighted impact, and finally anomaly detection. Jump to the Lab Setup section below. But the context helps you understand what you're building.
[Embedded video: “The Signal in the Storm: Practical Strategies for Managing Telemetry Overload” — Endre Sara] “Slow” isn't a single problem. It's a symptom with fundamentally different causes. A 50ms query might be fine for a reporting dashboard but catastrophic for checkout. As High Performance MySQL emphasizes, understanding why a query is slow determines how to fix it. Here are the most common problems that may cause slow queries: The database does more than necessary—typically full table scans due to missing or unusable indexes.
Without an index on customer_id, a simple SELECT * FROM orders WHERE customer_id = $1 grows from 20ms at 10K rows to minutes at 10M rows. The query didn't change; the data volume did. Aggregations and joins compound this. Even indexed queries can explode when the planner misjudges cardinality and chooses the wrong join strategy. Perfectly optimized queries can be slow when waiting for resources. Lock contention blocks queries until other transactions release rows. Connection pool exhaustion adds latency before the query even starts.