A sales proposal takes too long to prepare. The catalogue team says equipment lookup is fast. Discount calculation looks normal. Delivery estimates run in parallel. Every component seems acceptable on its own, while a salesperson waits in front of the customer without knowing when the answer will arrive.
Distributed tracing reconstructs the instrumented path of one operation across applications and dependencies. Instead of comparing unrelated averages, the team can inspect a particular execution: which steps ran, how long they took and how they were connected. “Instrumented” matters. A trace cannot invent measurements for work nobody recorded.
What makes up the history
An instrumented step is called a span. It might represent an incoming request, a database call or a pricing calculation. Each span has timing and context that connects it to other steps, forming a trace from the external request into internal work.
Library instrumentation often captures network and database calls automatically. It may not show an application waiting for a free connection or performing an important validation between those calls. Such gaps need explicit measurements. The OpenTelemetry traces documentation describes the underlying model.
Recording every internal method is rarely the best starting point. Instrument the boundaries where an explanation is missing: resource waits, expensive calculations and remote calls. Thousands of tiny spans can make an investigation unnecessarily difficult.
Do not add up all the bars
Catalogue and delivery requests may overlap. Adding their durations counts some elapsed time twice. Nested spans also sit inside their parents' duration. Customer latency depends on the path that determines when the response is ready, not on the sum of every visible operation.
A long span is not necessarily heavy CPU work. It may include waiting for a connection, lock or remote response. Even a long partner call only tells us what our client observed; without measurements on the other side, it does not establish why the provider took that long.
An empty gap is not automatically a network problem either. It may be missing instrumentation. Cross-machine timestamps also require care around clock synchronisation. Tracing narrows an investigation; resource metrics, logs or profiling may still be needed to explain it.
Keep the relationship across a queue
Suppose proposals are calculated asynchronously. The initial request finishes before the job starts. Context can travel in message metadata, and separate executions can be associated through span links. There is no need to keep one incoming request open for the entire workflow.
Queue wait and processing time must remain distinct. Measuring only the worker can show a fast calculation while hiding the customer's long wait for that worker. Retries should also appear as separate attempts rather than one ambiguous call.
Keep the proposal's business identifier independently of the trace ID: separate attempts may have separate traces. Correlation helps find the work; it does not authorise someone to read another customer's proposal. The context propagation guide covers these connections and trust boundaries.
The trace you need may not have been kept
At volume, teams commonly retain a sample. Head sampling decides early, before knowing whether the operation will fail. Tail sampling uses collected spans to select by outcomes or duration, but needs buffering and processing capacity. It cannot restore data discarded upstream. The sampling documentation explains the trade-offs.
A missing trace therefore does not prove that an operation never happened. Nor does an error-heavy diagnostic sample measure the failure rate of the whole service. Overall quality needs measurements with understood coverage.
Test what support can still establish when a rare failure has no retained trace. The business record of a proposal must remain available independently of diagnostic sampling.
Useful detail has limits
Attributes can accidentally capture full requests, confidential prices or authentication tokens. Operation names, service versions and approved characteristics are often enough. Sensitive-data controls must cover automatically captured fields as well as custom instrumentation.
Watch for dropped telemetry and ensure exporting traces does not block proposal generation. A failed diagnostics backend should not halt the product, but the team should know which evidence is currently unavailable.
Ask for a complete example
Inspect one deliberately slow test proposal from acceptance through waiting, calculation, retries and publication. Change one dependency in a controlled test and check whether the difference is visible. If the trace breaks at the queue, fix propagation before replacing the storage platform.
The eventual remedy may be a product decision: prepare some calculations in advance, stop waiting for optional information or remove a redundant call. Compare those changes by when the customer gets a useful result. More stored spans alone will not help the salesperson waiting for it.