OpenTelemetry: Instrument AI-Built Features You Can Actually Trust

There is a specific kind of dread that sets in when an AI-built feature hits production and you have no idea what it is actually doing. The code looked fine in review — clean, even — but the moment real traffic touches it, you are flying blind. No spans, no metrics, no structured logs. Just vibes and a prayer. OpenTelemetry observability is the fix I now reach for before any AI-assisted feature ships, and after living with it for several months, I would not go back.

What OpenTelemetry Actually Is (Past the Buzzwords)

OpenTelemetry (OTel) is a CNCF project — now at graduated status — that provides a unified, vendor-neutral API, SDK, and wire protocol for collecting traces, metrics, and logs from your application. The key word is unified. Before OTel, you picked a vendor (Datadog, Jaeger, Zipkin, whatever) and your instrumentation was glued to them. With OTel, you instrument once and route your telemetry wherever you like: your own Jaeger instance, Grafana Tempo, Honeycomb, or a local OTEL Collector running in Docker.

The SDKs cover almost every runtime: Node.js, Python, Go, Java, .NET, Ruby, PHP, and more. The JavaScript/TypeScript SDK is the one I use most day-to-day, so that is where this post focuses, but the mental model transfers directly.

Why AI-Built Code Needs This More Than Hand-Written Code

When I write a function myself, I carry a rough mental model of what it does, how often it should be called, and what counts as a slow response. When an AI generates a service, that mental model does not exist yet. The function might look idiomatic and pass every test, but:

• It might be making three database round-trips where one would do.
• It might be retrying silently on failure, masking errors upstream.
• It might be allocating objects in a hot path in a way no linter catches.
• It might call an external API in a loop you did not notice during review.

None of these surface in a code review. They surface under load, at 2 a.m., in an incident. Instrumentation is the layer that makes AI-generated behavior legible — not just to you right now, but to whoever is on-call three months from now.

Getting Up and Running: Node.js in Under 15 Minutes

Install the minimum required packages:

npm install @opentelemetry/sdk-node \ @opentelemetry/auto-instrumentations-node \ @opentelemetry/exporter-trace-otlp-http 

Create a tracing.js file that you load before anything else:

// tracing.js const { NodeSDK } = require('@opentelemetry/sdk-node'); const { getNodeAutoInstrumentations } = require('@opentelemetry/auto-instrumentations-node'); const { OTLPTraceExporter } = require('@opentelemetry/exporter-trace-otlp-http'); const sdk = new NodeSDK({ serviceName: 'my-ai-feature-service', traceExporter: new OTLPTraceExporter({ url: 'http://localhost:4318/v1/traces', // local OTEL Collector }), instrumentations: [getNodeAutoInstrumentations()], }); sdk.start(); 

Then start your app with:

node --require ./tracing.js src/index.js 

That one --require flag gives you automatic instrumentation for Express, HTTP, fetch, PostgreSQL, Redis, gRPC, and a dozen other libraries — zero changes to the AI-generated code itself. The auto-instrumentations package monkey-patches the libraries at startup, so spans appear for every inbound request, every DB query, and every outbound HTTP call. That last one is where the "calling an API in a loop" problem becomes immediately visible in a waterfall trace.

Going Deeper: Manual Spans for Business Logic

Auto-instrumentation gets you the plumbing. For the AI-built business logic itself, I add manual spans so I can see the why, not just the what:

const { trace } = require('@opentelemetry/api'); const tracer = trace.getTracer('my-ai-feature-service'); async function processUserRequest(userId, payload) { return tracer.startActiveSpan('processUserRequest', async (span) => { span.setAttribute('user.id', userId); span.setAttribute('payload.size', JSON.stringify(payload).length); try { const result = await doWork(payload); span.setAttribute('result.status', 'ok'); return result; } catch (err) { span.recordException(err); span.setStatus({ code: 2 }); // ERROR throw err; } finally { span.end(); } }); } 

This pattern — wrap the AI-generated function, attach business-level attributes, record exceptions — is the one I paste into every significant function boundary. It is the difference between a trace that says "HTTP 500 in 340ms" and one that says "processUserRequest failed for user 4821 after a 340ms Postgres timeout on the recommendations query."

Routing Telemetry: The Collector Pattern

Rather than exporting directly to a cloud vendor from every service, I run a local OpenTelemetry Collector as a Docker container. It receives all telemetry, does some filtering and batching, then fans it out — traces to Jaeger for local dev, and (in production) to whatever backend is paying the bills. This means I can switch backends without touching application code. That portability has already saved me once when we migrated away from a vendor mid-project.

Rough Edges and Real Limitations

OpenTelemetry is not painless. Here is what I have actually hit:

• SDK version churn. The JS ecosystem especially has seen frequent breaking changes between minor versions. Pin your versions and read the changelog before upgrading.
• Context propagation across async boundaries can break if a library is not OTel-aware and swallows the async context. You may need to manually propagate context in worker threads or certain queue consumers.
• Log correlation is still maturing. The Logs SDK for Node.js was experimental for a long time. It is stable now, but the ergonomics for correlating logs to traces are not as seamless as traces-to-metrics yet.
• The Collector is another service to operate. It is worth it at scale, but for a tiny side project it can feel like infrastructure for infrastructure's sake.
• Auto-instrumentation overhead at startup is real. Cold start times on serverless functions can increase noticeably. In that environment, selective manual instrumentation is often a better trade-off.

What I Look For Once It Is Running

After instrumenting an AI-built feature, I let it run under real traffic for 24 hours, then open Jaeger and look for: unexpectedly deep span trees (hidden loops), spans with high p99 vs. p50 divergence (inconsistent behavior), and exception events that never surfaced as HTTP errors (silent failures). Nine times out of ten, I find at least one thing the code review missed. That is not a knock on AI coding assistants — it is just a reminder that observability and code review answer different questions, and you need both.

If you are evaluating local tracing backends, check out my post on self-hosted DevOps tooling — I cover Jaeger and Grafana Tempo as companion pieces to an OTel setup.