Why are structured JSON logs more useful than free-text console.log messages during incidents?

Consistent fields (tenant, request and correlation IDs, durations, error types) can be filtered and aggregated with queries across many invocations and functions, answering questions such as which tenants are affected; free text can only be searched line by line.

What is CloudWatch Embedded Metric Format and why use it in Lambda?

A structured log format from which CloudWatch extracts metrics automatically, so functions can emit custom metrics by writing a log line instead of calling the PutMetricData API, avoiding latency and API limits.

Why should tenant ID be a log field but usually not a metric dimension?

Each unique dimension combination creates a separate metric time series, which adds cost and can hit limits when there are thousands of tenants. Logs handle high-cardinality fields well; metrics should use bounded dimensions such as tier or region.

Observability: Structured Logs, Metrics and Traces — Node.js Serverless on AWS Lambda

Scenario

Example app: a multi-tenant SaaS runs 40 Node.js functions. Logging is console.log with free-text messages; there are no custom metrics or traces.

What you see in production

During an incident, the on-call engineer needs to know which tenants are affected and which downstream call is slow. Searching logs for "error" returns thousands of unstructured lines across log groups, none containing a tenant ID or correlation ID. The incident takes three hours to scope.

The naive code

export const handler = async (event) => {
  console.log("processing order " + event.detail.orderId);
  try { await charge(event.detail); }
  catch (e) { console.log("error charging: " + e); throw e; }
};

Reproduce it

The same events logged both ways; only structured logs can answer the question:

Answering an incident question from logsTry it yourself

const events = [
  { tenant: "acme", orderId: "o1", ok: true, ms: 120 }, { tenant: "globex", orderId: "o2", ok: false, ms: 3000 },
  { tenant: "acme", orderId: "o3", ok: true, ms: 140 }, { tenant: "globex", orderId: "o4", ok: false, ms: 3000 },
  { tenant: "initech", orderId: "o5", ok: true, ms: 110 },
];
const textLogs = events.map((e) => e.ok ? `processed order ${e.orderId}` : `error charging: Error: timeout`);
const jsonLogs = events.map((e) => JSON.stringify({ level: e.ok ? "INFO" : "ERROR", message: e.ok ? "order charged" : "charge failed",
  tenantId: e.tenant, orderId: e.orderId, durationMs: e.ms, correlationId: `c-${e.orderId}` }));

console.log("text logs, errors by tenant:", textLogs.filter((l) => l.startsWith("error")).map(() => "unknown tenant"));
const errors = jsonLogs.map((l) => JSON.parse(l)).filter((l) => l.level === "ERROR");
const byTenant = errors.reduce((acc, l) => ({ ...acc, [l.tenantId]: (acc[l.tenantId] || 0) + 1 }), {});
console.log("JSON logs, errors by tenant:", byTenant, "| slow orders:", errors.map((l) => l.orderId));

console.log("text logs, errors by tenant:", textLogs.filter((l) => l.startsWith("error")).map(() => "unknown tenant"));
const errors = jsonLogs.map((l) => JSON.parse(l)).filter((l) => l.level === "ERROR");
const byTenant = errors.reduce((acc, l) => ({ ...acc, [l.tenantId]: (acc[l.tenantId] || 0) + 1 }), {});
console.log("JSON logs, errors by tenant:", byTenant, "| slow orders:", errors.map((l) => l.orderId));

Output

text logs, errors by tenant: [ 'unknown tenant', 'unknown tenant' ]
JSON logs, errors by tenant: { globex: 2 } | slow orders: [ 'o2', 'o4' ]

Root cause

Unstructured logs are written for humans reading one request, not for querying many. Without consistent fields (tenant, request and correlation IDs, durations), metrics derived from them, or distributed traces across functions and services, the system cannot be observed; it can only be grepped.

The fix

Use Powertools for AWS Lambda (TypeScript):

Logger: JSON logs with Lambda context (request ID, cold start, function version), persistent keys such as tenantId, correlation IDs extracted from events, and sampling of debug logs.
Metrics: business and technical metrics written in CloudWatch Embedded Metric Format (no extra API calls), with dimensions such as service and tenant tier.
Tracer: AWS X-Ray traces with annotations, capturing AWS SDK calls and HTTP requests automatically. OpenTelemetry (through the AWS Distro for OpenTelemetry) is an alternative when you send telemetry to other backends.

import { Logger } from "@aws-lambda-powertools/logger";
import { Metrics, MetricUnit } from "@aws-lambda-powertools/metrics";
import { Tracer } from "@aws-lambda-powertools/tracer";

const logger = new Logger({ serviceName: "billing" });
const metrics = new Metrics({ namespace: "Shop", serviceName: "billing" });
const tracer = new Tracer({ serviceName: "billing" });

export const handler = async (event, context) => {
  logger.addContext(context);
  logger.appendKeys({ tenantId: event.detail.tenantId, orderId: event.detail.orderId });
  try {
    await charge(event.detail);
    metrics.addMetric("OrderCharged", MetricUnit.Count, 1);
  } catch (error) {
    logger.error("charge failed", { error });
    metrics.addMetric("ChargeFailed", MetricUnit.Count, 1);
    throw error;
  } finally {
    metrics.publishStoredMetrics();
  }
};

Also enable Lambda's JSON log format and log-level controls, set log retention (the default keeps logs forever), and build dashboards and alarms on SLO-relevant metrics, not just function errors.

Trade-offs

Logs and custom metrics cost money at scale: sample debug logs, avoid logging large payloads, and choose metric dimensions with bounded cardinality (tenant tier, not tenant ID, for metrics).
Tracing adds a little overhead; sample in high-volume paths.
Never log secrets or full personal data; add redaction.

Detect it in production

Can you answer within minutes: which tenants, which function versions, which dependency? If not, instrumentation is missing.
CloudWatch Logs Insights queries on structured fields; service maps in X-Ray or your tracing tool.
Log retention and ingestion cost reports.

Key points

Structured JSON logs with consistent keys and correlation IDs.
Metrics via Embedded Metric Format; traces with X-Ray or OpenTelemetry.
Powertools provides all three; mind cost, cardinality and sensitive data.

Further reading: Powertools for AWS Lambda (TypeScript) · Configuring advanced logging controls for Lambda · CloudWatch Embedded Metric Format