The Handler, Init Code and Leaking State

Scenario

Example app: a Python function returns a customer's recommended products. To "cache" work, a developer stores intermediate results in module-level variables, and creates a boto3 client inside the handler for every request.

What you see in production

A customer reports seeing another customer's recommendations. Latency also has a long tail: some requests take 300 ms longer than others, and DynamoDB metrics show far more new connections than expected.

The naive code

import boto3

seen_products = []                     # module level: shared by every request in this environment

def handler(event, context):
    dynamodb = boto3.resource("dynamodb")          # new client and connection every invocation
    table = dynamodb.Table("products")
    for item in table.query(**query_for(event["customer_id"]))["Items"]:
        seen_products.append(item["sku"])          # grows forever, mixes customers
    return {"customer": event["customer_id"], "recommended": seen_products[-5:]}

Reproduce it

Two customers' requests land in the same execution environment, one after the other:

seen_products = []          # module level, like the naive code

def naive_handler(event):
    seen_products.extend(event["skus"])
    return {"customer": event["customer"], "recommended": seen_products[-3:]}

def fixed_handler(event):
    recent = list(event["skus"])               # request-scoped
    return {"customer": event["customer"], "recommended": recent[-3:]}

requests = [{"customer": "alice", "skus": ["A1", "A2", "A3"]}, {"customer": "bob", "skus": ["B1"]}]
for r in requests:
    print("naive:", naive_handler(r))
for r in requests:
    print("fixed:", fixed_handler(r))

naive: {'customer': 'alice', 'recommended': ['A1', 'A2', 'A3']}
naive: {'customer': 'bob', 'recommended': ['A2', 'A3', 'B1']}
fixed: {'customer': 'alice', 'recommended': ['A1', 'A2', 'A3']}
fixed: {'customer': 'bob', 'recommended': ['B1']}

Root cause

Lambda imports your module once per execution environment and calls the handler many times. Module-level objects therefore live across requests from different users. That is exactly what you want for SDK clients and configuration, and exactly wrong for request data. Creating clients inside the handler repeats expensive setup (credential resolution, TLS connections) on every call.

The fix

Put reusable, request-independent objects at module level; keep everything about the request inside the handler:

import os
import boto3

table = boto3.resource("dynamodb").Table(os.environ["TABLE_NAME"])   # created once per environment

def handler(event, context):
    items = table.query(**query_for(event["customer_id"]))["Items"]
    recommended = [i["sku"] for i in items][:5]                     # request-scoped
    return {"customer": event["customer_id"], "recommended": recommended}

If you really need a cache across invocations, key it explicitly (for example by product ID, never mixed per user), bound its size (functools.lru_cache(maxsize=...) or a TTL cache), and accept that each environment has its own copy.

Trade-offs

• Module-level clients make the first request in each environment slower (init), and every later one faster.
• Per-environment caches are not shared across environments; for a shared cache use ElastiCache or DynamoDB with a TTL.
• Avoid doing network calls at import time that can fail; a failing import makes every invocation in that environment fail.

Detect it in production

• Code review rule: module-level mutable containers ([], {}) that are written inside the handler.
• Memory growth over the life of an environment (Max Memory Used rising per invocation in REPORT lines).
• Init Duration versus handler duration; many new connections relative to concurrency.

Key points

• The module is imported once per environment; the handler runs per request.
• Clients and configuration at module level; request data inside the handler.
• Caches must be keyed, bounded and treated as per-environment.

The execution-environment model is the same in every runtime; the Node.js course's cold-start lesson explores it further.

Further reading: Define Lambda function handler in Python · Lambda execution environment lifecycle