SnapStart, Uniqueness and Priming

Scenario

Example app: a Java order service enables Lambda SnapStart and cold starts drop from 4 seconds to under 500 ms. The service generates order reference codes with a java.util.Random created in a static field, and opens a database connection during initialisation.

What you see in production

Duplicate-key errors appear: different execution environments generate identical order references. After a few hours, requests in freshly restored environments also fail with stale database connections.

The naive code

public class OrderHandler implements RequestHandler<Map<String, Object>, String> {
    private static final Random RANDOM = new Random();                // seeded once, before the snapshot
    private static final Connection DB = Database.connect();          // captured in the snapshot

    @Override
    public String handleRequest(Map<String, Object> input, Context ctx) {
        String reference = "ORD-" + Integer.toHexString(RANDOM.nextInt());
        save(DB, reference, input);
        return reference;
    }
}

Reproduce it

Three environments are restored from the same snapshot; each takes its random generator's state from it:

import java.util.*;

public class Main {
    public static void main(String[] args) {
        long snapshotSeed = 20261019L;                    // state captured in the snapshot

        System.out.println("state restored from snapshot:");
        for (int env = 1; env <= 3; env++) {
            Random restored = new Random(snapshotSeed);    // every environment resumes from identical state
            System.out.println("  env " + env + " first reference: ORD-" + Integer.toHexString(restored.nextInt()));
        }

        System.out.println("re-seeded after restore (or SecureRandom / UUID):");
        for (int env = 1; env <= 3; env++) {
            Random afterRestore = new Random(snapshotSeed + env * 7919L);   // stands in for fresh entropy per environment
            System.out.println("  env " + env + " first reference: ORD-" + Integer.toHexString(afterRestore.nextInt()));
        }
    }
}

state restored from snapshot:
  env 1 first reference: ORD-92b01e76
  env 2 first reference: ORD-92b01e76
  env 3 first reference: ORD-92b01e76
re-seeded after restore (or SecureRandom / UUID):
  env 1 first reference: ORD-8a286a13
  env 2 first reference: ORD-c395231d
  env 3 first reference: ORD-18fc3d21

Root cause

SnapStart runs your initialisation once when you publish a version, takes a snapshot of the memory and disk state of the initialised environment, and restores new environments from that snapshot. Anything computed during initialisation is therefore identical in every environment: random generator state, generated IDs, cached timestamps, and open network connections (which are dead after restore). Temporary credentials cached at init may also expire.

The fix

• Do not generate unique values during initialisation; use java.security.SecureRandom or UUID.randomUUID() (SnapStart-aware for Java), or create generators after restore.
• Use runtime hooks (the CRaC API, org.crac.Core.getGlobalContext().register(resource)) to refresh state: beforeCheckpoint() to close connections, afterRestore() to re-establish them and re-seed.
• Prime the code path before the snapshot: in beforeCheckpoint(), exercise serialization, class loading and SDK calls (with dummy or read-only requests) so those classes are loaded and initialised in the snapshot.
• Remember SnapStart works with published versions and aliases, not $LATEST, and is not compatible with provisioned concurrency, Amazon EFS or ephemeral storage larger than 512 MB.

public class OrderHandler implements RequestHandler<Map<String, Object>, String>, Resource {
    private Connection db;

    public OrderHandler() {
        Core.getGlobalContext().register(this);
    }

    @Override
    public void beforeCheckpoint(org.crac.Context<? extends Resource> context) throws Exception {
        primeSerialization();            // load and initialise hot classes before the snapshot
        if (db != null) db.close();
    }

    @Override
    public void afterRestore(org.crac.Context<? extends Resource> context) throws Exception {
        db = Database.connect();         // fresh connection per restored environment
    }

    @Override
    public String handleRequest(Map<String, Object> input, Context ctx) {
        String reference = "ORD-" + UUID.randomUUID();
        save(db, reference, input);
        return reference;
    }
}

Trade-offs

• Snapshots are taken at publish time, so deployments take longer.
• Priming with real calls must not cause side effects; use read-only or dummy operations.
• For Python and .NET, SnapStart has extra charges for caching and restoring snapshots; for Java managed runtimes it is free.

Detect it in production

• Restore Duration in REPORT logs instead of Init Duration confirms SnapStart is active.
• Duplicate unique values or collisions across environments.
• Connection errors on the first request after restore.

Key points

• SnapStart restores identical memory into many environments.
• Generate unique values after restore; refresh connections with CRaC hooks.
• Prime hot paths before the snapshot; use published versions.

Further reading: Improving startup performance with Lambda SnapStart · Handling uniqueness with Lambda SnapStart