JVM Cold Starts: Where the Time Goes

Scenario

Example app: a Java 21 function exposes a small API. It uses a dependency-injection framework, the AWS SDK v2 with default settings, and Jackson. Warm requests take 20 ms.

What you see in production

The first request after a scale-out takes 3 to 5 seconds. Users notice it every morning and after each deployment. Init Duration in the REPORT logs is consistently above 3,000 ms.

The naive code

public class ApiHandler implements RequestHandler<APIGatewayV2HTTPEvent, APIGatewayV2HTTPResponse> {
    @Override
    public APIGatewayV2HTTPResponse handleRequest(APIGatewayV2HTTPEvent event, Context context) {
        var container = DependencyContainer.boot();            // classpath scanning, reflection
        var dynamo = DynamoDbClient.create();                   // default credential + region chain, Apache HTTP client
        return container.get(Router.class).route(event, dynamo);
    }
}

Reproduce it

A breakdown of a cold start, before and after the common optimisations (illustrative values):

import java.util.*;

public class Main {
    public static void main(String[] args) {
        Map<String, Integer> naive = new LinkedHashMap<>();
        naive.put("JVM start + class loading", 700);
        naive.put("DI container: classpath scan + reflection", 1600);
        naive.put("SDK client: default credential/region chain", 450);
        naive.put("Apache HTTP client + TLS setup", 350);
        naive.put("first request JIT warm-up", 600);

        Map<String, Integer> tuned = new LinkedHashMap<>();
        tuned.put("JVM start + class loading (smaller jar)", 450);
        tuned.put("compile-time DI or plain constructors", 120);
        tuned.put("SDK client: explicit region + env credentials", 60);
        tuned.put("lightweight HTTP client, created once at init", 120);
        tuned.put("C1-only JIT (TieredStopAtLevel=1)", 250);

        for (var plan : List.of(Map.entry("naive", naive), Map.entry("tuned", tuned))) {
            int total = plan.getValue().values().stream().mapToInt(Integer::intValue).sum();
            System.out.println(plan.getKey() + ": " + total + " ms  " + plan.getValue());
        }
    }
}

naive: 3700 ms  {JVM start + class loading=700, DI container: classpath scan + reflection=1600, SDK client: default credential/region chain=450, Apache HTTP client + TLS setup=350, first request JIT warm-up=600}
tuned: 1000 ms  {JVM start + class loading (smaller jar)=450, compile-time DI or plain constructors=120, SDK client: explicit region + env credentials=60, lightweight HTTP client, created once at init=120, C1-only JIT (TieredStopAtLevel=1)=250}

Root cause

A Java cold start loads and verifies many classes, runs static initialisers, and executes code in the interpreter until the JIT compiler kicks in. Frameworks that scan the classpath and use reflection at start-up multiply that cost, and SDK clients with default settings probe several credential and region providers. Doing any of this inside the handler repeats it on every request.

The fix

In order of typical impact:

1. SnapStart (next lesson): snapshot the initialised JVM and restore it, often cutting cold starts dramatically, at no extra charge for Java.
2. Initialise once, outside the handler (constructor or static fields), and give the function enough memory, since CPU scales with memory and start-up is CPU-heavy.
3. Lighter frameworks: plain constructors or compile-time DI (Dagger, Micronaut, Quarkus) instead of runtime classpath scanning; or GraalVM native images.
4. Configure SDK clients explicitly: region from the AWS_REGION environment variable, EnvironmentVariableCredentialsProvider, and a lightweight HTTP client (UrlConnectionHttpClient or the AWS CRT client).
5. JVM options: Lambda's recent Java runtimes already default to C1-only tiered compilation, which suits short functions; you can set it explicitly with JAVA_TOOL_OPTIONS=-XX:+TieredCompilation -XX:TieredStopAtLevel=1 on older setups.

Trade-offs

• C1-only compilation starts faster but reaches lower peak performance; long-running, CPU-heavy functions may prefer full tiered compilation.
• More memory shortens cold starts but costs more per millisecond (often recovered by shorter durations).
• Provisioned concurrency removes cold starts for a fixed capacity at a fixed cost.

Detect it in production

• Init Duration per function and the share of invocations with cold starts.
• p99 latency spikes after deployments and scale-out.
• Profile start-up locally with -Xlog:class+load or JFR to find expensive initialisation.

Key points

• Java cold starts are dominated by class loading, framework initialisation and JIT warm-up.
• Initialise once, outside the handler; configure SDK clients explicitly.
• SnapStart, lighter frameworks or native images for the biggest wins.

Further reading: Optimizing Java performance on Lambda (AWS Compute Blog) · Best practices for working with Lambda functions