Memory, Heap and Garbage Collection

Scenario

Example app: a Java function transforms large JSON documents. It runs with 1,024 MB of memory. A developer sets -Xmx1024m in JAVA_TOOL_OPTIONS "to use all the memory".

What you see in production

Some invocations end with Runtime exited with error: signal: killed even though heap usage reported by the application peaks around 700 MB. Others show long pauses where nothing happens for hundreds of milliseconds.

The naive code

JAVA_TOOL_OPTIONS=-Xmx1024m -XX:+UseG1GC
MemorySize: 1024

Reproduce it

The JVM needs memory outside the heap: metaspace for classes, thread stacks, code cache, direct buffers and the runtime itself:

import java.util.*;

public class Main {
    public static void main(String[] args) {
        int functionMb = 1024;
        Map<String, Integer> nonHeap = new LinkedHashMap<>();
        nonHeap.put("metaspace (classes)", 110);
        nonHeap.put("code cache", 48);
        nonHeap.put("thread stacks", 40);
        nonHeap.put("direct buffers (HTTP client, I/O)", 64);
        nonHeap.put("runtime + native libraries", 90);
        int other = nonHeap.values().stream().mapToInt(Integer::intValue).sum();

        for (int xmx : List.of(1024, 640)) {
            int worstCase = xmx + other;
            System.out.printf("-Xmx%dm: worst case %d MB of %d MB -> %s%n", xmx, worstCase, functionMb,
                worstCase > functionMb ? "killed when the heap fills" : "fits, GC reclaims before OOM");
        }
        System.out.println("non-heap total: " + other + " MB " + nonHeap);
    }
}

-Xmx1024m: worst case 1376 MB of 1024 MB -> killed when the heap fills
-Xmx640m: worst case 992 MB of 1024 MB -> fits, GC reclaims before OOM
non-heap total: 352 MB {metaspace (classes)=110, code cache=48, thread stacks=40, direct buffers (HTTP client, I/O)=64, runtime + native libraries=90}

Root cause

The function's memory setting limits the whole process, not just the Java heap. Setting the maximum heap equal to the function memory leaves no room for metaspace, thread stacks, code cache and native buffers, so the operating system kills the process before the JVM can throw a catchable OutOfMemoryError or collect garbage effectively. The Lambda Java runtimes already size the heap sensibly by default. Collector choice also matters: concurrent collectors use extra threads and memory that small functions with one or two vCPUs cannot spare.

The fix

• Do not override heap size unless you have measured; the runtime's defaults leave room for non-heap memory. If you must, keep -Xmx well below the function memory (or use -XX:MaxRAMPercentage).
• Reduce allocation: stream large documents (Jackson streaming API) instead of building whole object trees; reuse buffers and the ObjectMapper.
• Leave the default collector for small functions (the runtime's defaults are tuned for short-lived processes); consider others only after measuring pauses at larger memory sizes.
• Increase function memory when the working set is genuinely large; it also adds CPU.

Trade-offs

• More memory costs more per millisecond but often shortens duration (more CPU), so cost may not rise.
• Streaming parsers are more code than data binding but keep memory flat for large inputs.
• GC tuning is workload-specific; measure with JFR or GC logs before changing defaults.

Detect it in production

• Max Memory Used close to the configured memory; signal: killed without a Java exception.
• GC logs (-Xlog:gc) in a test environment to see pause times and heap behaviour.
• Duration outliers that correlate with large inputs.

Key points

• Function memory limits the whole JVM process, not just the heap.
• Keep heap settings at defaults or well below the limit.
• Stream large data; measure before tuning the collector.

Further reading: Configuring Lambda function memory · Customize Java runtime startup behaviour for Lambda functions