Fix: Linux OOM Killer Killing Processes (Out of Memory)

The Error

A process is killed unexpectedly and you find this in the kernel logs:

Mar 19 03:42:11 server kernel: Out of memory: Killed process 12345 (node) total-vm:2048000kB, anon-rss:1800000kB, file-rss:0kB, shmem-rss:0kB, UID:1000 phandle:0x3f7
Mar 19 03:42:11 server kernel: oom_kill_process+0x2c8/0x2f0

Or an application exits with no error but the system log shows:

kernel: oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=/,mems_allowed=0,global_oom,task_memcg=/user.slice,task=node,pid=12345,uid=1000
kernel: Out of memory: Killed process 12345 (node)

Or a container is terminated with exit code 137:

Container exited with code 137
# Exit code 137 = 128 + 9 (SIGKILL), which is the OOM killer's signal

Why This Happens

Linux allocates memory optimistically. When a process calls malloc(), the kernel promises virtual address space without backing it with physical RAM immediately. This is called overcommit. It works because most programs allocate more memory than they actually touch. The problem starts when enough processes touch enough of their allocated memory at once, and the kernel runs out of real pages to hand out. At that point, the kernel cannot simply fail a memory access for a process that was already promised memory. Instead, it invokes the OOM (Out of Memory) killer, which selects a process to terminate and sends it SIGKILL (signal 9).

The OOM killer scores every process based on its current resident set size, the number of child processes, and adjustments set via oom_score_adj. The process with the highest composite oom_score (visible in /proc/<pid>/oom_score, ranging 0 to 1000) gets killed first. This scoring biases toward large memory consumers, but there is no guarantee the killed process is the one “responsible” for the pressure. A small daemon that happened to allocate a few pages at the wrong time can survive while a large database gets terminated.

Common causes:

• Memory leak — a process’s memory usage grows unbounded until the system runs out.
• Insufficient RAM for the workload — the total memory demand of all processes exceeds physical RAM plus swap.
• No swap space — without swap, the kernel has no overflow buffer. A single large memory spike can trigger the OOM killer.
• cgroup memory limit — in containers or cgroup-constrained environments, the OOM killer activates when the process exceeds the cgroup limit, even if system memory is available.
• Memory fragmentation — the system has free memory but not in large enough contiguous blocks (rare, but causes OOM-like behavior).
• vm.overcommit_memory=2 — disables overcommit. Allocations fail when total committed memory exceeds physical RAM + swap, even before memory is exhausted.

How Other Tools Handle This

Linux’s global OOM killer is one way to deal with memory exhaustion, but every major platform and orchestration layer has its own mechanism. Understanding the differences helps you choose the right boundary for your workload.

cgroups v2 memory.max is the kernel-level per-group limit. When a cgroup hits memory.max, the kernel invokes the cgroup-level OOM killer, which only kills processes inside that cgroup. System-wide memory may still be plentiful. You configure it via a single file write (echo 2G > /sys/fs/cgroup/myapp/memory.max). The softer memory.high throttles allocations by stalling the process in kernel space before the hard limit is reached, giving backpressure without killing. This two-tier approach (high + max) is unique to cgroups v2 and has no direct equivalent in Docker flags or Kubernetes specs.

Docker --memory is a wrapper around cgroups. docker run --memory=512m sets both memory.max (cgroup v2) or memory.limit_in_bytes (cgroup v1) and optionally memory.swap.max. The Docker daemon adds an OOM-kill event to docker inspect output and to docker events. The key difference from raw cgroups is that Docker also supports --oom-kill-disable, which prevents the OOM killer from acting on the container. Use this with extreme caution: a container that hits its memory limit with OOM kill disabled will hang indefinitely as allocations stall.

Kubernetes memory limits (resources.limits.memory) translate to cgroup limits on the node. When a Pod exceeds its memory limit, the kubelet does not kill it directly. Instead, the cgroup-level OOM killer terminates the container, and the kubelet reports the reason as OOMKilled in the Pod status. Kubernetes adds another layer: the eviction manager. When node-level memory pressure crosses the memory.available threshold (default 100Mi), the kubelet evicts Pods by priority. Pods without requests set are evicted first (BestEffort QoS class), then Burstable, then Guaranteed. Setting accurate requests and limits is the primary defense against unexpected evictions.

systemd MemoryMax / MemoryHigh works at the service level and maps directly to cgroups v2 properties. MemoryMax=2G in a unit file sets a hard limit. MemoryHigh=1.5G applies backpressure. systemd also exposes OOMPolicy=continue|stop|kill to control what happens after an OOM kill: continue restarts the service, stop stops it, and kill lets the default OOM behavior proceed. This is more granular than Docker’s binary --oom-kill-disable.

Windows job objects are the rough equivalent on Windows. A job object groups processes and enforces a JobObjectExtendedLimitInformation.ProcessMemoryLimit. When a process in the job exceeds the limit, Windows terminates it with STATUS_COMMITMENT_LIMIT. Unlike Linux, Windows does not have a global OOM killer. Instead, individual allocation calls (VirtualAlloc, HeapAlloc) fail with ERROR_COMMITMENT_LIMIT, and the process is expected to handle the failure. This makes Windows more predictable but requires applications to handle allocation failures gracefully, which many do not.

oom_score_adj tuning vs systemd MemoryMax represents two philosophies. oom_score_adj lets you influence which process the OOM killer picks, but it does not prevent OOM kills from happening. MemoryMax prevents a process from consuming more than its share, but it shifts the OOM risk to that specific process. In production, the best practice is to combine both: set MemoryMax to enforce a ceiling, and set OOMScoreAdjust=-500 on critical services so that if global memory pressure somehow occurs, the critical service is the last to be killed.

Fix 1: Identify What the OOM Killer Killed

Start by confirming an OOM kill occurred and identifying the victim:

# Check kernel logs for OOM events
sudo dmesg | grep -i "oom\|killed process\|out of memory"

# Or check system journal (systemd systems)
sudo journalctl -k | grep -i oom

# Or check syslog
sudo grep -i oom /var/log/syslog
sudo grep -i oom /var/log/kern.log

Interpret the OOM log:

Out of memory: Killed process 12345 (node) total-vm:2048000kB, anon-rss:1800000kB
#                              ^^^^^  ^^^^  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
#                              PID    Name  Memory usage at time of kill
#
# total-vm: Total virtual memory (may be much larger than RAM used)
# anon-rss: Actual RAM used by anonymous mappings (heap, stack) — this is the key number
# file-rss: RAM used for file-backed pages (libraries, mmap files)

Check memory state at time of kill:

# OOM log includes system memory state — look for lines like:
# [ 1234.567] Mem-Info:
# [ 1234.567] Node 0 DMA free:15360kB min:256kB low:320kB high:384kB
# [ 1234.567] active_anon:450000kB inactive_anon:200000kB active_file:10000kB

Monitor memory usage in real time to find leaking processes:

# Sort by memory usage
watch -n 2 "ps aux --sort=-%mem | head -20"

# Or use top — press M to sort by memory
top

# smem gives more accurate real memory usage (PSS)
smem -t -k -c "pid name pss" | sort -k 3 -n | tail -20

Fix 2: Add or Increase Swap Space

Swap gives the kernel an overflow buffer and significantly reduces OOM kills for temporary memory spikes:

# Check current swap
free -h
swapon --show

# Create a 4GB swap file
sudo fallocate -l 4G /swapfile
sudo chmod 600 /swapfile
sudo mkswap /swapfile
sudo swapon /swapfile

# Verify swap is active
free -h
# Swap: 4.0G

# Make swap permanent across reboots
echo '/swapfile none swap sw 0 0' | sudo tee -a /etc/fstab

Tune swappiness — controls how aggressively the kernel uses swap:

# Check current swappiness (default is 60)
cat /proc/sys/vm/swappiness

# Lower values prefer RAM; higher values prefer using swap
# For servers with large RAM, set lower (10-20)
sudo sysctl vm.swappiness=10

# Make permanent
echo 'vm.swappiness=10' | sudo tee -a /etc/sysctl.d/99-memory.conf
sudo sysctl -p /etc/sysctl.d/99-memory.conf

Note: Swap prevents OOM kills but trades memory for disk I/O. If a process is actively swapping, performance degrades significantly. Swap buys time — the real fix is reducing memory usage or adding RAM.

Fix 3: Adjust OOM Score to Protect Critical Processes

The OOM killer selects its victim using oom_score (0-1000). Higher scores are killed first. You can adjust a process’s score using oom_score_adj (-1000 to 1000):

# Check current OOM score for a process
cat /proc/$(pgrep nginx)/oom_score
# e.g., 150

# Check the adjustment value
cat /proc/$(pgrep nginx)/oom_score_adj
# 0 (default)

Make a critical process less likely to be killed:

# Lower the OOM score for nginx (less likely to be killed)
echo -500 | sudo tee /proc/$(pgrep nginx)/oom_score_adj

# Make a process nearly immune to OOM kill
echo -1000 | sudo tee /proc/$(pgrep nginx)/oom_score_adj

# Make a process the first target (useful for sacrificial processes)
echo 1000 | sudo tee /proc/$(pgrep nginx)/oom_score_adj

Set oom_score_adj for a systemd service permanently:

# /etc/systemd/system/myapp.service
[Service]
OOMScoreAdjust=-500   # Lower score = less likely to be killed

# Or disable OOM kill entirely for this service (use carefully)
OOMPolicy=continue    # Continue running even if OOM kill is attempted

sudo systemctl daemon-reload
sudo systemctl restart myapp

Set oom_score_adj when starting a process:

# Start a process with a specific OOM score adjustment
sudo -u myuser bash -c 'echo $$ > /tmp/myapp.pid && exec nice -n 10 /usr/bin/myapp'

# Or use the oom_score_adj directly with a wrapper
sudo sh -c 'echo 500 > /proc/self/oom_score_adj && exec sudo -u myuser /usr/bin/background-job'

Fix 4: Set Memory Limits with cgroups

Instead of letting the OOM killer react, proactively limit how much memory a process can use:

Using systemd service limits:

# /etc/systemd/system/myapp.service
[Service]
MemoryMax=2G          # Hard limit — process is OOM-killed if it exceeds this
MemoryHigh=1.5G       # Soft limit — triggers memory reclaim before hard limit
MemorySwapMax=0       # Disable swap for this service (0 = no swap allowed)

Using cgroups v2 directly:

# Create a cgroup
sudo mkdir /sys/fs/cgroup/myapp

# Set memory limit to 2GB
echo "2147483648" | sudo tee /sys/fs/cgroup/myapp/memory.max

# Set soft limit for backpressure at 1.5GB
echo "1610612736" | sudo tee /sys/fs/cgroup/myapp/memory.high

# Add a process to the cgroup
echo <PID> | sudo tee /sys/fs/cgroup/myapp/cgroup.procs

Docker container memory limits:

# Limit container to 512MB RAM and 512MB swap
docker run --memory=512m --memory-swap=1g myimage

# Prevent container from using swap
docker run --memory=512m --memory-swap=512m myimage

Pro Tip: Always set both memory.high and memory.max (or MemoryHigh and MemoryMax in systemd). The high limit applies backpressure by slowing allocations, giving the application time to free memory. The max limit is the hard kill boundary. Without the soft limit, your process goes from “normal” to “dead” with no intermediate warning.

Fix 5: Tune vm.overcommit Settings

Linux’s memory overcommit behavior controls whether malloc() can succeed when physical memory isn’t available:

# Check current setting
cat /proc/sys/vm/overcommit_memory
# 0 = heuristic overcommit (default)
# 1 = always allow overcommit (never fail malloc)
# 2 = strict — fail when committed memory > (RAM * overcommit_ratio/100 + swap)

vm.overcommit_memory=0 (default): The kernel guesses whether to allow allocation. Can lead to OOM kills when guesses are wrong.

vm.overcommit_memory=1: Always allow malloc() to succeed. Maximum memory efficiency but maximum OOM risk.

vm.overcommit_memory=2: Strict — fail malloc() early instead of OOM-killing later. Use when you’d rather have processes fail predictably than have random OOM kills:

# Set strict overcommit
sudo sysctl vm.overcommit_memory=2
sudo sysctl vm.overcommit_ratio=80  # Allow RAM * 80% + swap as committed memory

# Make permanent
echo 'vm.overcommit_memory=2' | sudo tee -a /etc/sysctl.d/99-memory.conf
echo 'vm.overcommit_ratio=80' | sudo tee -a /etc/sysctl.d/99-memory.conf

Real-world scenario: A Redis server running with vm.overcommit_memory=0 uses fork-based snapshotting (BGSAVE). Fork briefly doubles memory usage. If the system is tight on memory, the fork triggers the OOM killer. Redis’s own documentation recommends setting vm.overcommit_memory=1 to prevent this.

Fix 6: Find and Fix Memory Leaks

If the OOM killer repeatedly targets the same process, that process likely has a memory leak:

Monitor memory growth over time:

# Watch a specific process's memory usage every 30 seconds
PID=$(pgrep myapp)
while true; do
    RSS=$(awk '/VmRSS/{print $2}' /proc/$PID/status)
    echo "$(date): RSS=${RSS}kB"
    sleep 30
done

Use valgrind for C/C++ memory leak detection:

valgrind --leak-check=full --track-origins=yes ./myapp

For Node.js — use the built-in heap profiler:

// Add to your Node.js application
const v8 = require('v8');
const fs = require('fs');

// Dump heap snapshot (analyze with Chrome DevTools)
setInterval(() => {
    const snapshot = v8.writeHeapSnapshot();
    console.log(`Heap snapshot written to ${snapshot}`);
}, 60000);  // Every minute

For Python — use tracemalloc:

import tracemalloc

tracemalloc.start()

# ... run your code ...

snapshot = tracemalloc.take_snapshot()
top_stats = snapshot.statistics('lineno')
for stat in top_stats[:10]:
    print(stat)

For Java — capture a heap dump when memory is high:

# Trigger heap dump on OOM automatically
java -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/tmp/heapdump.hprof -jar myapp.jar

# Or dump manually from a running JVM
jmap -dump:format=b,file=/tmp/heapdump.hprof <PID>

Set up automatic memory monitoring with alerting:

# Simple script — alert when any process exceeds 80% of total RAM
THRESHOLD=$(($(grep MemTotal /proc/meminfo | awk '{print $2}') * 80 / 100))

ps aux --sort=-%mem | awk -v threshold=$THRESHOLD 'NR>1 {
    rss = $6  # RSS in kB
    if (rss > threshold) {
        print "WARNING: Process " $11 " (PID " $2 ") using " rss "kB"
    }
}'

Still Not Working?

Check if the OOM kill is from a cgroup, not the global OOM killer:

# cgroup OOM kills show different messages
sudo dmesg | grep "Memory cgroup"
# Memory cgroup out of memory: Kill process 12345 (myapp)...

For cgroup OOM kills, increase the cgroup memory limit or reduce the process’s memory usage — adding system swap won’t help.

Check for huge pages fragmentation:

# If the system uses huge pages and fragmentation prevents allocation
cat /proc/meminfo | grep -i huge
# HugePages_Total: 100
# HugePages_Free: 0   ← All huge pages are in use

Enable panic on OOM instead of killing (for debugging only):

# Don't use in production — causes a kernel panic instead of OOM kill
echo 1 | sudo tee /proc/sys/vm/panic_on_oom
# 0 = OOM kill (default)
# 1 = panic if OOM kill fails
# 2 = always panic on OOM

Review application memory configuration:

• JVM: Set -Xmx to limit the heap. Without it, the JVM can grow to consume all available RAM.
• Node.js: Use --max-old-space-size=4096 to limit heap to 4GB.
• PostgreSQL: Review shared_buffers, work_mem, and max_connections.
• Redis: Set maxmemory and maxmemory-policy in redis.conf.

Check NUMA topology on multi-socket servers:

# OOM can trigger on one NUMA node even if the other has free memory
numactl --hardware
# If you see unbalanced "free" across nodes, pin your process to a specific node
numactl --membind=0 --cpunodebind=0 /usr/bin/myapp

Verify no tmpfs or shared memory is consuming RAM silently:

# tmpfs mounts use RAM, not disk
df -h --type=tmpfs
# /dev/shm is often 50% of RAM by default — large files there eat physical memory
du -sh /dev/shm/*

For related Linux issues, see Fix: Linux Disk Full — No Space Left on Device, Fix: Linux Too Many Open Files, Fix: Docker Exited 137 OOMKilled, and Fix: Kubernetes Pod OOM Killed.