Quick answer

Exit code 137 alone does not prove an out-of-memory kill: Docker returns the container command’s exit status, and several paths can end at 137. Before restarting or raising memory, save the container ID, inspect state, Docker events, configured limits, and matching kernel log. Correlate State.OOMKilled with the same time window; never treat it as the whole diagnosis.

Contents

Provenance

  • Oct 3, 2026Published
  • Nov 5, 2026Last substantive update

By LK Wood IV · Published 2026-10-03 · Updated 2026-11-05 · St. Louis County, MO

Docker exit code 137 is a clue, not an OOM verdict. Under GNU Bash’s exit-status rules, a command terminated by fatal signal number N is reported as 128 + N. Linux lists SIGKILL as signal 9, which is why a shell-observed SIGKILL commonly appears as 137. That convention is not proof of the sender or cause: Docker returns the container command’s exit status, so a command can also return 137 itself. Docker documents at least two non-memory ways to send SIGKILL: docker kill uses it by default, and docker stop escalates to it when the process does not exit before its timeout.

Do not restart, recreate, or raise the memory limit yet. Those changes can erase the state and timing that separate a container limit from host pressure, a stop-timeout escalation, an operator kill, or an application-selected exit code.

The job is to bind every observation to the same container ID and time window. Capture first. Change one thing only after the evidence names a cause.

Use the Hardware Diagnostic Workbench to retain that container ID, bounded time window, configured limits, event sequence, kernel or daemon evidence, cgroup counters, and each later test. The record does not interpret exit 137 as an OOM verdict.

The four records to save first

First, find the historical container ID on the daemon that ran the failing service. A name such as worker-1 can already belong to its replacement. Set a narrow interval that has already ended; use UTC or an explicit offset. The example times below are placeholders.

SINCE='2026-10-03T18:20:00-05:00'
UNTIL='2026-10-03T18:30:00-05:00'
(
  umask 077
  set -C
  docker events --since "$SINCE" --until "$UNTIL" --filter type=container \
    --format '{{json .}}' > docker-137-events.jsonl
)

Run that from a private local folder. set -C prevents Bash from overwriting an existing capture. Keep the file private: event attributes can contain names and labels for other containers on this daemon. Docker keeps only the last 256 events; an empty window is not proof that no container died. If the command fails, fix the capture error before interpreting the file.

Find the die line whose exitCode is 137 at the incident time. In that line’s Actor object, copy the full ID. If several exits match, keep them separate. You can read the JSON directly or use the local timeline reader and practice capture below. The remaining commands use that ID, even if the name now points somewhere else.

Second, inspect that exact container. Set CONTAINER to the ID you copied; the placeholder below is intentionally not a valid ID.

CONTAINER='paste-the-full-64-character-ID-from-the-die-event'
docker inspect --type=container \
  --format 'id={{.Id}} name={{.Name}} state={{.State.Status}} image={{.Config.Image}} exit={{.State.ExitCode}} oom_killed={{.State.OOMKilled}} started={{.State.StartedAt}} finished={{.State.FinishedAt}} error={{json .State.Error}} memory={{.HostConfig.Memory}} reservation={{.HostConfig.MemoryReservation}} memory_swap={{.HostConfig.MemorySwap}} oom_kill_disable={{json .HostConfig.OomKillDisable}} restart_count={{.RestartCount}}' \
  "$CONTAINER"

If inspection says No such object, save that result and continue with the historical events and host logs. Do not replace CONTAINER with the service name. If the ID still exists, compare StartedAt and FinishedAt with the incident. State from a run that started after the failure cannot explain the earlier exit. Even when the ID matches, a later run’s exit code, OOM flag or current memory setting is not a historical reading.

Third, if the original ID still exists, save its application output from the same interval. This can show an orderly shutdown request, an application-level allocation failure, or the last completed operation. If the object is gone, use any existing external logs for that ID. Missing logs do not prove a kernel kill.

docker logs --timestamps --since "$SINCE" --until "$UNTIL" "$CONTAINER"

Fourth, check the host kernel and Docker daemon around those timestamps. journalctl accepts space-separated date and time values for --since and --until; the separate UTC values below describe the same interval as the Docker variables above and work on systemd releases that do not accept that RFC 3339 form. These read-only queries explicitly select the current boot:

JOURNAL_SINCE='2026-10-03 23:20:00 UTC'
JOURNAL_UNTIL='2026-10-03 23:30:00 UTC'

journalctl -k --boot=0 --since "$JOURNAL_SINCE" --until "$JOURNAL_UNTIL" --no-pager \
  | grep -Ei 'out of memory|oom-kill|killed process|memory cgroup'

journalctl -u docker.service --boot=0 --since "$JOURNAL_SINCE" --until "$JOURNAL_UNTIL" --no-pager

If the incident occurred before the current boot, run journalctl --list-boots, choose the boot ID whose recorded interval contains the incident, and replace --boot=0 in both commands with --boot=THE_32_CHARACTER_BOOT_ID. If the host does not use systemd, use its actual kernel and daemon log mechanism. Do not substitute an unbounded dmesg | tail and then assume the last message belongs to this incident.

Same name, different run: replay an exit

Docker Compose can recreate a container when its image or configuration changes. A restart poses a different problem: the ID can stay the same, while the Engine resets the exit code and OOM flag when the container starts. Neither a healthy replacement nor OOMKilled=false on a later run clears an earlier incident.

Save the Python 3 timeline reader and synthetic event file in one folder. This is a practice incident, not an outage recorded on this site. The reader opens the local file you name; it does not connect to Docker or require extra packages. List exits of 137, compare records carrying the old service name, then replay a single full ID:

python3 docker-137-window.py docker-137-synthetic.jsonl
python3 docker-137-window.py docker-137-synthetic.jsonl --name worker-1
python3 docker-137-window.py docker-137-synthetic.jsonl --id a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0
Practice incident (UTC)What the capture actually showsNext move
23:22:10 · first exit, ID beginning a0The same ID has an oom event one second before die exit=137.Check the matching kernel record or cgroup counters to locate the memory boundary. The event alone does not tell you what limit to raise.
23:24:08 · second exit, same IDA start at 23:22:11 separates the two runs. The second run shows kill signal=9 followed by die exit=137.Check who requested a stop or kill. Do not carry the first run’s OOM event across the intervening start; this capture does not identify the sender.
23:24:12 · worker-1 is running againThe old ID was destroyed. The new create and start use a different ID beginning b1 with the same name.Keep the old events. Inspecting worker-1 now tells you about the replacement. If the relevant old evidence is missing, stop at “cause unknown.”

For your own incident, substitute docker-137-events.jsonl for the practice filename and copy the full ID from your own output. Match the individual die by time, not just the container. A Moby user’s actual incident report includes ExitCode: 137 and OOMKilled: false together; that is a useful counterexample, not a diagnosis of your host.

Read the evidence as a set

What you capturedWhat it establishesWhat it does not establish
Exit code 137 onlyThe container command reported 137Who or what chose that value
State.OOMKilled=trueDocker recorded an OOM-killed stateWhich memory boundary caused it, or how much memory is safe
Docker oom event in the same windowDocker emitted an OOM event for that containerWhether the configured limit is simply too small, the host is overcommitted, or the workload is leaking
kill followed by die, with the signal and matching timeA signal was sent in Docker’s event sequenceWho initiated it unless the surrounding daemon/orchestrator record says so
Kernel OOM record naming the victim/cgroupThe kernel selected a process under memory pressureThat blindly raising this container’s limit protects the host
The exact cgroup’s local oom counter increasedAn allocation in that cgroup was about to fail because of that cgroup’s memory limitWhich process was selected, or how much memory is safe
cgroup oom_kill counter increaseAn OOM killer killed a process in that cgroupWhich cgroup supplied the limiting boundary, or that the killed process was the container’s PID 1
State.OOMKilled=false and no memory evidenceNo OOM cause has been demonstratedThat memory pressure is impossible

That last row prevents a reasoning error. A false flag is not permission to declare “not memory,” just as a 137 is not permission to declare “memory.” Keep the cause unknown until another record closes the gap.

Separate a container boundary from host pressure

The memory value in the inspect record is the hard memory limit in bytes. A zero means Docker did not configure a per-container hard limit. It does not mean the host, VM, systemd unit, parent cgroup, or orchestrator is unlimited.

Docker’s resource documentation distinguishes a hard --memory limit from a softer --memory-reservation. It also warns that without constraints a container can use as much memory as the host kernel scheduler allows, and that host OOM can select processes when the machine cannot perform essential work.

Use this split:

  1. A nonzero container limit and matching Docker oom event do not identify the limiting boundary by themselves. A kernel OOM record naming that cgroup as the limiting boundary, or a same-window increase in oom from that exact cgroup’s memory.events.local, corroborates the container boundary. An oom_kill increase alone only says a process in the cgroup was killed by an OOM killer. Once the boundary is identified, measure the workload and decide whether the limit, the application’s own memory configuration, or a leak is wrong.
  2. No container hard limit plus a host kernel OOM record points to host or parent-boundary pressure. Giving one container more memory is not a fix; it has no private boundary to raise.
  3. A kill or stop-timeout sequence with no correlated memory evidence points away from an OOM diagnosis. Docker documents that docker stop sends the configured stop signal first and forcibly sends SIGKILL after the timeout.
  4. No correlated record stays unresolved. Preserve the worksheet, increase observability, and wait for a second incident rather than editing limits by guesswork.

Read cgroup counters only where the host exposes them

Do not paste a path from someone else’s machine. Docker’s cgroup driver, systemd integration, parent cgroup and cgroup version determine the location. Locate the container’s cgroup through that host’s runtime configuration, replace the placeholder assignment below with the path you verified, then read the files that actually exist there. These commands only read those files.

For cgroup v2, read the following Linux kernel memory-interface files with the read-only commands below:

CGROUP_PATH='/replace/with/verified/container-cgroup-path'
cat "$CGROUP_PATH/memory.max"
cat "$CGROUP_PATH/memory.high"
cat "$CGROUP_PATH/memory.events"
cat "$CGROUP_PATH/memory.events.local"

memory.max is the hard limit. memory.high applies reclaim pressure but does not itself invoke the OOM killer. In memory.events, oom counts occasions when allocation was about to fail at the limit, while oom_kill counts processes in the cgroup killed by an OOM killer. The regular file is hierarchical; memory.events.local is local to that cgroup.

For a host still using cgroup v1, the kernel documents the older files below. The same documentation marks the v1 soft-limit and OOM-control interfaces deprecated, so read them as legacy evidence rather than advice for new configuration.

CGROUP_PATH='/replace/with/verified/container-cgroup-path'
cat "$CGROUP_PATH/memory.limit_in_bytes"
cat "$CGROUP_PATH/memory.usage_in_bytes"
cat "$CGROUP_PATH/memory.failcnt"
cat "$CGROUP_PATH/memory.oom_control"

If the container was removed or its cgroup disappeared, say that the counters were unavailable. Do not replace missing evidence with a guessed path or a counter from a new container.

Copy this incident worksheet before changing anything

Docker exit-137 incident

Observed at (timestamp + timezone):
Host name / VM:
Docker Engine version:
Container name:
Full container ID:
Incident die timestamp (UTC):
Any later start for the same ID, or replacement ID using the same name?:
Image + immutable digest if available:
Compose project / service or launcher:

State
  ExitCode:
  OOMKilled:
  StartedAt:
  FinishedAt:
  RestartCount:
  State.Error:

Configured boundary (bytes as reported by inspect)
  Memory:
  MemoryReservation:
  MemorySwap:
  OomKillDisable:
  Parent/orchestrator limit, if separately proven:

Same-window evidence
  Docker oom event: yes / no / unavailable
  Docker kill event + signal:
  Docker die event + exitCode:
  Stop/restart event immediately before death:
  Kernel OOM record naming process/cgroup: yes / no / unavailable
  cgroup version and exact path:
  memory.events(.local) before change, if retained:
  Application's last timestamped log line:

Classification
  [ ] container/parent OOM is correlated
  [ ] host OOM is correlated
  [ ] stop timeout or external signal is correlated
  [ ] application returned 137 without proven SIGKILL
  [ ] unknown — preserve evidence and observe again

One change to test next:
Result and timestamp:

The empty boxes matter. “Unavailable” is evidence quality; it is not a “no.”

Choose the next action from the proven branch

Container or parent memory boundary proven: compare the configured boundary with the application’s measured peak and memory model. A Java heap, Node heap, database cache, shared memory, native allocation and filesystem cache do not all appear under one application setting. Fix the thing consuming memory or set a tested boundary with headroom. The Docker RAM planner estimates future host capacity for a known stack; it does not diagnose this incident and should not replace the capture above.

Host pressure proven: inventory every workload and host reserve before changing one container. Docker warns that the Linux kernel can kill processes, including Docker or other important applications, when the host cannot satisfy essential memory work. Moving the failure from one container to the whole host is regression, not remediation.

Stop-timeout escalation proven: find why PID 1 did not handle the configured stop signal or finish within the timeout. Docker notes that shell-form ENTRYPOINT and CMD run below /bin/sh -c, which does not pass signals to the executable. Fix signal handling or set a justified grace period; do not change memory because the final number happened to be 137.

Manual/orchestrator kill proven: trace the actor from the daemon, service, scheduler or automation log. The container behaved as instructed.

Cause still unknown: keep the worksheet and improve capture for the next occurrence. A restart may restore service, but it does not turn an unknown cause into an OOM diagnosis.

What I verified and what I did not

I verified the command syntax and field semantics against Docker’s current documentation and the cgroup file meanings against current Linux kernel documentation on 2026-10-03. I did not run an OOM experiment on a Docker host for this article, so none of the worksheet values or branches is presented as a TechFuelHQ measurement.

The downloadable practice capture exercises the local event reader with two exits on the same container and another container reusing its name. It is not a live Docker or OOM test.

The article also does not assume Kubernetes. Pod eviction, Kubernetes OOMKilled status, pod limits and node-pressure decisions add another control plane and need their own evidence path.

Sources


If the incident is actually disk pressure rather than memory pressure, the Docker cleanup guide separates disposable cache and images from volumes that can contain the only copy of data.

Frequently asked questions

Does Docker exit code 137 always mean out of memory?
No. Docker returns the container command’s exit status, so the command can return 137 itself. A SIGKILL can also produce 137, and Docker documents more than one route to SIGKILL: docker kill sends it by default, while docker stop sends a stop signal and escalates to SIGKILL if the process outlives its timeout. Save the state, event timeline and kernel log before assigning a cause.
Does State.OOMKilled true prove the container memory limit was too low?
No. It is strong evidence that Docker recorded an OOM kill, but it does not by itself identify whether a container boundary, a parent cgroup, or host pressure was responsible. Correlate the flag with the configured limit, an oom event, the kernel log and, where retained, the cgroup’s OOM counters.
Can a container be killed by memory pressure when State.OOMKilled is false?
Do not use false as an all-clear. Docker’s flag records container state, while the kernel log and cgroup counters describe memory events at other layers. It does not replace either record. A false flag plus no correlated memory evidence means the cause remains unproven.
My service is running again. Can its current OOMKilled flag explain yesterday’s exit 137?
Match the full ID on the old die event and the time of that particular run. A restart can keep the ID but reset the inspected state; a replacement can keep the name but have a different ID. Use the old run’s events and same-window host logs. If those records are gone, the cause stays unknown.
Should I increase the memory limit after an exit 137?
Not until you capture the evidence and identify the boundary. Raising a limit can move pressure onto the host, and it does nothing for a manual kill, a stop-timeout escalation, or an application that returned 137 itself. Once a memory boundary is proven, compare the workload’s measured peak with the configured limit and leave headroom for non-heap memory and bursts.
Should I use oom_kill_disable to stop Docker OOM kills?
Not as a general fix. Docker warns against disabling the OOM killer without a memory limit because host processes may then be selected, and the Linux cgroup v1 documentation says tasks can hang in the OOM wait queue when the killer is disabled. Diagnose the boundary and workload first.

Updates and corrections

Last updated
Methodology
See our methodology for research and review standards.
Update log
  • 2026-11-05 — Changed the capture order so a reused service name cannot silently select a replacement. Added a local event-file reader and a synthetic replay of two exits on one ID followed by a new ID. The replay tests event selection; it does not reproduce a real OOM.
  • 2026-10-03 — Created a read-only incident-capture path for Docker exit 137. The article deliberately treats 137, State.OOMKilled, Docker oom events, kernel logs and cgroup counters as separate evidence instead of turning any one value into a universal diagnosis. Docker Engine CLI/resource documentation and Linux cgroup v1/v2 documentation were opened on 2026-10-03. No Docker daemon was available here, so no example output is presented as a TechFuelHQ measurement.
Corrections
Spotted an error? Email a correction with the affected line and what you found. The draft includes this page’s address. Verified changes are recorded here. Contact details.

About the author

I am Lowell K. Wood IV. I research, write and maintain everything on TechFuelHQ from St. Louis County, Missouri.