runtimes need to know about the cgroup limit. Go doesnt size its heap from the cgroup limit by itself, so set GOMEMLIMIT (Go 1.19+) to roughly 80-90% of the container limit and the GC works harder before the kernel kills it. modern JVMs respect the limit, but set heap explicitly with -XX:MaxRAMPercentage=75 so heap plus metaspace, threads and buffers fit. For Node.js, set --max-old-space-size below the limit. the rest of teh container needs headroom beyond the heap.
i raised the limit for the Go microservice to 256Mi after teh first OOMKill. then it happened again:
im afraid to just keep doubling it, my team lead asked is it a leak? 👍 and i honestly dont know how to tell.
I dont want to just throw more memory at the problem every time, especially if its a real leak. How do ppl tell the difference in practice? Which metrics should I actually be looking at for teh Go microservice? what am i doing wrong??
yeah so confirmed: Reason: OOMKilled and dmesg shows the cgorup OOM line for our process.
ok so page cache from file I/O counts toward teh cgroup too, though reclaimable cache is normally freed before an OOM. If the app writes lots of files or uses tmpfs/emptyDir with medium: Memory, those bytes count against the limit. hope that helps :)
page 1 lets gooo
runtimes need to know about the cgroup limit. Go doesnt size its heap from the cgroup limit by itself, so set GOMEMLIMIT (Go 1.19+) to roughly 80-90% of the container limit and the GC works harder before the kernel kills it. modern JVMs respect the limit, but set heap explicitly with -XX:MaxRAMPercentage=75 so heap plus metaspace, threads and buffers fit. For Node.js, set --max-old-space-size below the limit. the rest of teh container needs headroom beyond the heap.
@stackranger yesss ran a load test with a big limit, saw it plateau, then set requests and limits based on that. no more 137s. thx! owe u a beer 🍺
Sign in or create an account to reply.
