Restart an OOM killed docker automatically (lemmy.world)

submitted 5 months ago by RustyNova@lemmy.world to c/selfhosted@lemmy.world

29 comments fedilink hide all child comments

I got an home server that is running docker for all my self hosted apps. But sometimes I accidentally trigger Earlyoom by remotely starting expensive docker builds, which kill docker.

I don't have access to my server outside of my home network, so I can't manually restart docker in those situations.

What would be the best way to restart it automatically? I don't mind doing a full system restart if needed

you are viewing a single comment's thread
view the rest of the comments

[-] Badabinski@kbin.earth 16 points 5 months ago

The other person may have responded with a fair amount of hostility, but they're absolutely correct. I run Kubernetes clusters hosting millions of containers across hundreds of thousands of VMs at my job, and OOMKills are just a fact of life. Apps will leak memory, and you're powerless to fix it unless you're willing to debug the app and fix the leak. It's better for the container to run out of memory and trigger a cgroup-scoped OOM kill. A system-wide OOM kill will murder the things you love, shit in your hat, and lick your face like David Tennant licked Krysten Ritter.

[-] RustyNova@lemmy.world 1 points 5 months ago

Oh that's not a problem to let a container get killed. It's perfectly fine. What I want is just not crippling my whole server because one container did a funny.

If it keeps docker and the portainer VM I'll be 100% ok, because I can just restart it. I don't want to have remote access to my server outside of my home for security reasons, so this is just the bare minimum

[-] Findmysec@infosec.pub 5 points 5 months ago

Those remote access fears can be solved with a wireguard VPN

[-] null@slrpnk.net 0 points 5 months ago

I don't want to have remote access to my server outside of my home for security reasons, so this is just the bare minimum

What are your security concerns?

this post was submitted on 12 Jul 2024

25 points (83.8% liked)

Selfhosted

40782 readers

1626 users here now

A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.

Rules:

Be civil: we're here to support and learn from one another. Insults won't be tolerated. Flame wars are frowned upon.
No spam posting.
Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it's not obvious why your post topic revolves around selfhosting, please include details to make it clear.
Don't duplicate the full text of your blog or github here. Just post the link for folks to click.
Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).
No trolling.

Resources:

selfh.st Newsletter and index of selfhosted software and apps
awesome-selfhosted software
awesome-sysadmin resources
Self-Hosted Podcast from Jupiter Broadcasting

Any issues on the community? Report it using the report flag.

Questions? DM the mods!

founded 2 years ago

MODERATORS

HybridSarcasm@lemmy.world

HybridSarcasm@lemmy.hybridsarcasm.xyz