I’m using linux mint 21.3, and a process (brave aka chrome) sometimes memory leaking, so eats all the RAM, and then linux goes into swap death loop, when everything freezes (sometimes the mouse cursor is moving), and nothing can’t be done, i can just see the HDD led blinking, and do a reset. Is there a way to make the system automatically detect swap death loop, and close the biggest ram user process, and so on?
Not sure if your distro version has a new enough version of systemd, but newer versions have a systemd-oomd service for that. It may not be enabled by default. On older versions you could try early-oom which is not part of systemd. OOM stands for Out-Of-Memory.
I think that systemd oomd is pretty slow. I get a quite a few minutes of unresponsive system while waiting for oomd to do something. Honestly rebooting is faster than waiting for it to work.
I recently installed and enabled earlyoomd on fedora, and on an initial test, I still got an unresponsive system with the default settings, but after around 20 seconds or so, it killed the responsible process and I was able to continue working. Not perfect, but much better.