Can antiX be configured to still be responsive when running out of memory?

[Member rayluo]

Currently, antiX runs fine even on low end computers. But then when you (accidentally?) open one more tab in your browser, it would become the last straw and then your entire computer becomes very unresponsive (a.k.a. thrashing), and at that point you would likely need a hard shutdown and reboot. Such symptom would happen, seemingly regardless of whether I enable swap on hard disk, or use no swap at all.

I remember that, back in my Windows XP days, I could disable swap (it is named “pagefile” in Windows), so that my computer was fully operating on RAM only. When I’m about to open a new app Xyz, and if the computer was running out of memory, I would get a pop-up error window saying something like “there is not enough memory to run your program Xyz”. Of course, if that “offending” app happens to be my browser attempting to open a new tab, that entire browser process would be killed, and effectively free up all its previously consumed RAM. At least my entire desktop session remains as responsive as usual, therefore I would not lose – for example – my Word document being edited in a separated Word app.

Can antiX (or Linux, for that matter) be configured to still be responsive when running out of memory?

[Member PPC]

Helpful

Up

0

::

Hi. You can disable swap, if you want to. I think the command is:

swapoff -a

But I have no idea what the OS will do if the memory limit is exceeded…

Some time ago I remember reading about something that does exactly what you want: when the available is fully used, I think the last opened program/more memory using program was closed- but I have no idea where I found that info…
My point: it can be done, and in antiX. Google “linux close program when out of memory”, try to find that info. If I recall where the link, I’ll post it here (I’m not even sure it was in a thread here or not).

P.

[Member sybok]

Helpful

Up

0

::

Recently, Fedora implemented a so called Early OOM (Out of Memory) to deal with this issue.
My quick search lead me to find that there is a package ‘earlyoom’ in the repositories.
It is described at e.g. Debian packages and/or linuxx.

Changing parameter swappiness is also possible.

• This reply was modified 2 years, 11 months ago by sybok.

[Member olsztyn]

Helpful

Up

0

::

Thanks rayluo for reporting results of this very interesting stress test. I believe this is not specific to antiX but I had no idea Linux would not be able to handle such simple condition… Googling around seems to confirm this is happening, so it is not just a unique condition.
I was trying to test this on my laptop(s) but not able to exhaust memory (4GB) so I must rely on such reports, but as soon as I dig out 2GB laptop from my closet I will be glad to re-create this condition and experience this system failure myself… IMHO this seems to expose some fault (or sloppiness) in design of Linux core system as it is nowadays. I do not mean to start a heated controversy here so please disregard this my statement in case it would…
As this is a very interesting discovery (thanks rayluo!) I would like to just bring some wild (and possibly incorrect, as disclaimer) general theories which perhaps could be confirmed or corrected:
– Linux used to be cleaner in design thanks to original simplicity.
– Linux system used to be more separated from user. Such folders as /etc folder used to be just for system use, while nowadays they appear to be full of custom, non-system files.
– Proliferation of distros with their own approach to use of parts of system just worsen system reliability but typical abundance of memory helped to not see these issues too often…

This situation seems to be recognized by some and architecture re-designed, such as Intel’s Clear Linux completely separates Linux system from the user files, so as to make the system stable and able to boot even if user messes things up. To accomplish this they abandoned /etc for system use and placed system files somewhere not polluted by user installs. The system no longer uses /etc folder.

Out of curiosity I will remove some memory from a laptop running Clear Linux to test the ‘out of memory’ unresponsive system condition reported by rayluo to check if Intel was able to remedy this issue with this ‘rock solid’ as claimed Linux distro…
Thanks rayluo and Regards…

• This reply was modified 2 years, 11 months ago by olsztyn.

Live antiX Boot Options (Previously posted by Xecure):
https://antixlinuxfan.miraheze.org/wiki/Table_of_antiX_Boot_Parameters

[Forum Admin Dave]

Helpful

Up

0

::

Hmm interesting point.
I have experienced this before as well. The easiest way from my experience is to setup a keyboard shortcut to run a killall command for firefox (as most of my out of memory usage comes from firefox on the one pc). I suppose the second easiest thing to do would be a similar keyboard shortcut. However rather than running killall NAMED_PROGRAM_HERE, you could run a script to check for the process using the highest amount of ram and kill it. For example:
ps -e -o size,pid,comm --sort=-size --no-header |head -1
Would list all processes with the format virtual size, process id, command, sort them by virtual size in reverse order then select the top line/process.
There are probably other items that would work better for a ram statistic in ps (man ps).
From here you can cut the pid column value and insert that into a kill command (or brutally with kill -9) within the script. This relies on you to be the judge of when the system is running out of memory by seeing it limp.

From here you could get more fancy in the script… monitoring the usage of memory via a watch script or a cron job or another program (conky maybe?) and running the script when the system determines high memory usage. This is overkill for me as the number of times and the reasoning I run out of memory are very low (normally catching me by surprise). However my *guess* at the most accurate way to monitor this would be to monitor/watch physical ram use vs the rate of swap within the script. If both are high then run the select highest ram use and kill part of the script. Perhaps with a notification sent when the kill command is successful. I think you can watch the rate of swap by looking at page swap values in /proc/vmstat
cat /proc/vmstat |grep "psw"
of course that would give you an instantaneous look a the total since boot. You would then need to poll this value in your script keeping record of the past two, three, or more times to determine a swap rate / spike. You could probably watch memory usage through /proc/meminfo or through programs like ps_mem.py or free (man free). Maybe there is a program that already does this…

• This reply was modified 2 years, 11 months ago by Dave.
• This reply was modified 2 years, 11 months ago by Dave.

Computers are like air conditioners. They work fine until you start opening Windows. ~Author Unknown

[Member rayluo]

Helpful

Up

0

::

Thanks for all the valuable feedback in such a short time! I did not realize this turns out to be a much more common topic. 🙂 Please keep your opinions/workarounds coming.

PPC wrote:

Hi. You can disable swap, if you want to. I think the command is: “swapoff -a” But I have no idea what the OS will do if the memory limit is exceeded…

Yes, that was my first attempt to address this. Disabling swap on a hard disk has an interesting effect in that the computer would remain responsive when you reach to roughly 90% or even 95% of memory being used (as reported by antiX’s Task Manager). The computer remains responsive at this level, because there is no hard disk swap I/O to be conducted. If a swap on hard disk was being used, thrashing would likely already happen. However, unfortunately and disappointingly, somehow the unresponsiveness would still happen once you get closer to 100%. And, given that you did not enable swap in the first place, there is less overall memory available for your desktop session. So, that is not really a solution.

olsztyn wrote:

Thanks rayluo for reporting results of this very interesting stress test. I believe this is not specific to antiX but I had no idea Linux would not be able to handle such simple condition… Googling around seems to confirm this is happening, so it is not just a unique condition.
I was trying to test this on my laptop(s) but not able to exhaust memory (4GB) so I must rely on such reports, but as soon as I dig out 2GB laptop from my closet I will be glad to re-create this condition and experience this system failure myself… IMHO this seems to expose some fault (or sloppiness) in design of Linux core system as it is nowadays. …

This situation seems to be recognized by some and architecture re-designed, such as Intel’s Clear Linux completely separates Linux system from the user files, so as to make the system stable and able to boot even if user messes things up. To accomplish this they abandoned /etc for system use and placed system files somewhere not polluted by user installs. The system no longer uses /etc folder.

Out of curiosity I will remove some memory from a laptop running Clear Linux to test the ‘out of memory’ unresponsive system condition reported by rayluo to check if Intel was able to remedy this issue with this ‘rock solid’ as claimed Linux distro…

It was not a stress testing. 🙂 You would easily reproduce this if you are surfing heavy websites in multi tabs, on a 2GB ram computer.

I don’t know how big or deep this issue is. I thought there were supposed to be a configuration for this, otherwise how those Linux servers being able to run stably in last couple decades?

In particular, I do not think Intel’s Clear Linux’s clever reconstructing directories would make any difference here. But I would be happy to be proved wrong here. 🙂

Dave wrote:

I have experienced this before as well. The easiest way from my experience is to setup a keyboard shortcut to run a killall command for firefox (as most of my out of memory usage comes from firefox on the one pc).

Thanks for confirmation!

In one of my recent reply to the One-Tab plugin on Firefox, I mentioned to use just one mouse click to do the same job. Sometimes it would work.

Dave wrote:

I suppose the second easiest thing to do would be … you could run a script to check for the process using the highest amount of ram and kill it.

Sure that would mitigate a little bit. But there are supposed to be a better way for such a common need…

sybok wrote:

Recently, Fedora implemented a so called Early OOM (Out of Memory) to deal with this issue.
My quick search lead me to find that there is a package ‘earlyoom’ in the repositories.
It is described at e.g. Debian packages and/or linux.

Interesting. This sounds like an automated version of the self-developed script approach that Dave suggested. But does that also hint that there is NOT a simple configuration to turn on the “Windows XP behavior” that I described in OP?

sybok wrote:

Changing parameter swappiness is also possible.

I doubt whether that would help, given that the situation would still happen when using no swap at all. But still thanks.

[Member mroot]

Helpful

Up

0

::

I think you may want to look at using zswap. Essentially zswap takes a portion of your ram (the default is 20%) and compresses it. This compressed memory can store a lot more information but is somewhat slower than regular uncompressed ram. It is however vastly faster than the swap on your hard drive. Here is a thread that talks about zswap.

https://www.antixforum.com/forums/topic/using-zswap-instead-of-zram/

Another option is zram which works in a similar way although I think zswap is probably a better choice for most people.

[Member rayluo]

Helpful

Up

0

::

UPDATE: Since mroot’s hint above, I’ve been experimenting zswap with its default setup (i.e. up to 20% of ram being used for zswap, using lzo compression). Due to the lack of quantitative approach to monitor how a zswap setup performs, I do not have a scientific way to judge how much (or little) it helps. Some commenters – who presumably had much more ram available – in those external links referenced by mroot’s zswap post hinted some placebo effect. LOL. In my case, it seems the thrashing would still happen BUT not as bad as before: my computer would slow down noticeably but not halt completely, so I’ve been able to at least switch to my roxterm, run htop, find and kill the offending firefox-esr process, thus bring my computer back to normal without needing a hard reboot. I did that several times today already, so that alone seems like an improvement. Presumably, the cpu busy on compressing and decompressing, is relatively more responsive than a cpu halt for I/O.

Quoted from mroot’s comment from another post:

mroot wrote:

You also might consider posting the output of inxi -Fxzto this thread or the other thread you started as it is easier for people to help you if they have an idea of what you system is since how good the advice they give is system dependent.

Of course, the other approach to improve performance is to take a hardware approach. We don’t talk about it much on this forum because we like to keep hardware for a long time (in my case sometimes way way too long lol). … Sometimes it is better to buy newer hardware even though we would like to avoid doing so.

Sure. I do have other computers that are more recent and performant. It is still a surprise to me though, that my beloved OS would potentially start thrashing at any time when/if I accidentally approaching its ram limit. In that sense, my better computer’s seemingly more reliable performance might just be a false sense of reliability. And all these were exactly the question asked in this thread’s title:

Can antiX be configured to still be responsive when running out of memory, especially if/when swap is disabled? Why wouldn’t it simply give an “Out of memory” error and continue to work?

PS: my inxi output.

$ inxi -Fxz
System:    Host: antix1 Kernel: 4.9.212-antix.1-486-smp i686 bits: 32 compiler: gcc v: 8.3.0 Desktop: IceWM 1.6.5 
           Distro: antiX-19.2_386-full Hannie Schaft 27 March 2020 base: Debian GNU/Linux 10 (buster) 
Machine:   Type: Laptop System: LENOVO product: 1709A33 v: ThinkPad X60 serial: <filter> 
           Mobo: LENOVO model: 1709A33 serial: <filter> BIOS: LENOVO v: 7BET49WW (1.09 ) date: 07/27/2006 
Battery:   ID-1: BAT0 charge: 4.7 Wh condition: 4.8/34.6 Wh (14%) model: SONY 93P5028 status: Unknown 
CPU:       Topology: Dual Core model: Intel T2300 bits: 32 type: MCP arch: M Yonah rev: 8 L2 cache: 2048 KiB 
           flags: nx pae sse sse2 sse3 bogomips: 6650 
           Speed: 1000 MHz min/max: 1000/1667 MHz Core speeds (MHz): 1: 1000 2: 1667 
Graphics:  Device-1: Intel Mobile 945GM/GMS 943/940GML Express Integrated Graphics vendor: Lenovo ThinkPad R60/T60/X60 series 
           driver: i915 v: kernel bus ID: 00:02.0 
           Display: server: X.Org 1.20.4 driver: intel unloaded: fbdev,modesetting,vesa 
           resolution: 1024x768~60Hz, 1920x1080~60Hz 
           OpenGL: renderer: Mesa DRI Intel 945GM x86/MMX/SSE2 v: 1.4 Mesa 18.3.6 direct render: Yes 
Audio:     Device-1: Intel NM10/ICH7 Family High Definition Audio vendor: Lenovo ThinkPad R60/T60/X60 series 
           driver: snd_hda_intel v: kernel bus ID: 00:1b.0 
           Sound Server: ALSA v: k4.9.212-antix.1-486-smp 
Network:   Device-1: Intel 82573L Gigabit Ethernet vendor: Lenovo ThinkPad X60/X60s driver: e1000e v: 3.2.6-k port: 2000 
           bus ID: 02:00.0 
           IF: eth0 state: up speed: 100 Mbps duplex: full mac: <filter> 
           Device-2: Intel PRO/Wireless 3945ABG [Golan] Network driver: iwl3945 v: in-tree:s port: 2000 bus ID: 03:00.0 
           IF: wlan0 state: up mac: <filter> 
           IF-ID-1: irda0 state: down mac: <filter> 
Drives:    Local Storage: total: 63.10 GiB used: 52.66 GiB (83.4%) 
           ID-1: /dev/sda vendor: Hitachi model: HTS541060G9SA00 size: 55.89 GiB 
           ID-2: /dev/sdb type: USB vendor: Toshiba model: MSFT NORB size: 4.00 GiB 
           ID-3: /dev/sdc type: USB vendor: Toshiba model: MSFT NORB size: 3.22 GiB 
Partition: ID-1: / size: 1.54 GiB used: 666.9 MiB (42.2%) fs: overlay source: ERR-102 
           ID-2: swap-1 size: 256.0 MiB used: 252.6 MiB (98.7%) fs: swap dev: /dev/sda6 
Sensors:   System Temperatures: cpu: 80.0 C mobo: 68.0 C 
           Fan Speeds (RPM): fan-1: 2818 
Info:      Processes: 175 Uptime: 2d 18h 31m Memory: 1.96 GiB used: 711.4 MiB (35.5%) Init: SysVinit runlevel: 5 Compilers: 
           gcc: 8.3.0 Shell: bash v: 5.0.3 inxi: 3.0.36

[Moderator Brian Masinick]

Helpful

Up

0

::

My guess is that I/O ontext switching in the majority of scenarios has greater impact than CPU context switching.

Simple reason is that the CPU is almost always quite a bit faster than the peripheral devices.

The only exception I can think of would be on a computer that is heavily CPU bound, a computational server with very little IO.

--
Brian Masinick

[Member rayluo]

Helpful

Up

0

::

rayluo wrote:

UPDATE: Since mroot’s hint above, I’ve been experimenting zswap with its default setup (i.e. up to 20% of ram being used for zswap, using lzo compression). Due to the lack of quantitative approach to monitor how a zswap setup performs, I do not have a scientific way to judge how much (or little) it helps. Some commenters – who presumably had much more ram available – in those external links referenced by mroot’s zswap post hinted some placebo effect. LOL. In my case, it seems the thrashing would still happen BUT not as bad as before: my computer would slow down noticeably but not halt completely, so I’ve been able to at least switch to my roxterm, run htop, find and kill the offending firefox-esr process, thus bring my computer back to normal without needing a hard reboot. I did that several times today already, so that alone seems like an improvement. Presumably, the cpu busy on compressing and decompressing, is relatively more responsive than a cpu halt for I/O.

Yet another UPDATE: my same laptop ran stably for last 38 days of using the following settings:

* (Up to) 40% of my 2 GB ram being used for zswap, using the default lzo compression (Couldn’t get lz4 to work from liveUSB)
* Use 3 GB swap file (Perhaps 2 GB would also be enough, because frequent htop shows consistent around 1GB occupation based on my usage.)

In my workload, I visited some resource-heavy websites frequently. My 2 GB RAM machine would noticeably slow down when the memory consumption shown in htop goes above 1.7-ish GB (I don’t know why that particular number). But it never completely halted in last 38 days. And it can always recover when I close some resource-heavy tabs (using the OneTab plugin for Firefox). So, it is confident to say, zswap+swap helps!

The conclusion:

* zswap + swap is better than no swap at all, especially on low memory machines.
* zswap + swap is technically optional if you happen to have plenty of RAM. A swap-less setup will remain blazing fast until it suddenly passes the no-return point. So, choose wisely.

[Member ModdIt]

Helpful

Up

0

::

Thanks@all for interesting info.
I asked around, very few of us have more than 2 Gig Memory, Firefox seems to be part of the puzzle as is ads tracking telemetry.

We get the system going non responsive with Tor Browser. CPU @100% Pages full of blinking ads, trackers firing away, CSS running rife
and suspect we are also running other more hidden nefarious things in background as network traffic is very high, swap too.

With Latest Firefox, a comprehensive hosts file to reduce ads and tracking. Hidden extensions removed from Fox. No Pocket, No Safe Browsing,
No Snippets and search suggestions, Policy file (from Enterprise policy Generator Addon), Ghacks user.js are used as is u block origin.
Memory usage very rarely reaches swap level even with 5 or 6 (sometimes more) tabs open. No direct google search as the first 15 or 20 results
are always crappy paid redirecting tracking rubbish. Mostly use Metager, Ecosia, Duck.

I also look where popup ads and trash are coming from in the firefox java inspector keep an eye out for tracking stuff and extend blocklist.
For Netflix and Amazon video an extra user is created as hardened browser will not allow drm..

We have Claws on desk 1, Palemoon on 2, Calc on 3 Writer, on 4 Firefox opens on 6 and is the only non auto start mentioned.

@rayluo Maybe list standard pages each in own tab used to reach and test the condition as well as firefox setup would help others to replicate
as far as browser is concerned. Is only firefox open, What is running in background.

• This reply was modified 2 years, 9 months ago by ModdIt. Reason: spelling again

[Member rayluo]

Helpful

Up

0

::

ModdIt wrote:

@rayluo Maybe list standard pages each in own tab used to reach and test the condition as well as firefox setup would help others to replicate
as far as browser is concerned. Is only firefox open, What is running in background.

Well, I don’t think it is necessary to “replicate as far as browser is concerned”. Because:

1. On my machine I use an almost-original antix image, so I don’t think there are much “running in background”. “htop” also shows no other activities, besides Firefox and some extremely-lightweight terminal apps (tmux, vim, etc.)

2. And I don’t even think it has much to do with Ads, because the memory-intensive web pages I referred to, was just a couple big-name online email or online chat application inside a web page. Those pages contains no ads by its nature. Perhaps still contains some tracking, but I doubt we can – or should – mess up with a modern rich-client application in a web page.

So, perhaps these all boil down to what particular web pages an individual would need to visit in a whatever browser, with some particular computer specs. If the workload is approaching the (ram) limit, then so be it. To that end, the title and the original intention of this thread, was to see whether antiX can be configured to NOT getting into thrashing. I found no answer to that specific question, but the zswap+swap seems a reasonable workaround.

[Member AR]

Helpful

Up

0

::

In my antiX 19.2 full I never had the lack of memory on my 2Gb RAM EeePC 1215P. With IceWM (without ROX and wallpapers), and Chrome browser (where I has suppressed all ads), my using of swap had maximum of approximately 50Mb with about 4-6 opened tabs in Chrome plus 1-2 other applications opened. Even if one of other applications was GIMP, and the second was Double Commander. And yes, I use zswap with lz4, but I suspect that my 50Mb of swap doesn’t fit in zswap thresold.

[Forum Admin rokytnji]

Helpful

Up

0

::

I used to fight this stuff back in the day when my gear was on the edge of decay.

https://www.tapatalk.com/groups/antix/limiting-writes-to-ssd-on-eeepc-t2389.html

Sometimes I drive a crooked road to get my mind straight.
Not all who Wander are Lost.
I'm not outa place. I'm from outer space.

Linux Registered User # 475019
How to Search for AntiX solutions to your problems

[Anonymous]

Helpful

Up

0

::

rayluo, from the forum mainpage, perform a search for sysctl and skim/read the tips mentioned
especially the SamK post #4905 (January 9, 2018) and the skidoo post #3555 (December 9, 2017)

[Author]

Posts