- This topic has 10 replies, 5 voices, and was last updated Mar 4-5:42 pm by Luciad10.
-
Posts
-
seaken64
I’ve been working on setting up a couple of Live USB drives with persistence. It’s working good but I am unsure of what I am seeing in the Conky.
At first I chose the Persist-all selection with the F5 and after the live system was loaded I tried to do a dist-upgrade. It failed with not enough disk space. I learned from DO’s video that I should select Persist-static before doing the upgrade so I did that. I chose Persist-Static on reboot and after the live system was up I had about 300 MB used on the disk according to Conky. I went ahead and did the dist-upgrade and this time it completed. But my used disk space is now about 1.1GB.
Why did I go from 300MB to 1.1GB? Over 700MB was used by the dist-upgrade? When I setup the persist-all the first time it was not enough space. Why? I don’t know why the disk space used in “static” is so much more than with “all”. And if I re-master and go back to “all” what will happen?
Thanks for any knowledge you can share.
Seaken64
christophe
I’d say that your live system had to pull in a lot of upgrades, and a lot of those files are duplicates (or rather upgrades of others) that are being replaced. But because the “original files” are squashed on the linuxfs on the USB, your new, upgraded files still take up the extra space on your persistence file.
This situation will be rectified once you do a remaster. Then your new linuxfs will be only a little larger than it was before. (Depending on the compression option you choose, it could be a lot more if you use the “faster” compression method or just a little (but will take longer) if you use the one that makes the new linuxfs file the smallest (compresses the best). Those options are given with explanations during the remaster program, so it’s easy to follow.)
But according to my experience, you’ll recover much of that disk space once you remaster. And once you remaster, your root persistence file will go back down to (about) zero.
As a side note: I always run my live USBs and frugal installs with static persistence, so it leaves me more RAM for memory (as opposed to disk storage).
confirmed antiX frugaler, since 2019
BitJam
I chose Persist-Static on reboot and after the live system was up I had about 300 MB used on the disk according to Conky. I went ahead and did the dist-upgrade and this time it completed. But my used disk space is now about 1.1GB.
Why did I go from 300MB to 1.1GB? Over 700MB was used by the dist-upgrade? When I setup the persist-all the first time it was not enough space. Why? I don’t know why the disk space used in “static” is so much more than with “all”. And if I re-master and go back to “all” what will happen?
The root file system on our live system is combined from two parts. There is read-only compressed squashfs file (called linuxfs) that holds the bulk of the file system. Changes to the file system are stored in RAM and these two are combined using a “Union” file system called overlayfs. It appears to be a full read-write file system but it takes up much less space because most of it is compressed. The compression ratio is better than 2.5:1.
With (normal) dynamic root persistence the changes in RAM are copied to the rootfs file at shutdown and are copied back to RAM at start up. The amount of RAM you have available limits the amount of changes you can make to the root file system. If you don’t have oodles of RAM then you will likely run out of space on the root file system when you do a dist-upgrade.
Static root persistence bypasses the RAM and stores changes directly inside the rootfs file. Now the amount of RAM no longer limits the amount of file system changes you can make. Back in the days of usb-1 and usb-2 this could be slow or even painfully slow (with usb-1). If you have usb-3 then static is the way to go.
I believe conky uses the output of the df command (or something similar) to tell you how much space is available on the root file system. With the fancy overlayfs, the entire situation can’t be encapsulated in two numbers. The numbers reported by df and conky tell you the amount of RAM space used by the root file system and the amount of RAM available, or in the case of static root persistence, the amount of space used in the rootfs file and the amount of space available there.
When you do a dist-upgrade, a lot of packages get upgraded which means a lot file system changes need to be stored in RAM or in rootfs. In your case doing the dist-upgrade caused 800 MB to be used. The original size of the entire file system (stored in the compressed linuxfs file) was 2.5G so this was a big upgrade changing roughly 33% of the file-system.
As christophe said, almost all of the rootfs/RAM space will be recovered when you do a live-remaster where we create a new compressed linuxfs file that contains all of your changes so the amount of changes in RAM/rootfs goes back close to zero. If running with static root persistence is too slow then doing a live-remaster will let you go back to using dynamic root persistence again. If you try to got back to dynamic persistence with “persist_all” then we will probably refuse to enable persistence. If we don’t refuse then your system will run out of memory early in the boot process.
One possible difficulty with the live-remaster is it will require roughly 1G or more free space on your live-usb because we make a 2nd copy of the large linuxfs file. If you don’t have enough live-usb space to live-remaster then you can use live-usb-maker to clone your current system to a larger live-usb. The latest version of live-usb-maker in my git repo will offer to copy over the persistence files too but they have to first be disabled. Just boot without persistence enabled to make the clone. You could also manually copy over /antiX/rootfs and /antiX/homefs to the new system.
HTH
Context is worth 80 IQ points -- Alan Kay
Yes, thank you both. This does help. I also found a video from DO that covers the remaster. I still don’t quite get it yet but I will experiment some more so I can see better what is happening.
I have noticed the lag since I switched to “static”. I was hoping to go back to “all” since it seems to be snappier. But some of my machines have only 512Mb RAM, most are 1Gb, and some are 2Gb or more. Not sure how the ram available will affect the persistence. I don’t think it will break the LiveUSB so I will give it a try.
Thanks for the help.
Seaken64
Okay, I did the remastering. But there were a few things that were not clear.
First, it asks to choose Personal or General. I have no idea what to choose. I chose personal. What difference does it make? and why is there a choice?
Second, the compression choices. Does it matter what I choose here if I am trying to make sure I can use this LiveUSB in 512Mb RAM? Or is it just about file size on the USB drive?
(I remastered this on my i7 3.2 with quad cores and 8mb RAM so resources are not an issue here but I want to use this on a lot of systems, one is a Celeron M 1.4 with only 480Mb RAM).
Third, the size of the rootfs file. I got confused here. When I made this LiveUSB persistant I think I chose 3.5 Gb for the size. Now the choice defaults to 250Mb and the selection tops out at about 1.3 Gb or “Customized”. I chose 500Mb because I was wondering if this relates to the persist-all choice and fitting in RAM.
But I really do not understand how this file is used in RAM or on the USB drive. It can’t be directly loaded into ram if I had 3.5 Gb size when I first made it. I originally had this setup as persist-all and it ran fine. I only ran into problems when I went to upgrade.
Anyway. the Conky now says 2.8M/476M. Before I did the remaster it was 1.1Gb/3.5Gb. So it definitely shrank as you said t would. But why don’t I want it to be 3.5Gb?
I am going to try using persist-all just to see what happens. I would like to run mostly from RAM and have less lag. This is a USB 2.0.
Oh, and can I delete the linuxfs.old file once I confirm the boot up?
Seaken64
I rebooted and chose persist-all on the F5. I noted that the F8 save was off, although I had set that to save once before. Does the F8 have to be set each time?
The Conky shows the “/ Disk” as 220K/2.11G
This laptop has 8Gb RAM but the RAM shows 256M/2.66G with Seamonkey open. Maybe because this is 32-bit Live.
Seaken64
Anonymous
F8 should only be used when you want to lock in, make permanent, a changed settings configuration.
On a given boot, you might want to try “feature X” ~~ by skipping F8, featureX is only activated for the current boot session.Yeah, both the onscreen labeltext and the FAQ are quite vague about personal vs general
Personal uses your personal settings General does notcompression choices [..] is it just about file size on the USB drive?
Yes, plus one (minor) side consideration ~~ a more-highly compressed file takes a few (5-10?) extra seconds to unpack during each boot session.
.When I made this LiveUSB persistant I think I chose 3.5 Gb for the size.
Now the choice defaults to 250Mb and the selection tops out at about 1.3 Gb or “Customized”.
I chose 500Mb because I was wondering if this relates to the persist-all choice and fitting in RAM.The size initially selected, bear in mind that you can choose to later increase it.
“Now the choice defaults to 250Mb”
I’m fairly sure it gauges the free space available on the device and sets a “max” accordingly.
On you later run(s), if some older, pre-existing, files were on the drive… with less free space available, a lower max would be offered.Okay, so maybe having the “*.old” files affects the recommended size. I am thinking I want a larger rootfs file but maybe it doesn’t matter? Is the system in the linuxfs file and only the changes are in rootfs? So the size of the linuxfs is more relevant to what I have installed, such as LibreOffice and SeaMonkey, etc.
I’m still confused about this. Does it matter if I set the rootfs to 250Mb or 1.3G or 3.5G?
Seaken64
Confusing because there are (can be) so many permutations
…what I have installed, such as LibreOffice and…
Yes, linuxfs size is relevant to what is pre-installed.
It is also revevant to “Intalled by you and then you performed remaster, which caused it to be moved into the linuxfs”
Unless you choose the “toram” option, the linuxfs content will never occupy RAM.=============================================
rootfs can be RAM-resident (aka “dynamic”) or not, your choice.
homefs storage (which is separate, and is optional) always resides on disk.=============================================
“Does it matter if I set the rootfs to 250Mb or 1.3G or 3.5G?”
Where are you gonna store it?
If you intend to use dynamic persistence “size of rootfs matters” in relation to how much RAM is avialable.
If you intend to use static persistence, yaddyadda only matters in relation to the capacity of your storage media.Thank you skidoo, I THINK I understand now. I will try it out. I was thinking in the right direction I just wasn’t sure which file size mattered for toram running. I haven’t tried toram yet but I’ve been trying to plan for that but was not clear on how to do it. I will keep the file small enough for 1GB RAM machines.
I have an old Pentium M laptop where the hard disk has died. I setup an antiX LiveUSB and it runs fine but I wanted to see if I could put it all in RAM. I have been experimenting with another LiveUSB key with various machines from 480MB to 2GB of RAM. Once I get it in my head how to set it up correcctly I will put the P4 M laptop back into service with a antiX Live and no hard disk.
Thanks for the guidance.
I am watching DO’s video again to see what I missed and re-reading the FAQ pages. After the assistance here it is coming more clear on how to do this.
Seaken64
Luciad10
I’d say that your live system had to pull in a lot of upgrades, and a lot of those files are duplicates (or rather upgrades of others) that are being replaced. But because the “original files” are squashed on the linuxfs on the USB, your new, upgraded files still take up the extra space on your persistence file.
This situation will be rectified once you do a remaster. Then your new linuxfs will be only a little larger than it was before. (Depending on the compression option you choose, it could be a lot more if you use the “faster” compression method or just a little (but will take longer) [url=http://www.comparateur-mutuelle-assurance-sante.com/assurance-obseques]assurances obsèques[/url] if you use the one that makes the new linuxfs file the smallest (compresses the best). Those options are given with explanations during the remaster program, so it’s easy to follow.)
But according to my experience, you’ll recover much of that disk space once you remaster. And once you remaster, your root persistence file will go back down to (about) zero.
As a side note: I always run my live USBs and frugal installs with static persistence, so it leaves me more RAM for memory (as opposed to disk storage).
Thank you very much for the information you provided us. It is true that you must know the tool before using it and that’s what you did.
- You must be logged in to reply to this topic.
