Forum › Forums › General › Tips and Tricks › Small HDDs and btrfs (antiX 17.1 full on 1.5GB drive)
- This topic has 13 replies, 6 voices, and was last updated Mar 10-8:52 am by christophe.
-
AuthorPosts
-
April 24, 2018 at 1:55 am #9658Member
rob
The btrfs filesystem’s compression feature is quite handy to shoehorn things onto the small drives in early netbooks and other older systems.
There are three compression modes available in btrfs. Lzo is fast but the compression is not great. Zlib compresses very well but it is comparatively slow. Zstd is available from kernel 4.14 onwards and compresses about as well as zlib, and about as fast as lzo. (Zstd has one big disadvantage in that grub will not boot from a drive that is using zstd on the files contained in /boot/).
I run btrfs with zstd compression on my Asus eee pcs with their tiny and sluggish 4 GB SSDs. On a system with such small and slow storage, it helps spending less time reading and writing from the drive, albeit with a slight increase in cpu usage during access. It will however be slower to do the initial install as it tries to compress every file.
This modified version of the cli-installer from 17.1 has a couple of extra options for btrfs available in the file system selection menu.
There are only two very minor changes involved, both optional and in addition to a standard btrfs install.
The mount options include compress-force=zlib during the installation and in the fstab it creates, enabling and selecting the compression type.
# mount -o defaults,relatime,compress-force=zlibOptionally, during the filesystem creation, options are better tuned to small hard drives (<~16GB). This allows any sector to be used by data or metadata, (preventing erroneous drive-full errors which can occur on small drives otherwise).
# mkfs.btrfs -f -M -O skinny-metadata /dev/sdXHere are the results comparing installation of antiX-full to a 4GB drive. This is using the modified cli-installer, otherwise installed as standard from antiX-17.1-386-full.iso.
Ext 4 Filesystem 1K-blocks Used Available Use% Mounted on /dev/sda1 4061888 2991020 844820 78% /mnt Btrfs Filesystem 1K-blocks Used Available Use% Mounted on /dev/sda1 4061888 2972980 1037839 75% /mnt Btrfs with zlib compression Filesystem 1K-blocks Used Available Use% Mounted on /dev/sda1 4193280 1452200 2557720 37% /mnt Btrfs with zlib compression & small drive options Filesystem 1K-blocks Used Available Use% Mounted on /dev/sda1 4193280 1345712 2858836 33% /mntI also decided to see if it was possible to get a complete install of antiX 17.1-full with all subsequent updates installed onto a 1.5 GB drive, in a usable state.
After the initial installation, packages were updated individually using apt list –upgradable, then apt install for each one. (Some of them will pull others along with them). This was necessary as with the number and size of the updates available, apt upgrade as one operation would have filled the drive. Then the kernel was upgraded to 4.15.14 from the antiX repo. Finally, I installed bleachbit to remove the localizations that i did not require.
In the end, it got there with a touch of room to spare, at around 1.2 GB used for the install and around 300 MB available:
Filesystem 1K-blocks Used Available Use% Mounted on /dev/sda1 1534976 1210212 337056 79% /Although the 1.5GB drive was just as an exercise, this sort of install is useful on early netbooks with their minimal storage, and other older hardware with small slow drives. Once grub is updated to support zstd, it will bring easy performance benefits as well as size savings.
Updated cli-install
#!/usr/bin/env bash # cli-installer for antiX # written by Burt Holland # heavily edited by anticapitalista # btrfs tweaks for small systems added by rob TEXTDOMAINDIR=/usr/share/locale TEXTDOMAIN=cli-installer #Default DISTR0=antiX-17 LIVEUSER=demo ##functions #title #header #help_text #gethome #getroot #repartition #setfs #setpw #yn(yes/no) #getfs #encrypt title() { echo -ne "\e[32m" echo $"CL installer for $DISTRO" echo $"Version 4.1 2017/12/27" echo -ne "\e[0m" echo '' } header() { echo -ne "\e[36m" echo $"Requirements for minimum* installation:" echo -ne "\e[0m" echo '' echo $"antiX-full: hard-disk 4.0GB* RAM 64MB" echo '' echo $"antiX-base: hard-disk 3.0GB* RAM 48MB" echo '' echo $"antiX-core: hard-disk 1.0GB* RAM 48MB" echo '' echo $"antiX-net: hard-disk 0.7GB* RAM 48MB" echo '' echo $"Make sure you are connected to " echo '' echo $"the net BEFORE running this installer" echo '' } help_text() { echo $"Usage: cli-installer [drive]" echo $"Where drive is the name of the drive $DISTRO is to be installed on." echo $" For example: cli-installer sdb" echo $"The default is correct if the computer has only one drive." echo '' # tips for this version echo -ne "\e[36m" echo $"Pre-installation tips:" echo -ne "\e[0m" echo '' echo $"Set language at the boot screen by pressing F2" echo $"or by adding the line lang=xx where xx is your language code." echo $"eg lang=gr for Greek." echo '' echo $"Set timezone at the boot screen by pressing F3" echo '' echo $"Use kbd=dvorak for dvorak keyboard before installing" # end tips echo '' echo $"Press F1 at the boot screen for Help and list of cheatcodes for live use." } gethome() { ans=1 while [[ "$ans" -ne 0 ]] do read -p $"/home partition (hda1, sda2, etc): " hdrv echo '' if [[ ("$hdrv" == "$rdrv") || !( -e "/dev/$hdrv" ) ]] then echo $"$hdrv invalid. Retry:" else ans=0 fi done } getroot() { ans=1 while [[ "$ans" -ne 0 ]] do read -p $"Root partition (hda1, sda2, etc): " rdrv if [[ "$rdrv" > "hd" && "$rdrv" < "he" ]]; then rpre=hd elif [[ "$rdrv" > "sd" && "$rdrv" < "se" ]]; then rpre=sd else rpre=x; fi if [[ !("$rpre" = "x") ]] then gdrv1=-1 for var in "a" "b" "c" "d" "e" "f" "g" "h" "i" "j" "k" do if [[ "$rdrv" > "$rpre$var" ]] then gdrv1=$((gdrv1+1)) gdrv="$var" fi done gpart=-1 var1=-1 for var in "1" "2" "3" "4" "5" "6" "7" "8" "9" "10" "11" "12" "13" "14" "15" "16" "17" "18" "19" do var1=$((var1+1)) if [[ "$rdrv" == "$rpre$gdrv$var" ]] then gpart=$var gpart1=$var1 fi done fi if [[ "$rpre" = "x" || "$gdrv1" -lt 0 || "$gdrv1" -gt 18 || "$gpart1" -lt 0 ]] then echo $"$rdrv invalid. Retry" else ans=0 fi done } repartition() { if [[ -n "$1" ]] then cfdisk /dev/$1 else cfdisk fi } setfs() { umount /dev/$1 > /live/aufs/dev/null 2>&1 echo $"Available file systems for $1 are:" echo " 1)ext2 2)ext3 3)ext4 4)jfs 5)xfs 6)btrfs 7)btrfs with zlib compression 8)btrfs with zlib compression optimized for small drives 9)reiserfs" echo '' ans=1 while [[ $ans -ne 0 ]] do read -p $"Enter your choice 1-9: " fs if [[ "$fs" == "" ]] then fs=ext4 fi ans=0 echo $"You have chosen $fs for $1" echo '' mntopt="defaults,relatime" case $fs in 1) mkfs.ext2 /dev/$1 > /live/aufs/dev/null 2>&1;; 2) mkfs.ext3 /dev/$1 > /live/aufs/dev/null 2>&1;; 3) mkfs.ext4 /dev/$1 > /live/aufs/dev/null 2>&1;; 4) mkfs.jfs -q /dev/$1 > /live/aufs/dev/null 2>&1;; 5) mkfs.xfs -f /dev/$1 > /live/aufs/dev/null 2>&1;; 6) mkfs.btrfs -f /dev/$1 > /live/aufs/dev/null 2>&1;; 7) mntopt="defaults,relatime,compress-force=zlib" mkfs.btrfs /dev/$1 > /live/aufs/dev/null 2>&1;; 8) mntopt="defaults,relatime,compress-force=zlib" mkfs.btrfs -f -M -O skinny-metadata /dev/$1 > /live/aufs/dev/null 2>&1;; 9) mkreiserfs -q /dev/$1 > /live/aufs/dev/null 2>&1;; 10) mkfs.reiser4 -f -y /dev/$1 > /live/aufs/dev/null 2>&1 echo $"Ignore any 'barrier' lines";; *) echo $"$fs invalid. Retry:"; ans=1 esac; done } setpw() { ans=1 while [[ $ans -ne 0 ]] do if (chroot /media/$rdrv passwd $1) then ans=0 else echo -ne "\e[31m" echo $"Passwords are not identical. Retry:" echo -ne "\e[0m" fi done } yn() { x=1 while [[ "$x" -eq 1 ]] do x=0 read -p "$*? " if [[ ("$REPLY" > "x~" && "$REPLY" < "z" ) || ("$REPLY" > "X~" && "$REPLY" < "Z") ]] then ans=1 # yes elif [[ ("$REPLY" > "m~" && "$REPLY" < "o" ) || ( "$REPLY" > "M~" && "$REPLY" < "O") ]] then ans=0 # no elif [[ -z "$REPLY" ]] then ans=-1 # default else x=1 echo $"Invalid; retry:" fi done } getfs() { fs1=$(blkid /dev/$rdrv $a|sed -e "s/.*TYPE=\"//"|sed -e "s/\".*//") fs2=$(blkid /dev/$hdrv $a|sed -e "s/.*TYPE=\"//"|sed -e "s/\".*//") } ##End of functions #Run Help if [[ "$1" == "-h" || "$1" == "--help" ]] then help_text echo '' read -p $"Press Enter to exit this script." exit fi # Run as root check if [[ $UID -ne 0 ]] then echo -ne "\e[31m" echo $"Please run this script as root." echo -ne "\e[0m" exit fi # Make sure /live/aufs/dev, /live/aufs/sys, /live/aufs/proc exist when starting script mkdir -p /live/aufs/dev /live/aufs/sys /live/aufs/proc # Make sure fstab is ok and start swap - (needed for antixsnapshot) make-fstab swapon -a # Show requirements kernel="-$(uname -r)" echo '' header echo '' # Repartition or not and set file up system via cfdisk yn $"Do you want to repartition the disk (y/N)" if [[ $ans -eq 1 ]] then repartition $1 echo '===='; echo '' getroot echo '' setfs $rdrv else echo '' getroot echo '' echo $"Deleting the contents of the $rdrv partition." echo $"This may take some time. Please wait...." fi mkdir /media/$rdrv > /live/aufs/dev/null 2>&1 mount -o $mntopt /dev/$rdrv /media/$rdrv > /live/aufs/dev/null 2>&1 rm -r /media/$rdrv/* > /live/aufs/dev/null 2>&1 echo '' # Set up separate /home and mount on /media/$hdrv yn $"Use separate '/home' partition (y/N)" if [[ "$ans" -eq 1 ]] then gethome if [[ !(-e /media/$hdrv) ]] then mkdir /media/$hdrv fi yn $"Is $hdrv a new '/home' partition (y/N)" if [[ "$ans" -eq 1 ]] then hmtp=2 echo '' yn $"Set file system for $hdrv (y/N)" if [[ "$ans" -eq 1 ]] then setfs $hdrv fi mount /dev/$hdrv /media/$hdrv > /live/aufs/dev/null 2>&1 rm -r /media/$hdrv/* > /live/aufs/dev/null 2>&1 else hmtp=1 mount /dev/$hdrv /media/$hdrv fi else hmtp=0 fi getfs #Install grub-pc and clean out downloaded debs yn $"Are you running antiX-net (y/N)" if [[ $ans -eq 1 ]] then echo '' echo $"Time to download grub-pc, locales, keyboard-configuration, console-setup, console-data, lsb-release, sysv-rc-conf" apt-get update && apt-get -y install grub-pc locales keyboard-configuration console-setup console-data lsb-release sysv-rc-conf apt-get -y clean fi #We are now in $rdrv cd /media/$rdrv echo '' echo $"antiX will now be copied to $rdrv." echo $"This may take some time. Please wait...." cp -a /live/aufs/* . #set up fstab echo '# Pluggable devices are handled by uDev, they are not in fstab' > etc/fstab.new echo "/dev/$rdrv / $fs1 $mntopt 0 1" >> etc/fstab.new grep swap etc/fstab >> etc/fstab.new if [[ "$hmtp" -gt 0 ]] then echo "/dev/$hdrv /home $fs2 defaults,relatime 0 2" >> etc/fstab.new arg1=$hdrv else arg1='xxxx' fi echo '# Dynamic entries below' >> etc/fstab.new grep -v '#' etc/fstab|grep -v swap|grep -v proc|grep -v devpts|grep -v $rdrv|grep -v $arg1 >> etc/fstab.new rm etc/fstab mv etc/fstab.new etc/fstab cp etc/group etc/group.bak cp etc/gshadow etc/gshadow.bak echo $"File copy done" echo '' yn $"Install GRUB on MBR (Y/n). No will install to root partition" if [[ $ans -eq 0 ]] then grub=$rdrv else grub=sd$gdrv fi # Copy over non-live boot parameters to the installed system (FIX ME) #cmdline=$(/live/bin/non-live-cmdline | sed -e 's/\\/\\\\/' -e 's/[|]/\\|/') #[ $(uname -m) = x86_64 ] && cmdline="$cmdline" #cmdline="quiet $cmdline" #sed -r -i "s|^(GRUB_CMDLINE_LINUX_DEFAULT=).*|\1\$$cmdline|" /media/$rdrv/etc/default/grub #grub2 must set up bind mountpoints # mount stuff so grub will behave (so chroot will work) mount -o tmpfs --bind /dev/ /media/$rdrv/dev/ mount -o proc --bind /proc/ /media/$rdrv/proc/ mount -o sysfs --bind /sys/ /media/$rdrv/sys/ chroot /media/$rdrv grub-install --target=i386-pc --recheck --no-floppy --force /dev/$grub > /dev/null 2>&1 chroot /media/$rdrv update-grub $grub > /dev/null 2>&1 #/sbin/make-fstab --install /media/$rdrv --mntpnt=/media > /dev/null 2>&1 chroot /media/$rdrv update-initramfs -u -t > /dev/null 2>&1 echo $"GRUB installed on ($grub)" echo '' chroot /media/$rdrv userdel $LIVEUSER rm -r home/$LIVEUSER > /dev/null 2>&1 read -p $"Computer name (default is 'antix1')? " cnam name="" if [[ -n "$cnam" ]] then echo "$cnam" > etc/hostname sed -i s/antix1/$cnam/ etc/hosts fi echo '' echo $"Time to set up localisation" echo $"System locale is set to ..." chroot /media/$rdrv cat /etc/default/locale yn $"Do you want to set up system localisation (y/N)" if [[ $ans -eq 1 ]] then chroot /media/$rdrv dpkg-reconfigure locales fi echo '' echo $"Time to set keyboard layout" echo $"System keyboard is set to ..." chroot /media/$rdrv cat /etc/default/keyboard yn $"Do you want to set up keyboard (y/N)" if [[ $ans -eq 1 ]] then chroot /media/$rdrv dpkg-reconfigure keyboard-configuration fi echo '' echo $"Time to set console layout" echo $"System console is set to ..." chroot /media/$rdrv cat /etc/default/console-setup yn $"Do you want to set up console layout (y/N)" if [[ $ans -eq 1 ]] then chroot /media/$rdrv dpkg-reconfigure console-setup fi echo '' echo $"Time to set timezone" echo $"System timezone is set to ..." chroot /media/$rdrv cat /etc/timezone yn $"Do you want to set up system timezone (y/N)" if [[ $ans -eq 1 ]] then chroot /media/$rdrv dpkg-reconfigure tzdata fi echo '' echo $"Choose which services to run" yn $"Do you want to enable/disable startup services (y/N)" if [[ $ans -eq 1 ]] then chroot /media/$rdrv sysv-rc-conf fi echo '' sleep 1 yn $"Is this a remastered/snapshot install (y/N)" if [[ $ans -eq 1 ]] then # Write code to get buildfstab -r to start on first boot via rc.local (if user wants it?) mv etc/rc.local etc/rc.local2 mv etc/udev/rules.d/90-fstab-automount.rules etc/udev/rules.d/90-fstab-automount.rules.live cp usr/share/antiX/rc.local.install etc/rc.local # (also)Remove live system if it exists chroot /media/$rdrv dpkg -r live-init-antix 2>/dev/null rm /media/$hdrv/$LIVEUSER umount -l /media/$rdrv umount -l /dev/$hdrv echo $"Installation of $DISTRO finished!" echo '' echo $"Reboot computer without CD to start program. ('reboot')" exit fi sleep 1 echo $"Setting up user and root/admin accounts" name='' while [[ -z "$name" ]] do read -p $"Type in your default user name? " name done case $hmtp in 0)chroot /media/$rdrv adduser $name;; 1)echo '';echo $"Note: $hdrv must contain a folder named '$name'." chroot /media/$rdrv adduser $name;; 2)chroot /media/$rdrv adduser $name;; esac echo '' echo $"Type your Password for root:" setpw root sed -i "s/$LIVEUSER/$name/" etc/group sed -i "s/$LIVEUSER/$name/" etc/gshadow sed -i "s/$LIVEUSER/$name/" usr/share/slim/themes/antiX/slim-install.conf 2>/dev/null yn $"Set autologin for $name: (y/N)" if [[ $ans -eq 1 ]] then sed -i "s/#auto_login/auto_login/" usr/share/slim/themes/antiX/slim-install.conf 2>/dev/null sed -i "s/#default_user/default_user/" usr/share/slim/themes/antiX/slim-install.conf 2>/dev/null fi # Copy live configurations to install. For all. echo '' echo $"Cleaning up" /usr/sbin/live-to-installed /media/$rdrv mv etc/rc.local etc/rc.local2 mv etc/X11/xorg.conf etc/X11/xorg.conf.live 2>/dev/null cp usr/share/antiX/rc.local.install etc/rc.local cp usr/share/slim/themes/antiX/slim-install.conf etc/slim.conf 2>/dev/null chroot /media/$rdrv dpkg -r live-init-antix 2>/dev/null cp -r etc/skel/.[a-zA-Z]* home/*/ 2>/dev/null cp -r etc/skel/* home/*/ 2>/dev/null if [[ $hmtp -gt 0 ]] then if [[ $hmtp -eq 2 ]] then mv home/* /media/$hdrv fi rm -r home mkdir home cp -r /media/$rdrv/etc/skel/.[a-zA-Z]* /media/$hdrv/*/ 2>/dev/null cp -r /media/$rdrv/etc/skel/* /media/$hdrv/*/ 2>/dev/null chown -R $LIVEUSER.users /media/$hdrv/* 2>/dev/null fi chown -R $LIVEUSER.$LIVEUSER home/* 2>/dev/null rm -rf media/sd* rm -rf media/hd* rmdir live umount -l /media/$rdrv/dev/ umount -l /media/$rdrv/proc/ umount -l /media/$rdrv/sys/ umount -l /media/$rdrv if [[ "$hmtp" -ne 0 ]]; then umount -l /dev/$hdrv;fi echo '' echo $"Installation of antiX finished!" echo '' echo $"Reboot computer without CD to start program. ('reboot')"- This topic was modified 5 years ago by rob. Reason: included cli-installer
April 24, 2018 at 10:51 am #9670Forum Admin
anticapitalista
::Excellent!
I tested this out in Virtualbox and it works very well.
I’ll include the changes to the cli-installer and see about adding it to the gui installer.Thanks rob
Philosophers have interpreted the world in many ways; the point is to change it.
antiX with runit - leaner and meaner.
April 25, 2018 at 6:58 am #9682Memberrob
::I’ve also patched the gui installer.
The btrfs mixed mode is automatically set on small drives, and there is a choice of no compression, zlib, or lzo compression.
There are changes to minstall.cpp, minstall.h and meinstall.ui….
Which it turns out I can’t upload here.
- This reply was modified 5 years ago by rob. Reason: uploads unsuccessful
Attachments:
April 25, 2018 at 7:02 am #9685Forum Admin
anticapitalista
::Will it upload as a zip/tar file?
If not, could you email it to me at anticapitalista at riseup dot netPhilosophers have interpreted the world in many ways; the point is to change it.
antiX with runit - leaner and meaner.
April 25, 2018 at 11:12 am #9688Forum Admin
dolphin_oracle
::if you forked the code with github, you could also do a pull request.
but a zip file will work as well.
You’re timing is good…we are examining new features for the installer.
April 27, 2018 at 10:01 pm #9749Memberrob
::if you forked the code with github, you could also do a pull request.
Thanks. Just done that now.
April 29, 2018 at 1:23 pm #9776Moderator
Brian Masinick
::Rob, thanks for the work on this! I don’t happen to have an EEE style netbook right now, but you know us “scavengers” who tend to run our fine distribution: I’m sure that there are those who can use this immediately. As always, ideas and contributions – even if not directly included in the distribution – may still be quite useful to those who want to customize their system. In this case, it looks like this plays into the designs already in consideration! Fabulous! (big grin)
--
Brian MasinickOctober 8, 2018 at 5:44 am #12398Memberdelix02
::I’m going to install the new 17.2 .iso on an old Wyse thin client with a 2GB SSD drive on a chip….
Guess, rob’s work would be an excellent option for that machine.
However, the thin client cannot be booted by Grub, so I had and have to use the extlinux/syslinx package.For antiX 16 (based on Debian Jessie) these packaged had been removed. It was a specific bug in Jessie as the warning was never removed even when the original problem causing it was solved. Are they included in 17.2 again ?
As I remember, the earlier versions of the cli-installer had an option to use extlinux instead of Grub as bootloader.
Did you bring back this option in 17.2. ?
Actually, I know how to install and run extlinux on the client, but having this option in the cli-installer would be much more convinient.Great work anyway.
October 8, 2018 at 12:09 pm #12448Memberdelix02
::Bingo !!
17.2 is installed without problems from a 2GB USB stick using the .iso file from the mx server.
EXTLINUX is already included, only an extlinux.conf is missing.
The client booted just as it should do.
The full install occupies 1.27 GB disk space.Seems to be perfect 🙂 – thx guys, you’ve done a perfect job !
In the next days I’ll correct the German translation of the cli-installer as I’m not really happy with te current version.
October 15, 2018 at 7:21 am #12564Moderator
Brian Masinick
::Great collective teamwork!
These are great examples of what this community does well.
Keep up the GREAT work! 😀😎
--
Brian MasinickMarch 9, 2019 at 7:08 pm #19284Moderator
christophe
::I read above that this was going to be incorporated into the cli installer. I tried to use it on the 17.4 cli-installer, but didn’t see the compression options. Is it an oversight, or is there some bug that had it kept out? Is it safe to copy this script, make executable, and run it to install from it?
confirmed antiX frugaler, since 2019
March 10, 2019 at 1:27 am #19290Memberdelix02
::The compression slows down the btrfs file system (at least on elder machines) beyond usability.
I guess, you can test the script but after testing the option I think it is in most cases not worth the effort.March 10, 2019 at 7:10 am #19297Forum Admin
anticapitalista
::I read above that this was going to be incorporated into the cli installer. I tried to use it on the 17.4 cli-installer, but didn’t see the compression options. Is it an oversight, or is there some bug that had it kept out? Is it safe to copy this script, make executable, and run it to install from it?
some bug that had it kept out
for the other question, give it a try and let us know.Philosophers have interpreted the world in many ways; the point is to change it.
antiX with runit - leaner and meaner.
March 10, 2019 at 8:52 am #19301Moderator
christophe
::some bug that had it kept out
for the other question, give it a try and let us know.I tried but couldn’t get it to offer btrfs compression options, & cp error (drive full). That’s ok, because…
The compression slows down the btrfs file system (at least on elder machines) beyond usability.
I guess, you can test the script but after testing the option I think it is in most cases not worth the effort.Makes sense. Oh well – nice thought.
confirmed antiX frugaler, since 2019
-
AuthorPosts
- You must be logged in to reply to this topic.