.dotfiles management, backup, deployment strategies…

Forum Forums General Software .dotfiles management, backup, deployment strategies…

This topic contains 43 replies, has 6 voices, and was last updated by BobC Feb 19-11:24 pm.

Viewing 14 posts - 31 through 44 (of 44 total)
  • Author
    Posts
  • #18554
    Member
    Avatar
    BobC

    I got kdiff3-qt installed and found/modified a .desktop entry for it to get it onto the menu.

    Reading about rsync, as it seems a simple solution, and given my directory structured approach, should work, I think…

    Still thinking about frugal and the “overlay” file… It seems that might be the “elegant” solution, if I could figure it all out…

    Aptik can save or restore specified files or folders or scripts. I am guessing it would have those lists stored in files that could be generated from my changes folder, but haven’t fully investigated the idea.

    #18559
    Member
    Avatar
    BobC

    Ok, i tried the frugal with persistence route tonight on the Dell D620. I gave it a 15 gb partition and gave 8 gb of that for root persistence and 1.5 gb for home persistence. I ran out of disk trying to install my package list which was 321 new and about 100 upgraded and was 490 mb of package downloads. The machine has 2 gb of memory and a 4 gb swap partition.

    I had planned to put my changes in the gz file skidoo and bitjam were talking about.

    My only idea left to go that route is the install to hard drive add my package list do my aptik install and restore of dot files, and then make a new live USB from it and then do a frugal install with persistence of that, and put my changes in above it to load at boot. I could always backup my home stuff with aptik, and would create a script to update the boot time overlay of changes and run it anytime I made more changes that I wanted to make permanent.

    Any thoughts or suggestions?

    #18561
    Moderator
    masinick
    masinick

    @BobC: If the total available size is 15 GB, then redistribute the file systems you’re using a bit. 4GB is needed for swap unless you do not use double the size of memory for your swap partition (but that can affect hibernation and suspend operations. Perhaps you can increase the home partition by 1-2 GB and subtract a small amount from the root so that it all fits, or if possible, grant 20-30 GB, then enlarge the space of root and home. Then you should have plenty of room.

    For what it’s worth I have a 10 GB partition on my hard drive and I’m only using 25% of it (except perhaps in the middle of in-memory updates that add disk space temporarily). I’ve used moderate-sized root and home partitions when I’ve run live. Big updates can fill up volatile memory in such configurations; more than likely that is where it is actually having a problem. 2 GB isn’t a huge amount to easily handle big in memory package updates.

    Brian Masinick

    #18564
    Member
    Avatar
    BobC

    It did complete on a machine with 16 gb of memory. I think anti told me before that more memory was needed to do a big software install on a frugal-persist system.

    On the 16 gb machine, I thought I had done it all correctly, but for some strange reason the /home persistence didn’t create on the first shutdown, so I lost all my /home stuff on the next boot. I also must have missed a step, because there is no way to boot into it without booting from the flashdrive that I could find. I was looking for a file to add to my grub 40-custom file in the boot setup on hard drive, but didn’t find the file.

    Not sure where to go next with it because it won’t let me setup or configure /home persist, saying failed to run persist-config as user root when I ran it from control centre, so I think I’m at reload, and don’t know where I goofed.

    #18566
    Member
    Avatar
    BobC

    I erased the HD on the old Dell D620 and started fresh with a 9 gb antiX boot, 4 gb swap, 30 gb for isofiles, and 100 gb for persistence.
    I loaded and installed antiX-17.3.1_x64-full, then used my packagecomp and added all my packages, and installed aptik and aptik-gtk.

    Note that I didn’t restore my home stuff or my changes yet, so the system is pretty clean at this point.

    Next I will restore my /home and then make another snapshot, and then create a live USB of it, do a frugal install of it with persist_all, and then I will try to overlay my changes…

    #18567
    Member
    Avatar
    BobC

    I got all my changes done and made a Live USB of it.

    I want put the frugal copy and persistence files in a folder of a partition on the hard drive so it can run from there when selected by grub at boot.

    Does anyone have an example of doing that or a how to?

    #18574
    Member
    Avatar
    skidoo

    The FAQ page provides an example docs-antiX-17/FAQ/frugal.html
    and search within page boot-params.html search for “frugal” shows a few more details.
    You might find further examples by searching “fdev”, “flab”, and/or “fuuid” within the forum archives.

    • This reply was modified 4 months ago by skidoo.
    #18586
    Member
    Avatar
    BobC

    I got it onto a 3rd machine, completely different than the other two, this one an Alienware fast I7 laptop which I boosted with a very fast 250 gb Samsung EVO 850 SSD to make it really, really screaming fast.

    So my method of saving and deploying did work, and I installed frugal_all to a 2nd partition, and added the grub.entry to my 40_custom on the main partition, as well as a live ISO loopback with persist_all also, and now have grub able to boot any of the three…

    Thanks for everyone’s help 🙂 All is well!

    40_custom

    #!/bin/sh
    exec tail -n +3 $0
    # This file provides an easy way to add custom menu entries.  Simply type the
    # menu entries you want to add after this comment.  Be careful not to change
    # the 'exec tail' line above.
    
    menuentry "antiX 17.3.1 (Helen Keller) Frugal Install with persist_all" {
        search --no-floppy --set=root --fs-uuid 3c26982a-b823-44c1-8c02-3df3dd1fa51f
        linux /antiX-Frugal-4.9.146-antix.1-amd64-smp/vmlinuz bdir=antiX-Frugal-4.9.146-antix.1-amd64-smp hostname=alien buuid=3c26982a-b823-44c1-8c02-3df3dd1fa51f nowicd vga=791 persist_all tz=America/Chicago quiet splash=v disable=lx
        initrd /antiX-Frugal-4.9.146-antix.1-amd64-smp/initrd.gz
    }
    
    menuentry "antiX 17.3.1 ISO Loopback with persist_all" {
        set isofile='/antiX-17.3.1_x64-full/antiX-17.3.1_x64-full.iso'
        loopback loop (hd0,3)$isofile
        linux (loop)/antiX/vmlinuz hostname=alien fromiso=$isofile bootdev=/dev/sda3 persist_all pdev=/dev/sda3 tz=America/Chicago nowicd vga=791
        initrd (loop)/antiX/initrd.gz
    }
    
    #18598
    Member
    Avatar
    BobC

    I was trying to use the frugal with persist_all method, and use the xtra.tgz tarball in the /antiX folder described in the thread posted on page 1 here

    Synch between live USB and installed system?

    I definitely like the general idea, and got it to work as designed, but couldn’t figure out how to use it for my situation. It needs to be there on the USB before the frugal copy is made is the problem.

    I was trying to figure out how I could have a way to put a tarball somewhere after the frugal is copied to the drive, and when the frugal boots it would automatically overlay the files in at boot time. Or maybe find a way to create a third overlay that loads on top of the others that I could load after booting. I am looking for a way to test something new without messing up my persistence root and home.

    Any thoughts? And did you like my persist_all loopback setup? I am trying to figure a way that I can test various scenarios without needing to allocate 10 or 20 gb of SSD to each test. So if I load an antiX Lumina, Openbox, LXDE, XFCE, Cinnamon, Enlightenment, or Budgie system (all of which I have) for example, I can keep it out there in a compressed format, fully configured and ready to boot, in case someone wants to know what I did to make it work, or be able to test something else on it again 6 months later.

    #18599
    Member
    Avatar
    skidoo

    suggested background reading:
    https://www.antixforum.com/forums/topic/init-a-modular-linuxfs/
    https://www.slax.org/customize.php
    https://www.slax.org/blog.php
    websearch “pivot_root” (cannot use in initramfs context) and “switch_root” and “overlayroot” and/or “ubuntu overlayroot”

    can do some fancy stuff with bind mounts, just bear in mind that only the topmost overlay layer will be writable.

    “compressed format, fully configured and ready to boot”
    already linked in my previous message, quoting here the seemingly relevant part

    http://download.tuxfamily.org/antix/docs-antiX-17/FAQ/boot-params.html

    By default, we look for the linuxfs file and and persistence files in the antiX directory on the boot device. If you want to boot more than one Live system on the same device or if you want to do a frugal install, you should change that directory and use the bdir parameter to point to the new directory.

    bdir=<dir> The boot directory. This is the directory where we look for the linuxfs (squashfs) file, persistence files and live-remaster files. The default is /antiX. The leading slash is optional.

    “So if I load an antiX Lumina, Openbox, LXDE, XFCE, Cinnamon, Enlightenment, or Budgie system”
    bdir=/my/lumina_linuxfs
    bdir=/path/to/my/openbox
    bdir=/path_to_budgie_system

    #18602
    Member
    Avatar
    skidoo

    another potential source of inspiration for ya:
    run FatDog64 in virtualbox https://distrowatch.com/table.php?distribution=fatdog
    and inspect the various /usr/sbin/fatdog*.sh scripts, e.g.
    fatdog-remaster.sh
    fatdog-restore-basepkg.sh
    fatdog-save-multisession.sh
    fatdog-merge-layers.sh

    related note:
    the FatDog ISO builder (analagous to antiX Build-iso) tool is available here http://distro.ibiblio.org/fatdog/iso/builder/

    #18605
    Member
    Avatar
    BobC

    Skidoo, you are a wealth of knowledge!

    If sounds like I was thinking in the right general direction but just didn’t have the correct terminalogy, or knowledge of what actually exists that could possibly work. I was reading about overlayfs but will switch to your reading list. I need to boot in normal, frugal and loop back modes and try to see if I can find an overlay method that will work with all 3, and then test it…

    And bdir… I did see that but didn’t find examples where I was looking. I will definitely give it some tries.

    Thanks for all the ideas. I’ll post what I find that works so others looking for similar functionality can benefit as well.

    #18646
    Member
    Avatar
    skidoo

    trying to figure a way that I can test various scenarios without needing to allocate 10 or 20 gb

    live init already has recognition of, special handling of, the presence of multiple rootfs files.
    If a *.new is present, that indicates live-remaster was performed during the prior session ~~ the current rootfs gets renamed to .old and .new (generated during the live-remaster operation) gets renamed to rootfs.

    (The .old remains in place just in case you want/need to rollback. It is never automagically removed; it’s on you to perform housecleaning. Ummm, during a subsequent live-remaster operation, IIRC, you’ll be prompted to rename/remove .old if its still present.)

    Surely the handling could be expanded, so that it would present a chooser menu when multiple rootfs files are present (e.g. rootfs.cinnamon, rootfs.myopenbox, rootfs.openboxblue)… and would store the choice so that when you perform a persist-save operation during the session, changes are written back to the appropriate savefile.

    You would setup semi-automatic mode persistence, customize the system (not necessarily all in one session) and establish a “base” rootfs safefile. Maybe (probably) you would configure your $HOME to be mounted from a separate drive. During a given session, after installing, say, cinnamonIsAbigMamaDesktopEnvironment… you would first copy the base “rootfs” to “rootfs.bak” (or “rootfs.generic”), then perform a persist-save, then (manually) copy rootfs to “rootfs.cinnamon”. At shutdown, decline the persist-save prompt. During future liveboot, you could/would choose from a menu which of the rootfs.* files to use for the current session.

    Later, if you wanted to create an openbox variant, you could choose “rootfs.generic” during boot, install openbox-n-stuff… and repeat the process described in the prvious paragraph. Each of the variants will probably be VERY small (compared to your “10 or 20 gb” example). Even KDE with trimmings could be only 1Gb or so.

    Back in 2015, for a few weeks I juggled three rootfs files ~~ experimenting to determine how significant an impact the atime mount option (and a few other variable factors) had on livesession usage. Within less than a month I learned that doing so “is. just. not. worth. the. aggravation.” Inevitably, you’ll repeatedly realize that you forgot to add something(s) into the “base” savefile & wind up re-installing same, redundantly, in each of the clone savefiles (bloating their size). A 32Gb pendrive is US$8 nowadays. Notwithstanding the “thought experiment” I’ve described in this post, I suggest: just use live-usb-maker to clone the base system & each time you wanna dance with a different bigMama visit an alternate universe, pick which of 2-6 pendrives you’ll boot.

    #18731
    Member
    Avatar
    BobC

    Yes, I looked at the filesystem options, and watched videos, but to be honest, I looked and see how each OS maps things differently, and think I will eventually have problems as a result.

    I was/am considering different persistence files for different purposes, but its probably more than I need because it seems large software changes cause trouble blowing up using persistence, so maybe I need to just keep it simple. My thought is a separate bdir for each test environment as you mentioned a couple posts back, and then just use a simple rsync
    to overlay my changes outside of /home, as there really aren’t that many, and I should try to keep changes to a minimum anyway. I can always use persistence when I want to to avoid making full reloads like I do now to do small change tests.

    For my rsync idea, which I think Dave suggested when I created antiX-OpenBox 14 once upon a time, I could have a “changes-HOSTNAME” folder under ~/ and in it i put /etc and /usr in the same structure they exist off /, then I can use the following code to globally overwrite my code. Of course I could be smarter and back any of those existing files up first, and do a diff of some sort like apt-get does when it finds it is installing a fiole over top of a changed file, and those would be useful enhancements, but I think I need to keep it simple to avoid eventually landing myself in a pit of snakes.

    # rsync -avh ./ /

    What do you think? Any big disadvantages besides not being totally cool? I have tested it, and it works as long as I’m religious about copying changed files into there. At this point, everything works, btw. With the rsync I no longer need to copy in my changes a folder at a time. Thanks for all the help and ideas…..

Viewing 14 posts - 31 through 44 (of 44 total)

You must be logged in to reply to this topic.