Automated installs

Forum Forums antiX-development antiX Respins Automated installs

  • This topic has 20 replies, 4 voices, and was last updated Dec 1-12:57 pm by Keeely.
Viewing 15 posts - 1 through 15 (of 21 total)
  • Author
    Posts
  • #44519
    Member
    Keeely

    Hi everyone,

    I’ve looked through the customisation documentation and it seems that there’s nothing to do exactly what I’m wanting, which is an auto-install.

    What I’ve done so far is as follows:

    1) Starting with core, extracted contents
    xorriso -osirrox on -indev antiX-19.3_x64-core.iso -extract / extracted_iso_image

    2) Replaced isolinux.cfg with something that will boot immediately into text mode. This seemed to do the job:
    timeout 10
    default safe

    MENU TITLE Welcome to antiX-19.3_x64-core (Manolis Glezos)

    LABEL safe
    MENU LABEL Safe_Video_Mode
    KERNEL /antiX/vmlinuz
    APPEND disable=lx xorg=safe
    INITRD /antiX/initrd.gz

    3) Remastered with
    xorriso -as mkisofs -D -r -J -joliet-long -l -V “Custom antiX core” -b boot/isolinux/isolinux.bin -c boot/isolinux/boot.cat -iso-level 3 -no-emul-boot -partition_offset 16 -boot-load-size 4 -boot-info-table -o ./new_antix.iso ./extracted_iso_image

    I appreciate this may not be the way the developers intend me to customise the system, however I need this customisation step to work on a Mac so I was going to use the Brew port of squashfs-tools for the customisation.

    So this is a good start, as it takes me immediately into the login prompt, however I’d also like to automate the execution of cli-installer. I’ve been looking through the squashfs and initrd and I’m just wondering where I would run the cli-installer from. Should I put it somewhere in /etc/rc.local? Is there a better place? Should I be trying to run it at run level 1, or level 5?

    Thanks for any help!

    #44521
    Moderator
    christophe
    Helpful
    Up
    0
    :D

    What about autologin as root from /etc/inittab?
    1:2345:respawn:/sbin/getty -a root 38400 tty1

    Then add cli-installer at the end of root’s .profile?

    I haven’t tested this exact thing, but it’s how I set demo to autologin on antiX core, and then startx from .profile. So I figure this should work. 🙂

    #44527
    Moderator
    Brian Masinick
    Helpful
    Up
    0
    :D

    The first couple of references refer to automatic login.

    For autologin, if you happen to use LightDM, this *might* be useful:
    https://www.addictivetips.com/ubuntu-linux-tips/enable-automatic-login-on-linux/

    Another solution is along the lines of what Christophe just suggested:
    http://littlesvr.ca/linux-stuff/articles/autologinconsole/autologinconsole.php

    I know that Red Hat Linux offers “Kickstart” automated installations. There may be some ideas you can bring into an antiX installation from them as well. Kickstart definitely deals with the installation aspect.

    https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/6/html/installation_guide/sn-automating-installation

    This guy talks about using a PXE – The Preboot Execution Environment (PXE) is an industry standard client/server interface that allows networked computers that are not yet loaded with an operating system to be configured and booted remotely by an administrator.

    https://www.leifove.com/2016/11/fully-automated-linux-mint-desktop.html

    Combining an automated installation and an automatic login may also be potentially useful.

    One last reference is a project that contains an ISO image for Fully Automated Installation (FAI):
    https://fai-project.org/

    This ought to give you a few options to consider. If you need additional background details, please let us know.

    • This reply was modified 7 months, 2 weeks ago by Brian Masinick.

    Brian Masinick

    #44720
    Member
    Keeely
    Helpful
    Up
    0
    :D

    Thanks for the suggestions. Unfortunately pxe is not an option because I’d have to automatically create that as well leading to a chicken-and-egg situation with the OS install! I think auto-login is the right way, at least that’s what I’ll try first.

    BTW: If anyone else is attempting this the mistake I was making in the beginning was to use squashfs-tools. For re-packing an existing squashfs it seems squashfs-tools-ng is much more appropriate, better written, and has the right feature-set for round-trip unpack-> pack. Really struggled doing the same with basic squashfs-tools, even when I switched to v4.4 (Unfortunately v4.3 is in MX linux stable at time of writing but that’s another story).

    #44723
    Moderator
    Brian Masinick
    Helpful
    Up
    0
    :D

    Were any of the other links helpful? PXE was only one of the possibilities and the least plausible for most people; the other items are pretty easy to implement, but maybe they weren’t what you were looking for.

    When you get a chance, let us know what you came up with and if you achieved a complete solution or simply went in another direction?

    Thanks!

    Brian Masinick

    #44743
    Member
    Keeely
    Helpful
    Up
    0
    :D

    I only skimmed those links, but they all seemed to need either PXE setup or some input after power-on, whereas I want a simple power-on -> install -> power-off. Of course this is quite dangerous, because without care if you leave the disk in your main desktop machine and boot it it’s going to re-partition your hard disk but I plan to add some safeguards (e.g. check that mbr is blank before doing anything).

    #44764
    Moderator
    Brian Masinick
    Helpful
    Up
    0
    :D

    Unless I am reading these references incorrectly, both the Red Hat Kickstart and the FAI offer the PXE service as one of the alternative ways to perform an unattended installation. I think that it bears more investigation because there seem to be about a half-dozen (at least) different ways to perform the installation. While both unattended and PXE are among the choices, as I was reading it there seemed to be multiple ways to do it.

    Shortly after I finished working in a UNIX development group I had a couple of contracts doing various activities. One was a Year 2000 project, another was an assignment evaluating and testing daily builds of a Real Time Operating System. A third project was daily testing of a supercomputer project with redundant network paths, CPU processors and input/output bus and peripheral components.

    All three of these efforts had some potential for writing scripts with reuse. Another guy I worked with wrote a lot of Perl code for the Y2K stuff; I used mostly shell scripts myself. On the other projects, I added some TCL with Tk and later Expect. It’s the Expect – Send capability of TCL with Expect that could also be useful; you could automate as much or as little as you want if the environment is reasonably “deterministic”; if it changes, you can also modify what’s expected and send a different response. That’s another idea I had thought to share previously. I have a feeling that the FAI project may incorporate that type of interaction. It’s up to you, but I think they are at least worth looking into a bit more deeply. I’m almost certain that PXE isn’t the only protocol that can be used; for example, I believe that DHCP, which is quite common, is also usable in these environments, and some variations of FTP – hopefully SFTP, (and maybe SSL and a few others) are available as alternative network methods.

    I can understand the reluctance to have something running with no control at all over it; I know that I could stop those Expect scripts if I wanted to change something along the way; hopefully you can find something acceptable too.

    Brian Masinick

    #46139
    Member
    Keeely
    Helpful
    Up
    0
    :D

    This ended up being a lot more challenging than I had expected. I used squashfs-tools-ng to convert the squashfs to a tarball, so I could edit it in tar form, however then I found that just as there’s nothing to round-trip decompress->compress a squashfs (and have it recreated the same way) there isn’t really anything to recreate a tarball either.

    But if I can get away without changing the size of any files, then I can just update them in-place in the tarball without a re-pack. There is plenty of space in inittab, and also plenty of space in /root/.bashrc, the latter is composed mainly of comments in antiX in case you want to ‘comment-in’ various features, so no problem there, inittab with Christophe’s changes results in a smaller file anyway, so just some whitespace padding needed there.

    That’s all very well, but I still need to find out the offset of the files I need to modify in the tarball. I did this with a simple string search of the tarball. I’m making the assumption that the inittab and bashrc content doesn’t somehow appear in more than one place in the tarball, but that’s not unreasonable I feel.

    Here is the code to do this, in case someone else needs it.

    
    #!/bin/bash
    
    # Name of the antiX core iso of interest.
    ISO_NAME=antiX-19.3_x64-core.iso
    # Subdirectory where we'll extract the iso - THIS WILL GET CLOBBERED!!!!
    EXTRACT_DIR=iso_content
    
    # A python3 script to figure out the offset in a large file where a small file is found.
    offset_script="
    import sys
    small = open(sys.argv[1], 'rb').read()
    blocks = 
    base = 0
    while True:
        base += len(blocks[0])
        del blocks[0]
        blk = sys.stdin.buffer.read(len(small))
        if not blk:
             sys.exit(0)
        blocks.append(blk)
        offset = b''.join(blocks).find(small)
        if offset != -1:
            print(base + offset)
            sys.exit(0)
    "
    
    echo "Removing old extraction directory $EXTRACT_DIR"
    rm -Rf $EXTRACT_DIR
    
    echo "Extracting ISO..."
    xorriso -osirrox on -indev $ISO_NAME -extract / $EXTRACT_DIR
    
    echo "Updating isolinux.cfg for text mode boot and no delays..."
    cat << EOFEOFEOF > $EXTRACT_DIR/boot/isolinux/isolinux.cfg
    
    timeout 10
    default safe
    
    MENU TITLE Welcome to antiX-19.3_x64-core (Manolis Glezos)
    
    LABEL safe
    MENU LABEL Safe_Video_Mode
    KERNEL /antiX/vmlinuz
    APPEND disable=lx xorg=safe
    INITRD /antiX/initrd.gz
    EOFEOFEOF
    
    echo "Converting squashfs to tarball..."
    sqfs2tar -r . -s $EXTRACT_DIR/antiX/linuxfs > out.tar
    
    echo "Extracting inittab..."
    tar xfO out.tar ./etc/inittab > inittab
    echo "Finding offset of inittab..."
    OFFSET=$(python3 -c "$offset_script" inittab < out.tar)
    echo "initab found at $OFFSET in tarball"
    
    echo "Generating new inittab with auto-login enabled..."
    # Note:  these two sed strings, the search and replace must be identical sizes, hence the padding in the 2nd string.
    cat inittab | sed "s|1:2345:respawn:/sbin/getty --noclear 38400 tty1|1:2345:respawn:/sbin/getty -a root   38400 tty1|" > inittab2
    
    echo "Re-inserting modified inittab into tarball..."
    dd if=inittab2 of=out.tar bs=1 seek=$OFFSET conv=notrunc
    
    echo "Extracting tarball /root/.bashrc to bashrc..."
    tar xfO out.tar ./root/.bashrc > bashrc
    echo "Finding offset of /root/.bashrc..."
    OFFSET=$(python3 -c "$offset_script" bashrc < out.tar)
    echo "bashrc found at $OFFSET"
    
    LENGTH=$(wc -c bashrc | awk '{print $1}')
    echo "bashrc is $LENGTH bytes long"
    
    echo "Generating new bashrc with automation hooks into ISO image"
    cat << EOFBASHRC > bashrc2
    . /live/boot-dev/setup.sh
    EOFBASHRC
    
    echo "Padding bashrc up to original size with whitespace"
    LENGTH2=$(wc -c bashrc2 | awk '{print $1}')
    TOPAD=$((LENGTH-LENGTH2))
    printf "%${TOPAD}s" >> bashrc2
    
    echo "Re-inserting modified bashrc in tarball..."
    dd if=bashrc2 of=out.tar bs=1 seek=$OFFSET conv=notrunc
    
    echo "Recreating squashfs from tarball..."
    tar2sqfs -q -s -f $EXTRACT_DIR/antiX/linuxfs < out.tar
    
    echo "Add our new setup.sh in the root of the ISO image..."
    cat << EOFSETUP > $EXTRACT_DIR/setup.sh
    echo "This is an example of running automated setup..."
    EOFSETUP
    
    echo "Recreating ISO image..."
    cd $EXTRACT_DIR && xorriso -as mkisofs -D -r -J -joliet-long -l -V \
            "Custom antiX core" -b boot/isolinux/isolinux.bin \
            -c boot/isolinux/boot.cat -iso-level 3 -no-emul-boot \
            -partition_offset 16 -boot-load-size 4 \
            -boot-info-table -o ../new_antix.iso .
    
    
    #46141
    Moderator
    Brian Masinick
    Helpful
    Up
    0
    :D

    I like the solution you came up with; nice combination of Bash and Python scripting!

    Thanks for sharing this!

    Brian Masinick

    #46150
    Member
    skidoo
    Helpful
    Up
    0
    :D

    if I can get away without changing the size of any files, then I can just update them in-place in the tarball without a re-pack.

    I don’t understand why you are avoiding a repack.
    ?Are you unaware of the easy provided utility /usr/local/bin/ps_initrd.sh

    Regarding the automated install, I watched this topic to see whether you had, as a first step, managed to revise the cli-installer script so that it will run unattended. AFAIK, if (conditional, different hardware?) installation of any additional debian packages will be involved during the live-to-installed procedure, several apt//dpkg tweaks would be required, ala

    export DEBIAN_FRONTEND=noninteractive
    apt-get update -q
    apt-get install -q -y -o Dpkg::Options::=”–force-oldconf”

    In advance, prior to placing the seeding ISO onto live media, you might need to install those conditional packages and “dpkg-get-selections $myfile” save the debconf details… then, during the install livesession run “dpkg-set-selections $myfile” prior to invoking any “apt install” operation(s).

    #46180
    Member
    Keeely
    Helpful
    Up
    0
    :D

    Are you unaware of the easy provided utility /usr/local/bin/ps_initrd.sh

    I’m not sure how that helps me. I need to change inittab which is in the squashfs, not the initrd as far as I could see. Also, remember I need all this working on a Mac.

    I watched this topic to see whether you had, as a first step, managed to revise the cli-installer script so that it will run unattended.

    I haven’t started writing the automation of cli-installer, however a cursory look indicated that with the correct kernel options for locale/TZ and pre-created partition correctly formatted a simple answerfile may suffice, e.g.

    cli-installer < answerfile.txt

    There is one portion of cli-installer that switches to a curses interface, which could be a problem, I just have to remove that, either by setting stuff up so cli-installer doesn’t run that section, or by installing the expect package to automate it.

    But that’s going to be a whole lot easier than the squashfs side of things. Now that inittab and bashrc are changed, I can place any other ‘additions’ directly into the root of the ISO image, so my altered cli-installer can reside there, or I can add any packages I want to install before running there as well, e.g. expect.

    if (conditional, different hardware?) installation of any additional debian packages will be involved during the live-to-installed procedure

    Thanks very much for pointing this out it could be useful later. I’m running on virtualised hardware for this but I will be installing other packages after cli-installer. I planned to add them to the ISO image, and install them from a local ‘repo’. I’ve absolutely no idea how to do that at present. I didn’t plan on using any live-to-installed functionality, just the cli-installer and a couple of extra packages and I don’t really want my substituted inittab appearing in the final product, or my altered bashrc, although I could fix them up prior to doing that.

    • This reply was modified 6 months, 3 weeks ago by Keeely.
    #46182
    Member
    Keeely
    Helpful
    Up
    0
    :D

    I like the solution you came up with; nice combination of Bash and Python scripting!

    Thanks, but I’m not sure that ‘nice’ is the word I’d use! Bash has so many limitations and is hard to read (for me at least), however my experience tells me that doing things the other way up (Python calling out to the various utilities) ends up being even worse.

    #46183
    Moderator
    Brian Masinick
    Helpful
    Up
    0
    :D

    Your solution is a good and common way, using a variety of scripting tools to solve common problems.

    Just over a decade ago I was working on some financial services projects and I came across a data analysis issue that would have been pretty time consuming to solve with either a printout and a highlighted analysis or repeatedly searching using an editor.

    I did begin the search with an editor and even considered using Emacs to repeat my search. While I could have written an Emacs macro that would even write out the results, I thought that it would be a good idea to try a scripting language to solve the problem. Perl or Python seemed perfect for the job but I wasn’t very good with either of them.

    I did, however, have a clear design in mind so I created some fake mock-up data and contacted a forum friend who knew Python. We started with Email or text and may have even spoken by phone and an hour or two later we had a solution.

    I modified his solution to match the real data and the problem was solved.

    The reason I tell this story is that code sharing and reuse are two of the fundamentals of open software and I encourage sharing and reuse of code, ideas, and practices to make effective software.

    Brian Masinick

    #46186
    Member
    Keeely
    Helpful
    Up
    0
    :D

    These days I seem to create projects on github for everything I do, so I’m sharing a lot, but I’m not sure the code gets picked up and used at all. One of the other reasons I used Bash here (besides it being fewer lines of code) is that it’s more likely other people will use it in this form. My first instinct is to do everything in Python because you can do everything in Python, and it would be faster for me but many people don’t know Python, yet have a reasonably clear idea what Bash commands do, so they can get the gist of things from my shell script, and they’re prepared to read it because it’s just one file and it’s a sequence of instructions, rather than some class-based abstractions. They can gloss over a few lines of Python as being ‘magic’ and it’s still OK. Hopefully.

    So yeah, it’s here with the hope someone will use it, however it would be also great if the antiX developers could put some hooks in that allow this kind of thing to happen more easily, for instance an extra kernel option that would run a script automatically. Once I have a clear idea of what I want to do, I’ll figure out where/how to log a feature request, but for the moment I’m very much an antiX noob, having recently converted (or trying to convert?) from Slackware, so before I start asking for any changes I feel I have some learning to do. Also feature requests are much more interesting for developers if they come with patches, so there’s that :).

    #46187
    Moderator
    Brian Masinick
    Helpful
    Up
    0
    :D

    I really like your ideas and I hope that at least the concept that you have can be a template for others. That’s often how I code; I get useful stuff from others and modify it to suit my specific need.

    EDIT: taking out my comments about the tool; if I want to do something else with it, I’ll create my own version of it for personal use and do whatever I want with it instead of distracting and confusing people with my “off the wall” idea. If I ever come up with something that is actually useful for others, I’ll consider sharing it, but it won’t be a Ceni-related item.

    ed

    • This reply was modified 6 months, 3 weeks ago by Brian Masinick.

    Brian Masinick

Viewing 15 posts - 1 through 15 (of 21 total)
  • You must be logged in to reply to this topic.