I want to control and use command line history better

[Moderator BobC]

1. ~./bash_history appears to be limited to 500 lines

2. I want to save the history from multiple terminal and terminal with midnight commander shell sessions. I understand that each of those should have its own command history, ie previous commands

3. I would like the command history from the various windows to accumulate into a larger repository which could then be organized and searched/replayed. I think it would be best to keep it in a raw form and have a cron job that would process the raw data into an organized form, and then a pop-up search, edit, execute program or script

Why? I find myself up arrowing 20 times quite often to get back to a series of commands. Sometimes I don’t find it because it didn’t get saved (I don’t know why). It would be real nice if I could pop-up a screen or window full, arrow or page up/down and/or incremental search it in reverse for a string, and then be able to go up or down from there, and press enter to execute, or have a key to edit the line first, then optionally execute, and then when it completes, it pauses to tell me it completed and I press enter to return to my list where I told it to execute so I can do whatever next (maybe repeat the next command I had processed after that). Maybe it would page/search this session first, and when it runs out of history of this session to search, it would switch to the larger repository of all sessions.

Any ideas? I think I need help setting it up better to avoid the 500 line issue, and helper scripts or programs to give me some real command line efficiency…

• This topic was modified 2 years, 2 months ago by Brian Masinick. Reason: Move to Tips and Tricks
• This topic was modified 2 years, 2 months ago by Brian Masinick.

[Member sybok]

Helpful

Up

0

::

@1)
Add/Modify paremeter HISTSIZE in your ~/.bashrc or other files that are sourced by it.
Then source and verify

source ~/.bashrc
echo "$HISTSIZE"

@2)
I believe that they should accumulate once you (correctly) close all current sessions.

@3)
Search: I use ‘Ctrl+R’; the combination ‘Esc’ + ‘>’ resets the position in the history to the end.
Don’t know about the different sessions.
E.g. you can call commands from ‘vim’ (‘:! <command>’); this would get complicated to keep them separated.

• This reply was modified 2 years, 2 months ago by sybok.

[Member ModdIt]

Helpful

Up

0

::

Hi BobC

Set command should also work. History is always written to file on correctly closing the console as sybok pinted out.

demo@antix1:~
$ set HISTSIZE=550
demo@antix1:~
$ set HISTFILESIZE=550

[Moderator Brian Masinick]

Helpful

Up

0

::

You may already know this (or be able to locate this). I quote directly from the bash man page:

HISTFILE
The name of the file in which command history is saved (see HIS‐
TORY below). The default value is ~/.bash_history. If unset,
the command history is not saved when a shell exits.
HISTFILESIZE
The maximum number of lines contained in the history file. When
this variable is assigned a value, the history file is trun‐
cated, if necessary, to contain no more than that number of
lines by removing the oldest entries. The history file is also
truncated to this size after writing it when a shell exits. If
the value is 0, the history file is truncated to zero size.
Non-numeric values and numeric values less than zero inhibit
truncation. The shell sets the default value to the value of
HISTSIZE after reading any startup files.

This description is found in the Shell Variables section somewhere between lines 1100 and 1200.

There are a great variety of keystroke sequences to navigate through Bash history and the man page certainly covers them in detail.
However, in practical experience, a couple things make it pretty easy. Depending on which mode you have Bash editing set, you can perform a variety of commands for simple navigation. To simply go back one or a couple of entries back, arrow keys do the job. I typically have a history alias, h, defined, so I can type h to see a history list. This list provides a number followed by the command. So if I want to run a previous command, for example:
494 sudo /etc/rc5.d/S04bluetooth restart
I simply type !494 and press Enter and the command sudo /etc/rc5.d/S04bluetooth restart is executed. You can also go back to an earlier command and edit the line slightly; all of these details, of course, are provided in detail on the bash man page.

Some people never do anything other than use the arrow keys; that’s legitimate, but the further back the keys are, the more impractical this becomes.

There are also some really creative ways that you can take history, perform creative editing on it and actually turn certain sequences, that you find yourself repeating, very easily into a reproducible script, simply by copying those commands into a file, making the file executable, and running it as often as needed.
The possibilities are endless, limited only by what you are willing to experiment and try.

--
Brian Masinick

[Moderator BobC]

Helpful

Up

0

::

If I can get the code below working, all I need in addition is a terminal command history utility/launcher program. Anyone know of anything along those lines? I’m not looking to do it by hand, and would rather use a program that exists if I can find one that meets my needs…

My current ~/.bashrc has:

# don't put duplicate lines in the history. See bash(1) for more options
# don't overwrite GNU Midnight Commander's setting of ignorespace.
HISTCONTROL=$HISTCONTROL${HISTCONTROL+:}ignoredups
# ... or force ignoredups and ignorespace
HISTCONTROL=ignoreboth

# append to the history file, don't overwrite it
shopt -s histappend

# for setting history length see HISTSIZE and HISTFILESIZE in bash(1)
export HISTCONTROL
export HISTSIZE=9999
export HISTFILESIZE=999999
# PROMPT_COMMAND set so that Midnight Commander commands and subshell commands are also included
export PROMPT_COMMAND="history -a; history -c; history -r; $PROMPT_COMMAND"

• This reply was modified 2 years, 2 months ago by BobC.

[Moderator Brian Masinick]

Helpful

Up

0

::

That’s cool, BobC, so you already have quite a few of the useful things at your disposal, and since you use it with Midnight Commander (or your PROMPT_COMMAND values “suggest” that possibility), you have a lot of capabilities already. I’ll do a few searches and see if I can find any other useful apps, scripts, or tools that Bash users use and if I find anything interesting I’d be happy to share it.

Also, if I think of or remember anything creative I’ve done in the past, I’ll share that too.

--
Brian Masinick

[Moderator Brian Masinick]

Helpful

Up

0

::

This may not solve all of your “problems”, but there are enough example scripts and explanation of Bash features that the motivated, interested person will be able to do a LOT with it; hope it helps someone:

https://tldp.org/LDP/abs/html/
Advanced Bash-Scripting Guide
An in-depth exploration of the art of shell scripting
Mendel Cooper

Another one is
https://tldp.org/HOWTO/Bash-Prog-Intro-HOWTO.html#toc
BASH Programming – Introduction HOW-TO
by Mike G mikkey at dynamo.com.ar

--
Brian Masinick

[Moderator Brian Masinick]

Helpful

Up

0

::

https://doc.lagout.org/operating%20system%20/linux/Unix%20Power%20Tools.pdf

PDF version of a handy book that may provide a lot of good scripting ideas.

--
Brian Masinick

[Moderator Brian Masinick]

Helpful

Up

0

::

https://linuxhint.com/30_bash_script_examples/

--
Brian Masinick

[Anonymous]

Helpful

Up

0

::

3. I would like the command history from the various windows to accumulate into a larger repository which could then be organized and searched/replayed. I think it would be best to keep it in a raw form and have a cron job that would process the raw data into an organized form, and then a pop-up search, edit, execute program or script

At a bash prompt, press Ctrl+R
^—> as you type, a filtered match drawn from the most recent matching commandline from .bash_history will be displayed.

While said match is displayed, pressing Ctrl+R again (reteatedly, if desired) displays the previous matching commandline. By default, I don’t think we have a Ctrl+R keybind sibling which invokes to “jump to next (filtered match)”. You can enable additional keybinds by activating the “vi mode” bash option (set -o vi) or by declaring shell keybinds within your dot files. Ref: https://unix.stackexchange.com/questions/541/best-way-to-search-through-shells-history

_______________________________

hstr

HSTR is a command line utility that brings improved BASH command completion from the history.
It aims to make completion easier and more efficient than Ctrl-r.

view, navigate, search, and use your command history
In addition to completion, hstr enables you to manage your history list.
You can bookmark your favorite commands, and can remove commandstrings
that were mistakes, or are now obsolete, or contain sensitive//password details.

It will be avialable in debian bullseye https://packages.debian.org/search?keywords=hstr&searchon=names&suite=all&section=all
In the meantime, you can

cd /tmp
git clone https://github.com/dvorka/hstr
sudo apt install autotools-dev libreadline-dev libncursesw5-dev
cd /tmp/hstr
cd ./build/tarball && ./tarball-automake.sh && cd ../..

#./configure && make
/tmp/hstr/src/hstr

###     OR

#./configure && make && make install
hstr

.
animated giffy screenshot here:
https://raw.githubusercontent.com/dvorka/hstr/master/doc/hstr-v2.gif
same is embedded withing the hstr github project page https://github.com/dvorka/hstr

hstr –help
Usage: hstr [option] [arg1] [arg2]…
Shell history suggest box:

–favorites -f … show favorites view
–kill-last-command -k … delete last command in history
–non-interactive -n … print filtered history and exit
–show-configuration -s … show configuration to be added to ~/.bashrc
–show-zsh-configuration -z … show zsh configuration to be added to ~/.zshrc
–show-blacklist -b … show commands to skip on history indexation
–version -V … show version details
–help -h … help

___________________________________________

fzf
version 0.24.3 will be available in debian (bullseye)
Due to its various golang build dependencies, I would not recommend attempting to compile it yourself.

It is an interactive filter for commandline that can be used with any list;
files, command history, processes, hostnames, bookmarks, git commits, etc.
Refer /usr/share/doc/fzf/README.Debian for quick instructions
on how to add keybindings for Bash, Zsh, Fish to call fzf.

The fzf project page https://github.com/junegunn/fzf
contains an embedded yt intro video https://www.youtube.com/watch?v=qgG5Jhi_Els
and
due to its popularity, websearch will find plenty of other fzf videos + articles.

[Anonymous]

Helpful

Up

0

::

> I understand that each of those should have its own command history
>
> I would like the command history from the various windows to accumulate into a larger repository
.

that already happens (or can optionally be made to happen) each time you exit an interactive shell, e.g.
— closing a terminal emulator
— closing a tab within a tabbed terminal emulator
— exiting an interactive bash subshell which was initiated via console
— (?) logging out from console (if your user’s default login shell is bash)

YMMV based on which tools (mc,urxvt,roxterm,lxterminal) you are using
and might involve setting a preference or editing a config file
to specify “use a login shell”

[Moderator Brian Masinick]

Helpful

Up

0

::

This link explains some ways to utilize recent history.

https://unix.stackexchange.com/questions/5684/history-command-inside-bash-script

I’m wondering what actual value the new scripts will add, and at what cost.

If you’re using Emacs mode in Bash you can access however many history lines you save according to the environment variable values for history.
Even the default settings give you the most recent 500 lines of history.

I’ll share an old .bashrc file I used 20-25 years ago on UNIX systems. Originally it was a .kshrc script. By carefully noting differences between sh, ksh, bash and zsh it’s reasonably easy to make a shell initiation script that can work on any of them with little to no change except for the preamble specifying the script (if any).

--
Brian Masinick

[Moderator Brian Masinick]

Helpful

Up

0

::

#
# File: .bashrc
#
# Author:
#  Brian W. Masinick
# Purpose:
#  Defines variables, aliases, and functions for shortening typing
#  and tasks.  Used with the Bourne Again Shell, bash, from the GNU
#  project of the Free Software Foundation.
#
# NOTE: This file is an adaptation of my .kshrc file, removing things
# that are peculiar to the Korn Shell, ksh.
#
# Conventions:
#  I define every procedure and function using UpperCase Words Like
#  this and prepend each name with TheMas to reduce the likelihood of
#  a naming collision with a command, script, or program.

#  The procedures are invoked at the bottom of the file.  The
#  functions or procedures can be called at any time by any
#  procedure, which is why breaking the procedures and functions into
#  similar groupings makes sense.  The organization of the procedures
#  may need to be adjusted periodically to keep in step with my
#  current work.

# exit if not interactive

case "$-" in
*i*) ;;
*) return ;;
esac

TheMasPrinters()    # define printer aliases
{

    #
    # Printing aliases...
    #

    alias lpi='lpr  -K2 -N1'
 

}

TheMasMisc()        # miscelaneous aliases I like to use
{
    alias bye='clear; kill -1 0'
    alias c="clear;pwd"
    alias cls=clear sd=cd
    alias lo='clear;  kill -1 0'
    alias m='more'

    alias help=apropos
    alias wn='who -uT | fgrep .'

    # history aliases...

    alias h='fc -l'
    alias re='fc -e -'
    alias zaphist="rm -f /tmp/.bwm_history"

    # host aliases...

    alias host=hostname
    alias rw="rwho | sort +1"

    # desktop office tool aliases...

    alias e=emacs
    alias kh='exec /usr/bin/ksh'
    alias n=next
    alias p=prev
    alias R='repl -cc all -filter ~/.mhfilter'

}

TheMasFileManagement()    # commands I use to operate on files and dirs.
{
    # directory movement aliases...

    alias b="cd ~/bin"
    alias cd='g'
    alias home=cd

    # file management aliases...

    alias del="rm -i"
    alias copy=cp
    alias del='rm -i'
    alias dir="ls -aqFC"
    alias type=cat
    alias l='ls -aqCF'
    alias lsd='ls -algFqd'
    alias lsf='ls -F'
    alias lsi='ls -algFqi'
    alias lsl="ls -algFq"
    alias lsr='ls -aqCFR'
    alias lst="ls -algsFqt"
    alias lst10="lst | head -10"
    alias lst20="lst | head -20"
    alias lst30="lst | head -30"
    alias lst40="lst | head -40"
    alias lst50="lst | head -50"
    alias lst60="lst | head -60"
    alias lst70="lst | head -70"
    alias pdw=pwd
    alias pd=pwd
    alias wpd=pwd
}

TheMasTerm()
{

    alias vt100="TERM=vt100; export TERM"
    alias vt102="TERM=vt102; export TERM"
    alias vt200="TERM=vt200; export TERM"

    alias xrs='set noglob; eval <code>resize -s \!\*</code>; unset noglob'
    alias xs='set noglob; eval <code>resize</code>; unset noglob'
}

TheMasWorkon()
{

####################################################################
    #
    # Tom Woodburn's stuff (adapted) below...
    #

####################################################################

    export HISTFILE=/tmp/.bwm_bash_history
    export HISTSIZE=1500

    HOST=<code>hostname</code>
    # Strip off domain name; e.g., change flume.zk3.dec.com to flume.
    HOST=<code>expr &quot;$HOST&quot; : '\([^\.]*\)'</code>
    export HOST

    case "$WORKON" in
    "")
        PS1="$HOST:<code>whoami</code>> "
        ;;
    *)
    PATH=$SDETOOLSBIN:$PATH:
    PATH=$PATH:$HOME/bin:
    PATH=$PATH:$HOME/tools/bin:
        PATH=$PATH:/usr/projects/sa/i18neng/tools/bin
    PATH=$PATH:/usr/dt/bin:
    PATH=$PATH:/usr/bin/X11:
    PATH=$PATH:/usr/bin/mh:
    PATH=$PATH:/usr/local/bin:
    PATH=$PATH:/contrib/bin:
    PATH=$PATH:/usr/java/bin:
    export PATH
    CLASSPATH=/usr/java/classes
    export CLASSPATH
        case "$SANDBOX" in
        "")
        SANDBOX=<code>dirname  $SOURCEBASE</code>
        SANDBOX=<code>basename $SANDBOX</code>
        ;;
    esac
    export SBTOPDIR=$HOME/$SANDBOX/src
    case "$BCSSET"  in
    "")
        SET=$SANDBOX
        ;;
    *)
        SET=<code>expr &quot;$BCSSET&quot; : &quot;${PRINCIPAL}_\(.*\)&quot;</code>
        ;;
    esac
    PS1="[$HOST:(<code>whoami</code>) $SANDBOX($SET)] "
    # if DISPLAY exists, set titlebar and icon.
    if [ -n "$DISPLAY" ]; then
        ~/bin/title "Sandbox set $PS1 for $USER" $HOST
    fi
    unset LANG
    ;;
    esac
}

TheMasHosts()        # define host aliases...
{

    # rl is a procedure in my bin directory that alters the title bar,
    # then calls rlogin.

    alias quarry='rl quarry'
    alias themas='rl themas'
    alias xappl='rl xappl'
}

########################################################################
#
#
# Bill Gray's functions (adapted) below...
#
########################################################################
#

# directory stack functions

declare -i DNUM=0
DLIST[DNUM]=<code>pwd</code>

function g        # go to a directory
{
    if builtin cd "$@" >/dev/null  &&  [ ${DLIST[DNUM]} != "$PWD" ]
    then
        DNUM=DNUM+1
        DLIST[DNUM]=$PWD
    fi
    case $WORKON in
    "")
    # if DISPLAY exists, then set titlebar and icon.
    if [ -n "$DISPLAY" ]; then
        titlebar="$USER @ $HOST : $PWD"
        if [ "$OS" = "OSF1" ]; then
            ~/bin/title "$titlebar" "$HOST"
        else
            ~/bin/utitle "$titlebar" "$HOST"
        fi
    fi
    ;;
    *)
    ;;
    esac
    pwd
}

function gb        # go back
{
    if (( $DNUM > 0 ))
    then
        DNUM=DNUM-1
    fi
    g ${DLIST[DNUM]}
}

function gn        # go to selected (nth) dir
{
    select DIR in <code>echo ${DLIST[*]} | tr &quot; &quot; &quot;\012&quot; | sort -u -y0</code>
    do
        if [ "$DIR" ]
        then
            g $DIR
        else
            g $REPLY
        fi
        break
    done
}

function up        # go up n levels
{
    declare -i levels

    levels=${1}

    if [ -z "${1}" ] && [ ${PWD} != "/" ]
    then
      g ..
      return $?
    fi

    while [ ${levels} -gt 0 ] && [ ${PWD} != "/" ]
    do
      g ..
      levels=levels-1
    done
}

function lpit
{
    for file in $*
    do
        pr -e8 -f $file | lpr -J $file -P doc1
    done
}

function lpsrc
{
    for file in $*
    do
        pr -e8 -f -n5 $file | lpr -J $file -P doc1
    done
}

function lp_2sides
{
    for file in $*
    do
        pr -e8 -f -n5 $file | lpr -J $file -P doc1 -K 2
    done
}

function lp_2sides_2up
{
    for file in $*
    do
        pr -e8 -f -n5 $file | lpr -J $file -P doc1 -K 2 -N 2
    done
}

function t
{
    export TERM=$1
    xs
}

function addpath
{
    PATH=$PATH:$1
    echo $PATH
}

function log
{
    echo <code>date '+%D %H:%M'</code> >> ~/.logfile
    echo "$*" >> ~/.logfile
    echo >> ~/.logfile
    date
}

function todo
{
    echo <code>date '+%D %H:%M'</code> >> ~/todo
    echo "$*" >> ~/todo
    echo >> ~/todo
}

function rot13
{
    tr [a-z][A-Z] [n-z][a-m][N-Z][A-M]
}

function unrot13
{
    tr [a-m][n-z][A-M][N-Z] [n-z][a-m][N-Z][A-M]
}

# Remove files that are a symbolic link to some other file, (eg. UnLink)

function UL
{
    for i in $*
    do
        if [ -L $i ]; then
            f=<code>ls -al $i | sed &quot;s/^.*-> //&quot;</code>
            rm -f $i
            cp $f $i
        fi
    done
}

########################################################################
#
#
# Call the routines that I use each time that I login.
#
########################################################################
#

# Main            # <--- Main

# Set the default directory and file protection mask.  By default, do
# not mask any protection on my ownership, but remove default write
# access for the group, and do not give "world" any default access.

export SHELL=~/bin/bash
umask 022

OS=$(uname)
myterm=$(who am i)
myterm=$(echo $myterm | awk -F" " '{ print $2 }')
echo "The current HOST is $HOST"
echo "The current terminal is $myterm"
echo "The current shell is $SHELL"
echo "The terminal type is $TERM";echo "";echo ""
isLAT=$(echo $myterm | awk '{ print substr($0,1,3) }')

# The following condition checks to see if we are operating in a CDE
# "DT", that is, a CDE Desktop environment.  If we are, then we do not
# want to set terminal characteristics.  But if we are running a
# terminal emulator, such as dtterm, dxterm, or xterm, then we DO want
# to perform the following steps:

if [ ! "$DT" ]; then
    tty -s
    if test $? = 0
    then
        stty dec crt
    fi
    tset -Q -I

    #
    # Define the default editing mode
    #

    set editing-mode emacs
    set show-all-if-ambiguous
fi

TheMasPrinters
TheMasMisc
TheMasFileManagement
TheMasTerm
TheMasWorkon
TheMasHosts

return    # get out of any functions I may be in

# Finished .bashrc

--
Brian Masinick

[Moderator BobC]

Helpful

Up

0

::

Ahhhhh, skidoo, thank you so much. You chose well. Can I ask how you searched? I need to improve my search skills.

hstr is what I was hunting for. It even looks a lot like what I was thinking of. I had to build it, bu you must have done that, because you corrected the Debian install list.

I need to play with it more, and figure out how I want to call it up, but it definitely will make it easy to repeat command line tasks with dramatically less effort and fewer goofs.

Brian, Yes, I need to learn more. I appreciate the suggestion texts. I’m not the greatest coder on these systems, but I have dogged determination and focus, which makes up for that a bit. I like creating tools, but I knew someone had to have noticed and addressed these inefficiencies before. Maybe if I find any features that I can improve, I will contribute them if I can handle creating them.

[Moderator BobC]

Helpful

Up

0

::

Brian, That’s quite a script. I hadn’t seen those last 2 posts as it looks they crossed while I was working on skidoo’s post ideas. I used ksh on a system once, maybe an ATT Unix 3B2 box back in the 80’s or an SCO Xenix system, my memory has faded. But there is always something to learn from looking at other people’s code and gleaning for applicable ideas. Thanks for finding and posting.

Skidoo, the hstr is definitely what I had in mind. I watched the video on fzf and it wasn’t what I had pictured, but it has some features that would help minimize typing.

[Author]

Posts