Manipulating the past (or unix shell histories)

Engineers spend a lot of time in their shells. A lot. Which means a lot of history, a lot of context in that history, and a lot of repeating yourself (or, more accurately, a lot of nearly repeating yourself). There are a lot of ways to save time in your shell; aliases, shell scripts, etc, and of course your shell’s history.

/INSERT NECK-BEARD JOKE HERE/

I started using shells before they had much in the way of making life easier, and in fact before shells like zsh and bash even existed. As a result, I learned some optimisations that have stuck in my mind, muscle memory, and (more importantly), remained as available functionality in shells as they exist today.

Which shells know about history

Shells (can be configured to) store a history of all the commands you execute. I'm using zsh, but this is also true for bash, tcsh etc.

How much history is stored and how the shell manages history from a concurrency point of view is down to your shell config.

For zsh, this is the configuration I use:

# Don't ask for verification on history manipulation 
unsetopt HIST_VERIFY

# Share history between shells 
setopt SHARE_HISTORY

# Additional history data stored in the history file 
setopt EXTENDED_HISTORY

# Don't replace history files 
setopt APPEND_HISTORY

# Add to the history file as commands are entered (not at shell exit) 
setopt INC_APPEND_HISTORY

# Don't show dups when searching history 
setopt HIST_FIND_NO_DUPS

First up, to show all your history, use the history built-in command

history

So you can do things like

history | grep document1

TIP: For most shells (might need turning on), you can prevent a command from being added to history by starting it with one or more spaces.

Using your shell's history

Before the arrival of new kids on the block like readline made it possible to use your arrow keys to iterate through your history, make some changes and run the new commands, your shell had ways of accessing and manipulating those commands without using those newfangled arrow keys.

Basic format

ADDRESS:OPERATION:MODIFIER

To a large extent, you can optionally leave off portions from the right (as you'll see below, sometime just using the ADDRESS portion is fine - depending on what you're trying to do).

Addressing

Use the ! (bang) operator to find the item in your history to manipulate.

Most recent command

The !! address will retrieve your most recent command, so to repeat the last command executed

!!

As an example

$ rm tempfile.doc
Permission denied
$ sudo !!

Execute a specific item in your history

To execute a specific numbered item in your history, use the !n address. You can also use !-n to reference the nth most previous command in your history

$ history| tail -n 8
 1449  cd electronics/z80/stage2-io
 1450  ls
 1451  minipro --help
 1452  minipro -p AT28C256 -r rom.bin
 1453  ls
 1454  diff rom.bin stage2b.bin
 1455  make program IMG=stage2a.bin
 1456  make program IMG=stage2b.bin

 !1454

You could also do this using

!-3

Repeat a command, start with a specific prefx

You can use the !word to execute the most recent command starting with word.

!diff

Better searching

To find and execute a command out of your history that contains a specific word, use the !?search? address

!?rom.bin?

The trailing ? is optional, but is important, as you'll see when we add the p modifier. The p modifier just prints out the matched command (or portion of it - see operations below) instead of executing it. This is really useful in the case where you're unsure that you're going to successfully identify the command you want to execute (especially if it is destructive).

Find the last command you executed on rom.bin, validate it's the right one, and then actually execute it (with the !! manipulation).

$ !?rom.bin?:p
diff rom.bin stage2b.bin
$ !!

Operations

The OPERATION portion of a manipulation is used to do things like find an argument from the addressed history command, perform a substitution etc.

Fixing a typo

To quickly fix a typo (or repeat a command with a different argument), use the ^SEARCH^REPLACE manipulation

$ mv docment.txt backup/
docment.txt: File not found
$ ^docment^document

More complicated search and replace

You can use the regexps to search and replace parts of a previous command

$ ls docs/z80cpu_um.pdf
docs/z80cpu_um.pdf
$  !!:s/docs/man/:p
ls man/z80cpu_um.pdf

To execute the replace, on all occurrences of a match, use gs instead of s.

Accessing command line arguments

You can use $ and ^ to reference the last and first arguments of a command in your history. This works when prefixed by an addressing manipulation. The simplest uses are !$ which pulls in the last argument from the previous command, and !^ which does the same for the first argument

$ mv document.txt backup/
 $ ls !$

You can use this with more complicated addresses as well

$ ls !?rom.bin?:$:p
stage2b.bin

I used the (p) modifier to just print out the last argument of the most recent command in my history that contained rom.bin

Accessing a specific argument

If you're manipulating a command in your history with multiple arguments, and you want to pull out a specific one, you can use the :n modifier to get to the nth argument

$ ls one.doc two.doc three.doc
$ ls -l $:2
-rw-rw-r--   1 timmy timmy  2223 Jan  5 12:17 two.doc

Modifying parts of a path

Sometimes you need to use the filename portion of a previous argument, or the directory portion. You can use the :h and :t modifiers to extract the directory or the filename part respectively

$ ls docs/z80cpu_um.pdf
docs/z80cpu_um.pdf
$ ls !$:h
ls docs
z80cpu_um.pdf

As you can see, these modifiers need an address and argument to operate on.

You can also use :r to strip off the extension of a path, or :e to leave only the extension.

Other useful expansions

Technically, not really manipulating your history, but these expansions are still really useful

$ echo {one,two,three}.doc
one.doc two.doc three.doc

Or, more usefully - to copy a file to a backup

$ cp document.{doc,bak}

Wrapping up

These are pretty much only the tip of the iceberg. Some of these may seem super complicated, but if you pick up a few every week or so, you’ll soon find the subset of all these that makes you more efficient, and pretty soon they’ll form part of your muscle memory.

Your shell is way more powerful than you think it is, so keep diving, and when you find the next cool tip, share it.