Inline Filtering

sed

Linux for Programmers and Users, Section 4.12.2.

sed

A programmable stream editor for filtering and transforming text.

SYNOPSIS

sed [-e script] [-f scriptfile] [FILE ...]

-e script, --expression=script

add the script to the commands to be executed

-f script-file, --file=script-file

add the contents of script-file to the commands to be executed

-i[SUFFIX], --in-place[=SUFFIX]

edit files in place (makes backup if extension supplied)

The sed command is often used as a one line script, but may also be invoked with a file containing a multi-line script – much like a programming language.

Sed Script Commands

The general format of sed commands is:

[address-spec]  command

Commands that operate on whole lines (insert or delete lines) require an address (line) specification. The ‘s’ command, which substitutes text with a line, may either operate on all of the lines processed or use an optional address (line) specification.

/regex/

Apply the command to any line matching the regular expression.

n

Apply the command to line (number) n in the data.

start, stop

Apply the command between lines start and stop in the data.

The text editing commands are as follows.

address a\text

Append line(s) of text after address

address c\text

Replace line(s) of text

address d

Delete line(s)

address i\text

Insert line(s) of text before address

address r filename

Append a file after the address. Note: use cat to insert a file before the first line.

[address] s/regex/string/

Replace the first instance of text matching the regular expression on the applied lines with the string.

[address] s/regex/string/g

Replace all instances of text matching the regular expression on the applied lines with the string.

Note

One common usage of sed is to generate edited string variable names in a shell script program. For example:

newname=$(echo $oldname | sed 's/regex/str/')

See the text book for more examples.

tr

Linux for Programmers and Users, Section 4.12.3.

tr

Replace characters from one sequence of characters with characters from another sequence. The most common usage of tr is to convert text data to either all capital letters or all lower case letters.

SYNOPSIS

tr [-cds] sequence1 sequence2

-c, -C, --complement

first complement SET1

-d, --delete

delete characters in SET1, do not translate

-s, --squeeze-repeats

replace each input sequence of a repeated character that is listed in SET1 with a single occurrence of that character

Example:

input=$(echo input | tr A-Z a-z)