Mohammedz.com

For Linux and Shell scripting.


Leave a comment

SED: newline and embedded newline characters

The multiline Next (N) command creates a multiline pattern space by reading a new line of input and appending it to the contents of the pattern space. The original contents of pattern space and the new input line are separated by a newline. The embedded newline character can be matched in patterns by the escape sequence “\n”. In a multiline pattern space, the metacharacter “^” matches the very first character of the pattern space, and not the characters(s) following any embedded newline(s). Similarly, “$” matches only the final newline in the pattern space, and not any embedded newline(s). After the Next command is executed, control is then passed to subsequent commands in the script.

The Next command differs from the next command, which outputs the contents of the pattern space and then reads a new line of input. The next command doesn’t create a multiline pattern space.

This is taken from O’Reilly’s sed & awk.

~mohammed


3 Comments

How to remove non-printable/control characters from a file?

You may find difficulties with non-printables in your files. You can see such characters if you open your files in editors like vi. Eventhough commands like “cat” won’t output such non-printable characters into console by default, you can’t remove them by redirecting “cat” output to a different file.

Here is a way to remove non-printable characters with a combination of sed and tr commands.

Step 1:
Use sed with “l” (lower case L) option to print the file/line in a “visually unambiguous” form. From sed output, find the character notation that needs to be removed.

#sed -n ‘l’ filename.txt

Step 2:
Remove control characters that you found from sed output using “tr” or “sed”.

suppose you want to remove the form feed character “\f” lines from filename.txt, use any of the commands given below.

# tr -d ‘\f’ filename.txt
or
# sed ‘/\f/d’ filename.txt

You can any of the commands given below if you want to remove such control characters only, but not the entire line containing them.

# tr ‘\f’ ‘ ‘ filename.txt
or
# sed ‘s/\f//’ filename.txt

The control characters in ASCII still in common use include:

* 0 (null, , ^@), originally intended to be an ignored character, but now used by many programming languages to terminate the end of a string.
* 7 (bell, \a, ^G), which may cause the device receiving it to emit a warning of some kind (usually audible).
* 8 (backspace, \b, ^H), used either to erase the last character printed or to overprint it.
* 9 (horizontal tab, \t, ^I), moves the printing position some spaces to the right.
* 10 (line feed, \n, ^J), used as the end_of_line marker in most UNIX systems and variants.
* 12 (form feed, \f, ^L), to cause a printer to eject paper to the top of the next page, or a video terminal to clear the screen.
* 13 (carriage return, \r, ^M), used as the end_of_line marker in Mac OS, OS-9, FLEX (and variants). A carriage return/line feed pair is used by CP/M-80 and its derivatives including DOS and Windows, and by Application Layer protocols such as HTTP.
* 27 (escape, \e [GCC only], ^[).
* 127 (delete, ^?), originally intended to be an ignored character, but now used to erase a character (especially the one to the right of the cursor).

Read more about control characters at wikipedia.

~mohammed


Leave a comment

How to find past/future dates?

what day it was exactly one month ago? Are you counting down the days to find it? Or searching for a calender?

You can use “date” command in such situations. See the example below:

****************************************
-bash-3.1$ date +%F
2009-06-29
-bash-3.1$ date -d”1 day ago” +%F
2009-06-28
-bash-3.1$ date -d”1 week ago” +%F
2009-06-22
-bash-3.1$ date -d”1 month ago” +%F
2009-05-29
-bash-3.1$ date -d”1 year ago” +%F
2008-06-29
-bash-3.1$
****************************************

These examples will show you how to find the future dates.
****************************************
-bash-3.1$
-bash-3.1$ date +%F
2009-06-29
-bash-3.1$ date -d”-1 day ago” +%F
2009-06-30
-bash-3.1$ date -d”-1 week ago” +%F
2009-07-06
-bash-3.1$ date -d”-1 month ago” +%F
2009-07-29
-bash-3.1$ date -d”-1 year ago” +%F
2010-06-29
-bash-3.1$
****************************************

~mohammed


Leave a comment

Apache: how to disable directory listing

Disable directory listing using httpd.conf:
* Open your httpd.conf and look for “Options Indexes”
* Change “Indexes” to “-Indexes” if it exists.
* Or else add “Options -Indexes” as a new line
* Restart your apache webserver

Disable directory listing using .htaccess:-
* Open your .htacces file and look for “Options Indexes”
* Change “Indexes” to “-Indexes” if it exists.
* Or else add “Options -Indexes” as a new line

Regards,
Mohammed.


Leave a comment

Array manipulation in shell scripting

Best way to understand shell scripting is from examples. Find the example below to understand different options available in bash array manipulation.

You can down this script from here.
*********************************
#!/bin/bash

array=(one two three four [5]=five)

echo “Array size: ${#array[*]}”

echo “Array items:”
for item in ${array[*]}
do
printf ” %s\n” $item
done

echo “Array indexes:”
for index in ${!array[*]}
do
printf ” %d\n” $index
done

echo “Array items and indexes:”
for index in ${!array[*]}
do
printf “%4d: %s\n” $index ${array[index]}
#printf “%4d: %s\n” $index ${array[$index]}
done
*********************************

Regards,
Mohammed.


Leave a comment

HISTTIMEFORMAT: add execution time to Linux’s history file

As you might know, Linux logs all the commands executed on a shell prompt to its history file. The file is .bash_history by default, but you can change it. Also, you can change HISTTIMEFORMAT to enable timestamping to history.

If you add the following line to your bashrc or bash_profile, the time stamp information associated with each history entry will be written to the history file.

HISTTIMEFORMAT=”%h-%d-%Y – %H:%M:%S ”

Regards,
Mohammed.