Words


In the beginning was the Word
and the Word was with God
and the Word was God.
(John, 1:1)

In the beginning, the file was without form, and void;
and emptiness was upon the face of the bits.
And the Fingers of the Author moved upon the face of the keyboard.
And the Author said, Let there be words, and there were words."
( The Linux System Administrator's Guide)

The default Linux shell, bash, is an advanced command interpreter that allows a system administrator to program (or 'script') complex system administration tasks. Linux is pretty much a text oriented system, and lots (if not all) of the configuration is stored in text files. This allows for easy configuration with basic tools, such as a text editor. When the GNU project created the tools that were later incorporated in the Linux Operating System, they included many tools and features to make text processing easy. The gnu bash shell has built-in text processing operators, and there is a wealth of string manipulation and text sorting and filtering tools ...

What follows are some shell scripts, mainly exercises and 'just for fun' programs, exploring some bash and gnu/linux features in the field of string manipulation and text processing.


the "Change 1 Letter" game

The game : a player starts with a (random) word, and the next player needs to find a word by adding, removing, or replacing 1 letter in the given word. And so on.

So, we cheat :

Change Word
shell script that takes a given word as input (command line argument), and outputs all possible changes (add 1 letter anywhere, remove 1 letter anywhere, replace 1 letter anywhere
Find Words
Finds existing words by calling "changeword", pushes the result through a spell checker to filter (requires 'spell' to be installed). 'spell' lists incorrect words, so we use 'diff' to find words created by 'changeword' but not listed by 'spell" ; these are words that exist in the spell dictionary, so presumably good words. 'uniq' throws out any duplicates (or it should - some problems there still), and 'sort' sorts the output.
This is what it looks like :
me@kix:~$ ./findwords java
lava

me@kix:~$ ./findwords think
chink
thank
thick
thin
thine
thing
think
think
think
think
thinks
thins
thunk
		

You can play this game against yourself : start with a letter or a word, modify it, etc, and see how far you get. This script uses the above 2 scripts to let you play that game. It shows the history of the game (words already used), possible solutions for the last word so far, and a prompt to enter your solution.


Random Words

Need a list of random 'words', eg all possible combinations between 8 and 12 characters long ? Think of it as a brute force password checker, eg to check how long it would take to crack a short password if it doesn't exist in a dictionary or can be derived from an existing word. Generate Words lets you specify a specific word length or a range (eg 6 to 8 characters long), and which characters to use (lowercase, uppercase, numeric, punctuation, or a combination of any of those.

The program is build around a recursive function that takes the previous word, and adds or changes one character, until all possible combination of the specified lengths are generated

Example :

	# test password 'Z9X' by checking all combinations of 1 to 3 uppercase and numeric characters
	me@kix:~$ ./generatewords 3 1 A n | grep -w  Z9X
	Z9X
	

note that grep continues to look after the first match. For long lists, and if you want to time the test, this is inconvenient. 'grep -q' quits on the first match, so that's a better solution. It doesn't produce any output, but you can test it's return value (return 0 = match found). checkword.

	me@kix:~$ ./checkword
	Z9X
	test took 5 seconds
	

You can also use the 'time' command

	me@kix:~$ time ( ./generatewords 4 1 A a n | grep -q B1FF )

	real    1m21.160s
	user    1m12.753s
	sys     0m3.776s
	

How long does it take to try all combinations of 5 lowercase characters ?

	me@kix:~$ time ( ./generatewords 5 a )
	
	real    68m53.718s
	user    34m56.955s
	sys     2m34.030s

	

So, little over 1 hour. For 6 characters, that would be 26 times longer - a bit more than 1 day. On a not very new computer. Furthermore, this is a simple shell script. Compiled programs run much faster. Good programmers also now how to optimize for speed, and modern programs would use intelligent algorithms and parallel processing, while this script is just a sequential procedure. Make your passwords 6 or more characters long, use all types of characters (not just lowercase, also uppercase, numbers, and punctuation or special characters), and don't use words from a dictionary. Pussy99 is NOT a strong password. It's long, has lower and upper case and numbers, but pussy is a word from a dictionary, and any decent password cracker will also try common alterations such as trying capitals and adding numbers in from or at the end)

More on Password cracking


99 Bottles of beer

Bash shell script that generates the lyrics of the song "99 bottles of beer on the wall". (Also on 99-bottles-of-beer.net/).


Generating Text Files

Poor Man's Web Master Tools
Silly scripts for maintaining a static website


Koen Noens
December 2007