Automatic FTP

Linux ftp scripts


FTP is typically used to copy files from one computer to another, e.g. to upload your web pages to the web server of your internet provider. There are many ftp programs with a graphical user interface, but they usually come down to dragging files between a 'local' and a 'remote' file tree. The result : endless clicking as you navigate through the directories both on your computer and the ftp server. That's no way to upload multiple files.

What we want is a way to upload (a list of ) files automatically. The DOS ftp command has the option -s:file that allows you to supply a script to the (dos) ftp command. That option seems to be missing from the Linux version. However, Linux shells have extensive support for pipes and redirection, and it's quite easy to send input from a file to the ftp command.

FTP script in DOS (explain)

	open ftp.africaonline.com
	Pete
	3122
	prompt
	pwd
	lcd c:\mywebsite
	mput *.*
	lcd c:\mywebsite\pictures
	mkdir pictures
	cd pictures
	mput *.*

		[and so on ...]

	bye

similar FTP script in Linux :

	SERVER=ftp.africaonline.com
	USER=Pete
	PASSW=3122

	ftp -v -n $SERVER <<END_OF_SESSION
	user $USER $PASSW
	$FILETYPE
	lcd /home/pete/mywebsite
	mput *
	mput *.*
	bye
	END_OF_SESSION

Note how this is quite similar to the dos ftp -s:script format, but that the script uses variables, and that the ftp script is included in the shell script by means of a "here document" construct).

Apart from this so-called "Here Document" redirection (<<) , you could also use the program "expect" to create a scripted dialog with the ftp program. An other option is to use the kermit communication protocol.

However, redirection is quite common in Unix and Linux, and combined with shell variables and string manipulation, you can pretty much just write a shell script that acts as a simple ftp client that takes a list of local files, and uploads them to an ftp server, creating directories on the server as needed.

Here is an example of such a script : Silly Batch FTP client

extract :

## upload the files
  INPUT=/path/to/list_of_files_to_upload
  LOCALBASE=/home/pete/website		#local directory that corresponds to the root of the website

  for ITEM in $(cat $INPUT); do
        FILE=$(basename $ITEM)
        LOCALDIR=$(dirname $ITEM)

        if  [ "$LOCALDIR" == "$LOCALBASE" ]; then
                REMOTEDIR="/"
        else
                REMOTEDIR=${LOCALDIR#$LOCALBASE}
        fi

        ## ftp starts here
	ftp -v -n $SERVER <<END_OF_SESSION
	user $USER $PASSW
	$FILETYPE
	lcd $LOCALDIR
	cd $REMOTEDIR
	put $FILE
	bye
  END_OF_SESSION
  done

The core of the script is a loop that read a list of files, and uploads them to the ftp server, in the appropriate directory. In contrast to the 2 previous scripts, you do not have to create all the cd, lcd and put statements in advance, the script handles those based on the file paths from the list (a list that can be created with, eg, a find command). As a result, the script creates a new connection for every file. This produces some processing and network overhead, but reduces the risk for user error, and the amount of work you have to do to set it up : all you need is a list_of_files_to_upload.

Because files can not be uploaded to a directory that doesn't exist on the server, the script will also compare a remote directory listing to the local directory tree, and create directories where necessary.

Alternative : ftpmount

In stead of modifying local copies of your web pages, you can also mount the website locally by means of ftpmount or an other remote filesystem mount. This allows you to threat the remote files as if they're part of your local filesystem, and you could edit them 'locally' or run 'find and replace' scripts over entire directory trees without the need to upload all modified files later on : you'd be working on the remote files directly.


The Silly Software Company Silly Software Company
-=oOo=-


A poor man's way of doing things
is still a way to get things done