banner Debian GNU/Linux

Let the Machines do the Work

Unattended Installation for Debian GNU/Linux


Automatic Installations

When you're experimenting with Linux, or when you're maintaining a network with lots of Linux hosts, sooner or later you'll be thinking that there should be a way to automate the setup process, because you're going through the same installation process over and over again, repeating identical setups on multiple machines. Or you've gone to a lot of trouble to create an ideal server, a perfect baseline, or a highly customised workstation in a test environment, and you need to repeat that exact setup (just once, or several times) in a production environment.

These are exactly the kind the kind of repetitive actions that computers are supposed to be so good at. You need an automated installation procedure. It is also a great way to document and backup a system and make it reproducable (eg. for disaster recovery scenarios).

Of course, Linux has a sollution. Even more than one :

boot parameters
when booting the setup CD, you can add boot parameters (e.g. keyboard, dhcp or not, ...). This is mainly intended to customize the setup session itself, but can, to some extend, help to automate the installation procedure
preseeding
when preseeding, you supply an 'answer file', a text file that contains answers to the questions the installer normally asks. The installer reads the answers from the file, so you don't have to stay with it. When you use the same answers form multiple machines, you're guaranteed identical systems on all machines. It even offers some features not available during interactive installations.
FAI : Fully Automatic Installation
FAI is an automated installation tool to install Debian GNU/Linux and similar distributions. It's more powerful than preseeding, but requires more preparation. The intended audience is system administrators who need to set up Linux clusters or hundreds and more hosts.

Here, we investigate the easiest and most flexible : preseeding. It has some limitations (not always fully automatic, ...) but it's fairly easy to set up and maintain, so it's a good start, and most of the procedures discussed here can be incorporated in other Automatic Installation mechanisms as wel.

Preseeding : Methods and Media

Preseeding works with an answer file that is not standard included in your setup media. You can install from a CD or boot floppies, from a network source, from a filesystem. With any of these methods, the preseed file can be located on the CD, on a network location, ... Depending on this combanition, the setup will be more or less automatic : obviously, when booting from an installation CD and using a preseed file on a network, the installer will need answers to some questions (hostname, network configuration, ...) before it can load the preseed file. On the other hand, including a customized preseed file in a setup CD will allow for more automation, but requires re-mastering of the CD.

Appendix B of the Debian Installation Manual describes in detail which preseeding metothds are available with what setup media, and their respective limitations. We choose the easiest one : boot from a Debian netinstall CD, using a preseed file located on a web server. As it happens, we have a Linux server running http and ftp daemons, so that figures. Refer to Appendix B of the Debian Installation Manual for details about setting up and using preseeding. What follows is merely a summary, a short checklist, or a quick How To.

Overview

To automate a Debian setup, we will


Create a preseed file

The preseed file is a simple text file. You can find an example in Appendix B of the Debian Installation Manual, which can be easily modified to suit your intended setup.

Fore more elaborate installations, you can install debconf-utils on a working Debian system, then run debconf-get-selections . This will collect all possible questions the debian-installer may ask, and provide the answers as relevant to the system you run debconf-get-selections on. The output can be saved to a file which can then be used as a starting point for your own customized preseed file : Test your unattended setup, take note of the questions you still get asked, and answer them the way you'd like the unattended install deal with them. When the setup is completed, use debconf-get-selections to figure out how debian installer (d-i) and other config utilities recorded your answer, and add that to the answer file.

		debconf-get-selections --installer > file_selections
		debconf-get-selections >> file_selections
	
	#find out how to deal with installer questions about tasksel
	grep tasksel file_selections
	

see also Automatic System Documentation : scripts to create the files needed for automatic / unattended system setup.

Setup unattended

Boot the setup CD. When the boot prompt appears, type the preseed boot parameter with a protocol and url to your preseed file. You can add additional boot parameters (language, keyboard, framebuffers, network, ...) but the number is quite limited (8 for kernel 2.4 and early 2.6) and some are already in use by the bootstrap program so you may encounter error messages.

	boot: linux26 preseed/url=http://my.dom.com/debi/basicsetup.preseed 
	

You will still be prompted for language to be used during installation, country, keyboard, hostname, disk partitioning, and root password. Language, country and keyboard could be covered by boot parameters but you might exceed the maximum number allowed. hostname should be unique, so that's OK. Some control over partitioning is not too bad (although it is possible to automate this as well). Your network configuration will be dealt with by DHCP if you have a DHCP server handy. Overall, this is not too bad for an unattended setup with minimal preparations.

You can also automate this further but that would require a customized preseed file which is not on a network location; you'd need to build a custom install CD that contains your preseed fileso it can answer the questions that come up before the network configuration.


Post Installation Tasks

When you opt to install only the bare minimum (as opposed to 'select a task' [Workstation, Server, ...] which will install plenty packages), you may want to run a number of 'apt-get install' statements after the system setup. Likewise, you may want to further configure the system by replacing default certain configuration files by your own customized configs (eg a bastille config file, a firefox preferences script, an optimized XF86Config-4, apt sources and preferences ...)

With the "preseed/late_command" option you can execute a command (eg to load a script) that is executed at the end of the setup, but before the system reboots. Note that the system then is still using /target as target file system, which will become / after reboot. This is explained in more detail in Appendix B, Paragraph 5 of the Install Guide.

wget

To get additional (configuration) files copied to your newly installed system, wget is a beautiful sollution. It's a command line tool to get files from http and ftp servers, and as we're already using a http server to provide the preseed file, it is the obvious choice.

With the following statement, we get a script (my_post_install_tasks) from a web server. It will be stored in the current directory. We then make it executable, and run it. This is all done in one statement so that the statement can be used as a preseed/late_command value.

	
	wget http://my.dom.com/debi/my_post_install_tasks && chmod +x ./my_post_install_tasks && ./my_post_install_tasks
	

If you prefer to start the software setup after the post-install reboot, you can set a preseed/late_command to add the post-install script to the rc startup scripts, or to the root user's profile executed after root logs in. The preseed/late_command could look like this :

	wget http://.../tasks && chmod +x ./tasks && echo /root/tasks > /root/.bash_profile
	

Note that the difference is that the script is not executed rightaway, but a statement is echoed to .bash_profile so the script will be executed when root logs on (after the system has rebooted). To make it a 'run once', the post-installation script should remove references to itself from rc.d or root profile (see further).

The my_post_install_tasks script in turn may contain several wget statements to copy customized configuration files, run apt-get install statements, or whatever. You may even want to copy a customized /etc directory to import configuration files.


	# get customized config files from server
	# 	wget downloads to current directory and does not overwrite existing files, so
	# 	we pushd-popd between directories, and remove the old file first.

	pushd /etc/X11
	rm XF86Config-4
	wget http://my.dom.com/debi/XF86Config-4
	popd

	pushd /etc/apt
	rm sources.list preferences
	wget http://my.dom.com/debi/sources.list
	wget http://my.dom.com/debi/preferences
	popd

	# install packages
	apt-get -y install ssh vim

	

Automated software installation or reproducing an existing (model) configuration

To reproduce a baseline or model installation, what you need is to install a pre-defined list of packages, and apply a predefined configuration to them. In Debian, all configuration is kept in /etc. So you can setup 1 computer with a model configuration, then copy /etc to a server where you can wget, scp (ssh secure copy) or rsync (ssh secure remote synchronisation) it from.

To reinstall a list of packages, you can use a text file (eg. software.list) as input for apt-get, like this :

	for pkg in $(cat install_packages.list) ; do 
		apt-get -y install $pkg
	done
	

or do them all at once, which is faster, but may fail if any one of the packages can't be installed :

		apt-get -y install --force-yes $(cat install_packages.list)
	

You don't even have to create these lists by hand. From an existing model system, you can collect a list of installed packages with the following commands:

	dpkg -l
	

or filter out the generic package names like this (and later let sources.list and pinning decide what versions will get installed) :

	dpkg -l | awk '{print $2}' > ~/installed_packages.list	
	

see also Automatic System Documentation : scripts to create the files needed for automatic / unattended system setup.

Combined with a well-prepared sources.list and (optionally) a preferences file, which you have copied to the target system in advance (see post installation tasks), you can just sit back and watch the machine do all the work.

The only remaining problem is that the pre- and post install script in the debian packages may still ask input to (pre-)configure the packages during the install routine. You can reduce the required user interaction if you

See also Automatic System Documentation : scripts to create the files needed for automatic / unattended system setup)


sample post install script (my_post_install_tasks)

Assuming we have

we can execute the following script on the new system, to reproduce the configuration of the old system. Note that this is not yet a complete reproduction : we did not restore any data, and therefore it may also be possile that some references in configuration files (eg paths to data directories) can cause problems.

	#!/bin/bash
	
	SERVER="your_server"
	CONFIG_STORE="$SERVER:/root/inventory/baseline"

	echo starting reproduction script
	echo make sure we have the tools and have an up to date system to start with
	apt-get update
	apt-get upgrade
	apt-get dist-upgrade
	apt-get -y install perl debconf debconf-utils rsync

	echo;echo collecting files from server. 
	echo 	when prompted, enter root password on remote server
	cd /tmp
	scp -r $CONFIG_STORE/* /tmp
	
	echo;echo replacing files in /etc
	echo a copy of /etc is saved as /etc.old for recovery
	mkdir /etc.old
	cp -R /etc/* /etc.old

	
	echo;echo synchronising files in /etc
	# we could rsync directly with the files on the remote server as well, but
	# the files have been copied anyway because we used a wildcard
	# this way, we do not have to authenticate with the remote server multiple times
	# rsync i.s.o. cp because it creates directories if needed, before copying files

	rsync -r /tmp/etc/ /etc



	# resetting the  debian-installer selections 
	echo;echo resetting package options
	apt-get update
	debconf-set-selections installeroptions
	debconf-set-selections packageoptions
	apt-get -y dselect-upgrade
	apt-get check
	
	
	# do apt install from package list
	echo installing packages
	apt-get update
	for pkg in $(cat install_packages.list); do
		echo; echo processing package $pkg
		apt-get -y install --force-yes $pkg
	done
	apt-get check


	## asuming bash for root's shell and script started from /root/.bash_profile
	## remove the call to my_post_install_tasks
	## so it won't be executed again

	# rm /root/my_post_install_tasks
	# sed -e '@/root/my_post_install_tasks@d' /root/.bash_profile

	# depending on the situation, you may want to add statements to restore data here
	# (/home, /srv, maybe a selection from /var, ...)

	
	# finished
	echo 
	echo if there were problems with package installation and configuration due to
	echo missing data directories, create/restore those directories and 
	echo run apt-get install -f to fix things
	echo
	echo 
	echo you've made considerable changes to the system configuration
	echo you might want to reboot to make sure all services / daemons / applications
	echo are executed with their new configuration
	exit
	

Extras

Yet another way to automate the installation and configuration of software, is to create your own debian (.deb) packages with customized configuration files and dependencies to trigger the installation of those packages you want, the way you want them.

All of his should also work with debian-derived distributions such as Ubuntu or others as far as they use debian-installer. here's an ubuntu preseed file that worked form me - you may need to edit it a bit.

When maintaining or running multiple debian systems, apt-proxy, a local Debian mirror, may also be a good idea. You can also create a custom install CD that sets up your Linux the way you want it.

Preseeding can be combined with network boot. You set up a boot server with installation files; the target PC's boot (without operating system), download the installer from the network, and start the installation

And what if we would simply clone an entire system ?
and easily distribute changes to running systems ?

Links on Preseeding


Koen Noens
February 2006
updated June 2006