banner Debian GNU/Linux

My way or the Highway

Create your own Debian or Ubuntu packages


Roll you own

When you're discovering Linux, especially the system administration aspects of it, you'll sooner or later will want to create and maybe even publish your own packages. Here is a small introduction in the fine art of creating Debian packages. As it happens, this can also be used to create 'virtual' packages, packages that only contain references to other packages, or packages that only contain sutomized configuration files, which can be extremely useful in reproducing a system.

In this tutorial, we'll go step by step through the pocess of creating .deb packages, starting with really simple packages and building up to compile and build from source, and publishing packages in a repository.

1. A simple package

A debian package is basically a tar archive, and the essense of creating a debian package is that you

  1. create a directory structure in a directory called "debian"
  2. you populate the subdirectories of "debian" with files (binaries, configuration files, installation scripts, ...)
  3. add some 'control' files that contain information about the package itself
  4. pack this "debian" directory in an "archive" ; the deb package

1.1. the "debian" directory

create a directory, named "debian" in an appropriate location, eg your in home directory

inside this "debian" directory, create a directorie called "DEBIAN" : This is called the "controll area" and will contain "controll files", files that contain information about the package itself

Then, think of "debian" directory as the root of the filesystem your package will be installed on, and create subdirectories acoordingly. So, id you have a package that will be installed in /usr/sbin, create a directory debian/usr/sbin

1.2. put the files in the appropriate directories under /debian.

eg if your package should install a shell script in /usr.bin, copy myprogram.sh to .../debian/usr/bin/myprogram.sh

1.3. create a control file in debian/DEBIAN.

the file itself is called 'control', and contains a discription of the package.

1.4. build the package

to build the package, run dpkg-deb --build debian .
Obviously, "debian" refers to your debian directory, so you need to run this from the parent directory of debian. The result is a package called "debian.deb", in your current directory. You'll want to rename it to myprogram-versionso.and.so.deb, and you kan install it with dpkg -i ./myprogram-versionso.and.so.deb .

Basically, that's it. Refer to the Debian Binary Package Building HOWTO for details, and instructions on how to check and improve the quality of the package.

This method is sufficient to e.g. create a package that installs knoppix hardware recognition and xserver configuration on a Debian system, or to install one or more scripts in /usr/bin (or anywhere else) and handle any dependencies to packages the commands in the script may have. Support for customized configuration files is not yest included. That, we will handle in the next step.

2. A package to handle customized configuration files

re. Debian Policy : Handeling Configuration files

In the previous example, we've just installed a shell script. packages can also contain (customized) configuration files. Lets say you add your preferd apt/sources.list, apt/preferences file, /etc/ntp.conf, a collection of dns, dhcp or webserver configuration files, ... whatever, into a package ("mycustomconfig.deb"). Installing that package on an existing system would effortlesly convert that system to your preferred configuration. You could include a few scripts should you need them, or make it install additional software - see further: using dependencies).

Debian knows two ways of handling configuration files during software setup:

So we will take the dpkg approach. As before, create a "debian" and "debian/DEBIAN" directory. Now, collect copies of all customized files that ou want to be in this package, and place them into their corresponding location under "debian". So /etc/resolv.conf would go to debian/etc/resolv.conf, /etc/apt/sources.list is copied to debian/etc/apt/ sources.list, your dns zone files and other dns configuration goes under /debian/etc/bind/ , and so on.

Then, in debian/DEBIAN,

debian/DEBIAN/conffiles is a text file which list all configuration files that the package contains. The files should be referred to by their absolute path and file name on a normal system (so : /etc/resolv.conf, not: .../debian/etc/resolv.conf)
This can be done easily with a small script such as

		pushd debian
		find ./ -type f  > tmplist
		sed -i 's/\.//' tmplist
		mv tmplist ../tmplist
		popd
		cat tmplist |more
		mv tmplist debian/DEBIAN/conffiles
		

you should still review tmplist (replace the 'cat' statement with an 'edit' statement, eg "vim tmplist", and remove any /DEBIAN entries, trailing whitelines, ...).

Then, build and install the package.

3. using dependencies

The "Dependencies" in the control file of a package make sure that all other packages that your packes will needc, get installed automatically. This mechanism can also be used to just install a collection of existing packages. Say you really like Opera Web Browser, Thunderbird Email and Mplayer mediaplayer - 3 packages that Ubuntu, your preferred Linux distro, does not install by default. You could create a package 'mypreferredsoftware.deb' that depends on opera, thunderbird and mplayer, but contains no real files (just the control file with refers to these dependensies).

when you dpkg -i mypreferredsoftware.deb , you will get an error about unmet dependencies, which you can fix with apt-get install -f. so :

		dpkg -i mypreferredsoftware.deb
		apt-get install -f
	

Apt-get will now fix missing dependencies and complete pending installs, so those tree packages (and all other packages they require) will get installed automatically. Nice, no? And combined with your customized configuration files, you have there a very efficient way of rebuilding your system from scratch.

4. Building from source

The previous paragraphs handled small packages with scripts, custom configuration, and dependencies. Without even touching source code, this goes a long way in using the package system for setting up a customized system

Building .deb packages from source is basically the same as described so far, with one added complication : you have to compile the source code to create binary files. These binaries should, of course, end up in the appropriate subdirectory of the 'debian' directory.

Here are a couple of howto's that explain more about this :

5. Exercises

6. Installing packages from a server, with apt-get (or aptitude, ...)

Of course you can install packages with dpkg -i , and of course you will keep your packages in a save place so that ou can reinstall them easily after you've installed. But what if you could just keep them on your own "private" Debian mirror server and just install them with apt-get, as you usually do ? It's possible, it's not that difficult, and it also fixes the "apt-get install -f" workaround encountered earlier.

For one or a small bunch of packages, you can just set up your own web server to act as an apt source. For more extensive software collections, have a look at creating a partial Debian Mirror (people.debian.org). If your .deb packages are on a publically accessible server, others can use them in the same way, as in "people.debian.org" and lots of other "unofficial" debian and ubuntu repositiories (sometimes a single package, sometimes a rather large collections).

For a simple apt server :

  1. create a web server
  2. create a directory (mypackages) in web server root
  3. put your .deb packages in mypackages (or a subdirectory thereof)
  4. test if everything is good so far : browse to http://theserver/mypackages/ ; this should show the packages.
  5. create a Packages.gz in http://theserver/mypackages/ (see further)

To correctly interact with apt, your web server requires 1 additional file : Packages.gz. This is an archive that contains a textfile ("Packages"). The Packages text file has descriptions of the packages served by the server. These descriptions are very similar to those in each package's control file.

To create a Packages file, you can copy the contents of the control file of every package. A blank line in Packages separates one package description from the next. You have to at least add the Filename of the .deb package, including the path relative to the location of Packages.gz on the web server. So if Packages.gz is in //theserver/mypackages/ , and the .deb files ass well, you'd add something like
Filename: ./mycustomconfig.deb .

Also add entries Size: and Installed-Size: . Apt uses this to estimate download and installed sizes of selected packages and (at least on Debian, Ubuntu seems less strict) apt-get install may fail with 'Size mismatch' if those keywords are not given in the package description.

	Package: hwsetup
	Priority: optional
	Section: devel
	Maintainer: Klaus Knopper 
	Architecture: i386
	Version: 1.0-9
	Depends: libc6 (>= 2.2.5-13)
	Filename: packages/hwsetup_1.0-9_i386.deb
	Size: 51392
	Installed-Size: 148
	MD5sum: 013288d45e81494cc761f81e2777ac7e
	Description: Automatic hardware setup using the kudzu library
	 hwsetup is a non-interactive hardware setup tool that tries to read your
	 hardware setup and autoloads all necessary modules, plus generating the
	 symlinks for mouse and cdrom/scanner in /dev.
	

When Packages is ready, archive it into Packages.gz, and put it in //theserver/mypackages/

Debian has developped tools to automate the whole process, which sounds like a good idea if you're creating and maintaining lots of packages, or plan to distribute 'official' packages. See people.debian.org. For instance, to maintain a correct Packages.gz file on your apt server, use dpkg-scanpackages (provided by dpkg-utils) in stead of writing and gzipping it by hand.

For small scale home use, here's a small, simple script that you can use to automatically create a Packages file on a web server.

	PACKDIR=/srv/www/packages

	# create an 'override' file (required for dg-scanpackages)
	cd $PACKDIR
	ls -1 *.deb > override

	# create a package file
	dpkg-scanpackages ./ override | gzip > Packages.gz
	cd -
	

This 'updatepackages' script is then used in an other script that builds a package, moves it to the web server directory, and updates the override and Packages file. It takes the name of the package (= name of the directory where you keep the files of that package) as argument.

	## argument is required
	if [ "_$1" = "_" ] ; then
		echo "usage: $0 package_dir_name "
		exit 1
	fi

	WORKDIR=/home/jdoe/packaging
	PACKDIR=/srv/www/packages

	## build a package
	cd $WORKDIR
	dpkg -b debian  && \
	mv debian.deb $PACKDIR/$1.deb && \
	$PACKDIR/updatepackages
	cd -
	

This will only work if the directory where you do your packaging is on the same machine as the webs server you use as an apt source. You can replace the mv statement with an scp statement and call the (remote) updatepackages script through ssh to make it work between separate machines (or set up another way to move tha package to your web server, eg an automated ftpupload, a file sharing solution, or by mounting a remote filesystem.

Finally, for this to work, you need to modify the sources.list on the computers that will use your packages. Add something like

		deb http://theserver/mypackages/ ./
	

This entry points to the location of Packages.gz on the web server. You need to add "./" indicate the "current dir" of http://theserver/mypackages/, because sources.list expects an entry of minimal 2 components to reflect the standardized structure of a Debian mirror.

7. Customizing a setup CD to include your own packages

You can customize a Debian install CD image to include your own custom packages.

8. Introducing selfmade packages into the Debian package system

Technically, this is quite similar to uploading packages to your own web server. The main differences are "policy" : the Debian package management, patch management, standardized directory structure on mirror servers, quality control, security policy, etc... that you will have to deal with. That's far beyound scope of this tutorial - refer to Debian documentation. Likewise for Ubuntu.

additional documentation


Koen Noens
November 2006