My way or the Highway - Part 2

How to compile source for Debian and Ubuntu Linux



Background

I assume you know that binaries ('executables') are made from source code and that you more or less know how to use the Debian package system (also used by Ubuntu and other Debian derivatives). The question here is : how do we create a package from source. Read this mini-tutorial on "Creating your own Debian or Ubuntu packages" first to get some background on what a Debian package consists of.

If you're simply interested in installing a 'debianized' source package for use on your own computer, you don't need to go through all the steps of creating a new package. You can just 'install from source' with the following command :

	#This requires deb-src statements in /etc/apt/sources.list
	
	apt-get -b source some-package-name
	

re Apt HOWTO : wotking with source

What we'll be looking at is how to re-create a package (eg recompile a Debian package for use in Ubuntu, or "backport" a package - i.e. compile a source package for use in an earlier release than the one it was developed for), or how to create an installable .deb binary package from a source tar archive ('tarbal') you downloaded from a project's or developer's home page.

Get the Tools

You will need the following tools (re. Debian Maintainers Guide - chapter 1). Apart from some compilers and stuff, these are mainly scripts that automate common packaging tasks so ensure they are executed completely and an a standardised way (eg the creation of the debian/ tree in which the package will be created, the creation and chacking of the debian/DEBIAN/ control files, etc.)

	## list of development tools written as list to install with apt-get
  	
	apt-get install dpkg-dev file gcc g++ libc6-dev make patch perl autoconf automake
	apt-get install dh-make debhelper devscripts fakeroot lintian
	apt-get install pbuilder
	apt-get install gnupg
	apt-get install xutils
	

the Path of Least Resistance

When you download a source package from a Debian (Ubuntu ...) deb-src repository, it has already been debianized, i.e the source files have been placed in a suitable directory tree, appropriate configuration files have been added, and the package contains scripts to help with the building and installation of the package. (That's why you can just install from source with "apt-get -b source some-package-name" as mentioned before). The path of least resistance is to use thes debianization to create a binary package from a source package, without any customization, modification, or other tedious trouble. To do so, you do the following :

download the source package
apt-get source some-package-name
This will download the package, unpack it and create a directory tree in ./some-package-name_version.numbers, and apply any patches (diffs) included in the package.
You can now, if you want to, modify the source code files, the configure and makefile, ... to adapt the package to your wishes
satisfy the build-dependencies
apt-get build-dep some-package-name
This pulls in any additional packages you will need to be able to build the package (not necesarily the same as the dependencies required to run the resulting binaries)
build the binary package
from within the directory that was created for the package after downloading ( cd some-package-name_version.numbers/ )
dpkg-buildpackage -rfakeroot -uc -b

The result is a some-package-name_version.numbers.deb package that can be installed with dpkg --install, or added to an apt server.

Quick and Dirty Backport HOWTO

When you 'backport', you rebuild a package so that it can be used on an earlier release than it was intended for. You may want to do that because you prefer not to upgrade to a newer release (stick with the Dabian stable or Ubuntu LTS release) but still want to use a newer version of a given package. Sometimes this can be accomplished with the "path of least resistance" method - see previous paragraph. In other cases, you will need to modify the source package to resolve build-dependencies or dependencies of the resulting binary. Sometimes you'd have to rebuild from scratch, i.e. from a source tarbal.
We'll have a look at re-compiling an already debianized source package first.

An excellent description of thise procedure can be found in Using pbuilder to backport Debian packages. It uses pbuilder, one of those sets of scripts that automate the building process. pbuilder creates a building environment instite a directory - it's "chrooted", meaning that inside that directory, tyou create a new Linux filesystem against which the compilation occurs. This way you can compile for other systems than your own without interference.
To set up a pbuilder environment :

  1. apt-get install pbuilder
  2. edit /etc/pbuilderrc . Pay attention to things like the result directory, the "hooks" directory, the mirror url's etc that you want to use in the pbuilder environment, ...
  3. create "hook" scripts - these are scripts that will run in the pbuilder environment so any value in the script (eg sources, preferences, mirror servers ,...) should refer to the distribution you're compiling for
  4. initialise the pbuilder environment with pbuilder create --distribution xxxx

see man pbuilder and pbuilder --help for additional options, eg to update the environment, or to clean it after a build

building a package works as follows :

download the source package
apt-get source some-package-name . As before. Note that you will work from the directory that you unpacked the source tree to, not from the pbuilder chrooted directory; pbuilder does that for you.
make changes
see further
build
pbuilder build some_package_version.dsc .
The .dsc file contains a description of the source package + the binary packages that will result from the build. That, plus the debian control files and configure and make files are used by dpbuilder to build one or more binary packages (.deb) automatically

applying changes

You will find that (sometimes, in rare cases) you can just build the package, without any modifications, meaning that you've succesfully build or backported the package. Most of the time you will find that the build fails because of missing dependencies. To resolve this, you can :

  1. review the dependencies. sometimes you can just relax them a bit, e.g. allow a lower version af a dependency and see if it works. The dependencies are listed in debian/control in the directory where you unpacked the source. Go for the Build-dependencies first to get the package to compile. You may have to come back and modify binary dependencies as well if it turns out that the package compiles, but doesn't install because of (binary) "Depends".
    When the dependencies refer to compiler and header libraries or debian configuration tools, this often works if the functions that the package requires are available in earlier versions (but the maintainer used a newer version so he let it depend on that version). Try and see.
  2. If you can't resolve the dependencies by relaxing them, you'll need to backport (recompile ) the packages your package depends on. This step needs to be repeated untill all dependencies are satisfied. This also means that you create new versions (back-ports) of packages, so you need to change any version-dependencies of the packages that depend on it to refer to the backported package.
    You can try to anticipate these dependencies by looking at the control, install and readme files of your initial target package (both the source package and the binary package), and work back until you have a tree of dependencies. Useful command to see a package's dependencies: dpkg -s package | grep ^Depends:

There is a procedure to implement those changes. The idea is that you add files that describe the changes ("diffs") and document them in the changelog, because the software you're playing with will be developped further, and future developers will want to distinguish between the original version and your changes. E.g. when you backport, newer versions of the original package should not be based on your backport.

Often you will also find that others have already modified the package before you (eg to port from Debian to Ubuntu). So you'll need to handle those changes as well. The MOTU School explains how.

Ubuntu Backport : version numbers

appened "~${distro}1" or if already there, increment the number (re Ubuntu Backports Howto). Examples:

debianization and packaging from scratch

This is the method you'll use when you found it impossible to re-create an already debianized source package (because it does not yet exist, or because the previously described approaches resulted in problems you could not get around). In this case, you'll get the source code files (from a developer's website, from a source forge project, from the gnu Savannah website, ... ) and turn them in a ready-to install binary packake for Debian, Ubuntu, or another Linux distribution that uses the .deb packaging system. This procedure involves two steps : compiling the source code, with the relevant options/modifications suitable for Debian (eg the location of the executable, the path to the configuration files, ...), and adding the debian control files (conffiles, install and uninstall scripts, ...). Ideally, you'd also update the changelog file and do some other housekeeping so that others can (easily) use your work to build on.

needs work - complete "tarbal to deb" procedure

additional documentation


Koen Noens
December 2006