How to Build Your Own Linux Distribution

Many authors have written about UNIX® flavors, delving into the mysteries of scheduling, memory management, multiprocessing and threading, file systems, and the interaction between users and the kernel. The author writing about Linux has an advantage over UNIX authors: The Linux kernel is unlikely to split into competing forks -- corporate upheavals notwithstanding -- because the GNU Public License (GPL), the existence of a centralized research lab -- the Open Source Development Lab (OSDL) -- and Linus Torvalds' unassailable position make Linux, luckily, a slow-moving target.

Things are never quite as simple as that. Inspecting Linux kernel and system-level code is likely to be a time-intensive affair and of somewhat limited use in the real world. The LFS project aims to remedy the problem of limited system-level intelligibility on Linux. The very fact that the kernel needs a large number of libraries and tools to get a Linux system to perform even basic tasks has been commented upon, but what if a somewhat more sophisticated user who has a slim-line Linux distribution does not like to download several gigabytes of binaries that lock him out of any chance to optimize a system and do not allow him to throw out all those pesky, unnecessary tools? What if a very sophisticated user refuses to accept the diktat of various community distributions and wants to run a Linux/Apache/MySQL/PHP (LAMP)-type application stack from a CD? LFS comes to the rescue.

To get acquainted with the way a Linux system works, the creators of LFS decided that compiling the system by following the tree of module dependencies would be a natural way to get to know the mechanics of an operating system in general and Linux in particular. After users have mastered the compilation process, they can start eliminating those parts of the dependency tree connected to system components that are irrelevant to supporting the operating system's primary purpose. It is feasible, for instance, to eliminate the compiler tool chain itself after compilation is complete. Embedded LAMP stacks can make do without a full set of command-line utilities. Configuration utilities might be dropped, as well, and most users can make do with one, instead of the plethora of file systems Linux tends to support.

Linux parts
One important part of the LFS system is the large number of source files available as tar balls. Documentation is another vital part -- and the most important. Indeed, it would be perfectly possible to take an up-to-date LFS book file and create an LFS distribution, because each download location and the characteristics of each source file and its dependencies are described in the LFS book in detail. The procedures for compiling each group of source files from the kernel to the compiler to the shell have been written up, and you can find alternative routes -- where they are possible -- in LFS books describing systems with different characteristics. Another part of the LFS system that is unlikely to be in the toolkit of the average power user is the boot script needed to boot the system after the basic LFS system has been pieced together.

Now for the big caveat of LFS distributions: What a courageous distribution builder needs is a working Linux distribution, including a complete compiler tool chain and a suite of file system-creation utilities. Naturally, all source-based Linux distributions need to be bootstrapped using a particular compiler version, which is by no means identical from distribution release to distribution release. LFS is not the only system in this field, but it is the only system that allows you to work directly with individual source files. Most other source-based Linux systems, such as Sourcemage and MyGeOS, provide a complete download, which users are well advised to use. LFS makes no such assumption, and stripping down the LFS framework is encouraged.

Presuppose a functioning Linux distribution installed on nonexotic hardware, even though LFS is probably less demanding as far as configuration tools and scripting are concerned. To compile LFS, you need to prepare a partition and a file system, and you also need to compile a compiler and system library. It is a fairly nerve-racking procedure if done by hand, but it definitely increases your confidence in dealing with the rest of the installation. The compilation of the whole system tends to take from an hour to four days, depending on the age of the underlying hardware and your command-line dexterity.

If -- and this is a fairly big assumption -- you're willing to retain much of the book installation and keep changes to the installation proposed in the LFS book to a minimum, you could also use the automated installation routine to install an LFS-based distribution. The installation routine is not presented in the LFS book, but is available as an XML-based description under the name Automated Linux from Scratch (ALFS). The active installation is available as a C-based script that uses ncurses to give some semblance of a graphical installation. The installation is also known as nALFS and presents an extremely flexible package installation framework. It needs a functioning Linux system with a working C compiler and XML parser to work. A working LFS system would suffice.

Automated Linux From Scratch
ALFS has a purpose that goes beyond LFS itself. LFS on its own teaches the inner workings of a Linux-based operating system, but it does not include a single graphical user interface (GUI). Neither does LFS permit connecting to a network or, indeed, the Internet. ALFS can simplify extending the system -- for example, by adding the libraries enabling Internet access or by installing the X libraries required for graphical desktops.

Code audits are extremely time-consuming, and adding a large number of patches is advisable only if patch servers are maintained centrally by dedicated staff. It is, however, possible to replace some libraries that have been rewritten from the ground up to reflect new approaches to security problems. A good example is to make it extremely difficult to guess process identifiers by randomly allocating numbers from a reasonable large random number pool. The OpenBSD project has pioneered this method, which has found its way into various UNIX flavors and Linux distributions.

A fairly new project known as Hardened Linux From Scratch (HLFS) takes this approach to security under Linux. The project, which presupposes a fairly decent grasp of LFS and some parts of BLFS, uses several utilities and libraries that do not tend to be standard in most Linux systems.

Possibly the most important addition to HLFS is the Stack-Smashing Protector (SSP), which you enable by using a gcc directive. SSP was developed to protect against stack-smashing attacks, which belong to the most common class of security threats affecting Linux systems. Other security goodies include a first-class random number generator and the compilation of position-independent executables, where executable code typically turned into statically linked object code appears as shared libraries, and position-independent executables libraries can hide their addresses by randomizing them. Of course, a large number of patches are available and can be sourced from the HLFS Web site.

More generally, users do not need to be programmers to change the way a Linux distribution is built: The bit of scripting users learn by building an LFS system is sufficient. An LFS specialist can change and extend the very composition of a Linux distribution without impairing its basic structure. This functionality is particularly important for organizations that have the manpower and expertise to maintain Linux systems, but not the financial wherewithal to buy commercial support from consultancies and corporations. LFS-based Linux systems have been demonstrated for educational purposes and for large networks. It is likely they will be used in other areas, as well.