Parser Tools like lex/yacc
Be prepared that some portions of your grammar written on AIX or Solaris might not work directly on Linux. For example, some variables like yylineno (an undocumented lex scanner internal variable) might not be directly available on Linux by default. The following code snippet can be used to check if yylineno is directly supported or not. Open a file named a.l with following contents:
Then enter lex a.l. Search for “yylineno” in lex.yy.c. If the variable is not available, two possible solutions to supporting yylineno are to use the -l option for lex in Linux (in other words, do lex -l a.l) or change the code to the following:
Some distributions (such as SLES 9) do not come packaged with yacc but come packaged with bison by default. If the requirement is for yacc, it may need to be downloaded.
It is possible that some of the code pages are named differently on Linux. For example, IBM-850 on AIX could be aliased to ibm850 on Linux and ISO8859-1 could be aliased to ISO-8859-1. Scripts might have to be changed if the application-message catalogues depend on some of these code pages and code-page conversions are required (which can be accomplished by using the iconv tool). Most of the common locales like ja_Jp, en_US, etc., are available on Linux.
Communications over sockets is protected by default in the new distributions (RHEL AS 3), so if you are implementing an IP-based server kind of process listening on a particular port, you will need to add the new service into iptables. Iptables are used to set up, maintain, and inspect the tables of IP packet filter rules in the Linux kernel.
For example, for the first time you might have to add a new chain like /sbin/iptables -N RH-Firewall-1-INPUT and then add the new service in the chain like so: iptables -I RH-Firewall-1-INPUT -s 0/0 -i eth0 -m state –state NEW -p TCP –dport 60030 -j ACCEPT (where the new destination port 60030 is mapped to a service in /etc/services).
Locating Installed Packages and Variable Data
It is a good idea to follow the packaging recommendations produced by the Linux community — the recommendations help prevent the cluttering of /opt by code and /var by application data. These recommendations suggest including the vendor and package name in the location of the code and data. As an illustration, consider the following example.
Suppose IBM develops a new application called “abc.” The package should ideally be installed in /opt/ibm/abc. The related data should be located in /var/opt/ibm/abc and not simply /var.
File System, Usage Parameters, Stacks
My group discovered a number of diverse things during its porting activities, and because they are relatively concise, I’ve gathered them together here.
Support for the file system
If your application needs to use facilities such as logging and writing data files, file system-based support is easier to install, configure, and administer compared with the raw I/O.
Direct system calls to gather information about parameters (like memory heap usage) don’t seem to exist. /proc filesystem support needs to be used to determine such parameters.
Currently, support for calls like pstack is available only on the Intel architecture; it is in the process of being developed on other architectures. To get stack traces programmatically, programmers might have to implement their own versions using the ABI definitions for the yet-to-be-supported architectures.
Another option is to use gdb-based scripts to get the stack information. The stack information is usually required for servicability of the product. gdb is more standardized across different architectures and distributions.
Memory Maps and Using Shared Memory Segments
If the application uses shared memory segments, care has to be taken to place the starting addresses of the shared memory segments appropriately unless the user wants to rely on the system-provided initial addresses. Also, different architectures will have different memory-map support; the areas available for shared memory could be different.
For example, on Intel every process has the bottom three-quarters of the address space allocated to user land; the top piece is allocated to the kernel. This means that the total memory that any Linux process could ever have is 2 GB (390) or 3 GB (Intel). This total has to include the text, data, and stack segments, plus all shared memory regions. On the Linux/390, the area for shared memory starts at 0x50000000 and must end before 0x7fffa000. You have to consider all architectures before deciding on the addresses if you want to keep the starting addresses common for all the architectures that are going to be supported by the application.
Signaling — sending control signals that start and stop a transmission or other operation — is not much different on Linux as compared with other Unix platforms except that the signal numbers might differ or some of the signals, such as SIGEMT, are not available on some distributions (such as RHEL AS 3). (For more details on differences in signaling between Solaris and Linux, see the reference in the Resources section.)
Configure Kernel Karameters
The programmer might be required to tune some of the kernel parameters so that the application can scale at runtime. If that’s the case, some of the important kernel parameters to consider are threads-max (maximum threads per process), shmmax, msgmax, and so on. The list of configurable parameters is available in /proc/sys/kernel. The parameters can be configured using the /sbin/sysctl system call. The threads-max parameter can be particularly important if you are porting a large multi-threaded application.
Architecture-specific code in an application is typically limited to a few areas. I’ll look at some examples in this section.
The programmer does not have to worry about which architecture the code being written for. Linux provides a way to determine the endian-ness in /usr/include/endian.h. Following is a typical code snippet you can use to determine if the operating environment is big- or little-endian; you can set a specific flag for your convenience.
/• Are we big-endian? •/
#if __BYTE_ORDER == __LITTLE_ENDIAN
#elif __BYTE_ORDER == __BIG_ENDIAN
Determining the stack pointer
Inline assembly can be written to determine the stack pointer.
int get_stack(void ••StackPtr)
•StackPtr = 0;
__asm__ __volatile__ ("movl %%esp, %0": "=m" (StackPtr) );
#error No code for this architecture in __FILE__
Implementing compare and swap
Here’s an example of implementing compare and swap for the Intel architecture.
bool_t My_CompareAndSwap(IN int •ptr, IN int old, IN int new)
unsigned char ret;
/• Note that sete sets a 'byte' not the word •/
__asm__ __volatile__ (
" cmpxchgl %2,%1\n"
" sete %0\n"
: "=q" (ret), "=m" (•ptr)
: "r" (new), "m" (•ptr), "a" (old)
#error No code for this architecture in __FILE__
Choose an IPC Mechanism
Typically the choice for an interprocess communication (IPC) mechanism — mechanisms for facilitating communications and data sharing between applications — is between using signals, writing a loadable kernel extension, or using process-shared mutexes and condition variables.
Signals are the easiest to implement, but in a multi-threaded environment, care has to be taken that all the threads spawned have similar signal masks. The process structure should normally be modeled such that only one thread should handle the signals, or else the signal could be delivered to any of the threads and the results could be unpredictable. It is possible that threads are spawned in a process by other participating entities outside the control of the application and it might not be possible to control their signal masks. Signals might be an unpopular way of doing IPC in a large multi-threaded application for that reason. For example, applications running under an application server could spawn their own threads and could catch signals actually meant for the application server process.
Kernel extensions are not very easy to write and may not be easily portable across various architectures where Linux is supported.
With the advent of the POSIX draft 10 standard and its available implementation on Linux 2.6, process-shared mutexes (mutual exclusion object: a program that allows multiple programs to share the same resources but not simultaneously) and condition variables are a good choice for implementing the IPC mechanism in a multi-process environment. This mechanism would require setting up shared memory in which the mutexes and condition variables reside and all processes will have a common reference for these constructs.
Select the Threading Model
It is quite possible that some old application being ported to Linux are based on draft 4 of pthreads. The latest versions of Linux support pthreads draft 10, so care needs to be taken to map the calls appropriately. If the application is using some exception-handling mechanisms based on a third-party implementation (for example, TRY-CATCH macros provided by DCE), then the programmer needs to make sure that the exception-handling code is also compatible with draft 10 of pthreads.
Some examples of calls that have changed from draft 4 to 10 are in the following table.
Table 1. Calls changed from draft 4 to 10 of pthreads
|pthreads draft 4||pthreads draft 10|
|pthread_getspecific(key, value)||*value = pthread_getspecific(&key)|
The choice of threading model lies somewhere between native Linux threads and the Native POSIX Thread Library (NPTL) implementation, which provides a POSIX-compliant implementation for the threads in Linux. The Linux kernel (from 2.5 version onwards) has been modified to provide POSIX-compliant support. NPTL is available on SLES9. Red Hat has backported NPTL support for RHEL3 (which is based on the Linux 2.4 kernel). RHEL 3 has support for both NPTL and native Linux threads. You can switch to native Linux threads by setting an environment variable LD_ASSUME_KERNEL=2.4.1, but not many vendors have ported their software on RHEL3 using NPTL support.
Major drawbacks to using native Linux threads are the following:
• Child thread SIGCHILD comes to the thread, not the process (a non-POSIX behavior).
• getpid() is broken — it is very hard to get the pid for a group of threads constituting a process.
• Changing the userid in a thread doesn’t change it for all threads in a process.
In short, each thread looks like (and in some ways behaves like) a separate process.
There are also problems on the kernel side:
• Processes with hundreds or thousands of threads make the /proc filesystem barely usable. Each thread shows up as a separate process.
• The problems with the signal implementation are mainly due to missing kernel support. Special signals like SIGSTOP would have to be handled by the kernel and for all threads.
• The misuse of signals to implement synchronization primitives adds even more to the problems. Delivering signals is a heavy-handed approach to ensuring synchronization.
On the other hand, with NPTL:
• Signaling issues seem to have been resolved in the latest distribution (Linux 2.6 onwards). Now, signals can be delivered to the process as a whole.
• Futexes (fast userspace mutex, a basic tool to realize locking and building higher-level locking abstractions such as semaphores and POSIX mutexes on Linux) have been implemented, which helps callers wait in the kernel and be woken up explicitly. Thus, PTHREAD_PROCESS_SHARED and interprocess POSIX synchronization primitives can be implemented and are now available.
• It uses a 1:1 model (each user-level thread has an underlying kernel thread) and is pre-emptive (kernel threads can be pre-empted).
• It is Suitable for I/O- and CPU-intensive applications.
• The aim is to move towards 100 percent POSIX compliance.
Based on the improvements provided by various distributions, it would generally be advisable to use the version of Linux that supports NPTL.
Decide on a Viable Operating Environment
Key to the planning process is determining to which distribution of Linux the application is to be ported. You should make sure that all of the required software is available for the level to which you planning to port. For example, it may not be possible to release a middleware product for the Linux 2.6 distribution, because a key third-party database used in the most typical configuration is not available on that same distribution. The initial offering of your product or application might have to be based on a Linux 2.4 distribution instead.
It’s also possible that some of the software with which the application interacts may not be available for all distributions or architectures for which the application is intended. Make a careful study of the viability of your chosen operating environment.
Another issue to consider is whether the application is going to be 32- or 64-bit and whether it is going to coexist with other third-party software that could also operate in a 32- or 64-bit mode.
Get the Build System Working
Products that support multiple platforms typically require code that is specific to the particular operating system on which the product is running. This common code is typically held in a separate code component within the source directory structure.
For example the OS-specific code layout could look something like this:
src/operating_system_specific_code_component/aix (for AIX).
src/operating_system_specific_code_component/solaris (for Solaris).
src/operating_system_specific_code_component/UNIX (for other flavors of Unix).
The following figure presents a more “graphic” view of the OS-specific code layout.
Figure 1. Code organization layout
Make a Linux build system
For the first step, you’ll create a directory for Linux-specific code and populate it with files from one of the platforms. When you introduce a new directory for Linux, the layout might look like this:
src/operating_system_specific_code_component/linux (for Linux).
This in turn would give us a new code layout that would look like the following.
Figure 2. New code organization layout
Normally, much of the application code is common across all Unix flavors and will work on Linux, too. For Linux-specific code, experience shows that picking Solaris-specific files as an initial drop minimizes the effort to port the platform-specific code to Linux.
Next, change the makefiles and introduce Linux-specific items:
• Definitions for the compilers to use
• Library path
• Thread library path
• Compiler flag
• Include file path
• Preprocessor flags
• Whatever else you need
Many changes in the source files are related to changing the include file paths. For example, for the variable errno, <sys/errno.h> needs to be specifically included.
Care must be taken whereever possible that you don’t directly include architecture-specific include files, but do include the recommended files. For example, as mentioned in <bits/dlfcn.h>:
# error "Never use <bits/dlfcn.h> directly; include <dlfcn.h> instead."
You should use the directive -Dlinux or word “linux” carefully. The preprocessor on Linux translates the word “linux” to the numeral 1. For example, if there is a path in the file like /home/linux and the file is preprocessed using cpp, the path in the output file will look like /home/1. To avoid this substitution, the preprocessor directive could look like this: /lib/cpp -traditional -Ulinux <file_name>.
Common compilation commands
The compiler that programmers normally use is gcc. A typical compile line might look like this: gcc -fPIC -D_GNU_SOURCE -ansi -O2 -c <file_name.c> -I<include_path>. -fPIC helps generate position-independent code and is equivalent to -KPIC on Solaris. -ansi is equivalent to -Xa on Solaris.
For shared objects, a typical link time directive could be gcc -fPIC -shared -o <shared_object> <object_file> -L<library_search_path> -l<library_name>. -shared is equivalent to -G on Solaris.
For relocatable objects with entry points, a typical directive might be gcc -fPIC -shared -o <name> <object_file> -e entry_point -L<library_search_path> -l<library_name>.
Before moving on to choosing the best operating environment, I’ll examine the issues surrounding compiling the code on other architectures.
Compiling on other architectures
Another important consideration is that the programmer should be able to get the code to compile on other architectures as easily as possible. The build system should have separate definition files for each architecture involved. For example, the compiler directive for an x86 architecture could have a flag -DCMP_x86 with a directive like -DCMP_PSERIES for some code specific to Linux on pSeries servers. The compile lines in the specific build-definition files will look like this for compiling on an x86-architecture system:
gcc -fPIC -D_GNU_SOURCE -ansi -O2 -c <file_name.c> -I<include_path> -DCMP_x86
and this for compiling on the pSeries architecture:
gcc -fPIC -D_GNU_SOURCE -ansi -O2 -c <file_name.c> -I<include_path> -DCMP_PSERIES.
Both -CMP_x86 and -CMP_PSERIES are user-defined flags and should be used wherever the program is going to have architecture-specific code within Linux-specific code. My experience has been that most of the application code for Linux is architecture-independent, and the architecture-specific code comes into play in areas in which assembly code needs to be written. For example, architecture-specific code will be used if you’re going to exploit locks using an implementation of compare and swap instructions.
Code should be arranged so that there are no architecture-specific subdirectories within the Linux-specific directory in the code layout. Why? Because Linux already does a great job of segregating the architecture specifics and an application programmer typically should not have to care about which architectures the application is going to be compiled on. The aim should be to have the program written for a particular architecture to compile on another architecture with minimal effort and minimum changes to the code, the code layout, and the makefiles. By avoiding architecture-specific subdirectories within the linux directory, makefiles are greatly simplified.
Source files in the linux subdirectory could have a code layout with preprocessor directives as follows:
<x86 specific code>
<p-series specific code>
#error No code for this architecture in __FILE__
A Practical Checklist, Tips, and Insight Drawn from Experience
Much of today’s enterprise-level software on UNIX® caters to the business needs of large companies. And so it must support emerging technologies and follow the rapidly evolving market trends, such as the proliferation of the powerful, flexible Linux™ operating system. Because much of this software is large, multi-threaded, and multi-process, porting it to Linux presents challenges. In this article, get a checklist and advice derived from a real-world port of one piece of enterprise-level software to Linux.
One of the realities of current business IT practices is that many organizations are moving IT to Linux, given its flexibility and stability as a system platform. Another reality is that existing enterprise-level software is too valuable to be discarded. These two realities often crash into each other, but it is critical that they be resolved.
Porting enterprise-level software to Linux can present some interesting challenges. Care has to be taken at all stages — from making design choices to getting the build system to work to finally getting the OS-specific code to cooperate with Linux.
This article is based on my experiences on RHEL and SLES distributions running C applications on Intel and IBM eServer zSeries architectures, but the lessons could easily be extended to other distributions and architectures as well. I’ll talk about some of the planning and technical issues that need to be considered to make your application run on Linux, including the following:
• Getting the build system working
• Deciding on an a viable operating environment
• Minimizing the effort to get the product built on various architectures on which Linux is to be supported
• Identifying architecture-specific changes such as mutex locking
• Maintaining as thorough a common code base for various architectures as possible, using a new compiler
• Deciding on the IPC mechanism
• Choosing the appropriate threading model
• Changing install and packaging to comply with Linux-specific guidelines
• Deciding on signaling options
• Selecting parser tools such as lex/yacc
• Making globalization choices
Porting a new product to a new platform requires that the product be tested extensively. Some of the areas that can require special attention are interprocess communication, packaging, intersystem communication (client-server between AIX and Linux or Solaris and Linux), persistent storage (because of endian-ness), data-exchange formats, and so on.
It is possible that external documentation might change, so a thorough documentation review should be carried out.
The test-case porting to Linux needs to be staged properly along with the development work. A list of intermediate deliverables to be tested should be prepared before moving on to full product testing. This will help you find problems at earlier stages of product development. (For more on highly effective general testing methods, see reference to XP in the Resources section.)
There’s a Port in Every Storm
In this article I have touched upon various stages of porting, including design choices for various OS-specific areas, creating a suitable directory structure, creating a build system, making the code changes, and testing. I’ve highlighted those areas where effort needs to be concentrated, areas such as signaling, shared memory, mutexes and condition variables, threading, and architecture-specific changes. This article is based on real experience gained while porting a large multi-threaded application onto a Linux 2.6-based system, so I hope this checklist can be helpful in saving you time and effort.
The details will change with each port, but the principles I’ve outlined (combined with the material in the referenced below) will allow you to go through the process more easily.
• Ulrich Drepper offers an article on NPTL Design gives an insight into differences between NPTL and Linux threads (and lots of great stuff like futex programming and POSIX signals).
• Get information about the Linux filesystem hierarchy that outlines the set of requirements and guidelines for file and directory placement under the Linux operating system according to those of the FSSTND v2.3 final.
• The developerWorks Migration station, Linux track provides links to a wealth of tools and information for porting and migrating your applications to Linux — on multiple platforms — from Windows/.NET, Solaris, and OS/2.
• Guide to porting from Solaris to Linux on POWER (developerWorks, February 2005) presents reasons why you might want to port your application from Solaris to Linux, as well as guidelines, suggestions, and resources to help.
• For more details on differences in signaling between Solaris and Linux, read Technical guide for porting applications from Solaris to Linux, version 1.0 (developerWorks, February 2002).
• Get involved in the developerWorks community by participating in developerWorks blogs.
• Find more resources for Linux developers in the developerWorks Linux zone.
• Purchase Linux books at discounted prices in the Linux section of the Developer Bookstore.