Make shared memory work for you, not against you
In terms of time and space, shared memory is probably the most efficient inter-process communication channel provided by all modern operating systems. Shared memory is simultaneously mapped to the address space of more than one process: a process simply attaches to the shared memory and starts communicating with other processes by using it as it would use ordinary memory.
However, in the object-oriented programming world, processes prefer to share objects rather than raw information. With objects, there is no need to serialize, transport, and de-serialize the information contained within the object. Shared objects also reside in shared memory, and although such objects "belong" to the process that created them, all processes on the system can access them. Hence, all of the information within a shared object should be strictly process-neutral.
This is a direct contradiction to the C++ object model currently adopted by all popular compilers: C++ objects invariably contain pointers to various Vee-Tables and sub-objects that are process-specific. For such objects to be sharable, you need to make sure that the target of these pointers resides at the same address in all the processes.
With the help of a small sample program, this article illustrates cases where the C++ model succeeds, where it fails to work with the shared memory model, and where possible workarounds exist. The discussion and the sample program are limited to non-static data members and virtual functions. Other situations are not as relevant to the C++ object model as these are: static and non-static non-virtual member functions do not have any issues in shared environment. Per-process static members do not reside in shared memory (and thus have no issues), while shared static members have issues similar to those discussed here.
This text is highly specific to the Red Hat Linux 7.1 distribution for 32-bit x86 Intel architectures, as GNU C++ compiler version 2.95 and associated tools were used to build and test the sample program. However, you can apply the overall concepts equally well to any machine architecture, operating system, and compiler combinations.
The sample program consists of two clients, shm_client1 and shm_client2, and uses the shared objects services provided by the shared library shm_server. Object definitions reside in common.h: Listing 1. Definitions in common.h
Listing 1 defines three classes (A, B and C) with a common virtual function
WhoAmI(). The base class, A, has a member named
m_nA. The static member
m_sArena and overloaded operator
new() are there to enable the construction of objects in shared memory. Class B is simply derived from A, and class C is virtually derived from A. Both
C::m_nC are provided to ensure that the sizes of A, B, and C are distinct. This simplifies the implementation of
A::operator new(). The interface
GetObjects() returns the shared objects pointers.
The shared library implementation is in shm_server.cpp:
Let’s look at Listing 2 in more detail:
Lines 9-25: operator new ()
The same overloaded operator allows you to construct objects of class A, B, and C in shared memory. Object A starts right at the beginning of shared memory. Object B starts at offset 1024, and C at offset 2048.
Lines 26-34: Virtual functions
The virtual functions simply write a line of text to standard output.
Lines 35-39: GetObjects
GetObjects() returns the pointers to shared objects.
Lines 40-46: Initializer
This class stores the shared memory identifier. Its constructor creates the shared memory and objects in it. If the shared memory already exists, it simply attaches to it. The static member
m_sInitializer ensures that the constrictor is invoked prior to the
main() function of the client module that is using the shared library.
Lines 48-82: Initializer::Initializer()
Shared memory is created if it does not exist, and shared objects are created within it. The object construction is skipped if the shared memory already exists.
Initializer::m_shmid records the identifier and
A::m_sArena records the shared memory address.
The shared memory is not destroyed even after all processes detach from it. This allows you to explicitly destroy it using the
ipcrm command or to make some quick observations with the
The client process implementation is in shm_client.cpp:
The client process gets pointers to three shared objects, makes three references to their data members, and — depending on the command-line input — invokes three virtual functions.
pthread_create() function is used to force linking with another shared library. Any function from any shared library will serve this purpose.
The shared library and two instances of client executable are built as follows:
gcc shared g shm_server.cpp o libshm_server.so lstdc++
gcc -g shm_client.cpp -o shm_client1 -lpthread -lshm_server -L .
gcc -g shm_client.cpp -o shm_client2 -lshm_server -L . lpthread
Note that the link sequence of shm_server and pthread for shm_client1 and shm_client2 is swapped to ensure that the base address of the shm_server shared library in both executables is different. This can further be verified using the
ldd command. Sample output is usually something like the following:
The main aim here is to build two client binaries that have distinct base addresses for the server library. In the context of this sample program, using the uninvoked
pthread_create() function and the different link sequence for shared libraries works to accomplish this goal. However, there is no concrete rule or uniform procedure available that works across all linkers; different methods will need to be employed on a case-by-case basis.
Case 1: shm_client1 vs. shm_client1
In the following output, the shm_client1 is invoked first from the shell. Because no shared objects are present, it creates them, references their data members, invokes their virtual functions, and quits — leaving the objects behind in memory. The second time, the process simply references the data members and virtual functions.
When the second process tries to refer to data member
A::m_nA via the pointer of type C * (you will remember that C is virtually derived from A), the base sub-object pointer from within the shared object is read. The shared object was constructed in the context of a now non-existing process. Hence, garbage is read for both
This is because the Vee-Table and virtual functions lie within the shm_server shared library, which happens to be reloaded at the same virtual address. Hence, no problem is observed while de-referring pointers of type A * and B *.
Hence, the C++ object model adopted by GNU fails with virtual inheritance.
Case2: shm_client1 vs. shm_client2
In the next sample output, first the shm_client1 and then the shm_client2 are executed from the command line:
However, the Vee-Table lies within the shm_server shared library: it is loaded to different virtual addresses within shm_client1 and shm_client2. Hence, garbage is read for both
Making shared memory work
You should consider two major issues when instantiating C++ objects within shared memory. First, the Vee-Table pointer is used to access virtual functions, and data members are accessed directly using compile time offsets. Hence, for all such shared objects, Vee-Table and virtual functions should have the same virtual addresses in all processes. There is no hard-and-fast rule for doing this, but adopting a proper link sequence for dependent shared libraries should work most of the time.
In addition, keep in mind that virtually inherited objects have base pointers to address base objects. Base pointers refer to data sections of the process, and are always process specific. It is difficult to ensure the same numerical values for these across all client processes. Hence, construction of virtually inherited objects in shared memory should be avoided for the assumed C++ object model. But keep in mind, too, that different compilers adopt different models. For instance, Microsoft Compiler uses process-neutral offsets to address base objects for virtually inherited classes: thus, this issue does not arise. The important thing is to ensure the same addresses for shared libraries in all client-processes.
ld, ldd, ipcs, ipcrmand so on.