Improve collaborative build times with ccache
By Martin C. Brown2004-11-08
Using ccache
|
|
The ccache (short for compiler cache) tool caches the object files generated by a compilation. It uses a hash based on the output of CPP to provide a reliable method of determining whether the raw source before actual compilation has been changed. This method is more reliable than the timestamp system used by make. For example, using make without ccache on the source in Listing 2 would compile the file as normal.
Listing 2. Source file contents
#include "foobar.h"
void main(void)
{
}
If we edit one of the header files referenced by foobar.h, then standard make would definitely recompile the file in Listing 2, irrespective of the actual changes. With ccache, the output of the CPP version of the file would be checked with the hash of the previous compilation. If they matched, nothing would be done. If they didn't, it would recompile. It's as simple as that. On a single file, you are not going to see much advantage, but for over 20 or 30 or, certainly, hundreds, of files, the speed gains would be significant.
Installation
Installing ccache and using it is not as complicated as you might think. It doesn't replace or in any way affect the way you already use your compiler; instead, it acts as an interface between you and your compiler, so you can choose whether or not to use it according to your needs. To install ccache, download the source directly from the Samba group or a local mirror (see Resources at end of this article). Unpack the contents of the file:
$ bunzip2 -c ccache-2.3.tar.bz2|tar xf -
Change into the directory:
$ cd ccache-2.3
Configure:
$ ./configure
Build:
$ make
And finally install ccache:
$ make install
You're all ready to go!
Deployment
As mentioned above, ccache works by sitting between you and your normal compiler. Instead of calling gcc, you call ccache with gcc as the first argument. For example, to compile a file from the command line, you would normally use:
$ gcc foo.c
To use ccache you would type:
$ ccache gcc foo.c
On a single compilation of a file like this, especially if it is the first time the file has been compiled using ccache, you won't see any benefit, because the compilation information has not been cached yet. Therefore, it is generally more effective to configure ccache as a permanent replacement for your main compiler. To do this, set the value of the CC environment variable:
$ export set CC='ccache gcc'
If you want to enable ccache only on a project basis, say when compiling third-party tools such as Perl, then you can either use the environment trick or tell the configure script or make command what C compiler to use.
Controlling caches
By default, ccache uses a directory within the current user's home directory ($HOME/.ccache) to hold the cache information. In a team environment, you'll want to use a central location for the cache so everybody can use the cache information during builds. Another environment variable, CCACHE_DIR, specifies the location of the cache directory. In a single-machine environment, set this to a directory that is accessible to everybody who needs it. For a greater speed boost, and providing you have the memory to support it, use a directory mounted using tmpfs. You could get an additional speed boost of between 10 and 25 percent.
If you are using ccache across a network of machines, make sure the directory you share is exported over NFS and mounted on each client. Again, you can use a tmpfs filesystem if you want to get an extra speed boost.
Some additional options give you further control over the cache settings:
• The CCACHE_LOGFILE environment variable defines the location of a log file that will be populated as you use ccache.
• Use the -s command line option with ccache to get statistics about the cache performance (see Listing 3).
• Use the -M command line option to set the maximum size of the cache. The default is 1GB. The settings for the cache are written to the cache directory, so you can have different cache sizes in different locations for different users and groups.
• The -F option sets the maximum number of files, rounded to the nearest 16, for the cache directory. As with -M it should only need to be used when you want to change the configuration.
• The -c option cleans the cache. You shouldn't normally need to use this, as ccache updates the information during execution, but if you reuse a cache directory that has not been used for a file, you might want to try this option.
• The -C option completely clears the cache.
Listing 3. ccache cache statistics
cache hit 44
cache miss 152
called for link 107
compile failed 11
no input file 2
files in cache 304
cache size 8.8 MB
max cache size 976.6 MB
Once you've set the initial options and configured the directory and cache size you want, there's no need to change anything. There's also no need to perform any regular maintenance.
Tutorial Pages:
» How to squeeze more speed out of your compilations
» Using ccache
» Combining ccache and distcc
» Statistics
» Summary
» Resources
First published by IBM DeveloperWorks
|
| Related Tutorials: » How to Install PHP 5 on Linux » How to Install Apache 2 on Linux » How to Install MySQL 5.0 on Linux » SMB Caching » Mound --Bind » Tar Wild Card Interpretation |