Automate Perl Module Deployment

By Martin C. Brown
2005-04-27

Automating CPAN Across a Network

CPAN is an excellent tool, but the system has its limitations. One major drawback is that it's a single-system solution, which is fine if you want to manage your Perl installations on a single-computer basis. But if you have a set of Perl modules spread across your Web server farm or even across your lab on a range of platforms, trying to keep up to date across all the computers can be a time-consuming process -- even if you use the automation techniques I discussed above.

CPAN also consumes a lot of Internet bandwidth in that for each computer on which you use CPAN, CPAN expects to download a copy of the files from one of the central CPAN repositories. CPAN also relies on a configuration that has to be tailored to each computer: You can copy a configuration file, which resides in /perlinstalldirectory/CPAN/Config.pm, only if you can configure your platforms identically.

An obvious route to take when using CPAN is to use the autobundle function. This function generates a list of every installed module in a Perl installation, then dumps the list into a format that you can execute to re-install the selection either on the same computer (after an update to the latest Perl version) or onto another computer. The code in Listing 2 shows you how to create a new bundle.

Listing 2. Code to create a CPAN autobundle

$ perl -MCPAN -e autobundle

...



Package namespace         installed    latest  in CPAN file

AnyDBM_File                    1.00      1.00  N/NW/NWCLARK/perl-5.8.5.tar.gz

Apache::ASP                    2.57      2.57  C/CH/CHAMAS/Apache-ASP-2.57.tar.gz

Apache::ASP::ApacheCommon     undef     undef  C/CH/CHAMAS/Apache-ASP-2.57.tar.gz

Apache::ASP::Application      undef     undef  C/CH/CHAMAS/Apache-ASP-2.57.tar.gz

Apache::ASP::CGI              undef     undef  C/CH/CHAMAS/Apache-ASP-2.57.tar.gz

Apache::ASP::CGI::Table       undef     undef  C/CH/CHAMAS/Apache-ASP-2.57.tar.gz

Apache::ASP::CGI::Test        undef     undef  C/CH/CHAMAS/Apache-ASP-2.57.tar.gz

Apache::ASP::Collection       undef     undef  C/CH/CHAMAS/Apache-ASP-2.57.tar.gz

Apache::ASP::CollectionItem     undef     undef  C/CH/CHAMAS/Apache-ASP-2.57.tar.gz

Apache::ASP::Date             undef     undef  C/CH/CHAMAS/Apache-ASP-2.57.tar.gz

...





Wrote bundle file

	/export/build/solaris/cpan/Bundle/Snapshot_2004_08_08_00.pm

The file that is generated is created within your CPAN installation directory. You can copy it from that location or, if you're upgrading to a later version of Perl, you can simply run the bundle module to re-install everything:
$ perl -MCPAN -e 'install Bundle::Snapshot_2004_08_08_00'

Although the bundle function works fine, I have to admit that I'm not a huge fan of the bundling system as a network solution for a few reasons. First, bundling relies on having an identical configuration on each computer, which is fine if everything is identical. But if you have different platforms and environments, you can easily run into problems. Second, because the autobundle feature lists every module installed, it lists multiple packages (individual packages may contain multiple modules). However, the feature can also list references to later versions of Perl (as the source for some module updates).

The final problem, however, is that bundling relies on having a single computer responsible for containing the latest version of all the modules. You can't use the feature to upgrade a standard suite up to the latest versions -- whatever they may be -- and this stipulation makes the system complex to use in a network environment.

if you want to build a custom solution, two are available. You can use a central CPAN installation directory, or you can use a central CPAN source directory.

Use a central CPAN installation directory
A relatively straightforward way of configuring CPAN across a network -- and one that works well if all your computers are identical -- is to create a single CPAN directory that is shared over Network File System (NFS), copy the CPAN/Config.pm file onto each computer, then use the CPAN::Shell techniques discussed earlier to install the modules on each computer. You install the "base" module set on the main computer (usually the one sharing the CPAN directory over NFS). For every other computer, the process of building and installing each required module should be quick, because the source tarballs already exist. Therefore, you don't need to download the tarball again, saving precious Internet bandwidth and significant time.

The main problem with this technique is that for computers that are in all other ways identical, it's wasteful on processor time: You've already compiled it once, and it's wasteful to compile it all again. You also run into a dangerous situation in which the same build directory is being used for multiple computers, which really isn't a good idea because different computers could be compiling the source at the same time, causing a combination of timing errors and, in a cross-platform environment, bad object files when linking.

Use a central CPAN source directory
An adaptation of the previous solution is to use the same source directory and basic CPAN configuration but manually change the configuration file so that the bulk of the configuration information for each host is host-specific. In this way, you solve the build directory problem but still use a local (that is, on your network) cached version of the tarballs. Listing 3 provides such a configuration file.

Listing 3. Example of a CPAN::Config file

# This is CPAN.pm's systemwide configuration file. This file provides

# defaults for users, and the values can be changed in a per-user

# configuration file. The user-config file is being looked for as

# ~/.cpan/CPAN/MyConfig.pm.



$CPAN::Config = {

  'build_cache' => q[200],

  'build_dir' => q[/export/build/solaris/cpan/build],

  'cache_metadata' => q[1],

  'cpan_home' => q[/export/build/solaris/cpan],

  'dontload_hash' => {  },

  'ftp' => q[/usr/bin/ftp],

  'ftp_proxy' => q[proxy.mcslp.pri:8080],

  'getcwd' => q[cwd],

  'gpg' => q[],

  'gzip' => q[/usr/bin/gzip],

  'histfile' => q[/export/build/solaris/cpan/histfile],

  'histsize' => q[100],

  'http_proxy' => q[proxy.mcslp.pri:8080],

  'inactivity_timeout' => q[0],

  'index_expire' => q[1],

  'inhibit_startup_message' => q[0],

  'keep_source_where' => q[/export/build/solaris/cpan/sources],

  'lynx' => q[],

  'make' => q[/usr/local/bin/make],

  'make_arg' => q[-j3],

  'make_install_arg' => q[UNINST=1],

  'makepl_arg' => q[],

  'ncftp' => q[],

  'ncftpget' => q[],

  'no_proxy' => q[],

  'pager' => q[/usr/bin/less],

  'prerequisites_policy' => q[follow],

  'scan_cache' => q[atstart],

  'shell' => q[/usr/local/bin/bash],

  'tar' => q[/usr/local/bin/tar],

  'term_is_latin' => q[1],

  'unzip' => q[/usr/bin/unzip],

  'urllist' => [q[ftp://cpan.etla.org/pub/CPAN], q[ftp://cpan.teleglobe.net/pub/CPAN],

    q[ftp://ftp.demon.co.uk/pub/CPAN/], q[ftp://ftp.flirble.org/pub/languages/perl/CPAN/],

    q[ftp://ftp.mirror.ac.uk/sites/ftp.funet.fi/pub/languages/perl/CPAN/],

    q[ftp://ftp.mirror.anlx.net/CPAN/], q[ftp://ftp.plig.org/pub/CPAN/],

    q[ftp://usit.shef.ac.uk/pub/packages/CPAN/], q[http://cpan.hambule.co.uk/]],

  'wget' => q[],

};

1;

			__END__

You'll need to change all the configuration directory parameters to something local on each computer -- for example, change the line 'build_dir' => q[/export/build/solaris/cpan/build], to point to a directory on the local machine. Meanwhile, share the CPAN source directory on your main computer, then change the keep_source_where parameter to point to this NFS-mounted directory.

Unfortunately, a few issues remain with this solution. First, you have no way of conveniently recording which modules you want to install on each computer. You also run into the problem of keeping the information in the source directory (and the rest of the CPAN directories) correct and current without updating or downloading a newer version of the tarball and upsetting the status quo among computers. This issue can lead to minor but nearly fatal differences between the installed versions of the modules on each system, which is the key issue that we are trying to resolve.

Tutorial Pages:
» Scripts go Beyond CPAN to Ease Network Installations
» Perl Module Installation
» Automating CPAN
» Automating CPAN Across a Network
» Use CPAN to Create a Rigid Installation
» Conclusions
» Resources

First published by IBM DeveloperWorks

Related Tutorials:
» Random subroutines in Perl
» Log Script Use
» Creating Perl Modules for Web Sites
» Bit Vector, Using Perl Vec
» Build a Perl/CGI Voting System
» Perl Range Operator