Managing Perl Libraries

Overview

Maintaining Perl modules across our server farm can be a time consuming and tedious task because of the heterogeneity of our setup. Further, we often need to maintain unique sets of libraries for different applications with collisions.

To solve these problems, I maintain local copies of libraries for each application. Typically, these are found at

 /path/to/application/extlib

Installing modules in a local path is made significantly easier through the use of local::lib and Apache2/mod_perl2 "PerlSwitches" directive.

local::lib dispenses with the need to maintain multiple CPAN configurations or manual tweaks to your environment variables in order to recognize already installed modules.

Installing a local CPAN repository

I maintain a local mirror of CPAN. This makes installing our web application significantly faster.

$ sudo perl -MCPAN -e shell
$ cpan> install CPAN::Mini

Create a minicpanrc file at ~/

# CPAN::Mini configuration file
# to set up our local WormBase CPAN

local: /usr/local/wormbase/minicpan
remote: http://mirror.rit.edu/CPAN/
exact_mirror: 1

Set up your CPAN to use the local mirror:

$ cpan
cpan> o conf set urllist file:///usr/local/wormbase/minicpan/
cpan> o conf commit

Set up the minicpan to run once a day:

• 3 * * * minicpan

Required Perl modules

See the Makefile.PL for a complete list of required Perl modules.

Installing Modules

Location

By convention, 3rd party Perl modules are maintained in an "extlib" directory within each version of the site. This enables us to keep unique sets of libraries for each website.

website/production/extlib - The current production version of the website
website/tharris/extlib - My private development space

To set up your environment, copy the wormbase.env.template file to wormbase.env. Change the $APP variable to match your directory (ie tharris'). Then:

$ source wormbase.env

local::lib

The wormbase.env file uses local::lib to easily set up a local library path. local::lib is already installed into the system Perl path:

 sudo perl -MCPAN -e install 'local::lib'

To set a local installation environment, you would:

$ cd /usr/local/wormbase/website/tharris/extlib
$ perl -Mlocal::lib=./
export MODULEBUILDRC="/usr/local/wormbase/website/tharris/extlib/.modulebuildrc"
export PERL_MM_OPT="INSTALL_BASE=/usr/local/wormbase/website/tharris/extlib"
export PERL5LIB="/usr/local/wormbase/website/tharris/extlib/lib/perl5:/usr/local/wormbase/website/tharris/extlib/lib/perl5/x86_64-linux-gnu-thread-multi:$PERL5LIB"
export PATH="/usr/local/wormbase/website/tharris/extlib/bin:$PATH"

$ eval $(perl -Mlocal::lib=--self-contained,./)

To install a module:

$ perl -MCPAN -Mlocal::lib=--self-contained -e 'CPAN::install(MODULE)'

Installing a module without using local::lib

You may on occasion need to install something without using local::lib. For EUMM modules, use

 perl -I/usr/local/wormbase/[PROJECT]/extlib Makefile.PL INSTALL_BASE=/usr/local/wormbase/[PROECT]/extlib

And for Module::Build

 perl -I/usr/local/wormbase/[PROJECT]/extlib Makefile.PL --install_base /usr/local/wormbase/[PROJECT]/extlib 

Alternatively, you can manually edit ~/.cpan/CPAN/MyConfig.pm

Tips and Tricks

Some modules may require a little TLC. After running the script you can find source at ~/.cpan/build/module-*

BioPerl

Instead of building BioPerl from CPAN, it might be necessary to build from SVN:

 $ cd /usr/local/wormbase/build
 $ svn co svn+ssh://USERNAME@dev.open-bio.org/home/svn-repositories/bioperl/bioperl-live/trunk bioperl-live
 $ mv bioperl-live bioperl-live-tharris
 $ cd bioperl-live-tharris
 $ perl ./Build.PL install_base /usr/local/wormbase/extlib
 $ ./Build test
 $ ./Build install

DBD::mysql

You will need to specify a test user and test password for tests to pass.

ModPerl2

You will need to be sudo in order to install the dso.

Updating Modules

To show all modules that are out-of-date with those on CPAN:

$ cd /path/to/extlib
$ perl -Mlocal::lib=./
$ eval $(perl -Mlocal::lib=--self-contained,./)
$ perl -MCPAN -e 'CPAN::Shell->r'

To update all of these modules at once:

$ perl -MCPAN -e 'CPAN::Shell->install(CPAN::Shell->r)'

Managing libraries across architectures

Via CPAN bundles

# Create a bundle
$ cd /path/to/extlib
$ perl -Mlocal::lib=./
$ eval $(perl -Mlocal::lib=--self-contained,./)
$ perl -MCPAN -e autobundle   # The bundle is (typically) in extlib

# To reinstall:
$ perl -MCPAN -e 'install Bundle::Snapshot_YYYY_MM_DD_00'

(Or just use the install script).