Difference between revisions of "Managing Perl Libraries"

From WormBaseWiki
Jump to navigationJump to search
Line 136: Line 136:
 
  XML::Twig
 
  XML::Twig
 
  XML::Simple
 
  XML::Simple
 
== WormBase > 2.0 ==
 
 
WormBase2.0 and above uses the Catalyst web framework.  In addition to the modules described above we also need to install Catalyst.
 
 
These first two bundles will install the majority of the Catalyst web framework:
 
 
Task::Catalyst
 
Catalyst::Devel
 
 
  We also need:
 
 
Catalyst::Model::DBI
 
Catalyst::Model::Adaptor
 
Catalyst::Plugin::ConfigLoader
 
Catalyst::Plugin::Session
 
Catalyst::Plugin::Session::State
 
Catalyst::Plugin::Session::State::Cookie
 
Catalyst::Plugin::Session::Store::FastMMap
 
Catalyst::Action::RenderView
 
Catalyst::Action::Rest
 
Catalyst::Log::Log4perl
 
Test::YAML::Valid
 
  
 
= local::lib =
 
= local::lib =

Revision as of 16:36, 7 January 2011

Overview

Maintaining Perl modules across our server farm can be a time consuming and tedious task because of the heterogeneity of our setup. Further, we often need to maintain unique sets of libraries for different applications with collisions.

To solve these problems, I maintain local copies of libraries for each application. Typically, these are found at

 /path/to/application/extlib

Installing modules in a local path is made significantly easier through the use of local::lib and Apache2/mod_perl2 "PerlSwitches" directive.

local::lib dispenses with the need to maintain multiple CPAN configurations or manual tweaks to your environment variables in order to recognize already installed modules.

Installing a local CPAN repository

I maintain a local mirror of CPAN. This makes installing our web application significantly faster.

$ sudo perl -MCPAN -e shell
$ cpan> install CPAN::Mini

Create a minicpanrc file at ~/

# CPAN::Mini configuration file
# to set up our local WormBase CPAN

local: /usr/local/wormbase/minicpan
remote: http://mirror.rit.edu/CPAN/
exact_mirror: 1

Set up your CPAN to use the local mirror:

$ cpan
cpan> o conf set urllist file:///usr/local/wormbase/minicpan/
cpan> o conf commit

Set up the minicpan to run once a day:

  • 3 * * * minicpan

Private library locations

By convention, 3rd party Perl modules are maintained in an "extlib" directory within each project directory:

 website-classic/extlib - The "Classic" WormBase website
 website/extlib - 'The rearchitecture
 website-2.01/extlib - Version 2.01 of the rearchitecture

And so on...


Required Perl modules

WormBase "Classic"

Install the following Perl modules via CPAN. Note that you DO NOT AND SHOULD NOT be sudo.

cd /usr/local/wormbase/extlib/classic
perl -Mlocal::lib=./
eval $(perl -Mlocal::lib=--self-contained,./)
perl -MCPAN -e 'CPAN::install(MODULE)'
// Some basics...
YAML
LWP
ExtUtils::MakeMaker
Bundle::CPAN

// The rest...
Bio::Graphics
BioPerl   
   // When prompted, choose to download all optional modules. This will also install Ace:
Ace
   // During configuration, choose option (3), then set the remaining variables as follows:
   Site-specific configuration files:  /usr/local/wormbase/website-classic/conf
   CGI path:  /usr/local/wormbase/website-classic/cgi-bin
   HTML path:  /usr/local/wormbase/website-classic/html
   // For Catalyst-based apps:
   Site-specific configuration files:  /usr/local/wormbase/website/conf
   CGI path:  /usr/local/wormbase/website/root/cgi
   HTML path:  /usr/local/wormbase/website/root/static
Cache::Cache
Cache::FileCache
CGI
CGI::Session  
CGI::Cache
CGI::Toggle
Class::Base
Data::Stag
Date::Calc
Date::Manip
DB_File
DBI
DBD::mysql   (mysql devel package must be installed first)
    // To build by hand:
    perl Makefile.PL INSTALL_BASE=/usr/local/wormbase/extlib/classic --testuser=USER --testpassword=PASS
    make ; make test ; make install
Digest::MD5
Flickr::API
Flickr::API::Simple
    // This is a private library
    cd /usr/local/wormbase/build
    perl ./Build.PL --install_base /usr/local/wormbase/extlib/classic
   ./Build install
GD 
GD::SVG
GD::Graph
GD::Graph::pie
HTML::TokeParser
IO::Scalar
IO::String
Image::GD::Thumbnail
Log::Log4perl
MIME::Lite
mod_perl2   // Requires that apache2 already be installed; apxs is at /usr/local/apache2/bin/apxs
Net::FTP
Proc::Simple
Search::Indexer
SOAP::Lite
Statistics::OLS
Storable
SVG
SVG::Graph
Term::ReadKey
Test::Pod
Text::Shellwords
Time::Format
WeakRef
XML::SAX
XML::Parser
XML::DOM
XML::Writer
XML::Twig
XML::Simple

local::lib

Installation

local::lib is installed into the system Perl path:

 sudo perl -MCPAN -e install 'local::lib'

Setting a library path

$ cd /usr/local/wormbase/extlib/classic 
$ perl -Mlocal::lib=./
export MODULEBUILDRC="/usr/local/wormbase/extlib/classic/.modulebuildrc"
export PERL_MM_OPT="INSTALL_BASE=/usr/local/wormbase/extlib/classic"
export PERL5LIB="/usr/local/wormbase/extlib/classic/lib/perl5:/usr/local/wormbase/extlib/classic/lib/perl5/x86_64-linux-gnu-thread-multi:$PERL5LIB"
export PATH="/usr/local/wormbase/extlib/classic/bin:$PATH"

Take note: we will use some of these variables for, say, setting up apache.

$ eval $(perl -Mlocal::lib=--self-contained,./)

Installing a module

$ perl -MCPAN -Mlocal::lib=--self-contained -e 'CPAN::install(MODULE)'

Installing a module without using local::lib

You may on occasion need to install something without using local::lib. For EUMM modules, use

 perl -I/usr/local/wormbase/[PROJECT]/extlib Makefile.PL INSTALL_BASE=/usr/local/wormbase/[PROECT]/extlib

And for Module::Build

 perl -I/usr/local/wormbase/[PROJECT]/extlib Makefile.PL --install_base /usr/local/wormbase/[PROJECT]/extlib 

Alternatively, you can manually edit ~/.cpan/CPAN/MyConfig.pm

Tips and Tricks

Some modules may require a little TLC. After running the script you can find source at ~/.cpan/build/module-*

BioPerl

Instead of building BioPerl from CPAN, it might be necessary to build from SVN:

 $ cd /usr/local/wormbase/build
 $ svn co svn+ssh://USERNAME@dev.open-bio.org/home/svn-repositories/bioperl/bioperl-live/trunk bioperl-live
 $ mv bioperl-live bioperl-live-tharris
 $ cd bioperl-live-tharris
 $ perl ./Build.PL install_base /usr/local/wormbase/extlib
 $ ./Build test
 $ ./Build install

DBD::mysql

You will need to specify a test user and test password for tests to pass.

ModPerl2

You will need to be sudo in order to install the dso.

Updating Modules

To show all modules that are out-of-date with those on CPAN:

$ cd /path/to/extlib
$ perl -Mlocal::lib=./
$ eval $(perl -Mlocal::lib=--self-contained,./)
$ perl -MCPAN -e 'CPAN::Shell->r'

To update all of these modules at once:

$ perl -MCPAN -e 'CPAN::Shell->install(CPAN::Shell->r)'

Managing libraries across architectures

Via CPAN bundles

# Create a bundle
$ cd /path/to/extlib
$ perl -Mlocal::lib=./
$ eval $(perl -Mlocal::lib=--self-contained,./)
$ perl -MCPAN -e autobundle   # The bundle is (typically) in extlib
# To reinstall:
$ perl -MCPAN -e 'install Bundle::Snapshot_YYYY_MM_DD_00'

(Or just use the install script).