Managing Perl libraries across nodes

From WormBaseWiki
Revision as of 16:06, 24 September 2007 by Tharris (talk | contribs) (New page: Standardizing modules across nodes ---- In order to standardize modules across nodes (and avoid the headache of having to update each node separately with new libraries), I am now includ...)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigationJump to search

Standardizing modules across nodes


In order to standardize modules across nodes (and avoid the headache of having to update each node separately with new libraries), I am now including required perl modules with the WormBase software. This includes modules for both different versions of Perl and different architectures.

Configuring the primary build environment

Brie3 acts as the primary build environment. At the end of this configuration, we will be left with a new directory at /usr/local/wormbase/extlib containing all necessray Perl modules for a WormBase installation. We can then automatically sync this to our production nodes.

The intial configuration of the system followed these steps:

Upgrade Perl to 5.8.8

sudo perl -MCPAN -e shell cpan> force install threads (this will update to the latest version of Perl)

Select the following options:

 enable threads [y]

You can have perl look in other directories for extensions and modules in addition to those already specified. Enter


 /usr/local/wormbase/extlib/lib/perl5/site_perl/5.8.8/i686-linux-thread-multi:\
/usr/local/wormbase/extlib/lib/perl5/site_perl/5.8.8:\
/usr/local/wormbase/extlib/lib/perl5/5.8.8

This will add the WormBase to @INC automatically. (It's probably better just to do this in perl.startup)

Select [n] (the default) when asked if you want to install at /usr/bin/perl.

Finally, fix the shebang line:


What shall I put after the #! to start up perl ("none" to not use #!)?
[/usr/local/bin/perl] /usr/bin/perl

Create a symlink to the new Perl

cd /usr/bin
sudo ln -s /usr/local/bin/perl5.8.8 perl

Adjust @INC

If you followed the configuration above, it shouldn't be necessary for you to adjust @INC. But just in case, you can add the WormBase extlib directory to your PERL5LIB environment variable.

export PERL5LIB=/usr/local/wormbase/extlib/lib/perl/5.8.8:/usr/local/wormbase/exlib/perl/site_perl/5.8.8

Configure CPAN

The first time you launch CPAN, you will be prompted to configure it. Specially, we need to configure CPAN to install modules in our special WormBase directory.

$ perl -MCPAN -e shell (note: we do NOT need to be a privileged user) cpan> o conf init (only necessary if not prompted)

For the Makefile.PL arguments, enter

PREFIX=/usr/local/wormbase/extlib INSTALLMAN1DIR=/usr/local/wormbase/extlib/man/man1 INSTALLMAN3DIR=/usr/local/wormbase/extlib/man/man3

TH - Note to self: It might be cleaner to just flatten this whole structure into a single directory using something like...


PREFIX=/usr/local/wormbase/extlib \
INSTALLPRIVLIB=...
INSTALLSCRIPT=...
INSTALLSITELIB=...
INSTALLBIN=...
INSTALLMAN1DIR=...
INSTALLMAN3DIR=...

Intiial module list

Some of the following commands may be redundant depending on module dependencies

Note: Some of these modules may be architecture-dependent. For the most part, most of the WormBase servers are i686. If you encounter problems with the included modules and you are working on a non i686 system, try installing these modules first. The site-specific module paths for your Perl will be searched first.


cpan> install Bundle::CPAN
cpan> install LWP
cpan> install CGI
cpan> install CGI::Session
cpan> install CGI::Cache
cpan> install Cache::Cache
cpan> install Cache::Filecache
cpan> install SVG
cpan> install GD       <------ architecture dependent!
cpan> install GD::SVG  (forced)
cpan> install Digest::MD5
cpan> install HTML::Parser
cpan> install HTML::TokeParser
cpan> install IO::Scalar
cpan> install IO::String
cpan> install Net::FTP
cpan> install SOAP::Lite
cpan> install Statistics::OLS
cpan> install Storable
cpan> install Text::Shellwords
cpan> install WeakRef
cpan> install XML::SAX
cpan> install SVG::Graph
cpan> intsall XML::Writer
cpan> install XML::Twig
cpan> install Test::Pod
cpan>install DBI
cpan>install DBD::mysql
cpan>install GTop    <------ arch dependent
cpan>install Apache::GTopLimit  <------- arch dependent

Install AcePerl

This can probably be done through CPAN but I prefer to install from source. This might be architecture dependent....

perl Makefile.PL `cat /usr/local/wormbase/conf/Makefile.params`

Rebuild Apache and mod_perl

Now we need to rebuild apache and mod_perl against the new version of Perl.


cd mod_perl-1.29
perl Makefile.PL DO_HTTPD=1 USE_APACI=1 EVERYTHING=1 \
               APACHE_PREFIX=/usr/local/apache \
               APACI_ARGS='--enable-shared=info --enable-shared=status \
                           --enable-shared=proxy --enable-module=rewrite'

make
sudo make install

Build BioPerl

Upgrade BioPerl:


perl Makefile.PL `cat /usr/local/wormbase/conf/Makefile.params`
make
make install

Build GBrowse

Rebuild Gbrowse placing libraries into our extlib/.


perl Makefile.PL LIB=/usr/local/wormbase/extlib/lib/perl5/site_perl/5.8.8 \
   CONF=/usr/local/wormbase/conf \
   HTDOCS=/usr/local/wormbase/html \
   CGIBIN=/usr/local/wormbase/cgi-perl/seq \
    DO_XS=1

Preparing Nodes

To prepare individual production nodes:

  1. [[Managing_Perl_libraries_across_nodes#Upgrade_Perl_to_5.8.8 Upgrade Perl to 5.8.8]
  2. [[Managing_Perl_libraries_across_nodes#Create_a_symlink_to_the_new_Perl Create a symlink to the new Perl]

It shouldn't be necessary to [[Managing_Perl_libraries_across_nodes#Adjust_.40INC adjust your @INC], but it might be useful. You might also want to append

/usr/local/wormbase/extlib/bin

to your path.

Synchronizing libraries

Library syncrhonization is handled by two scripts.

1. Copy /usr/local/wormbase/extlib to /usr/local/wormbase-production/extlib

This step is not automated to avoid accidentally pushing libraries to production nodes. Remember, wormbase-production is the rsync target for all of the nodes.

2. New libraries will automatically be synced onto the production nodes during the next update. If you want to push software/libraries onto the nodes before then, see [[Updating_software_on_production_nodes Updating software on production nodes].