Difference between revisions of "Wormmart administration"
(7 intermediate revisions by the same user not shown) | |||
Line 263: | Line 263: | ||
location=biomart | location=biomart | ||
</pre> | </pre> | ||
+ | |||
+ | == Reinstalling BioMart == | ||
+ | |||
+ | So, sometimes wormmart gets a redirect problem that you can fix by reinstalling! | ||
+ | |||
+ | 1. reinstall biomart-perl | ||
+ | cd /usr/local/wormbase/biomart/ | ||
+ | cvs -d :pserver:cvsuser@cvs.sanger.ac.uk:/cvsroot/biomart login | ||
+ | The password you need to enter when prompted is CVSUSER. | ||
+ | cvs -d :pserver:cvsuser@cvs.sanger.ac.uk:/cvsroot/biomart \ | ||
+ | co -r release-0_7 biomart-perl | ||
+ | |||
+ | Note: you may not be able to connect to cvs from wb-biomart. If so, checkout code somewhere else and move over to /usr/local/womrbase/biomart/biomart-perl | ||
+ | |||
+ | 2. copy over necessary files | ||
+ | * conf/httpd.conf.bak | ||
+ | * conf/templates/default | ||
+ | * conf/settings.conf | ||
+ | |||
+ | 3. Change the permissions for conf/templates/cached | ||
+ | $ sudo chmod 777 conf/templates/cached | ||
+ | |||
+ | 3. Make sure you kill all previous wormmart processes | ||
+ | |||
+ | 4. Run the following from the biomart-perl directory: | ||
+ | $ sudo perl bin/configure.pl -r conf/wormMartRegistry.xml --clean; | ||
+ | $ cp conf/httpd.conf.bak conf/httpd.conf; | ||
+ | $ sudo /usr/local/apache2/bin/apachectl -d /path/to/biomart-perl -f conf/httpd.conf |
Latest revision as of 14:39, 4 November 2010
Contents
- 1 Overview
- 2 Basic Server Configuration
- 2.1 Create groups
- 2.2 Create directories
- 2.3 Install prerequisites (Debian)
- 2.4 Install Perl 5.10.1 or greater
- 2.5 Install Apache2
- 2.6 Install mod_perl
- 2.7 Downloading martj
- 2.8 Downloading biomart-perl
- 2.9 Install Requisite Perl Modules
- 2.10 MySQL
- 2.11 Berkeley DB
- 2.12 2.4.3 Configuring
- 2.13 Reinstalling BioMart
Overview
WormMart is WormBase's implementation of the BioMart data warehouse.
Official BioMart Documentation
Basic Server Configuration
Create groups
sudo addgroup wormbase sudo addgroup biomart # Add yourself to the groups sudo usermod -a -G biomart, wormbase acabunoc
Create directories
Create a directory for GBrowse. For now, we're maintaining GBrowse on dedicated hardware completely distinct from the rest of the web application. This makes updates significantly easier, although it increases maintenance (ie boilerplate headers and footers are maintained in two places). We'll also keep gbrowse inside a wormbase/ directory should we wish to install other components on these servers at a later date.
$ sudo mkdir -p /usr/local/wormbase $ sudo chgrp -R wormbase /usr/local/wormbase $ sudo chmod 2775 /usr/local/wormbase
Install prerequisites (Debian)
// Update apt $ sudo apt-get update
// Make tools $ sudo apt-get install gcc make
// expat $ sudo apt-get install expat libexpat1 libexpat1-dev
// GD support $ sudo apt-get install libgd2-noxpm libgd2-noxpm-dev
// Graphviz, now apparently required by BioPerl $ sudo apt-get install graphviz
// Curl $ sudo apt-get install curl
//CVS $ sudo apt-get install cvs
//Java $ sudo apt-get install sun-java5-jre
Install Perl 5.10.1 or greater
You'll need Perl version 5.10.1 or greater.
Debian:
$ sudo apt-get install perl
NOTE: This version of Perl on Debian Lenny doesn't ship with perl.h. You won't be able to build mod_perl without it! You should probably build from source.
Building from source:
bash> cd ~/src bash> curl -O http://www.cpan.org/src/perl-5.10.1.tar.gz bash> cd ~/build bash> tar xzf ../src/perl-5.10.1.tar.gz bash> cd perl-5.10.1 bash> ./Configure -des // Or, to install in a local path: bash> ./Configure -des -Dprefix=$HOME/website/perl/5.10.1 // if mod_perl doesn't work properly, I had to install like this: bash> CFLAGS='-m64 -mtune=nocona' ./Configure -des -A ccflags=-fPIC bash> make bash> make test bash> sudo make install
Install Apache2
If you're running Debian, get rid of the installed apache. The default layout is just plain annoying.
$ sudo apt-get remove apache2
# Build httpd 2.2.15 cd ~/src curl -O http://apache.raffsoftware.com/httpd/httpd-2.2.15.tar.gz tar xzf httpd-2.2.15.tar.gz cd httpd-2.2.15 ./configure --enable-mods-shared=all make sudo make install
Apache Configuration
Edit the primary httpd.conf file (/usr/local/apache2/conf/httpd.conf) with the appropriate port, virtual host, and fcgi settings:
- Set httpd to listen on your desired port. In the examples below, we assume port 8080.
# Edit "Listen 80" to read Listen 8080
Install mod_perl
cd ~/src wget http://perl.apache.org/dist/mod_perl-2.0-current.tar.gz tar zxvf mod_perl-2.0-current.tar.gz cd mod_perl-2.0.4 perl Makefile.PL \ PREFIX=/usr/local/apache2 \ MP_APXS=/usr/local/apache2/bin/apxs
Add to httpd.conf:
LoadModule perl_module modules/mod_perl.so
Then,
sudo make sudo make install
Downloading martj
Download: ftp://anonymous@ftp.ebi.ac.uk/pub/software/biomart/martj_current/martj-bin.tgz
tar -zxvf martj-bin.tgz
Downloading biomart-perl
cd /usr/local/wormbase/biomart/ cvs -d :pserver:cvsuser@cvs.sanger.ac.uk:/cvsroot/biomart login
The password you need to enter when prompted is CVSUSER.
cvs -d :pserver:cvsuser@cvs.sanger.ac.uk:/cvsroot/biomart \ co -r release-0_7 biomart-perl
Note: you may not be able to connect to cvs from wb-blast. If so, checkout code somewhere else and move over to /usr/local/womrbase/biomart/biomart-perl
Install Requisite Perl Modules
cd /usr/local/wormbase/biomart/biomart-perl perl bin/configure.pl -r conf/registryURLPointer.xml
my output:
Spreadsheet::WriteExcel Template::Constants Log::Log4perl Test::Exception Mail::Mailer DBI Template::Plugin::Number::Format XML::Parser List::MoreUtils Exception::Class CGI::Session::Driver::db_file LWP::UserAgent Spreadsheet::WriteExcel::Big Number::Format CGI::Session XML::DOM Template OLE::Storage_Lite Readonly
install the perl modules you are instructed to by this script
cpan -i Module::Name
MySQL
Installation (>= 5.1)
We need a newer version of mysql than is available from the package manager. We'lll install the binary for x86_64.
$ sudo groupadd mysql $ sudo useradd -g mysql mysql $ sudo usermod -a -G mysql tharris $ cd /usr/local $ sudo gunzip < /path/to/mysql-VERSION-OS.tar.gz | sudo tar xvf - $ sudo ln -s full-path-to-mysql-VERSION-OS mysql $ cd mysql $ sudo chown -R mysql . $ sudo chgrp -R mysql . $ sudo scripts/mysql_install_db --user=mysql // may be necessary to remove /etc/mysql/my.cnf first $ sudo chown -R root . $ sudo chown -R mysql data $ sudo chmod 2775 data // I like to be able to write to my datadir
// Copy the configuration file $ sudo cp support_files/my-medium.cnf /etc/my.cnf
// It may be necessary to add the following to my.cnf if back end nodes // don't have fully qualified domain names skip-name-resolve
// Set up mysql to start up automatically $ sudo cp support_files/mysql.server /etc/init.d/. $ cd /etc/rc3.d $ sudo ln -s ../init.d/mysql.server S90mysql $ cd ../rc5.d $ sudo ln -s ../init.d/mysql.server S90mysql
// Start it up and set a root password. $ sudo /etc/init.d/mysql.server start $ mysqladmin -u root password 'PASSWORD'
Databases
MySQL databases will be mirrored over directly as part of the production update process. They should be located in /usr/local/mysql/data. Symlinks, using the symbolic name of the database, point to the current version:
c_elegans -> c_elegans_WS211
Privileges
$ mysql -uroot -p $ mysql> create use mart_admin $ mysql> grant select on `wormmart_%`.* to 'mart_admin'@'localhost'
Berkeley DB
Install Berkeley DB
save to /usr/local/src http://www.oracle.com/technology/software/products/berkeley-db/htdocs/popup/db/5.0.21/db-targz.html
cd /usr/local/src/ sudo gunzip < /path/to/berkleydb.tar.gz | sudo tar xvf - cd build_unix ../dist/configure make make install cd ../.. echo /usr/local/BerkeleyDB.3.2/lib >> /etc/ld.so.conf ldconfig
Then install the following perl modules:
DB_File XML::Simple DBD::mysql <-- may have to install from source Apache2::PerlSections
2.4.3 Configuring
2.4.3.1 Configuring the BioMart Perl API
Configuration of the Perl API requires a single step. Change into the biomart-perl directory, then type:
perl bin/configure.pl -r conf/registryURLPointer.xml
where registryURLPointer.xml is the registry file you wish to use from the conf folder. The first question the configure script will ask is:
Do you want to install in API only mode [y/n] [n]:
Type y to install the API only. During configuration it may point out that required Perl modules are missing. If this happens, follow the steps detailed in the prerequisites section above to install these missing Perl modules. When it has completed successfully, you will see this final message:
Looks good.... you are done.
2.4.3.2 Configuring MartView
in conf/settings.conf
[httpdSettings] apacheBinary=/usr/local/apache2 serverHost=localhost port=9002 # to use proxying set the proxy port below (even if its 80) and set serverHost as your proxy host proxy= # if apxs or apxs2 is not in default directory where httpd lives, you may set this param to point to apxs/2 apxs= location=biomart
Reinstalling BioMart
So, sometimes wormmart gets a redirect problem that you can fix by reinstalling!
1. reinstall biomart-perl
cd /usr/local/wormbase/biomart/ cvs -d :pserver:cvsuser@cvs.sanger.ac.uk:/cvsroot/biomart login
The password you need to enter when prompted is CVSUSER.
cvs -d :pserver:cvsuser@cvs.sanger.ac.uk:/cvsroot/biomart \ co -r release-0_7 biomart-perl
Note: you may not be able to connect to cvs from wb-biomart. If so, checkout code somewhere else and move over to /usr/local/womrbase/biomart/biomart-perl
2. copy over necessary files
- conf/httpd.conf.bak
- conf/templates/default
- conf/settings.conf
3. Change the permissions for conf/templates/cached
$ sudo chmod 777 conf/templates/cached
3. Make sure you kill all previous wormmart processes
4. Run the following from the biomart-perl directory:
$ sudo perl bin/configure.pl -r conf/wormMartRegistry.xml --clean; $ cp conf/httpd.conf.bak conf/httpd.conf; $ sudo /usr/local/apache2/bin/apachectl -d /path/to/biomart-perl -f conf/httpd.conf