Difference between revisions of "GBrowse Administration"
Line 1: | Line 1: | ||
− | GBrowse is one of the central -- and most heavily used -- components of the WormBase site. To meet this demand, we use a number of dedicated servers using GBrowse's built in master/slave configuration. This document describes how to build these servers | + | = Overview = |
+ | |||
+ | GBrowse is one of the central -- and most heavily used -- components of the WormBase site. To meet this demand, we use a number of dedicated servers using GBrowse's built in master/slave configuration. This document describes how to build and configure these servers. | ||
= Hardware Topology = | = Hardware Topology = | ||
+ | The GBrowse topology looks like this: | ||
+ | |||
+ | Master | ||
+ | ------------- ----------------------------- | ||
+ | | gb1 | \ \ | ||
+ | | | ------------- ------------- | ||
+ | |* * * * * *| <----- | gb2 | | gb2 | Slaves | ||
+ | ------------- ------------- ------------- | ||
+ | |||
+ | Where: | ||
+ | * * * * * = relational databases running under mysql on gb1. | ||
+ | |||
= Node Configuration = | = Node Configuration = | ||
+ | |||
+ | == Create groups == | ||
+ | |||
+ | sudo addgroup wormbase | ||
+ | sudo addgroup gbrowse | ||
+ | # Add yourself to the groups | ||
+ | sudo usermod -a -G gbrowse, wormbase tharris | ||
== Create Directories == | == Create Directories == | ||
− | ''Create a directory for GBrowse. For now, we're maintaining GBrowse distinct from the rest of the web application. This makes updates significantly easier.'' | + | ''Create a directory for GBrowse. For now, we're maintaining GBrowse on dedicated hardware completely distinct from the rest of the web application. This makes updates significantly easier, although it increases maintenance (ie of boilerplate). We'll also keep gbrowse inside a wormbase/ directory should we wish to install other components on these servers at a later date.'' |
Let $ROOT = /usr/local/wormbase | Let $ROOT = /usr/local/wormbase | ||
− | $ sudo mkdir -p ${ROOT} | + | $ sudo mkdir -p ${ROOT} |
$ sudo chgrp -R wormbase $ROOT | $ sudo chgrp -R wormbase $ROOT | ||
$ sudo chmod 2775 $ROOT | $ sudo chmod 2775 $ROOT | ||
+ | |||
+ | == Debian preparation == | ||
+ | |||
+ | $ sudo apt-get update | ||
+ | $ sudo apt-get install gcc | ||
== Install Perl 5.10.1 or greater == | == Install Perl 5.10.1 or greater == | ||
− | You'll need Perl version 5.10.1 or greater. | + | ''You'll need Perl version 5.10.1 or greater.'' |
− | + | Debian: | |
+ | $ sudo apt-get install perl | ||
+ | |||
+ | Or from source: | ||
bash> mkdir ~/website/perl | bash> mkdir ~/website/perl | ||
Line 27: | Line 56: | ||
bash> tar xzf ../src.perl-5.10.1.tar.gz | bash> tar xzf ../src.perl-5.10.1.tar.gz | ||
bash> cd perl-5.10.1 | bash> cd perl-5.10.1 | ||
+ | bash> ./Configure -des | ||
+ | // Or, to install in a local path: | ||
bash> ./Configure -des -Dprefix=$HOME/website/perl/5.10.1 | bash> ./Configure -des -Dprefix=$HOME/website/perl/5.10.1 | ||
bash> make | bash> make | ||
Line 37: | Line 68: | ||
== Install Apache == | == Install Apache == | ||
+ | |||
+ | Debian (including it's annoying directory layout): | ||
+ | $ sudo apt-get install apache2 | ||
+ | |||
+ | Or from source: | ||
== Install mod perl == | == Install mod perl == | ||
− | == Install MySQL == | + | == Install MySQL > 5.1 == |
+ | |||
+ | ''We need a newer version of mysql than is available from the package manager. We'lll install the binary for x86_64.'' | ||
+ | |||
+ | ''For now, only the master server will run mysql. In the future, each dedicated slave could as well in order to meet growing demand.'' | ||
+ | |||
+ | $ groupadd mysql | ||
+ | $ useradd -g mysql mysql | ||
+ | $ usermod -a -G mysql tharris | ||
+ | $ cd /usr/local | ||
+ | $ sudo gunzip < /path/to/mysql-VERSION-OS.tar.gz | sudo tar xvf - | ||
+ | $ sudo ln -s full-path-to-mysql-VERSION-OS mysql | ||
+ | $ cd mysql | ||
+ | $ sudo chown -R mysql . | ||
+ | $ sudo chgrp -R mysql . | ||
+ | $ sudo scripts/mysql_install_db --user=mysql // may be necessary to remove /etc/mysql/my.cnf first | ||
+ | $ sudo chown -R root . | ||
+ | $ sudo chown -R mysql data | ||
+ | $ sudo chmod 2775 data // I like to be able to write to my datadir | ||
+ | |||
+ | // Copy the configuration file | ||
+ | $ sudo cp support_files/my-medium.cnf /etc/my.cnf | ||
+ | |||
+ | // Set up mysql to start up automatically | ||
+ | $ sudo cp support_files/mysql.server /etc/init.d/. | ||
+ | $ cd /etc/rc3.d | ||
+ | $ sudo ln -s ../init.d/mysql.server S90mysql | ||
+ | $ cd ../rc5.d | ||
+ | $ sudo ln -s ../init.d/mysql.server S90mysql | ||
+ | |||
+ | // Start it up and set a root password. | ||
+ | $ sudo /etc/init.d/mysql.server start | ||
+ | $ mysqladmin -u root password 'PASSWORD' | ||
== Install Requisite Perl Modules == | == Install Requisite Perl Modules == | ||
+ | |||
+ | All libraries specific for GBrowse are maintained in a local directory, specific to the version of gbrowse being run. This means that prerequisites need to be installed for each version of Gbrowse. | ||
+ | |||
+ | $ cd /usr/local/wormbase | ||
+ | $ mkdir gbrowse-2.01 // eg | ||
+ | $ ln -s gbrowse-2.01 gbrowse-current | ||
+ | $ mkdir gbrowse-current/extlib | ||
+ | |||
+ | Install local::lib to make things easier: | ||
+ | |||
+ | $ sudo perl -MCPAN -e 'CPAN::install(local::lib)' | ||
+ | $ cd /usr/local/wormbase/gbrowse-current/extlib | ||
+ | |||
+ | To install modules to this path: | ||
+ | |||
+ | $ perl -Mlocal::lib=. | ||
+ | $ eval $(perl -Mlocal::lib=--self-contained,./) | ||
+ | $ sudo rm -rf ~/.cpan | ||
+ | $ perl -MCPAN -e shell | ||
+ | |||
+ | // You need libexpat first. On debian: | ||
+ | $ sudo apt-get install expat libexpat1 libexpat1-dev | ||
+ | |||
+ | cpan> install Bio::Perl // may be necessary to install from source | ||
+ | cpan> install Bio::Graphics // may be necessary to install from source | ||
+ | |||
+ | |||
+ | == Install SamTools == | ||
+ | |||
== Install GBrowse2 == | == Install GBrowse2 == |
Revision as of 21:21, 2 March 2010
Contents
Overview
GBrowse is one of the central -- and most heavily used -- components of the WormBase site. To meet this demand, we use a number of dedicated servers using GBrowse's built in master/slave configuration. This document describes how to build and configure these servers.
Hardware Topology
The GBrowse topology looks like this:
Master ------------- ----------------------------- | gb1 | \ \ | | ------------- ------------- |* * * * * *| <----- | gb2 | | gb2 | Slaves ------------- ------------- -------------
Where:
* * * * * = relational databases running under mysql on gb1.
Node Configuration
Create groups
sudo addgroup wormbase sudo addgroup gbrowse # Add yourself to the groups sudo usermod -a -G gbrowse, wormbase tharris
Create Directories
Create a directory for GBrowse. For now, we're maintaining GBrowse on dedicated hardware completely distinct from the rest of the web application. This makes updates significantly easier, although it increases maintenance (ie of boilerplate). We'll also keep gbrowse inside a wormbase/ directory should we wish to install other components on these servers at a later date.
Let $ROOT = /usr/local/wormbase
$ sudo mkdir -p ${ROOT} $ sudo chgrp -R wormbase $ROOT $ sudo chmod 2775 $ROOT
Debian preparation
$ sudo apt-get update $ sudo apt-get install gcc
Install Perl 5.10.1 or greater
You'll need Perl version 5.10.1 or greater.
Debian:
$ sudo apt-get install perl
Or from source:
bash> mkdir ~/website/perl bash> cd ~/src bash> curl -O http://www.cpan.org/src/perl-5.10.1.tar.gz bash> cd ~/build bash> tar xzf ../src.perl-5.10.1.tar.gz bash> cd perl-5.10.1 bash> ./Configure -des // Or, to install in a local path: bash> ./Configure -des -Dprefix=$HOME/website/perl/5.10.1 bash> make bash> make test bash> make install
Be sure to set up your your system to preferentially use this newly installed Perl:
bash> export PATH=~/wormbase/perl/5.10.1/bin:${PATH}
Install Apache
Debian (including it's annoying directory layout):
$ sudo apt-get install apache2
Or from source:
Install mod perl
Install MySQL > 5.1
We need a newer version of mysql than is available from the package manager. We'lll install the binary for x86_64.
For now, only the master server will run mysql. In the future, each dedicated slave could as well in order to meet growing demand.
$ groupadd mysql $ useradd -g mysql mysql $ usermod -a -G mysql tharris $ cd /usr/local $ sudo gunzip < /path/to/mysql-VERSION-OS.tar.gz | sudo tar xvf - $ sudo ln -s full-path-to-mysql-VERSION-OS mysql $ cd mysql $ sudo chown -R mysql . $ sudo chgrp -R mysql . $ sudo scripts/mysql_install_db --user=mysql // may be necessary to remove /etc/mysql/my.cnf first $ sudo chown -R root . $ sudo chown -R mysql data $ sudo chmod 2775 data // I like to be able to write to my datadir
// Copy the configuration file $ sudo cp support_files/my-medium.cnf /etc/my.cnf
// Set up mysql to start up automatically $ sudo cp support_files/mysql.server /etc/init.d/. $ cd /etc/rc3.d $ sudo ln -s ../init.d/mysql.server S90mysql $ cd ../rc5.d $ sudo ln -s ../init.d/mysql.server S90mysql
// Start it up and set a root password. $ sudo /etc/init.d/mysql.server start $ mysqladmin -u root password 'PASSWORD'
Install Requisite Perl Modules
All libraries specific for GBrowse are maintained in a local directory, specific to the version of gbrowse being run. This means that prerequisites need to be installed for each version of Gbrowse.
$ cd /usr/local/wormbase $ mkdir gbrowse-2.01 // eg $ ln -s gbrowse-2.01 gbrowse-current $ mkdir gbrowse-current/extlib
Install local::lib to make things easier:
$ sudo perl -MCPAN -e 'CPAN::install(local::lib)' $ cd /usr/local/wormbase/gbrowse-current/extlib
To install modules to this path:
$ perl -Mlocal::lib=. $ eval $(perl -Mlocal::lib=--self-contained,./) $ sudo rm -rf ~/.cpan $ perl -MCPAN -e shell // You need libexpat first. On debian: $ sudo apt-get install expat libexpat1 libexpat1-dev
cpan> install Bio::Perl // may be necessary to install from source cpan> install Bio::Graphics // may be necessary to install from source