Difference between revisions of "Administration:WormBase Production Environment"
Line 137: | Line 137: | ||
= Memached/libmemcached = | = Memached/libmemcached = | ||
− | |||
− | |||
sudo apt-get install memcached | sudo apt-get install memcached | ||
Line 160: | Line 158: | ||
libc6 | libc6 | ||
sudo apt-get install libc6-dev | sudo apt-get install libc6-dev | ||
− | |||
= Adjust iptables = | = Adjust iptables = |
Revision as of 19:05, 1 December 2010
Contents
Overview
The WormBase production environment consists of a series of a http servers glued to our webapp, all sitting behind a load-balancing reverse-proxy server (nginx).
This document describes the configuration of individual web nodes and the reverse proxy server.
Reverse Proxy and Load Balancing via nginx
Installation
We'll place nginx entirely within the wormbase root directory.
1. Install prerequisites
# Perl Compatabile Regular Expression libaray sudo apt-get install libpcre3 libpcre3-dev
# Fetch and unpack openssel wget http://www.openssl.org/source/openssl-0.9.8p.tar.gz tar -zxf openssl-0.9.8p.tar.gz
2. Get the nginx cache-purge module
cd src/ curl -O http://labs.frickle.com/files/ngx_cache_purge-1.2.tar.gz tar xzf ngx_cache_purge-1.2.tar.gz
3. Build and install nginx
curl -O http://nginx.org/download/nginx-0.8.53.tar.gz tar xzf nginx* ./configure \ --prefix=/usr/local/wormbase/nginx \ --error-log-path=/usr/local/wormbase/logs/nginx-error.log \ --http-log-path=/usr/local/wormbase/logs/nginx-access.log \ --with-http_stub_status_module \ --with-http_ssl_module \ --with-ipv6 \ --with-http_realip_module \ --with-http_addition_module \ --with-http_image_filter_module \ --with-http_sub_module \ --with-http_dav_module \ --with-http_flv_module \ --with-http_gzip_static_module \ --with-http_secure_link_module \ --with-openssl=../openssl-0.9.8p \ --add-module=../ngx_cache_purge-1.2 make make install
Get rid of the original configuration file and symlink to that in our source code repository:
cd /usr/local/wormbase/nginx mv conf conf.original ln -s /usr/local/wormbase/admin/conf/nginx conf
Configuration
Configuration files are stored in the wormbase-admin source code repository, under conf/nginx and symlinked as:
> cd /usr/local/wormbase/nginx ; ls conf conf -> /home/tharris/projects/wormbase/wormbase-admin/conf/nginx
In particular see the file conf/nginx/nginx.conf for details on configuration.
Test the configuration file syntax by:
$ nginx -t
Here's a more complicated example demonstrating caching and load balancing: http://nathanvangheem.com/news/nginx-with-built-in-load-balancing-and-caching
Load Balancing
nginx relies on the NginxHttpUpstreamModule for load balancing. It's built-in by default. The documentation contains a number of possibly useful configuration directives:
http://wiki.nginx.org/NginxHttpUpstreamModule
There are a number of other interesting load-balancing modules that might be of use:
http://wiki.nginx.org/3rdPartyModules
Starting the Server
Copy wormbase-admin/init/nginx.init to /etc/init.d/nginx to setup a suitable init script. (Re)start the server by:
$ /etc/init.d/nginx restart
Set nginx to start at server launch
sudo /usr/sbin/update-rc.d -f nginx defaults
The output will be similar to this:
Adding system startup for /etc/init.d/nginx ... /etc/rc0.d/K20nginx -> ../init.d/nginx /etc/rc1.d/K20nginx -> ../init.d/nginx /etc/rc6.d/K20nginx -> ../init.d/nginx /etc/rc2.d/S20nginx -> ../init.d/nginx /etc/rc3.d/S20nginx -> ../init.d/nginx /etc/rc4.d/S20nginx -> ../init.d/nginx /etc/rc5.d/S20nginx -> ../init.d/nginx
Webserver Nodes
PSGI/Plack + Starman
PSGI: specification for Perl superglue between frameworks and servers. Plack is an implementation of PSGI. Compare to Rack (Ruby) or Jack (Javascript).
http://plackperl.org/
Starman is a high performance pre-forking Perl PSGI server. We're using it in place of Apache+fastcgi.
https://github.com/miyagawa/Starman
Install Plack:
- Catalyst::Controller::Metal
- Catalyst::Engine::PSGI
- Catalyst::Helper::PSGI
- Plack::Test::Adopt::Catalyst
Instal CPANMinus
- App::cpanminus
Install Plack:
- cpanm Task::Plack
- cpanm Starman
Configuration
script/wormbase_psgi.psgi
Starting Starman
starman script/wormbase_psgi.psgi OR starman -MFindBin script/wormbase_psgi.psgi
Memached/libmemcached
sudo apt-get install memcached wget http://download.tangent.org/libmemcached-0.44.tar.gz tar -zxf libmemcached-0.44.tar.gz cd libmemcached-0.4 ./configure –prefix=/usr make make install
MISC:
Crypt: ssl sudo apt-get install libssl-dev
libc6 sudo apt-get install libc6-dev
Adjust iptables
We need to open port 8000, which should only be to the squid.
sudo emacs /etc/init.d/iptables
Add the following:
# The new website runs on port 8000. It SHOULD only be accessible by squid $BIN -A INPUT -p tcp --dport 8000 -m state --state NEW -j ACCEPT # ... or only accessible by proxy # $BIN -A INPUT -p tcp -s 206.108.125.175 --dport 8000 -m state --state NEW -j ACCEPT
Then
/etc/init.d/iptables.local restart
EVERYTHING BELOW HERE IS DEPRECATED
FastCGI
Installing fastcgi
curl -O http://www.fastcgi.com/dist/mod_fastcgi-2.4.6.tar.gz tar xzf mod_fastcgi* cd mod_fastcgi* cp Makefile.AP2 Makefile make top_dir=/usr/local/apache2 sudo make top_dir=/usr/local/apache2 install
If you get an error on make saying it can't find special.mk (which is supposed to be distributed with httpd but isn't on CentOS and is not part of httpd-devel, either), try:
sudo apxs -n mod_fastcgi -i -a -c mod_fastcgi.c fcgi_buf.c fcgi_config.c fcgi_pm.c fcgi_protocol.c fcgi_util.c
Add an entry to httpd.conf like this:
LoadModule fastcgi_module modules/mod_fastcgi.so // Note: if you use the apxs command above, it inserts an incorrect line into your httpd.conf file. // Edit it to read exactly as above.
Launch the fastcgi server
// as a socket server in daemon mode /usr/local/wormbase/website/script/wormbase_fastcgi.pl \ -l /tmp/wormbase.sock -n 5 -p /tmp/wormbase.pid -d // as a deamon bound to a specific port script/wormbase_fastcgi.pl -l :3001 -n 5 -p /tmp/wormbase.pid -d
Set up the fastcgi server to launch at boot
Symlink the webapp-fastcgi.init script to /etc/init.d
cd /etc/init.d sudo ln -s /usr/local/wormbase/website/util/init/webapp-fastcgi.init wormbase-fastcgi
Set up symlinks in runlevels:
cd ../rc3.d sudo ln -s ../init.d/wormbase-fastcgi S99wormbase-fastcgi cd ../rc5.d sudo ln -s ../init.d/wormbase-fastcgi S99wormbase-fastcgi
Add a cron job that keeps FCGI under control
The following cron job will kill off fcgi children that exceed the specified memory limit (in bytes).
sudo crontab -e */30 * * * * /usr/local/wormbase/website/util/crons/fastcgi-childreaper.pl \ `cat /tmp/wormbase.pid` 104857600
FastCGID
cd src tar xzf mod_fcid* cd mod_fcgid* APXS=/usr/local/apache2/bin/apxs ./configure.apxs make sudo make install
Apache
Configure Apache to connect to the fastcgi server
Edit /usr/local/apache2/conf/extra/httpd-vhosts.conf
<VirtualHost *:8000> # ServerName beta.wormbase.org ErrorLog /usr/local/wormbase/logs/wormbase2.error_log TransferLog /usr/local/wormbase/logs/wormbase2.access_log # 502 is a Bad Gateway error, and will occur if the backend server is down # This allows us to display a friendly static page that says "down for # maintenance" Alias /_errors /home/todd/projects/wormbase/website/trunk/root/error-pages ErrorDocument 502 /_errors/502.html # Map dynamic images to the file system # static images are located at img Alias /images /tmp/wormbase/images/ # <Directory /filesystem/path/to/MyApp/root/static> # allow from all # </Directory> # <Location /myapp/static> # SetHandler default-handler # </Location> # Static content served directly by Apache DocumentRoot /usr/local/wormbase/website/root # Alias /static /usr/local/wormbase/website-2.0/root # Approach 1: Running as a static server (Apache handles spawning of the webapp) # <IfModule fastcgi_module> # FastCgiServer /usr/local/wormbase/website-2.0/script/wormbase_fastcgi.pl -processes 3 # Alias / /usr/local/wormbase/website-2.0/script/wormbase_fastcgi.pl/ # </IfModule> # Approach 2: External Process (via mod_fcgi ONLY) <IfModule mod_fastcgi.c> # This says to connect to the Catalyst fcgi server running on localhost, port 777 # FastCgiExternalServer /tmp/myapp.fcgi -host localhost:7777 # Or to use the socket FastCgiExternalServer /tmp/wormbase.fcgi -socket /tmp/wormbase.sock # Place the app at root... Alias / /tmp/wormbase.fcgi/ # ...or somewhere else Alias /wormbase/ /tmp/wormbase.fcgi/ </IfModule> # fcgid configuration # <IfModule mod_fcgid> # # This should point at your myapp/root # DocumentRoot /usr/local/wormbase/beta.wormbase.org/root # Alias /static /usr/local/wormbase/beta.wormbase.org/root/static # <Location /static> # SetHandler default-handler # </Location> # # Alias / /usr/local/wormbase/beta.wormbase.org/script/wormbase_fastcgi.pl/ # AddType application/x-httpd-php .php # <Location /> # Options ExecCGI # Order allow,deny # Allow from all # AddHandler fcgid-script .pl # </Location> # </IfModule> </VirtualHost>
Edit /usr/local/apache2/conf/httpd.conf
Add the appropriate Listen PORT directive.