Design Specs: Scalability

Objectives

To build a robust and scalable platform for hosting the next generation of the WormBase website.

Limitations of the Current System

As of Jan 2010, the WormBase website resides on a heterogeneous cluster of web and database servers. A single server running the squid reverse proxy software load balances across these servers.

Although this system scales easily and has served WormBase well for six years, it is not ideal for the following reasons.

1. Multiple single points of redundancy that require manual intervention to resolve

2. Difficulty of configuration and management of reverse proxy software

3. Lack of true fault tolerance in system

4. Heterogeneity complicates management, backups, production releases

Page and Page Fragment Caching

CHI/PageCache or via Template Toolkit.

Catalyst Modules:

Catalyst::Plugin::Cache
Catalyst::Plugin::PageCache

Mailing list discussion: [Catalyst] Page fragment caching From Jan 20, 2010

use Digest::SHA1 'sha1_hex'; use base 'Catalyst::Controller';

sub foo : Local {

my ($self, $c) = @_;

my $name = $c->req->params->{name}; my $age = $c->req->params->{age}; my $page = $c->req->params->{page};

my $hash = sha1_hex('foo', $name, $age, $page);
my $cache = $c->cache;
my $page_part = $cache->get($hash);

unless ($page_part) {

1. Here make the selection from the database, get the data from somewhere else...

my $object = $c->model("DB::Table")->search({ name => {-like => "%$name%"}, age => $age, },{ order_by => 'name', rows => 20, page => $page, });

$c->stash(obj => $obj); $page_part = $c->view('TT')->render($c, 'template/for/that/part/of/the/page.tt');

$cache->set($hash, $page_part, 600); }

$c->stash(page_part => $page_part); }

This way it works fine, although if someone has an easier and/or nicer solution, please tell us.

PageCache has some conflicts with another plugin, if I remember well is Catalyst::Plugin::Unicode.

Template::Plugin::Cache that can be used for page parts caching very easy doesn't work well with templates that contain special UTF-8 chars (I was informed that this is because the FileCache module and I heard that it could work with a CHI object).

Anyway, if somebody has some recommendations for speed improvements with Catalyst, please tell us if there are (I have checked a PHP page with many dynamic types of data that have a 44 requests/second, but I wasn't able to create a much simpler page with Catalyst that displays so fast). Yes, I know, the PHP page is low level, harder to create and to maintain, gives bad errors, but I can't convince that Catalyst is good if it is not fast enough, so any tips for using caches or other ways of improving the speed are welcome.

> If you are using Template Toolkit, > > http://search.cpan.org/dist/Template-Plugin-Cache/

Agree, this will do what you want. To have it use your Catalyst cache, just do something like this:

 USE cache = Cache('cache' => c.cache);

You can use it to cache either a whole page or just part of one, since you can "cache.inc()" or "cache.proc()" to INCLUDE or PROCESS any template (or BLOCK? not sure...) from inside another one.

Catalyst::Plugin::Cache will happily use CHI, or you can bind CHI in as a model, whatever..

I don't see why there needs to be anything CHI specific?

(But I'd welcome doc patches / wiki / advent articles showing how to use CHI).