Difference between revisions of "Website:Application overview"
Line 7: | Line 7: | ||
-rw-rw-r-- Makefile.PL | -rw-rw-r-- Makefile.PL | ||
-rw-rw-r-- README | -rw-rw-r-- README | ||
− | |||
− | |||
drwxrwxr-x conf | drwxrwxr-x conf | ||
Line 42: | Line 40: | ||
Application test suite. | Application test suite. | ||
− | -rw-rw-rw- wormbase. | + | -rw-rw-rw- wormbase.conf |
Application-wide default configurations. | Application-wide default configurations. | ||
− | -rw-rw-r-- wormbase_local. | + | -rw-rw-r-- wormbase_local.conf |
Template for local configuration. Entries here will override defaults. | Template for local configuration. Entries here will override defaults. | ||
Line 56: | Line 54: | ||
Here, $ROOT refers to the document root of your application. | Here, $ROOT refers to the document root of your application. | ||
− | == $ROOT/wormbase. | + | == $ROOT/wormbase.conf == |
This file contains the default configuration for the application. The application defaults to using remote data sources. If you would like to override this, see wormbase_local.yml. | This file contains the default configuration for the application. The application defaults to using remote data sources. If you would like to override this, see wormbase_local.yml. | ||
Line 71: | Line 69: | ||
==Pages== | ==Pages== | ||
− | Pages are composed of widgets, in turn composed of fields. | + | Pages are composed of widgets, which are in turn composed of fields. |
===Widgets=== | ===Widgets=== | ||
Line 79: | Line 77: | ||
The widgets available for any page are specified in the primary configuration file. They are comprised of: | The widgets available for any page are specified in the primary configuration file. They are comprised of: | ||
− | * a | + | * suitable configuration in wormbase.conf. |
− | * a template file | + | * a (generic) REST controller action. |
+ | * and (usually) a template file. | ||
===Fields=== | ===Fields=== | ||
Line 88: | Line 87: | ||
* a Controller method implementing a chained action. This calls a... | * a Controller method implementing a chained action. This calls a... | ||
* a Model method(s) that collect and massage the appropriate data. The primary method should correspond to the name of the field. | * a Model method(s) that collect and massage the appropriate data. The primary method should correspond to the name of the field. | ||
− | + | * field names are: stash keys (dynamic), controller actions (dynamic), and section names in templates. | |
− | * field names are: stash keys (dynamic), controller actions (dynamic), | ||
* fields are also URL and REST targets. | * fields are also URL and REST targets. | ||
==Example== | ==Example== | ||
− | http://localhost/gene/* | + | http://localhost/reports/gene/identification/* |
Access a widget called identification that collects together the fields that comprise the widget. | Access a widget called identification that collects together the fields that comprise the widget. | ||
− | http://localhost/gene/* | + | http://localhost/reports/gene/ncbi/* |
Access the ncbi field of the identification widget. Note that it is not necessary to know the containing widget. | Access the ncbi field of the identification widget. Note that it is not necessary to know the containing widget. | ||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
= Setting up a new page = | = Setting up a new page = | ||
− | == | + | == Add Configuration == |
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | + | Add configuration specifying the name of the page, the order widgets should be loaded, and each widget and the fields it contains to '''wormbase.conf''': | |
− | + | <pages> | |
− | + | <account_details> | |
+ | widget_order profile | ||
+ | widget_order address | ||
+ | <widgets> | ||
+ | <profile> | ||
+ | name profile | ||
+ | title Account Profile | ||
+ | fields id | ||
+ | fields nickname | ||
+ | fields email | ||
+ | </profile> | ||
+ | <address> | ||
+ | name address | ||
+ | title Address | ||
+ | fields street | ||
+ | fields city | ||
+ | fields state | ||
+ | fields zip | ||
+ | </address> | ||
+ | </widgets> | ||
+ | </account_details> | ||
+ | </pages> | ||
− | + | See '''wormbase.conf''' and the documentation on [[Website:Configuration|configuration]] for full details. | |
− | |||
− | |||
This configuration is currently stored in WormBase.pm but is subject to change! | This configuration is currently stored in WormBase.pm but is subject to change! | ||
− | + | ==Create your WormBase::API::Object::* methods== | |
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | Remember: | + | Remember: The names of these methods should correspond to the names of your widget fields! If they do not, your template stash will not be populated. |
− | |||
− | |||
== Create templates (if necessary)== | == Create templates (if necessary)== | ||
Line 249: | Line 143: | ||
The widget template: | The widget template: | ||
− | root/templates/$class | + | root/templates/classes/$class/$your_widget_name.tt2 |
− | |||
− | |||
− | |||
= Caching = | = Caching = |
Revision as of 14:37, 9 August 2010
Directory Structure
-rw-rw-r-- Changes -rw-rw-r-- Makefile.PL -rw-rw-r-- README drwxrwxr-x conf Application and third-party library configuration. drwxrwxr-x design Design elements and ideas. drwxrwxr-x extlib Directory containing builds of third party libraries. drwxrwxr-x lib The core application modules. drwxrwxr-x logs drwxrwxr-x private Docs and presentations. drwxrwxr-x root /root contains static files and templates. drwxrwxr-x script Helper scripts and the stand-alone server. drwxrwxr-x sql SQL statements for various databases. drwxrwxr-x src Third party sources. drwxrwxr-x t Application test suite. -rw-rw-rw- wormbase.conf Application-wide default configurations. -rw-rw-r-- wormbase_local.conf Template for local configuration. Entries here will override defaults.
Configuration
Catalyst offers a powerful configuration system. We use it to provide application-wide, per-page, and per-session configuration. In addition, local configuration files can be used to override any configuration option for production or development deployment.
Here, $ROOT refers to the document root of your application.
$ROOT/wormbase.conf
This file contains the default configuration for the application. The application defaults to using remote data sources. If you would like to override this, see wormbase_local.yml.
$ROOT/wormbase.yml.template
Move this template file to wormbase_local.yml and edit values to locally override the default configuration variables. This file is maintainted in SVN as a template so that individual development preferences are not obliterated by an inadvertent svn commit.
Quick Overview
The general structure of the application mirrors the organization in Acedb. Separate Model::* packages correspond to classes in Acedb.
Pages
Pages are composed of widgets, which are in turn composed of fields.
Widgets
Widgets are small segments of a page comprised of a series of fields. Widgets roughly correspond to sections of the old WormBase site.
The widgets available for any page are specified in the primary configuration file. They are comprised of:
- suitable configuration in wormbase.conf.
- a (generic) REST controller action.
- and (usually) a template file.
Fields
Individual sections of each widget are referred to as fields. They are comprised of:
- a Controller method implementing a chained action. This calls a...
- a Model method(s) that collect and massage the appropriate data. The primary method should correspond to the name of the field.
- field names are: stash keys (dynamic), controller actions (dynamic), and section names in templates.
- fields are also URL and REST targets.
Example
http://localhost/reports/gene/identification/*
Access a widget called identification that collects together the fields that comprise the widget.
http://localhost/reports/gene/ncbi/*
Access the ncbi field of the identification widget. Note that it is not necessary to know the containing widget.
Setting up a new page
Add Configuration
Add configuration specifying the name of the page, the order widgets should be loaded, and each widget and the fields it contains to wormbase.conf:
<pages> <account_details> widget_order profile widget_order address <widgets> <profile> name profile title Account Profile fields id fields nickname fields email </profile> <address> name address title Address fields street fields city fields state fields zip </address> </widgets> </account_details> </pages>
See wormbase.conf and the documentation on configuration for full details.
This configuration is currently stored in WormBase.pm but is subject to change!
Create your WormBase::API::Object::* methods
Remember: The names of these methods should correspond to the names of your widget fields! If they do not, your template stash will not be populated.
Create templates (if necessary)
Create templates as necessary. These should be located in:
The widget template: root/templates/classes/$class/$your_widget_name.tt2
Caching
To ease server load and accelerate page load times, the web application implements a number of caching mechanisms of elements that are computationally intensive to generate.
File cache of computationally intensive methods
Required modules
- Catalyst::Plugin::Cache
- CHI
- CHI::Driver::File
Usage
# Check the cache for the presence of your data. # my_cache_id can be any string you want (see below for details). # $cache_id is the provided string with the current version of WormBase appended # $cached_data is data returned from the cache, if any my ($cache_id,$cached_data) = $c->check_cache("my_cache_id"); unless ($cached_data) { my $data = some_computationally_expensive_method(); $c->set_cache($cache_id,$data); }
Cache ID
When using the check_cache/set_cache approach, the initial cache ID that you provide will have the current version of WormBase appended to it. This lets us automatically expire entries when a new version of the database is released. Note that this ONLY works if you use both check_cache() and set_cache(). You should always follow this strategy as checking the cache is extremely efficient.
Here's an example from the REST controller for widgets:
sub widget_GET { my ($self,$c,$class,$name,$widget) = @_; # Does the data for this widget already exist in the cache? my ($cache_id,$cached_data) = $c->check_cache($class,$widget,$name); # The cache ONLY includes the field data for the widget, nothing else. # This is because most backend caches cannot store globs. if ($cached_data) { $c->stash->{fields} = $cached_data; } else { # Load the stash with the field contents for this widget. my @fields; # Widgets accessible by name if (ref $c->config->{pages}->{$class}->{widgets}->{$widget}->{fields} ne "ARRAY") { @fields = ($c->config->{pages}->{$class}->{widgets}->{$widget}->{fields}); } else { @fields = @{ $c->config->{pages}->{$class}->{widgets}->{$widget}->{fields} }; } foreach my $field (@fields) { my $data = {}; $data = $object->$field if defined $object->$field; # Conditionally load up the stash (for now) for HTML requests. # Alternatively, we could return JSON and have the client format it. $c->stash->{fields}->{$field} = $data; } # Cache the field data for this widget. $c->set_cache($cache_id,$c->stash->{fields}); } $self->status_ok($c, entity => { class => $class, name => $name, uri => "$uri" } ); }
Types of data to store in the cache
In the REST widget() example, we cache all data drawn from the database necessary for the widget. We do not cache generated HTML (template processing is very efficient). Should we deem it necessary, we can exploring caching of generated HTML using Template::Plugin::Cache.
Note that it is not possible to cache data structures containing globs.
Cache Duration/Expiry
Caching is enabled for *all* installations. This enables us to test the caching mechanism in development.
Cached items in development installations are set to 4 seconds to ensure that code changes are reflected in page reloads. Cached items in production installations are set to expire after 4 weeks. These values can be modified in lib/WormBase/Web.pm.
Cache Location
The cache is written to /tmp/wormbase/file_cache_chi. This is specified during in Web.pm during application setup.
NFS?
Component and page-level caching via reverse proxy
Production websites site behind a caching reverse proxy. This proxy caches many (but not all) dynamically generated content as well. Due to performance considerations, the front end proxy cache expires sooner than that of the back end application cache. See documentation on squid3 elsewhere on this wiki.