Unit Testing

Executing Unit Tests

1. change directory into your Github Website clone.
2. run tests of the REST API: perl t/rest.t
3. run tests of the WormBase Perl API: API_TESTS=1 perl t/api.t
4. run comparative GBrowse tests: perl --base http://your.url.org:1234/cgi-bin/gb2/gbrowse

Successful tests are preceded by an "ok". For example:

ok 1 - require 't/rest_tests/template.t';
    ok 1 - port open
    1..1
ok 2 - test_port_open

How to read the output: The first "ok 1" means that the first executed test require ... completed sucessfully. "ok 2" shows that the second test was also successful, where sub-tests (i.e., tests within the test_port_open Perl sub) are shown above as indented lines. The interval "1..1" denotes the tests were run within test_port_open. In this example file "t/rest_tests/template.t" contains one subroutine called "test_port_open" which contains an assertion labelled "port open" which returned as true. Subroutine names appear below the results of the tests they contain.

Failed tests are preceded by a "not ok" and a short description is displayed that explains why the test failed (lines preceded by "#"). For example:

    not ok 1 - port open
    #   Failed test 'port open'
    #   at t/rest_tests/template.t line 53.
    #          got: undef
    #     expected: anything else
    1..1
    # Looks like you failed 1 test of 1.
not ok 2 - test_port_open

API Unit Testing Framework: How it works / Write your own

Unit tests are

• based on Test::More.
• run on a fully populated WormBase database backend.
• API tests: start (and stop) a Catalyst web server (random port between 28,000 and 31,999)

Unit test templates are available as

• t/rest_tests/template.t for writing REST API tests
• t/api_tests/template.t for writing WormBase Perl API tests

New unit tests should be using the following workflow:

1. create a copy of template.t within the same directory; choose a descriptive filename
2. make appropriate changes to the newly created file

Comparative Testing Framework: GBrowse

Comparative GBrowse tests are

• based on a reference image set (one image per species, per track, per example landmark)
• comparing a live gbrowse_img rendering to the respective reference image

The reference image set can be created via

• perl --base http://your.url.org:1234/cgi-bin/gb2/gbrowse --reference

A summary log and a full disclosure of broken URLs is written to the logfile logs/gbrowse_test.log.

Reference Set Generation, Current Output on Staging, WS238

cd /var/lib/jenkins/jobs/gbrowse_track_tests/workspace
sudo -u jenkins perl t/gbrowse.t --cutoff 1 --reference --base http://staging.wormbase.org/tools/genome/gbrowse_img
Configuration        : a_suum_PRJNA62057
    Retrieved images : 82
    Broken URLs      : 0
Configuration        : a_suum_PRJNA80881
    Retrieved images : 160
    Broken URLs      : 80
Configuration        : b_malayi_PRJNA10729
    Retrieved images : 173
    Broken URLs      : 1
Configuration        : b_xylophilus_PRJEA64437
    Retrieved images : 82
    Broken URLs      : 0
Configuration        : c_angaria_PRJNA51225
    Retrieved images : 120
    Broken URLs      : 0
Configuration        : c_brenneri_PRJNA20035
    Retrieved images : 306
    Broken URLs      : 0
Configuration        : c_briggsae_PRJNA10731
    Retrieved images : 374
    Broken URLs      : 11
Configuration        : c_elegans_PRJNA13758
    Retrieved images : 7045
    Broken URLs      : 1035
Configuration        : c_japonica_PRJNA12591
    Retrieved images : 270
    Broken URLs      : 0
Configuration        : c_remanei_PRJNA53967
    Retrieved images : 293
    Broken URLs      : 1
Configuration        : c_sp11_PRJNA53597
    Retrieved images : 123
    Broken URLs      : 0
Configuration        : c_sp5_PRJNA194557
    Retrieved images : 82
    Broken URLs      : 0
Configuration        : h_bacteriophora_PRJNA13977
    Retrieved images : 82
    Broken URLs      : 0
Configuration        : h_contortus_PRJEB506
    Retrieved images : 123
    Broken URLs      : 0
Configuration        : l_loa_PRJNA60051
    Retrieved images : 123
    Broken URLs      : 0
Configuration        : m_hapla_PRJNA29083
    Retrieved images : 164
    Broken URLs      : 0
Configuration        : m_incognita_PRJEA28837
    Retrieved images : 120
    Broken URLs      : 0
Configuration        : p_pacificus_PRJNA12644
    Retrieved images : 180
    Broken URLs      : 0
Configuration        : s_ratti_PRJEA62033
    Retrieved images : 82
    Broken URLs      : 0
Configuration        : t_spiralis_PRJNA12603
    Retrieved images : 39
    Broken URLs      : 2