Unit Testing

From WormBaseWiki
Revision as of 13:36, 7 October 2013 by Joachim.baran (talk | contribs) (Added output of reference set generation for WS239.)
Jump to navigationJump to search

Executing Unit Tests

  1. change directory into your Github Website clone.
  2. run tests of the REST API: perl t/rest.t
    1. run a particular unit test: perl t/rest.t --test=gene
  3. run tests of the WormBase Perl API: API_TESTS=1 perl t/api.t
    1. run a particular unit test: API_TESTS=protein perl t/api.t
  4. run comparative GBrowse tests: perl --base http://your.url.org:1234/cgi-bin/gb2/gbrowse

Successful tests are preceded by an "ok". For example:

ok 1 - require 't/rest_tests/template.t';
    ok 1 - port open
    1..1
ok 2 - test_port_open

How to read the output: The first "ok 1" means that the first executed test require ... completed sucessfully. "ok 2" shows that the second test was also successful, where sub-tests (i.e., tests within the test_port_open Perl sub) are shown above as indented lines. The interval "1..1" denotes the tests were run within test_port_open. In this example file "t/rest_tests/template.t" contains one subroutine called "test_port_open" which contains an assertion labelled "port open" which returned as true. Subroutine names appear below the results of the tests they contain.

Failed tests are preceded by a "not ok" and a short description is displayed that explains why the test failed (lines preceded by "#"). For example:

    not ok 1 - port open
    #   Failed test 'port open'
    #   at t/rest_tests/template.t line 53.
    #          got: undef
    #     expected: anything else
    1..1
    # Looks like you failed 1 test of 1.
not ok 2 - test_port_open

API Unit Testing Framework: How it works / Write your own

Unit tests are

  • based on Test::More.
  • run on a fully populated WormBase database backend.
  • API tests: start (and stop) a Catalyst web server (random port between 28,000 and 31,999)

Unit test templates are available as

  • t/rest_tests/template.t for writing REST API tests
  • t/api_tests/template.t for writing WormBase Perl API tests

New unit tests should be using the following workflow:

  1. create a copy of template.t within the same directory; choose a descriptive filename
  2. make appropriate changes to the newly created file

Comparative Testing Framework: GBrowse

Comparative GBrowse tests are

  • based on a reference image set (one image per species, per track, per example landmark)
  • comparing a live gbrowse_img rendering to the respective reference image

The reference image set can be created via

A summary log and a full disclosure of broken URLs is written to the logfile logs/gbrowse_test.log.

Reference Set Generation, Current Output on Staging, WS239

Note: A. suum landmarks are still broken?

cd /var/lib/jenkins/jobs/gbrowse_track_tests/workspace
sudo -u jenkins perl t/gbrowse.t --cutoff 1 --reference --base http://staging.wormbase.org/tools/gen
ome/gbrowse_img
Configuration        : a_suum_PRJNA62057
    Retrieved images : 82
    Broken URLs      : 0
Configuration        : a_suum_PRJNA80881
    Retrieved images : 158
    Broken URLs      : 82
Configuration        : b_malayi_PRJNA10729
    Retrieved images : 174
    Broken URLs      : 0
Configuration        : b_xylophilus_PRJEA64437
    Retrieved images : 82
    Broken URLs      : 0
Configuration        : c_angaria_PRJNA51225
    Retrieved images : 120
    Broken URLs      : 0
Configuration        : c_brenneri_PRJNA20035
    Retrieved images : 306
    Broken URLs      : 0
Configuration        : c_briggsae_PRJNA10731
    Retrieved images : 385
    Broken URLs      : 0
Configuration        : c_elegans_PRJNA13758
    Retrieved images : 0
    Broken URLs      : 8080
Configuration        : c_japonica_PRJNA12591
    Retrieved images : 270
    Broken URLs      : 0
Configuration        : c_remanei_PRJNA53967
    Retrieved images : 294
    Broken URLs      : 0
Configuration        : c_sp11_PRJNA53597
    Retrieved images : 123
    Broken URLs      : 0
Configuration        : c_sp5_PRJNA194557
    Retrieved images : 82
    Broken URLs      : 0
Configuration        : h_bacteriophora_PRJNA13977
    Retrieved images : 82
    Broken URLs      : 0
Configuration        : h_contortus_PRJEB506
    Retrieved images : 123
    Broken URLs      : 0
Configuration        : h_contortus_PRJNA205202
    Retrieved images : 0
    Broken URLs      : 123
Configuration        : l_loa_PRJNA60051
    Retrieved images : 123
    Broken URLs      : 0
Configuration        : m_hapla_PRJNA29083
    Retrieved images : 164
    Broken URLs      : 0
Configuration        : m_incognita_PRJEA28837
    Retrieved images : 120
    Broken URLs      : 0
Configuration        : p_pacificus_PRJNA12644
    Retrieved images : 180
    Broken URLs      : 0
Configuration        : s_ratti_PRJEA62033
    Retrieved images : 82
    Broken URLs      : 0
Configuration        : t_spiralis_PRJNA12603
    Retrieved images : 41
    Broken URLs      : 0

Reference Set Generation, Current Output on Staging, WS238

cd /var/lib/jenkins/jobs/gbrowse_track_tests/workspace
sudo -u jenkins perl t/gbrowse.t --cutoff 1 --reference --base http://staging.wormbase.org/tools/genome/gbrowse_img
Configuration        : a_suum_PRJNA62057
    Retrieved images : 82
    Broken URLs      : 0
Configuration        : a_suum_PRJNA80881
    Retrieved images : 160
    Broken URLs      : 80
Configuration        : b_malayi_PRJNA10729
    Retrieved images : 173
    Broken URLs      : 1
Configuration        : b_xylophilus_PRJEA64437
    Retrieved images : 82
    Broken URLs      : 0
Configuration        : c_angaria_PRJNA51225
    Retrieved images : 120
    Broken URLs      : 0
Configuration        : c_brenneri_PRJNA20035
    Retrieved images : 306
    Broken URLs      : 0
Configuration        : c_briggsae_PRJNA10731
    Retrieved images : 374
    Broken URLs      : 11
Configuration        : c_elegans_PRJNA13758
    Retrieved images : 7045
    Broken URLs      : 1035
Configuration        : c_japonica_PRJNA12591
    Retrieved images : 270
    Broken URLs      : 0
Configuration        : c_remanei_PRJNA53967
    Retrieved images : 293
    Broken URLs      : 1
Configuration        : c_sp11_PRJNA53597
    Retrieved images : 123
    Broken URLs      : 0
Configuration        : c_sp5_PRJNA194557
    Retrieved images : 82
    Broken URLs      : 0
Configuration        : h_bacteriophora_PRJNA13977
    Retrieved images : 82
    Broken URLs      : 0
Configuration        : h_contortus_PRJEB506
    Retrieved images : 123
    Broken URLs      : 0
Configuration        : l_loa_PRJNA60051
    Retrieved images : 123
    Broken URLs      : 0
Configuration        : m_hapla_PRJNA29083
    Retrieved images : 164
    Broken URLs      : 0
Configuration        : m_incognita_PRJEA28837
    Retrieved images : 120
    Broken URLs      : 0
Configuration        : p_pacificus_PRJNA12644
    Retrieved images : 180
    Broken URLs      : 0
Configuration        : s_ratti_PRJEA62033
    Retrieved images : 82
    Broken URLs      : 0
Configuration        : t_spiralis_PRJNA12603
    Retrieved images : 39
    Broken URLs      : 2