Difference between revisions of "Unit Testing"

From WormBaseWiki
Jump to navigationJump to search
(Added information about the logfile that is kept for GBrowse tests.)
 
(9 intermediate revisions by 3 users not shown)
Line 1: Line 1:
 +
=== Installing Required Perl Modules ===
 +
 +
The following Perl modules need to be installed in order to run GBrowse tests in particular:
 +
 +
<pre>
 +
sudo cpan install Bio::Graphics
 +
sudo cpan install Config::Simple
 +
sudo cpan install boolean
 +
sudo cpan install --force AnyEvent::CouchDB
 +
</pre>
 +
 
=== Executing Unit Tests ===
 
=== Executing Unit Tests ===
  
 
# change directory into your [https://github.com/WormBase/website Github Website] clone.
 
# change directory into your [https://github.com/WormBase/website Github Website] clone.
 
# run tests of the REST API: <code>perl t/rest.t</code>
 
# run tests of the REST API: <code>perl t/rest.t</code>
 +
## run a particular unit test: <code>perl t/rest.t --test=gene</code>
 
# run tests of the WormBase Perl API: <code>API_TESTS=1 perl t/api.t</code>
 
# run tests of the WormBase Perl API: <code>API_TESTS=1 perl t/api.t</code>
 +
## run a particular unit test: <code>API_TESTS=protein perl t/api.t</code>
 
# run comparative GBrowse tests: <code>perl --base http://your.url.org:1234/cgi-bin/gb2/gbrowse</code>
 
# run comparative GBrowse tests: <code>perl --base http://your.url.org:1234/cgi-bin/gb2/gbrowse</code>
  
Line 15: Line 28:
 
</pre>
 
</pre>
  
'''How to read the output:''' The first "ok 1" means that the first executed test <code>require ...</code> completed sucessfully. "ok 2" shows that the second test was also successful, where sub-tests (i.e., tests within the <code>test_port_open</code> Perl sub) are shown ''above'' as indented lines. The interval "1..1" denotes the tests were run within <code>test_port_open</code>.
+
'''How to read the output:''' The first "ok 1" means that the first executed test <code>require ...</code> completed sucessfully. "ok 2" shows that the second test was also successful, where sub-tests (i.e., tests within the <code>test_port_open</code> Perl sub) are shown ''above'' as indented lines. The interval "1..1" denotes the tests were run within <code>test_port_open</code>.  In this example file "t/rest_tests/template.t" contains one subroutine called "test_port_open" which contains an assertion labelled "port open" which returned as true.  Subroutine names appear below the results of the tests they contain.
  
 
'''Failed tests''' are preceded by a "not ok" and a short description is displayed that explains why the test failed (lines preceded by "#"). For example:
 
'''Failed tests''' are preceded by a "not ok" and a short description is displayed that explains why the test failed (lines preceded by "#"). For example:
Line 47: Line 60:
 
# create a copy of <code>template.t</code> within the same directory; choose a descriptive filename
 
# create a copy of <code>template.t</code> within the same directory; choose a descriptive filename
 
# make appropriate changes to the newly created file
 
# make appropriate changes to the newly created file
 +
 +
It is good to check out the testing section of this example [[Adding Data to a Widget (Example)#Formal Testing]]
  
 
=== Comparative Testing Framework: GBrowse ===
 
=== Comparative Testing Framework: GBrowse ===
Line 57: Line 72:
 
The reference image set can be created via
 
The reference image set can be created via
  
* <code>perl --base http://your.url.org:1234/cgi-bin/gb2/gbrowse</code> --reference
+
* <code>perl --cutoff 1 --reference --base http://your.url.org:1234/cgi-bin/gb2/gbrowse</code>
  
 
A summary log and a full disclosure of broken URLs is written to the logfile <code>logs/gbrowse_test.log</code>.
 
A summary log and a full disclosure of broken URLs is written to the logfile <code>logs/gbrowse_test.log</code>.
  
==== Current Output on Staging, WS238 ====
+
==== Reference Set Generation, Current Output on Staging, WS239 ====
 +
 
 +
'''Note:''' A. suum landmarks are still broken?
  
 
<pre>
 
<pre>
time perl t/gbrowse.t --base http://staging.wormbase.org/tools/genome/gbrowse_img --reference
+
cd /var/lib/jenkins/jobs/gbrowse_track_tests/workspace
 +
sudo -u jenkins perl t/gbrowse.t --cutoff 1 --reference --base http://staging.wormbase.org/tools/gen
 +
ome/gbrowse_img
 
Configuration        : a_suum_PRJNA62057
 
Configuration        : a_suum_PRJNA62057
 
     Retrieved images : 82
 
     Retrieved images : 82
 
     Broken URLs      : 0
 
     Broken URLs      : 0
 
Configuration        : a_suum_PRJNA80881
 
Configuration        : a_suum_PRJNA80881
     ABORTED: too many broken URLs (>= 10%)
+
     Retrieved images : 158
 +
    Broken URLs     : 82
 
Configuration        : b_malayi_PRJNA10729
 
Configuration        : b_malayi_PRJNA10729
 
     Retrieved images : 174
 
     Retrieved images : 174
Line 83: Line 103:
 
     Broken URLs      : 0
 
     Broken URLs      : 0
 
Configuration        : c_briggsae_PRJNA10731
 
Configuration        : c_briggsae_PRJNA10731
     Retrieved images : 375
+
     Retrieved images : 385
     Broken URLs      : 10
+
     Broken URLs      : 0
 
Configuration        : c_elegans_PRJNA13758
 
Configuration        : c_elegans_PRJNA13758
     ABORTED: too many broken URLs (>= 10%)
+
     Retrieved images : 0
 +
    Broken URLs     : 8080
 
Configuration        : c_japonica_PRJNA12591
 
Configuration        : c_japonica_PRJNA12591
 
     Retrieved images : 270
 
     Retrieved images : 270
Line 105: Line 126:
 
     Retrieved images : 123
 
     Retrieved images : 123
 
     Broken URLs      : 0
 
     Broken URLs      : 0
 +
Configuration        : h_contortus_PRJNA205202
 +
    Retrieved images : 0
 +
    Broken URLs      : 123
 
Configuration        : l_loa_PRJNA60051
 
Configuration        : l_loa_PRJNA60051
 
     Retrieved images : 123
 
     Retrieved images : 123
Line 121: Line 145:
 
     Broken URLs      : 0
 
     Broken URLs      : 0
 
Configuration        : t_spiralis_PRJNA12603
 
Configuration        : t_spiralis_PRJNA12603
     ABORTED: too many broken URLs (>= 12.1951219512195%)
+
     Retrieved images : 41
 +
    Broken URLs     : 0
 +
</pre>
 +
 
 +
==== Reference Set Generation, Current Output on Staging, WS238 ====
  
real 150m58.484s
+
<pre>
user 0m20.461s
+
cd /var/lib/jenkins/jobs/gbrowse_track_tests/workspace
sys 0m3.096s
+
sudo -u jenkins perl t/gbrowse.t --cutoff 1 --reference --base http://staging.wormbase.org/tools/genome/gbrowse_img
 +
Configuration        : a_suum_PRJNA62057
 +
    Retrieved images : 82
 +
    Broken URLs      : 0
 +
Configuration        : a_suum_PRJNA80881
 +
    Retrieved images : 160
 +
    Broken URLs      : 80
 +
Configuration        : b_malayi_PRJNA10729
 +
    Retrieved images : 173
 +
    Broken URLs      : 1
 +
Configuration        : b_xylophilus_PRJEA64437
 +
    Retrieved images : 82
 +
    Broken URLs      : 0
 +
Configuration        : c_angaria_PRJNA51225
 +
    Retrieved images : 120
 +
    Broken URLs      : 0
 +
Configuration        : c_brenneri_PRJNA20035
 +
    Retrieved images : 306
 +
    Broken URLs      : 0
 +
Configuration        : c_briggsae_PRJNA10731
 +
    Retrieved images : 374
 +
    Broken URLs      : 11
 +
Configuration        : c_elegans_PRJNA13758
 +
    Retrieved images : 7045
 +
    Broken URLs      : 1035
 +
Configuration        : c_japonica_PRJNA12591
 +
    Retrieved images : 270
 +
    Broken URLs      : 0
 +
Configuration        : c_remanei_PRJNA53967
 +
    Retrieved images : 293
 +
    Broken URLs      : 1
 +
Configuration        : c_sp11_PRJNA53597
 +
    Retrieved images : 123
 +
    Broken URLs      : 0
 +
Configuration        : c_sp5_PRJNA194557
 +
    Retrieved images : 82
 +
    Broken URLs      : 0
 +
Configuration        : h_bacteriophora_PRJNA13977
 +
    Retrieved images : 82
 +
    Broken URLs      : 0
 +
Configuration        : h_contortus_PRJEB506
 +
    Retrieved images : 123
 +
    Broken URLs      : 0
 +
Configuration        : l_loa_PRJNA60051
 +
    Retrieved images : 123
 +
    Broken URLs      : 0
 +
Configuration        : m_hapla_PRJNA29083
 +
    Retrieved images : 164
 +
    Broken URLs      : 0
 +
Configuration        : m_incognita_PRJEA28837
 +
    Retrieved images : 120
 +
    Broken URLs      : 0
 +
Configuration        : p_pacificus_PRJNA12644
 +
    Retrieved images : 180
 +
    Broken URLs      : 0
 +
Configuration        : s_ratti_PRJEA62033
 +
    Retrieved images : 82
 +
    Broken URLs      : 0
 +
Configuration        : t_spiralis_PRJNA12603
 +
    Retrieved images : 39
 +
    Broken URLs      : 2
 
</pre>
 
</pre>
 +
 +
 +
 +
[[Category:QA (Web Dev)]]

Latest revision as of 18:39, 18 June 2014

Installing Required Perl Modules

The following Perl modules need to be installed in order to run GBrowse tests in particular:

sudo cpan install Bio::Graphics
sudo cpan install Config::Simple
sudo cpan install boolean
sudo cpan install --force AnyEvent::CouchDB

Executing Unit Tests

  1. change directory into your Github Website clone.
  2. run tests of the REST API: perl t/rest.t
    1. run a particular unit test: perl t/rest.t --test=gene
  3. run tests of the WormBase Perl API: API_TESTS=1 perl t/api.t
    1. run a particular unit test: API_TESTS=protein perl t/api.t
  4. run comparative GBrowse tests: perl --base http://your.url.org:1234/cgi-bin/gb2/gbrowse

Successful tests are preceded by an "ok". For example:

ok 1 - require 't/rest_tests/template.t';
    ok 1 - port open
    1..1
ok 2 - test_port_open

How to read the output: The first "ok 1" means that the first executed test require ... completed sucessfully. "ok 2" shows that the second test was also successful, where sub-tests (i.e., tests within the test_port_open Perl sub) are shown above as indented lines. The interval "1..1" denotes the tests were run within test_port_open. In this example file "t/rest_tests/template.t" contains one subroutine called "test_port_open" which contains an assertion labelled "port open" which returned as true. Subroutine names appear below the results of the tests they contain.

Failed tests are preceded by a "not ok" and a short description is displayed that explains why the test failed (lines preceded by "#"). For example:

    not ok 1 - port open
    #   Failed test 'port open'
    #   at t/rest_tests/template.t line 53.
    #          got: undef
    #     expected: anything else
    1..1
    # Looks like you failed 1 test of 1.
not ok 2 - test_port_open

API Unit Testing Framework: How it works / Write your own

Unit tests are

  • based on Test::More.
  • run on a fully populated WormBase database backend.
  • API tests: start (and stop) a Catalyst web server (random port between 28,000 and 31,999)

Unit test templates are available as

  • t/rest_tests/template.t for writing REST API tests
  • t/api_tests/template.t for writing WormBase Perl API tests

New unit tests should be using the following workflow:

  1. create a copy of template.t within the same directory; choose a descriptive filename
  2. make appropriate changes to the newly created file

It is good to check out the testing section of this example Adding Data to a Widget (Example)#Formal Testing

Comparative Testing Framework: GBrowse

Comparative GBrowse tests are

  • based on a reference image set (one image per species, per track, per example landmark)
  • comparing a live gbrowse_img rendering to the respective reference image

The reference image set can be created via

A summary log and a full disclosure of broken URLs is written to the logfile logs/gbrowse_test.log.

Reference Set Generation, Current Output on Staging, WS239

Note: A. suum landmarks are still broken?

cd /var/lib/jenkins/jobs/gbrowse_track_tests/workspace
sudo -u jenkins perl t/gbrowse.t --cutoff 1 --reference --base http://staging.wormbase.org/tools/gen
ome/gbrowse_img
Configuration        : a_suum_PRJNA62057
    Retrieved images : 82
    Broken URLs      : 0
Configuration        : a_suum_PRJNA80881
    Retrieved images : 158
    Broken URLs      : 82
Configuration        : b_malayi_PRJNA10729
    Retrieved images : 174
    Broken URLs      : 0
Configuration        : b_xylophilus_PRJEA64437
    Retrieved images : 82
    Broken URLs      : 0
Configuration        : c_angaria_PRJNA51225
    Retrieved images : 120
    Broken URLs      : 0
Configuration        : c_brenneri_PRJNA20035
    Retrieved images : 306
    Broken URLs      : 0
Configuration        : c_briggsae_PRJNA10731
    Retrieved images : 385
    Broken URLs      : 0
Configuration        : c_elegans_PRJNA13758
    Retrieved images : 0
    Broken URLs      : 8080
Configuration        : c_japonica_PRJNA12591
    Retrieved images : 270
    Broken URLs      : 0
Configuration        : c_remanei_PRJNA53967
    Retrieved images : 294
    Broken URLs      : 0
Configuration        : c_sp11_PRJNA53597
    Retrieved images : 123
    Broken URLs      : 0
Configuration        : c_sp5_PRJNA194557
    Retrieved images : 82
    Broken URLs      : 0
Configuration        : h_bacteriophora_PRJNA13977
    Retrieved images : 82
    Broken URLs      : 0
Configuration        : h_contortus_PRJEB506
    Retrieved images : 123
    Broken URLs      : 0
Configuration        : h_contortus_PRJNA205202
    Retrieved images : 0
    Broken URLs      : 123
Configuration        : l_loa_PRJNA60051
    Retrieved images : 123
    Broken URLs      : 0
Configuration        : m_hapla_PRJNA29083
    Retrieved images : 164
    Broken URLs      : 0
Configuration        : m_incognita_PRJEA28837
    Retrieved images : 120
    Broken URLs      : 0
Configuration        : p_pacificus_PRJNA12644
    Retrieved images : 180
    Broken URLs      : 0
Configuration        : s_ratti_PRJEA62033
    Retrieved images : 82
    Broken URLs      : 0
Configuration        : t_spiralis_PRJNA12603
    Retrieved images : 41
    Broken URLs      : 0

Reference Set Generation, Current Output on Staging, WS238

cd /var/lib/jenkins/jobs/gbrowse_track_tests/workspace
sudo -u jenkins perl t/gbrowse.t --cutoff 1 --reference --base http://staging.wormbase.org/tools/genome/gbrowse_img
Configuration        : a_suum_PRJNA62057
    Retrieved images : 82
    Broken URLs      : 0
Configuration        : a_suum_PRJNA80881
    Retrieved images : 160
    Broken URLs      : 80
Configuration        : b_malayi_PRJNA10729
    Retrieved images : 173
    Broken URLs      : 1
Configuration        : b_xylophilus_PRJEA64437
    Retrieved images : 82
    Broken URLs      : 0
Configuration        : c_angaria_PRJNA51225
    Retrieved images : 120
    Broken URLs      : 0
Configuration        : c_brenneri_PRJNA20035
    Retrieved images : 306
    Broken URLs      : 0
Configuration        : c_briggsae_PRJNA10731
    Retrieved images : 374
    Broken URLs      : 11
Configuration        : c_elegans_PRJNA13758
    Retrieved images : 7045
    Broken URLs      : 1035
Configuration        : c_japonica_PRJNA12591
    Retrieved images : 270
    Broken URLs      : 0
Configuration        : c_remanei_PRJNA53967
    Retrieved images : 293
    Broken URLs      : 1
Configuration        : c_sp11_PRJNA53597
    Retrieved images : 123
    Broken URLs      : 0
Configuration        : c_sp5_PRJNA194557
    Retrieved images : 82
    Broken URLs      : 0
Configuration        : h_bacteriophora_PRJNA13977
    Retrieved images : 82
    Broken URLs      : 0
Configuration        : h_contortus_PRJEB506
    Retrieved images : 123
    Broken URLs      : 0
Configuration        : l_loa_PRJNA60051
    Retrieved images : 123
    Broken URLs      : 0
Configuration        : m_hapla_PRJNA29083
    Retrieved images : 164
    Broken URLs      : 0
Configuration        : m_incognita_PRJEA28837
    Retrieved images : 120
    Broken URLs      : 0
Configuration        : p_pacificus_PRJNA12644
    Retrieved images : 180
    Broken URLs      : 0
Configuration        : s_ratti_PRJEA62033
    Retrieved images : 82
    Broken URLs      : 0
Configuration        : t_spiralis_PRJNA12603
    Retrieved images : 39
    Broken URLs      : 2