Difference between revisions of "Unit Testing"
(Added information about the logfile that is kept for GBrowse tests.) |
|||
(9 intermediate revisions by 3 users not shown) | |||
Line 1: | Line 1: | ||
+ | === Installing Required Perl Modules === | ||
+ | |||
+ | The following Perl modules need to be installed in order to run GBrowse tests in particular: | ||
+ | |||
+ | <pre> | ||
+ | sudo cpan install Bio::Graphics | ||
+ | sudo cpan install Config::Simple | ||
+ | sudo cpan install boolean | ||
+ | sudo cpan install --force AnyEvent::CouchDB | ||
+ | </pre> | ||
+ | |||
=== Executing Unit Tests === | === Executing Unit Tests === | ||
# change directory into your [https://github.com/WormBase/website Github Website] clone. | # change directory into your [https://github.com/WormBase/website Github Website] clone. | ||
# run tests of the REST API: <code>perl t/rest.t</code> | # run tests of the REST API: <code>perl t/rest.t</code> | ||
+ | ## run a particular unit test: <code>perl t/rest.t --test=gene</code> | ||
# run tests of the WormBase Perl API: <code>API_TESTS=1 perl t/api.t</code> | # run tests of the WormBase Perl API: <code>API_TESTS=1 perl t/api.t</code> | ||
+ | ## run a particular unit test: <code>API_TESTS=protein perl t/api.t</code> | ||
# run comparative GBrowse tests: <code>perl --base http://your.url.org:1234/cgi-bin/gb2/gbrowse</code> | # run comparative GBrowse tests: <code>perl --base http://your.url.org:1234/cgi-bin/gb2/gbrowse</code> | ||
Line 15: | Line 28: | ||
</pre> | </pre> | ||
− | '''How to read the output:''' The first "ok 1" means that the first executed test <code>require ...</code> completed sucessfully. "ok 2" shows that the second test was also successful, where sub-tests (i.e., tests within the <code>test_port_open</code> Perl sub) are shown ''above'' as indented lines. The interval "1..1" denotes the tests were run within <code>test_port_open</code>. | + | '''How to read the output:''' The first "ok 1" means that the first executed test <code>require ...</code> completed sucessfully. "ok 2" shows that the second test was also successful, where sub-tests (i.e., tests within the <code>test_port_open</code> Perl sub) are shown ''above'' as indented lines. The interval "1..1" denotes the tests were run within <code>test_port_open</code>. In this example file "t/rest_tests/template.t" contains one subroutine called "test_port_open" which contains an assertion labelled "port open" which returned as true. Subroutine names appear below the results of the tests they contain. |
'''Failed tests''' are preceded by a "not ok" and a short description is displayed that explains why the test failed (lines preceded by "#"). For example: | '''Failed tests''' are preceded by a "not ok" and a short description is displayed that explains why the test failed (lines preceded by "#"). For example: | ||
Line 47: | Line 60: | ||
# create a copy of <code>template.t</code> within the same directory; choose a descriptive filename | # create a copy of <code>template.t</code> within the same directory; choose a descriptive filename | ||
# make appropriate changes to the newly created file | # make appropriate changes to the newly created file | ||
+ | |||
+ | It is good to check out the testing section of this example [[Adding Data to a Widget (Example)#Formal Testing]] | ||
=== Comparative Testing Framework: GBrowse === | === Comparative Testing Framework: GBrowse === | ||
Line 57: | Line 72: | ||
The reference image set can be created via | The reference image set can be created via | ||
− | * <code>perl --base http://your.url.org:1234/cgi-bin/gb2/gbrowse</code> | + | * <code>perl --cutoff 1 --reference --base http://your.url.org:1234/cgi-bin/gb2/gbrowse</code> |
A summary log and a full disclosure of broken URLs is written to the logfile <code>logs/gbrowse_test.log</code>. | A summary log and a full disclosure of broken URLs is written to the logfile <code>logs/gbrowse_test.log</code>. | ||
− | ==== Current Output on Staging, | + | ==== Reference Set Generation, Current Output on Staging, WS239 ==== |
+ | |||
+ | '''Note:''' A. suum landmarks are still broken? | ||
<pre> | <pre> | ||
− | + | cd /var/lib/jenkins/jobs/gbrowse_track_tests/workspace | |
+ | sudo -u jenkins perl t/gbrowse.t --cutoff 1 --reference --base http://staging.wormbase.org/tools/gen | ||
+ | ome/gbrowse_img | ||
Configuration : a_suum_PRJNA62057 | Configuration : a_suum_PRJNA62057 | ||
Retrieved images : 82 | Retrieved images : 82 | ||
Broken URLs : 0 | Broken URLs : 0 | ||
Configuration : a_suum_PRJNA80881 | Configuration : a_suum_PRJNA80881 | ||
− | + | Retrieved images : 158 | |
+ | Broken URLs : 82 | ||
Configuration : b_malayi_PRJNA10729 | Configuration : b_malayi_PRJNA10729 | ||
Retrieved images : 174 | Retrieved images : 174 | ||
Line 83: | Line 103: | ||
Broken URLs : 0 | Broken URLs : 0 | ||
Configuration : c_briggsae_PRJNA10731 | Configuration : c_briggsae_PRJNA10731 | ||
− | Retrieved images : | + | Retrieved images : 385 |
− | Broken URLs : | + | Broken URLs : 0 |
Configuration : c_elegans_PRJNA13758 | Configuration : c_elegans_PRJNA13758 | ||
− | + | Retrieved images : 0 | |
+ | Broken URLs : 8080 | ||
Configuration : c_japonica_PRJNA12591 | Configuration : c_japonica_PRJNA12591 | ||
Retrieved images : 270 | Retrieved images : 270 | ||
Line 105: | Line 126: | ||
Retrieved images : 123 | Retrieved images : 123 | ||
Broken URLs : 0 | Broken URLs : 0 | ||
+ | Configuration : h_contortus_PRJNA205202 | ||
+ | Retrieved images : 0 | ||
+ | Broken URLs : 123 | ||
Configuration : l_loa_PRJNA60051 | Configuration : l_loa_PRJNA60051 | ||
Retrieved images : 123 | Retrieved images : 123 | ||
Line 121: | Line 145: | ||
Broken URLs : 0 | Broken URLs : 0 | ||
Configuration : t_spiralis_PRJNA12603 | Configuration : t_spiralis_PRJNA12603 | ||
− | + | Retrieved images : 41 | |
+ | Broken URLs : 0 | ||
+ | </pre> | ||
+ | |||
+ | ==== Reference Set Generation, Current Output on Staging, WS238 ==== | ||
− | + | <pre> | |
− | + | cd /var/lib/jenkins/jobs/gbrowse_track_tests/workspace | |
− | + | sudo -u jenkins perl t/gbrowse.t --cutoff 1 --reference --base http://staging.wormbase.org/tools/genome/gbrowse_img | |
+ | Configuration : a_suum_PRJNA62057 | ||
+ | Retrieved images : 82 | ||
+ | Broken URLs : 0 | ||
+ | Configuration : a_suum_PRJNA80881 | ||
+ | Retrieved images : 160 | ||
+ | Broken URLs : 80 | ||
+ | Configuration : b_malayi_PRJNA10729 | ||
+ | Retrieved images : 173 | ||
+ | Broken URLs : 1 | ||
+ | Configuration : b_xylophilus_PRJEA64437 | ||
+ | Retrieved images : 82 | ||
+ | Broken URLs : 0 | ||
+ | Configuration : c_angaria_PRJNA51225 | ||
+ | Retrieved images : 120 | ||
+ | Broken URLs : 0 | ||
+ | Configuration : c_brenneri_PRJNA20035 | ||
+ | Retrieved images : 306 | ||
+ | Broken URLs : 0 | ||
+ | Configuration : c_briggsae_PRJNA10731 | ||
+ | Retrieved images : 374 | ||
+ | Broken URLs : 11 | ||
+ | Configuration : c_elegans_PRJNA13758 | ||
+ | Retrieved images : 7045 | ||
+ | Broken URLs : 1035 | ||
+ | Configuration : c_japonica_PRJNA12591 | ||
+ | Retrieved images : 270 | ||
+ | Broken URLs : 0 | ||
+ | Configuration : c_remanei_PRJNA53967 | ||
+ | Retrieved images : 293 | ||
+ | Broken URLs : 1 | ||
+ | Configuration : c_sp11_PRJNA53597 | ||
+ | Retrieved images : 123 | ||
+ | Broken URLs : 0 | ||
+ | Configuration : c_sp5_PRJNA194557 | ||
+ | Retrieved images : 82 | ||
+ | Broken URLs : 0 | ||
+ | Configuration : h_bacteriophora_PRJNA13977 | ||
+ | Retrieved images : 82 | ||
+ | Broken URLs : 0 | ||
+ | Configuration : h_contortus_PRJEB506 | ||
+ | Retrieved images : 123 | ||
+ | Broken URLs : 0 | ||
+ | Configuration : l_loa_PRJNA60051 | ||
+ | Retrieved images : 123 | ||
+ | Broken URLs : 0 | ||
+ | Configuration : m_hapla_PRJNA29083 | ||
+ | Retrieved images : 164 | ||
+ | Broken URLs : 0 | ||
+ | Configuration : m_incognita_PRJEA28837 | ||
+ | Retrieved images : 120 | ||
+ | Broken URLs : 0 | ||
+ | Configuration : p_pacificus_PRJNA12644 | ||
+ | Retrieved images : 180 | ||
+ | Broken URLs : 0 | ||
+ | Configuration : s_ratti_PRJEA62033 | ||
+ | Retrieved images : 82 | ||
+ | Broken URLs : 0 | ||
+ | Configuration : t_spiralis_PRJNA12603 | ||
+ | Retrieved images : 39 | ||
+ | Broken URLs : 2 | ||
</pre> | </pre> | ||
+ | |||
+ | |||
+ | |||
+ | [[Category:QA (Web Dev)]] |
Latest revision as of 18:39, 18 June 2014
Contents
Installing Required Perl Modules
The following Perl modules need to be installed in order to run GBrowse tests in particular:
sudo cpan install Bio::Graphics sudo cpan install Config::Simple sudo cpan install boolean sudo cpan install --force AnyEvent::CouchDB
Executing Unit Tests
- change directory into your Github Website clone.
- run tests of the REST API:
perl t/rest.t
- run a particular unit test:
perl t/rest.t --test=gene
- run a particular unit test:
- run tests of the WormBase Perl API:
API_TESTS=1 perl t/api.t
- run a particular unit test:
API_TESTS=protein perl t/api.t
- run a particular unit test:
- run comparative GBrowse tests:
perl --base http://your.url.org:1234/cgi-bin/gb2/gbrowse
Successful tests are preceded by an "ok". For example:
ok 1 - require 't/rest_tests/template.t'; ok 1 - port open 1..1 ok 2 - test_port_open
How to read the output: The first "ok 1" means that the first executed test require ...
completed sucessfully. "ok 2" shows that the second test was also successful, where sub-tests (i.e., tests within the test_port_open
Perl sub) are shown above as indented lines. The interval "1..1" denotes the tests were run within test_port_open
. In this example file "t/rest_tests/template.t" contains one subroutine called "test_port_open" which contains an assertion labelled "port open" which returned as true. Subroutine names appear below the results of the tests they contain.
Failed tests are preceded by a "not ok" and a short description is displayed that explains why the test failed (lines preceded by "#"). For example:
not ok 1 - port open # Failed test 'port open' # at t/rest_tests/template.t line 53. # got: undef # expected: anything else 1..1 # Looks like you failed 1 test of 1. not ok 2 - test_port_open
API Unit Testing Framework: How it works / Write your own
Unit tests are
- based on Test::More.
- run on a fully populated WormBase database backend.
- API tests: start (and stop) a Catalyst web server (random port between 28,000 and 31,999)
Unit test templates are available as
t/rest_tests/template.t
for writing REST API testst/api_tests/template.t
for writing WormBase Perl API tests
New unit tests should be using the following workflow:
- create a copy of
template.t
within the same directory; choose a descriptive filename - make appropriate changes to the newly created file
It is good to check out the testing section of this example Adding Data to a Widget (Example)#Formal Testing
Comparative Testing Framework: GBrowse
Comparative GBrowse tests are
- based on a reference image set (one image per species, per track, per example landmark)
- comparing a live
gbrowse_img
rendering to the respective reference image
The reference image set can be created via
perl --cutoff 1 --reference --base http://your.url.org:1234/cgi-bin/gb2/gbrowse
A summary log and a full disclosure of broken URLs is written to the logfile logs/gbrowse_test.log
.
Reference Set Generation, Current Output on Staging, WS239
Note: A. suum landmarks are still broken?
cd /var/lib/jenkins/jobs/gbrowse_track_tests/workspace sudo -u jenkins perl t/gbrowse.t --cutoff 1 --reference --base http://staging.wormbase.org/tools/gen ome/gbrowse_img Configuration : a_suum_PRJNA62057 Retrieved images : 82 Broken URLs : 0 Configuration : a_suum_PRJNA80881 Retrieved images : 158 Broken URLs : 82 Configuration : b_malayi_PRJNA10729 Retrieved images : 174 Broken URLs : 0 Configuration : b_xylophilus_PRJEA64437 Retrieved images : 82 Broken URLs : 0 Configuration : c_angaria_PRJNA51225 Retrieved images : 120 Broken URLs : 0 Configuration : c_brenneri_PRJNA20035 Retrieved images : 306 Broken URLs : 0 Configuration : c_briggsae_PRJNA10731 Retrieved images : 385 Broken URLs : 0 Configuration : c_elegans_PRJNA13758 Retrieved images : 0 Broken URLs : 8080 Configuration : c_japonica_PRJNA12591 Retrieved images : 270 Broken URLs : 0 Configuration : c_remanei_PRJNA53967 Retrieved images : 294 Broken URLs : 0 Configuration : c_sp11_PRJNA53597 Retrieved images : 123 Broken URLs : 0 Configuration : c_sp5_PRJNA194557 Retrieved images : 82 Broken URLs : 0 Configuration : h_bacteriophora_PRJNA13977 Retrieved images : 82 Broken URLs : 0 Configuration : h_contortus_PRJEB506 Retrieved images : 123 Broken URLs : 0 Configuration : h_contortus_PRJNA205202 Retrieved images : 0 Broken URLs : 123 Configuration : l_loa_PRJNA60051 Retrieved images : 123 Broken URLs : 0 Configuration : m_hapla_PRJNA29083 Retrieved images : 164 Broken URLs : 0 Configuration : m_incognita_PRJEA28837 Retrieved images : 120 Broken URLs : 0 Configuration : p_pacificus_PRJNA12644 Retrieved images : 180 Broken URLs : 0 Configuration : s_ratti_PRJEA62033 Retrieved images : 82 Broken URLs : 0 Configuration : t_spiralis_PRJNA12603 Retrieved images : 41 Broken URLs : 0
Reference Set Generation, Current Output on Staging, WS238
cd /var/lib/jenkins/jobs/gbrowse_track_tests/workspace sudo -u jenkins perl t/gbrowse.t --cutoff 1 --reference --base http://staging.wormbase.org/tools/genome/gbrowse_img Configuration : a_suum_PRJNA62057 Retrieved images : 82 Broken URLs : 0 Configuration : a_suum_PRJNA80881 Retrieved images : 160 Broken URLs : 80 Configuration : b_malayi_PRJNA10729 Retrieved images : 173 Broken URLs : 1 Configuration : b_xylophilus_PRJEA64437 Retrieved images : 82 Broken URLs : 0 Configuration : c_angaria_PRJNA51225 Retrieved images : 120 Broken URLs : 0 Configuration : c_brenneri_PRJNA20035 Retrieved images : 306 Broken URLs : 0 Configuration : c_briggsae_PRJNA10731 Retrieved images : 374 Broken URLs : 11 Configuration : c_elegans_PRJNA13758 Retrieved images : 7045 Broken URLs : 1035 Configuration : c_japonica_PRJNA12591 Retrieved images : 270 Broken URLs : 0 Configuration : c_remanei_PRJNA53967 Retrieved images : 293 Broken URLs : 1 Configuration : c_sp11_PRJNA53597 Retrieved images : 123 Broken URLs : 0 Configuration : c_sp5_PRJNA194557 Retrieved images : 82 Broken URLs : 0 Configuration : h_bacteriophora_PRJNA13977 Retrieved images : 82 Broken URLs : 0 Configuration : h_contortus_PRJEB506 Retrieved images : 123 Broken URLs : 0 Configuration : l_loa_PRJNA60051 Retrieved images : 123 Broken URLs : 0 Configuration : m_hapla_PRJNA29083 Retrieved images : 164 Broken URLs : 0 Configuration : m_incognita_PRJEA28837 Retrieved images : 120 Broken URLs : 0 Configuration : p_pacificus_PRJNA12644 Retrieved images : 180 Broken URLs : 0 Configuration : s_ratti_PRJEA62033 Retrieved images : 82 Broken URLs : 0 Configuration : t_spiralis_PRJNA12603 Retrieved images : 39 Broken URLs : 2