Difference between revisions of "Website:WormMine Builds"
(Added preliminary pre-processing. instructions) |
|||
(14 intermediate revisions by 3 users not shown) | |||
Line 1: | Line 1: | ||
+ | === WS244 === | ||
+ | |||
+ | ==== Build ==== | ||
+ | |||
+ | ===== AceDB Dump ===== | ||
+ | |||
+ | Host: dev.wormbase.org | ||
+ | |||
+ | User: intermine | ||
+ | |||
+ | Path: <code> /home/intermine</code> | ||
+ | |||
+ | Command: <code>./website-intermine/scripts/dump_ace.sh /mnt/ephemeral0/acedb_dump_ws244</code> | ||
+ | |||
+ | Note: DO NOT add a trailing backslash after the path | ||
+ | |||
+ | Output | ||
+ | |||
+ | <pre> | ||
+ | ./website-intermine/scripts/dump_ace.sh /mnt/ephemeral0/acedb_dump_ws244 | ||
+ | Did not specify AceDB dir in $ACEDB. Searching for tace... | ||
+ | ACEDB set to: /usr/local/wormbase/acedb/wormbase | ||
+ | Transcript | ||
+ | |||
+ | ... done. | ||
+ | Variation | ||
+ | ... done. | ||
+ | Phenotype | ||
+ | ... done. | ||
+ | Expr_pattern | ||
+ | ... done. | ||
+ | Expression_cluster | ||
+ | ... done. | ||
+ | Anatomy_term | ||
+ | ... done. | ||
+ | Expr_cluster | ||
+ | ... done. | ||
+ | Life_stage | ||
+ | ... done. | ||
+ | RNAi | ||
+ | ... done. | ||
+ | |||
+ | **** Program tace, compiled on: May 14 2010 09:21:02 **** | ||
+ | **** Using ACEDB 4.9.52, compiled on: May 14 2010 09:20:07 **** | ||
+ | |||
+ | Code by: Jean Thierry-Mieg (CNRS, France) mieg@crbm.cnrs-mop.fr | ||
+ | Richard Durbin (Sanger Centre, UK) rd@sanger.ac.uk | ||
+ | Ed Griffiths (Sanger Centre, UK) edgrif@sanger.ac.uk | ||
+ | |||
+ | You may redistribute this program and database subject to the | ||
+ | conditions in the accompanying copyright file. Anyone interested in | ||
+ | maintaining an up-to-date version should contact one of the authors | ||
+ | at the above email addresses. | ||
+ | |||
+ | // Database directory: /usr/local/wormbase/acedb/wormbase | ||
+ | // Session 594 is readlocked by acabunoc since 2014-08-11_18:29:04 using tace (process 31767 on machine dev.wormbase.org) | ||
+ | // Session 594 is readlocked by getLoginFailed since 2014-08-11_19:28:09 using tace (process 672 on machine dev.wormbase.org) | ||
+ | acedb> | ||
+ | // Found 223435 objects | ||
+ | // 223435 Active Objects | ||
+ | acedb> // 223435 objects dumped | ||
+ | // 223435 Active Objects | ||
+ | acedb> | ||
+ | // Found 199169 objects | ||
+ | // 199169 Active Objects | ||
+ | acedb> // 199169 objects dumped | ||
+ | // 199169 Active Objects | ||
+ | acedb> | ||
+ | // Found 201722 objects | ||
+ | // 201722 Active Objects | ||
+ | acedb> // 201722 objects dumped | ||
+ | // 201722 Active Objects | ||
+ | acedb> | ||
+ | // Found 239936 objects | ||
+ | // 239936 Active Objects | ||
+ | acedb> // 239936 objects dumped | ||
+ | // 239936 Active Objects | ||
+ | acedb> // ERROR - Failed to open for reading: /home/intermine/website-intermine/acedb-dev/acedb/species.ace (system error 2 - No such file or directory) | ||
+ | // Sorry, I could not open file /home/intermine/website-intermine/acedb-dev/acedb/species.ace | ||
+ | // 239936 Active Objects | ||
+ | acedb> // 239936 objects dumped | ||
+ | // 239936 Active Objects | ||
+ | acedb> // 239936 Active Objects | ||
+ | acedb> | ||
+ | // A bientot | ||
+ | </pre> | ||
+ | |||
+ | |||
+ | |||
+ | ===== Pre-Processing ===== | ||
+ | |||
+ | Host: WormMine dev server (206.108.125.174) | ||
+ | |||
+ | User: intermine | ||
+ | |||
+ | Path: <code>/nfs/wormbase/data/acedmp</code> | ||
+ | |||
+ | Commands: | ||
+ | |||
+ | <pre> | ||
+ | mkdir WS244 | ||
+ | cd WS244 | ||
+ | scp acabunoc@dev.wormbase.org:/mnt/ephemeral0/acedb_dump_ws244/* . | ||
+ | cd /home/intermine/website-intermine/acedb-dev/intermine/redeployment | ||
+ | # Set "release = WS244" in update.properties | ||
+ | ant | ||
+ | </pre> | ||
+ | |||
+ | Build failed with error: | ||
+ | <code>/nfs/wormbase/data/intermine/redeployment/build.xml:68: Property 'bpid-c_sp11' is not defined.</code> | ||
+ | |||
+ | Fixed by changing c_sp11 to c_tropicalis in <code>genomic-fasta-species.txt</code>, <code>protein-fasta-species.txt</code> and <code>gff3-species.txt</code> | ||
+ | |||
+ | |||
+ | |||
+ | |||
+ | ===== InterMine Build ===== | ||
+ | |||
+ | Host: WormMine dev server (206.108.125.174) | ||
+ | |||
+ | User: intermine | ||
+ | |||
+ | Path: <code>/home/intermine/website-intermine</code> | ||
+ | |||
+ | Command: <code>./scripts/build_intermine.sh WS240</code> | ||
+ | |||
+ | Output (stdout, stderr not shown): | ||
+ | |||
+ | <pre> | ||
+ | USING RELEASE: WS244 | ||
+ | DATABASE NAME: wormmine_ws244 | ||
+ | Updating ~/.intermine/wormmine.properties... | ||
+ | Updating project.xml... | ||
+ | Building the InterMine database... | ||
+ | </pre> | ||
+ | |||
+ | <span style="color: red">'''Quick fix for resolving missing organisms.xml file:'''</span> | ||
+ | |||
+ | <pre> | ||
+ | cd /nfs/wormbase/data/intermine/datadir | ||
+ | mkdir entrez-organism | ||
+ | cp /home/intermine/organisms.xml entrez-organism | ||
+ | </pre> | ||
+ | |||
+ | |||
+ | |||
=== WS240 === | === WS240 === | ||
Line 7: | Line 153: | ||
Host: Website dev server | Host: Website dev server | ||
− | Path: <code>/home/ | + | User: intermine |
+ | |||
+ | Path: <code>/home/intermine</code> | ||
Command: <code>./website-intermine/scripts/dump_ace.sh /mnt/ephemeral0/acedb_dump_ws240</code> | Command: <code>./website-intermine/scripts/dump_ace.sh /mnt/ephemeral0/acedb_dump_ws240</code> | ||
Line 81: | Line 229: | ||
Host: WormMine dev server (206.108.125.174) | Host: WormMine dev server (206.108.125.174) | ||
+ | |||
+ | User: intermine | ||
Path: <code>/nfs/wormbase/data/acedmp</code> | Path: <code>/nfs/wormbase/data/acedmp</code> | ||
− | |||
− | |||
Commands: | Commands: | ||
Line 96: | Line 244: | ||
ant | ant | ||
</pre> | </pre> | ||
+ | |||
+ | Output: ''too long to show here'' | ||
+ | |||
+ | ===== InterMine Build ===== | ||
+ | |||
+ | Host: WormMine dev server (206.108.125.174) | ||
+ | |||
+ | User: intermine | ||
+ | |||
+ | Path: <code>/home/intermine/website-intermine</code> | ||
+ | |||
+ | Command: <code>./scripts/build_intermine.sh WS240</code> | ||
+ | |||
+ | Output (stdout, stderr not shown): | ||
+ | |||
+ | <pre> | ||
+ | USING RELEASE: WS240 | ||
+ | DATABASE NAME: wormmine_ws240 | ||
+ | Updating ~/.intermine/wormmine.properties... | ||
+ | Updating project.xml... | ||
+ | Building the InterMine database... | ||
+ | |||
+ | action name time in seconds | ||
+ | ------------------------------------------------------------- | ||
+ | entrez-organism 130 | ||
+ | create-chromosome-locations-and-lengths 1075 | ||
+ | do-sources 51 | ||
+ | create-search-index 3747 | ||
+ | summarise-objectstore 740 | ||
+ | create-autocomplete-index 41 | ||
+ | create-attribute-indexes 191 | ||
+ | transfer-sequences 209663 | ||
+ | final-dump 1246 | ||
+ | |||
+ | total 216884 | ||
+ | </pre> | ||
+ | |||
+ | <span style="color: red">'''Quick fix for resolving missing organisms.xml file:'''</span> | ||
+ | |||
+ | <pre> | ||
+ | cd /nfs/wormbase/data/intermine/datadir | ||
+ | mkdir entrez-organism | ||
+ | cp /home/intermine/organisms.xml entrez-organism | ||
+ | </pre> | ||
+ | |||
+ | ===== Deployment ===== | ||
+ | |||
+ | TBD | ||
=== WS239 === | === WS239 === | ||
Line 142: | Line 338: | ||
1.3G joachim_ws239_2.final + 17G PostgreSQL DB | 1.3G joachim_ws239_2.final + 17G PostgreSQL DB | ||
+ | |||
+ | [[Category: Architecture (Web Dev)]] |
Latest revision as of 18:27, 12 August 2014
Contents
WS244
Build
AceDB Dump
Host: dev.wormbase.org
User: intermine
Path: /home/intermine
Command: ./website-intermine/scripts/dump_ace.sh /mnt/ephemeral0/acedb_dump_ws244
Note: DO NOT add a trailing backslash after the path
Output
./website-intermine/scripts/dump_ace.sh /mnt/ephemeral0/acedb_dump_ws244 Did not specify AceDB dir in $ACEDB. Searching for tace... ACEDB set to: /usr/local/wormbase/acedb/wormbase Transcript ... done. Variation ... done. Phenotype ... done. Expr_pattern ... done. Expression_cluster ... done. Anatomy_term ... done. Expr_cluster ... done. Life_stage ... done. RNAi ... done. **** Program tace, compiled on: May 14 2010 09:21:02 **** **** Using ACEDB 4.9.52, compiled on: May 14 2010 09:20:07 **** Code by: Jean Thierry-Mieg (CNRS, France) mieg@crbm.cnrs-mop.fr Richard Durbin (Sanger Centre, UK) rd@sanger.ac.uk Ed Griffiths (Sanger Centre, UK) edgrif@sanger.ac.uk You may redistribute this program and database subject to the conditions in the accompanying copyright file. Anyone interested in maintaining an up-to-date version should contact one of the authors at the above email addresses. // Database directory: /usr/local/wormbase/acedb/wormbase // Session 594 is readlocked by acabunoc since 2014-08-11_18:29:04 using tace (process 31767 on machine dev.wormbase.org) // Session 594 is readlocked by getLoginFailed since 2014-08-11_19:28:09 using tace (process 672 on machine dev.wormbase.org) acedb> // Found 223435 objects // 223435 Active Objects acedb> // 223435 objects dumped // 223435 Active Objects acedb> // Found 199169 objects // 199169 Active Objects acedb> // 199169 objects dumped // 199169 Active Objects acedb> // Found 201722 objects // 201722 Active Objects acedb> // 201722 objects dumped // 201722 Active Objects acedb> // Found 239936 objects // 239936 Active Objects acedb> // 239936 objects dumped // 239936 Active Objects acedb> // ERROR - Failed to open for reading: /home/intermine/website-intermine/acedb-dev/acedb/species.ace (system error 2 - No such file or directory) // Sorry, I could not open file /home/intermine/website-intermine/acedb-dev/acedb/species.ace // 239936 Active Objects acedb> // 239936 objects dumped // 239936 Active Objects acedb> // 239936 Active Objects acedb> // A bientot
Pre-Processing
Host: WormMine dev server (206.108.125.174)
User: intermine
Path: /nfs/wormbase/data/acedmp
Commands:
mkdir WS244 cd WS244 scp acabunoc@dev.wormbase.org:/mnt/ephemeral0/acedb_dump_ws244/* . cd /home/intermine/website-intermine/acedb-dev/intermine/redeployment # Set "release = WS244" in update.properties ant
Build failed with error:
/nfs/wormbase/data/intermine/redeployment/build.xml:68: Property 'bpid-c_sp11' is not defined.
Fixed by changing c_sp11 to c_tropicalis in genomic-fasta-species.txt
, protein-fasta-species.txt
and gff3-species.txt
InterMine Build
Host: WormMine dev server (206.108.125.174)
User: intermine
Path: /home/intermine/website-intermine
Command: ./scripts/build_intermine.sh WS240
Output (stdout, stderr not shown):
USING RELEASE: WS244 DATABASE NAME: wormmine_ws244 Updating ~/.intermine/wormmine.properties... Updating project.xml... Building the InterMine database...
Quick fix for resolving missing organisms.xml file:
cd /nfs/wormbase/data/intermine/datadir mkdir entrez-organism cp /home/intermine/organisms.xml entrez-organism
WS240
Build
ACeDB Dump
Host: Website dev server
User: intermine
Path: /home/intermine
Command: ./website-intermine/scripts/dump_ace.sh /mnt/ephemeral0/acedb_dump_ws240
Output:
Did not specify AceDB dir in $ACEDB. Searching for tace... ACEDB set to: /usr/local/wormbase/acedb/wormbase Transcript ... done. Variation ... done. Phenotype ... done. Expr_pattern ... done. Expression_cluster ... done. Anatomy_term ... done. Expr_cluster ... done. Life_stage ... done. RNAi ... done. **** Program tace, compiled on: May 14 2010 09:21:02 **** **** Using ACEDB 4.9.52, compiled on: May 14 2010 09:20:07 **** Code by: Jean Thierry-Mieg (CNRS, France) mieg@crbm.cnrs-mop.fr Richard Durbin (Sanger Centre, UK) rd@sanger.ac.uk Ed Griffiths (Sanger Centre, UK) edgrif@sanger.ac.uk You may redistribute this program and database subject to the conditions in the accompanying copyright file. Anyone interested in maintaining an up-to-date version should contact one of the authors at the above email addresses. // Database directory: /usr/local/wormbase/acedb/wormbase acedb> // Found 204526 objects // 204526 Active Objects acedb> // 204526 objects dumped // 204526 Active Objects acedb> // Found 186343 objects // 186343 Active Objects acedb> // 186343 objects dumped // 186343 Active Objects acedb> // Found 188862 objects // 188862 Active Objects acedb> // 188862 objects dumped // 188862 Active Objects acedb> // Found 221196 objects // 221196 Active Objects acedb> // 221196 objects dumped // 221196 Active Objects acedb> // ERROR - Failed to open for reading: /home/jbaran/src/website-intermine/acedb-dev/acedb/species.ace (system error 2 - No such file or directory) // Sorry, I could not open file /home/jbaran/src/website-intermine/acedb-dev/acedb/species.ace // 221196 Active Objects acedb> // 221196 objects dumped // 221196 Active Objects acedb> // 221196 Active Objects acedb> // A bientot
Pre-Processing
Host: WormMine dev server (206.108.125.174)
User: intermine
Path: /nfs/wormbase/data/acedmp
Commands:
mkdir WS240 cd WS240 scp jbaran@dev.wormbase.org:/mnt/ephemeral0/acedb_dump_ws240/* . cd /home/intermine/website-intermine/acedb-dev/intermine/redeployment # Set "release = WS240" in update.properties ant
Output: too long to show here
InterMine Build
Host: WormMine dev server (206.108.125.174)
User: intermine
Path: /home/intermine/website-intermine
Command: ./scripts/build_intermine.sh WS240
Output (stdout, stderr not shown):
USING RELEASE: WS240 DATABASE NAME: wormmine_ws240 Updating ~/.intermine/wormmine.properties... Updating project.xml... Building the InterMine database... action name time in seconds ------------------------------------------------------------- entrez-organism 130 create-chromosome-locations-and-lengths 1075 do-sources 51 create-search-index 3747 summarise-objectstore 740 create-autocomplete-index 41 create-attribute-indexes 191 transfer-sequences 209663 final-dump 1246 total 216884
Quick fix for resolving missing organisms.xml file:
cd /nfs/wormbase/data/intermine/datadir mkdir entrez-organism cp /home/intermine/organisms.xml entrez-organism
Deployment
TBD
WS239
Build
Host: WormMine dev server (206.108.125.174)
Path: /home/jbaran/src/website-intermine/acedb-dev/intermine/wormmine
Command: ../bio/scripts/project_build -l -v localhost joachim_ws239_2
Summary:
action name time in seconds ------------------------------------------------------------- entrez-organism 95 create-chromosome-locations-and-lengths 708 do-sources 25 create-search-index 983 summarise-objectstore 245 create-autocomplete-index 28 create-attribute-indexes 95 transfer-sequences 248429 final-dump 697 total 251305
Deployment
Host: WormMine production server (206.108.125.166)
Path: /nfs/wormbase/archive/jbaran/WS239-2
Commands:
dropdb -U intermine joachim-ws239-2 createdb -U intermine -E SQL_ASCII joachim-ws239-2 pg_restore -U intermine_admin -d joachim-ws239-2 joachim_ws239_2.final
Required Disk Space:
1.3G joachim_ws239_2.final + 17G PostgreSQL DB