OA-phenotype

From WormBaseWiki
Revision as of 19:01, 31 October 2017 by Cgrove (talk | contribs) (→‎TAB 1)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigationJump to search

package dump script

tazendra /home/acedb/work/allele_phenotype/use_package.pl

use_package.pl is a perl script that uses the perl module( ), to generate an error file and two .ace files (since May 2011)- one for phenotypes and one for molecule-phenotype.

Constraints:

  • does not dump record when value in Curation status is "down right disgusted", this acts as our no dump toggle
  • when there is no value for phenotype -this will stop non-curated NBP data from being dumped
  • will dump if phenotype is present regardless of curator

requested changes

5/17/2011

  • add a constraint - for rearrangement objects, dump all phenotype information except molecule values (molecule data should not be annotated for rearrangements so this is just a fail safe in case it does happen).
  • in addition to the varphene.ace output file, please output a separate file called mol_phene.ace, which contains:
  • molecule variation phenotype
  • molecule strain phenotype
  • molecule transgene phenotype

2/2012

I can't tell if all 5 of these are new fields, or just some. They are all new fields. 5 new tables + fields added -- J
If they're all fields, are they all at the bottom of tab2, or just the first 2 fields ? Yes, add them all to tab 2 bottom
Are you sure you want those field names, they're really long (which is fine by me, but will take up more space for you) -- J Good point, I shortened them

  • Add fields to OA - add to TAB2 at the bottom in the following order
    • (NEW FIELD) rescued by - multi-ontology from transgene tables, autocomplete on transgene name app_rescuedby
    • (NEW FIELD) legacy info - app_legacyinfo parsed data from legacy data- all entries with [celegans] from file on tazendra and mangolassi at /home/acedb/work/allele_phenotype This file ? /home/acedb/work/allele_phenotype/legacy_information.txt There are no entries with "[celegans]" Do you mean lines where it says --;"[C.elegansII]-- ? yes
      • What do you mean by parse ? Enter everything in each line into a new app_ OA line with its own pgid ? Or split on ";" and only enter stuff from the third column ? or something else ? -- J take everything in quotes starting with the third column where there is "[C. elegansII], in some cases there are more semi-colons, these will need to be ignored after the third column
        Do you mean that there are 4th and 5th+ columns, or that the third column contains semicolons ? The divider is --";"-- with the doublequotes, so if you meant the latter, it should be okay if you meant you want it all including the semicolons in the third column. If there are 4th and 5th columns, let me know -- J Oh, there are semicolons in within the quotes, so you are saying those don't count as columns? if not, then cool, we are good.
      • each entry gets its own pgid
      • add curation status (app_curation_status) of "down right disgusted" so lines that I have not touched do not get dumped.
      • make legacy data editable text or bigtext ? -- J bigtext
      • what other fields ? no app_name ? you for app_curator ? -- J
        • sure, me for app_curator
        • please add Jonathan Hodgkin (WBPerson261) to the person field
      • parsed into app_curator WBPerson712 ; app_curation_status down_right_disgusted ; app_legacyinfo "<wbgene> | <info>" (also history tables) -- J
    • (NEW FIELD) ES - drop down list, with values app_easescore
      • "ES0_Impossible_to_score", "ES1_Very_difficult_to_score", "ES2_Difficult_to_score", "ES3_Easy_to_score"
    • (NEW FIELD) ME - drop down list with values app_mmateff
      • "ME0_Mating_not_successful", "ME1_Mating_rarely_successful", "ME2_Mating_usually_successful", "ME3_Mating_always_successful"
    • (NEW FIELD) HME - drop down list with values app_hmateff
      • "HME0_Mating_not_successful", "HME1_Mating_rarely_successful", "HME2_Mating_usually_successful", "HME3_Mating_always_successful"


CHANGES TO DUMP SCRIPT

  • mating_efficiency
    • constrain lines with mating_efficiency values to be NOT NULL in app_curator, app_tempname (variation), app_person OR app_paper what does constrain mean ? check_data button on OA ? so not in dumping script ? I can't find stuff like this in the dumping script. Unless it's a new thing, but I thought this was in the check_Data button. Did we ever wiki the dumping script ? -- J https://bitbucket.org/kyook/ky_wbprojects/wiki/use_package.pl we did go over the dumping script, there are some constraints (rules? you probably have another term for this) that I thought were employed, see the bit bucket page. that's just the script that calls the module, the module is what generates the data, and we need to go over that if we're going to change stuff -- J ok
    • lines with mating_efficiency can be blank for phenotype (app_phenotype) Do you mean app_term ? yes, sorry
      This implies they can't be blank for other stuff, the script only does some stuff for pgids with data in app_term, does it already ever dump stuff when there isn't an app_term, and this new thing should join that, or is this the first time it will do that ? If we haven't gone the dumping script, we should, this seems like a pretty big change -- J You are correct the script already has rules in it to not dump data when there is no phenotype (app_term); however, lines with mating efficiency values (ME and or HME) will need to escape that rule. I hope this is not a big change(!) well, it's not huge, but it's not just adding another line and another tag, so we really should just wiki up the module so that future changes are more clear -- J ok
      • Don't know what the tag for non-Male mating efficiency should be (using Nonmale for now) It should be Hermaphrodite

couldn't find it here : http://cvs.sanger.ac.uk/cgi-bin/viewvc.cgi/wormbase/wspec/models.wrm?root=ensembl&view=markup&pathrev=WS229 Wrong release, it should be http://cvs.sanger.ac.uk/cgi-bin/viewvc.cgi/wormbase/wspec/models.wrm?root=ensembl&view=markup&pathrev=WS231

    • .ace should look like the example is good -- J
Variation : "WBVar00266499"
Male	"ME2_Mating_usually_successful"	Curator_confirmed	"WBPerson712"
Male	"ME2_Mating_usually_successful"	Person_evidence	"WBPerson261"


  • ease_of_scoring
    • .ace should be like
Variation : "WBVar00266499"
Species	"Caenorhabditis_elegans"
Phenotype	"WBPhenotype:0000456"	Curator_confirmed	"WBPerson712"
Phenotype	"WBPhenotype:0000456"	Person_evidence	"WBPerson261"
Phenotype	"WBPhenotype:0000456"	Remark	"touch-insensitive"	Curator_confirmed	"WBPerson712"
Phenotype	"WBPhenotype:0000456"	Remark	"touch-insensitive"	Person_evidence	"WBPerson261"
Phenotype	"WBPhenotype:0000456"	Ease_of_scoring	"ES2_Difficult_to_score"	Curator_confirmed	"WBPerson712"
Phenotype	"WBPhenotype:0000456"	Ease_of_scoring	"ES2_Difficult_to_score"	Person_evidence	"WBPerson261"
  • rescued_by_transgene
    • .ace should be like
Variation : "WBVar00266499"
Species	"Caenorhabditis_elegans"
Phenotype	"WBPhenotype:0000456"	Curator_confirmed	"WBPerson712"
Phenotype	"WBPhenotype:0000456"	Person_evidence	"WBPerson261"
Phenotype	"WBPhenotype:0000456"	Remark	"touch-insensitive"	Curator_confirmed	"WBPerson712"
Phenotype	"WBPhenotype:0000456"	Remark	"touch-insensitive"	Person_evidence	"WBPerson261"
Phenotype	"WBPhenotype:0000456"	Rescued_by_Transgene	"asIs432248"	Curator_confirmed	"WBPerson712"
Phenotype	"WBPhenotype:0000456"	Rescued_by_Transgene	"asIs432248"	Person_evidence	"WBPerson712"
legacy info

It is easiest just to do one final patch dump of updated legacy info rather than try to update it through -D and updates through each data dump .ace, so for now ignore this section until we have settled on the right course

  • legacy data- needs to be mapped to corresponding gene (app_wbgene)
    while the table app_wbgene still exists, it is 1) not in the OA, 2) does not have WBGene objects, it has data in what looks like a bad format : WBGene00003883 (osm-1 (WBGene00003883)) or WBGene00000058 (acr-19). I don't know where this came from, but it seems bad. This is most likely from the Variation_gene.txt I create after each build so alleles can be mapped to genes. The most recent entry is :
4606    | WBGene00002245 (lag-1 (WBGene00002245))     | 2008-01-18 20:00:41.652326-08   

What do you do with that file, do you run a script that does something with it ? -- J oh, then it is probably very old, that is in Jolene's time.

If you meant that we should dump whatever is in legacy pgids to existing app_wbgene, I don't see how since the legacy pgids are new, and the app_wbgene pgids are old and you can't add data to them without going to postgres directly. no, you are correct that is not what I meant.

If you know where this data came from, let me know if it's good. If it's not good, let's back it up and get rid of the table -- J if it isn't being used then yes, by all means we should get rid of it. Can you get rid of it on mangolassi so we can see what happens? As it is an app_wbgene table it is only looked at by the phenotype OA correct? yes, I think so, but if that Variation_gene.txt is used for some scripts you run manually, or a cronjob picks it up, getting rid of it on mangolassi won't show that anything changed because you wouldn't be dumping data there or any of the other stuff (cronjobs don't run on mangolassi). The second point is, if you were not thinking of the app_wbgene table with weird data, did you mean that you want a new app_wbgene table, and that you're doing to be populating it manually as you edit the legacy information ? And if so, ontology of multiontology -- J we can talk to chris and xiaodong about how variations are getting mapped to genes during the dump, this is the function that we would need in order to generate the .ace output for mating efficiency data. if that rings some bells, then great, otherwise we will need to wait for Chris and Xiaodong. this is no longer necessary as this problem was solved by including the WBGeneID in with the legacy data, piped before the legacy info text.

Legacy data is dumped whenever there is a value in the legacy data field.

  • make sure if lines are duplicated, the legacy value is deleted or the there will be more than one gene-legacy data entry in the .ace
  • if all legacy data is taken care of - it is no longer needed so delete all the values in the field or blanks will be dumped and an error message will pop up.
    • For the .ace
  • dump data when there is no 'down right disgusted tag' this is how the script works already
  • put in a -D for the legacy data and dump a legacy data line with the text in the legacy data field. Need to get original legacy data from legacy_information.txt, only information starting with "[C. elegansII]"
Gene :	"WBGene00006932"
-D Legacy_information	"[C.elegansII] h797 :  Mid-larval lethal. OA5: h351, etc. [KR]"
Legacy_information "[C.elegansII] OA5: h351, etc. [KR]"
  • for lines that have no legacy information but has a WBGene value | the .ace paragraph should be:
Gene :	"WBGene00001177"
-D Legacy_information	"n488 : transient variable bloating Type E. ES2 (adult). NA1."

--kjy 21:51, 9 March 2012 (UTC)


--kjy 23:33, 13 February 2012 (UTC)

Phenotype OA postgres tables (app_*)

postgres tables
app_allele_status app_func app_molaffectedquality_hst app_quantity_remark_hst
app_allele_status_hst app_func_hst app_molecule app_range_end
app_anat_term app_genotype app_molecule_hst app_range_end_hst
app_anat_term_hst app_genotype_hst app_nature app_range_hst
app_anatomy app_go_sug app_nature_hst app_range_start
app_anatomy_hst app_go_sug_hst app_nbp app_range_start_hst
app_anatomyquality app_gocomponent app_nbp_hst app_rearrangement
app_anatomyquality_hst app_gocomponent_hst app_needsreview app_rearrangement_hst
app_caused_by app_gocomponentquality app_needsreview_hst app_remark
app_caused_by_hst app_gocomponentquality_hst app_nodump app_rescuedby
app_caused_by_other app_gofunction app_nodump_hst app_rescuedby_hst
app_caused_by_other_hst app_gofunction_hst app_not app_reviewedby
app_child_of app_gofunctionquality app_not_complement app_reviewedby_hst
app_child_of_hst app_gofunctionquality_hst app_not_complement_hst app_rnai_brief
app_cold_degree app_goprocess app_not_hst app_rnai_brief_hst
app_cold_degree_hst app_goprocess_hst app_obj_remark app_strain
app_cold_sens app_goprocessquality app_obj_remark_hst app_strain_hst
app_cold_sens_hst app_goprocessquality_hst app_paper app_sug_ref
app_communitycurator app_haplo app_paper_hst app_sug_ref_hst
app_communitycurator_hst app_haplo_hst app_paper_remark app_suggested
app_communitycuratoremail app_heat_degree app_paper_remark_hst app_suggested_definition
app_communitycuratoremail_hst app_heat_degree_hst app_parentstrain app_suggested_definition_hst
app_complements app_heat_sens app_parentstrain_hst app_suggested_hst
app_complements_hst app_heat_sens_hst app_pat_effect app_temperature
app_control_isolate app_hmateff app_pat_effect_hst app_temperature_hst
app_control_isolate_hst app_hmateff_hst app_pathogen app_term
app_controlstrain app_intx_desc app_pathogen_hst app_term_hst
app_controlstrain_hst app_intx_desc_hst app_penetrance app_transgene
app_curator app_laboratory app_penetrance_hst app_transgene_hst
app_curator_hst app_laboratory_hst app_percent app_treatment
app_delivered app_legacyinfo app_percent_hst app_treatment_hst
app_delivered_hst app_legacyinfo_hst app_person app_unregpaper
app_easescore app_lifestage app_person_hst app_unregpaper_hst
app_easescore_hst app_lifestage_hst app_phen_remark app_unregtransgene
app_filereaddate app_lifestagequality app_phen_remark_hst app_unregtransgene_hst
app_filereaddate_hst app_lifestagequality_hst app_phenotype app_unregvariation
app_finalname app_mat_effect app_phenotype_hst app_unregvariation_hst
app_finalname_hst app_mat_effect_hst app_picture app_variation
app_finished_hst app_mmateff app_picture_hst app_variation_hst
app_flaggenereg app_mmateff_hst app_preparation_hst app_wbgene
app_flaggenereg_hst app_molaffected app_quantity app_wbgene_hst
app_flaggeneticintxn app_molaffected_hst app_quantity_hst
app_flaggeneticintxn_hst app_molaffectedquality app_quantity_remark

Phenotype OA TAB Summary

  • Updated for newest OA format 8-30-2015 (CG)


  • Line format: Field Name - postgres_tablename - NOT_DUMPED - ?WBClass/Entry-type (OA Field-type)
    • Notes

TAB 1

  • pgid - the postgres ID - NOT DUMPED
  • Curator - app_curator - ?Person (Dropdown)
  • Pub - app_paper - ?Paper (Ontology)
  • Person - app_person - ?Person (Ontology)
    • Note: Person providing evidence for the annotation (if not published)
  • Phenotype - app_term - ?Phenotype (Ontology)
  • NOT - app_not - (Toggle)
    • Note: triggers dump of Phenotype_not_observed in #Phenotype_info (Toggle)
  • Phenotype Remark - app_phen_remark - Text (Big Text)
  • Variation - app_variation - ?Variation (Ontology)
  • Transgene - app_transgene - ?Transgene (Ontology)
  • Strain - app_strain - ?Strain (Ontology)
  • Rearrangement - app_rearrangement - ?Rearrangement (Ontology)
  • Object Remark - app_obj_remark - NOT DUMPED - Text (Text)
  • Allele Status - app_allele_status - NOT DUMPED - Allele status (Dropdown)
    • Note: options are: new_gene_assignment, lost, other
  • Caused By Gene - app_caused_by - ?Gene (Ontology)
  • Caused By Other - app_caused_by_other - Text (Text)

TAB 2

  • Phenotype - app_term - ?Phenotype (Ontology)
  • NOT - app_not - (Toggle)
    • Note: triggers dump of Phenotype_not_observed in #Phenotype_info (Toggle)
  • Phenotype Remark - app_phen_remark - Text (Big Text)
  • Suggested - app_suggested - NOT DUMPED - Text (Text)
    • Note: field to suggest new phenotype term. Will replace whatever term(s) are currently in the Phenotype field once it is approved
  • Suggested Definition - app_suggested_definition - NOT DUMPED - Text (Big Text)
    • Note: definition of new suggested phenotype term
  • Child Of - app_child_of - NOT DUMPED - ?Phenotype (Ontology)
    • Note: list of parent phenotype term(s) for suggested phenotype
  • GO Process - app_goprocess - ?GO_term (Multi-ontology)
  • GO P Quality - app_goprocessquality - PATO term (Ontology)
  • GO Function - app_gofunction - ?GO_term (Multi-ontology)
  • GO F Quality - app_gofunctionquality - - PATO term (Ontology)
  • GO Component - app_gocomponent - ?GO_term (Multi-ontology)
  • GO C Quality - app_gocomponentquality - PATO term (Ontology)

TAB 3

  • Phenotype - app_term - ?Phenotype (Ontology)
  • NOT - app_not - (Toggle)
    • Note: triggers dump of Phenotype_not_observed in #Phenotype_info (Toggle)
  • Phenotype Remark - app_phen_remark - Text (Big Text)
  • Anatomy - app_anatomy - ?Anatomy (Multi-ontology)
  • Anatomy Quality - app_anatomyquality - PATO term (Multi-ontology)
  • Life Stage - app_lifestage - Life_stage - ?Life_stage (Multi-ontology)
  • Life Stage Quality - app_lifestagequality - PATO term (Multi-ontology)
  • Molecule Affected - app_moleculeaffected - ?Molecule (Multi-ontology)
  • Mol Aff Quality - app_molaffectedquality - PATO term (Multi-ontology)
  • Affected By Molecule - app_molecule - (in-line with Phenotype) Molecule - ?Molecule (Multi-ontology)
  • Affected By Pathogen - app_pathogen - ?Species (Multi-ontology)

TAB 4

  • Phenotype - app_term - ?Phenotype (Ontology)
  • NOT - app_not - (Toggle)
    • Note: triggers dump of Phenotype_not_observed in #Phenotype_info (Toggle)
  • Phenotype Remark - app_phen_remark - Text (Big Text)
  • Allele Nature - app_nature -
  • Functional Change - app_func -
  • Temperature - app_temperature - Temperature - ?Text (Text)
  • Treatment - app_treatment - Treatment description as text (Big text)
  • Control Isolate - app_control_isolate - NOT DUMPED - Text (Text)
  • Penetrance - app_penetrance - (Incomplete, Complete, High, Low) (Dropdown)
  • Penetrance Remark - app_percent - Text describing penetrance (Big text)
  • Cold Sensitive - app_cold_sens - (in-line with Phenotype) Temperature_sensitive - Toggle
  • Cold Sensitive Degree - app_cold_degree - (in-line with Phenotype) Temperature_sensitive - Toggle
  • Heat Sensitive - app_heat_sens - (in-line with Phenotype) Temperature_sensitive - Toggle
  • Heat Sensitive Degree - app_heat_degree - (in-line with Phenotype) Temperature_sensitive - Toggle
  • Maternal Effect - app_mat_effect - Tag for maternal effect (Dropdown)
    • Note: options are: Strictly_maternal, With_maternal_effect, Maternal
  • Paternal Effect - app_pat_effect (Toggle)
  • Haploinsufficient - app_haplo (Toggle)

TAB 5

  • Phenotype - app_term - ?Phenotype (Ontology)
  • NOT - app_not - (Toggle)
    • Note: triggers dump of Phenotype_not_observed in #Phenotype_info (Toggle)
  • Phenotype Remark - app_phen_remark - Text (Big Text)
  • Picture - app_picture - ?Picture (Ontology)
  • Parent Strain - app_parentstrain - ?Strain (Multi-ontology)
  • Genotype - app_genotype - Genotype (Big text)
  • Control Strain - app_controlstrain - Text (Text)
  • Rescued By - app_rescuedby - ?Transgene (Multi-ontology)
  • Complements - app_complements - NOT DUMPED - ?Variation (Multi-ontology)
  • Does Not Complement - app_not_complement - NOT DUMPED - ?Variation (Multi-ontology)
  • Flag Genetic Intxn - app_flaggeneticintxn - NOT DUMPED (Toggle)
  • Genetic Intx Desc - app_intx_desc - NOT DUMPED - Text (Big text)
  • Flag Gene Reg - app_flaggenereg - NOT DUMPED (Toggle)

TAB 6

  • Phenotype - app_term - ?Phenotype (Ontology)
  • NOT - app_not - (Toggle)
    • Note: triggers dump of Phenotype_not_observed in #Phenotype_info (Toggle)
  • Phenotype Remark - app_phen_remark - Text (Big Text)
  • NBP - app_nbp - NOT DUMPED - Text (Big text)
  • NBP / File Date - app_filereaddate - NOT DUMPED - Text (Text)
  • Laboratory Evidence - app_laboratory - ?Laboratory (Multi-ontology)
  • Legacy Info - app_legacyinfo - NOT DUMPED - Text (Big text)
  • ES - app_easescore - Ease of scoring (Dropdown)
  • ME - app_mmateff - Dumped in ?Variation object - Male mating efficiency (Dropdown)
  • HME - app_hmateff - Dumped in ?Variation object - Hermaphrodite mating efficiency (Dropdown)
  • Community Curator - app_communitycurator - ?Person (Ontology)
  • Community Curator Email - app_communitycuratoremail - Text (Text)
  • Unregistered Paper - app_unregpaper - NOT DUMPED - Text (Text)
  • Unregistered Variation - app_unregvariation - NOT DUMPED - Text (Text)
  • NO DUMP - app_nodump - NOT DUMPED (Toggle)
  • Needs Review - app_needsreview - NOT DUMPED (Toggle)