Difference between revisions of "RNAi"

From WormBaseWiki
Jump to navigationJump to search
Line 513: Line 513:
 
'''We're not keeping remark nor any other #Phenotype_info for the field Phenotype_not_observed right ?  There's no place to put it'''
 
'''We're not keeping remark nor any other #Phenotype_info for the field Phenotype_not_observed right ?  There's no place to put it'''
  
'''DNA_text is a double text field.  I'm assuming that the first part is always going to be a string of atcg so will never have a space.  In that case you could store the data in this field like qq(<atcg><space><next_thing>) but if you think you'd ever want spaces the better divider would be qq(<atcg>" "<next_thing")  I didn't see anything else in the model that stores two values in a row besides the From-To fields, if there are any, let me know and we'll talk about them.  I need to figure this out to know how to parse stuff'''
+
'''DNA_text is a double text field.  I'm assuming that the first part is always going to be a string of atcg so will never have a space.  In that case you could store the data in this field like qq(<atcg><space><next_thing>) but if you think you'd ever want spaces the better divider would be qq(<atcg>" "<next_thing")  I didn't see anything else in the model that stores two values in a row besides the From-To fields, if there are any, let me know and we'll talk about them.  I need to figure this out to know how to parse stuff.  We talked about Database which I'm treating as a triple text field (instead of ontology values) and I think you don't have spaces in those values either, but if you do we should talk about it.'''
  
 
'''The OAs that have lifestage store the WBls:####### ID instead of the name, but the .ace file has names, so either there was some kind of translation, or more likely the data in there is wrong (someone should have switched it when the ?Life_stage class changed from text to IDs).  I'm converting from names to IDs to store in postgres, but some names don't match the list of valid values in postgres in obo_name_lifestage.  The list of RNAi objects and their Life_stage text is at /home/postgres/work/pgpopulation/rna_rnai/bad_lifestage'''
 
'''The OAs that have lifestage store the WBls:####### ID instead of the name, but the .ace file has names, so either there was some kind of translation, or more likely the data in there is wrong (someone should have switched it when the ?Life_stage class changed from text to IDs).  I'm converting from names to IDs to store in postgres, but some names don't match the list of valid values in postgres in obo_name_lifestage.  The list of RNAi objects and their Life_stage text is at /home/postgres/work/pgpopulation/rna_rnai/bad_lifestage'''

Revision as of 11:44, 27 March 2012

RNAi Curation Mission Summary

The term "RNAi" stands for "RNA-interference" and refers to the targeted silencing of gene expression of a "target gene" via introduction of double stranded RNA (dsRNA) containing high degrees of sequence identity to the "target gene". In C. elegans, molecules of dsRNA may be introduced into the worm by a variety of different methods including direct micro-injection, soaking of worms in solution containing dsRNA, feeding worms bacteria that express dsRNA from a plasmid, and transgenic expression (within the worm) of dsRNA. For efficient and specific knockdown of gene expression, the dsRNA must have a minimum sequence identity with the target gene sequence. Any phenotype(s) resulting from the RNAi-mediated knockdown of a particular gene is thought to directly reflect the phenotype of a loss-of-function mutation in that gene, hence providing evidence as to the gene's biological function.

The goal of RNAi curation is to associate RNAi-mediated phenotypes with the target genes knocked down in RNAi experiments, as found in the literature pertaining to C. elegans and related species. There are some important things to consider when curating RNAi experiments. Some RNAi experiments are simple to curate as the strain is N2 (wild type genotype), only a single gene is targeted for RNAi, the phenotype is clearly stated, and the source and identity of the dsRNA is clearly stated by the authors. In many cases, however, the curation task is not so simple: there may be complex genotypes with multiple mutations in the strain receiving the dsRNA, multiple RNAi gene targets, complex genetic interactions, and/or missing descriptions from the authors as to controls, dsRNA delivery method, dsRNA identity, and/or phenotype directly resulting from the RNAi in question. Hopefully these issues will all be addressed below.

RNAi Data Model

This is the RNAi data model as of WormBase Release WS227:

//////////////////////////////////////////////////////////////////
//
// ?RNAi class
//
//////////////////////////////////////////////////////////////////

?RNAi   Evidence #Evidence
        History_name UNIQUE ?Text
        Homol Homol_homol ?Homol_data XREF RNAi_homol ?Method Float Int UNIQUE Int Int UNIQUE Int #Homol_info
        Sequence_info   DNA_text Text UNIQUE Text //stores actual probe sequence for automated mapping
						  // 1st Text is DNA, 2nd is probe name
                       Sequence ?Sequence XREF RNAi  //links to a real Sequence object used in the experiment 
                                                     // such as yk clone; not UNIQUE anymore
                       PCR_product ?PCR_product XREF RNAi // links to a PCR_product object used in 
                                                          // the experiment; not UNIQUE anymore
       Uniquely_mapped  //boolean; if present, signifies that ?RNAi object has a unique sequence 
                        // which maps to a single place in the genome
       Experiment      Laboratory ?Laboratory
                       Author ?Author
                       Date UNIQUE DateType
                       Strain UNIQUE ?Strain
                       Genotype UNIQUE ?Text   //used when no Strain object exists
                       Treatment UNIQUE ?Text
                       Life_stage UNIQUE ?Life_stage
                       Temperature UNIQUE Int
                       Delivered_by UNIQUE Bacterial_feeding      //RL [010327]
                                           Injection              //RL [010327]
                                           Soaking                //RL [010327]
                                           Transgene_expression   //RL [010327]
       Inhibits        Predicted_gene ?CDS XREF RNAi_result #Evidence // "gene" parent (unreliable)
                       Gene ?Gene   XREF RNAi_result #Evidence           //RL [010327]
                       Transcript ?Transcript XREF RNAi_result #Evidence // [021126 krb]
                       Pseudogene ?Pseudogene XREF RNAi_result #Evidence // [030801 krb]
       Supporting_data Movie ?Movie XREF RNAi    // Lincoln, krb [010807]
       DB_info         Database ?Database ?Database_field ?Accession_number //to link out to Phenobank ar2 02-DEC-05 
                                                                            //removed UNIQUE as reqs multiple connections
       Species         UNIQUE ?Species
       Gene_regulation ?Gene_regulation XREF RNAi  // this tag is used when an RNAi experiment describes 
                                                   // gene regulation ar2 29-MAR-06 for igor
       Interaction     ?Interaction
       Reference       UNIQUE ?Paper XREF RNAi //[070215 ar2] made reference unique so Paper sort of 
                                               // equates to a Study class for Will S
       Phenotype       ?Phenotype XREF RNAi #Phenotype_info
       Phenotype_not_observed ?Phenotype XREF Not_in_RNAi #Phenotype_info //added by Wen to separate 
                                                                          // Not phenotype from real phenotypes
       Expr_profile    ?Expr_profile XREF RNAi_result // connection added during build [030106 krb]
       Remark          ?Text #Evidence
       Method UNIQUE   ?Method

RNAi Curation Standard Operating Procedure (SOP)

In order to ensure consistency of RNAi curation across curators and WormBase releases, a set of standard procedures for RNAi curation are outlined below. The descriptions of these procedures include use of the two main methods for generating *.ace files for release submission: (1) the web-based CGI form for one-at-a-time RNAi object generation and (2) the batch form submission method.

Minimum Requirements for an RNAi Object

Regardless of which curation method you (the curator) choose, there are a set of minimum requirements in order to generate a complete RNAi experiment object:

1) A reference (e.g. WBPaperID)

2) A curator name

3) The sequence of the dsRNA used in the experiment to knockdown gene expression

4) A dsRNA delivery method (e.g. Injection) [Note: some authors omit this information. This can be curated arbitrarily and removed from the *.ace file once it has been generated]

5) A phenotype (observed or not_observed)

If any one of these five basic pieces of information are missing, the scripts that generate the *.ace file will fail, returning an error.

Two Methods of Curation: Web CGI & Batch Form

There are two basic methods of RNAi curation: (1) a web-based CGI form and (2) a batch form involving a Perl script (living on the elbrus@caltech machine) to convert a spreadsheet (tab-delimited file) into an *.ace file.

(1) Web-based CGI

The web-based RNAi curation form can be found at:

http://elbrus.caltech.edu/cgi-bin/igor/rnaitools/rnai_curation

Web Form Page One

The first page of the web form requests information about the publication and looks like this:


RNAi CGI 1.png


This page requests information about the paper (e.g. WBPaperID), laboratory two-letter code (usually senior/corresponding author; for a list of all labs' two-letter codes look here: http://www.wormbase.org/db/misc/laboratory?name=*;class=laboratory), date of publication (accepted date), and a curator name. The laboratory and publication acceptance date are optional.

Once you've input the relevant information, click on the "Submit" button. Note that this may take a few moments to load. Once loaded you will be at the second page of the web form.

Web Form Page Two

The second page of the web form looks like this:



RNAi CGI 2.png



This second page of the web form requests information about the RNAi probe(s) (i.e. the dsRNA sequence used to knockdown expression of the target gene). This information may be submitted in the form of a PCR product object (e.g. sjj_ZK617.1 or mv_CAA33463), PCR primers (it's usually good practice to check the primer sequences using e-PCR to confirm that they work properly), genomic coordinates, cDNA/EST/OST clone (note that many of these clones may be problematic; in such a case it is best to determine the sequence to the best of your ability and submit that instead), or actual sequence (e.g. "ACGT") of the probe.

An important point here is that RNAi target genes for an RNAi experiment are not submitted directly using the name of the target gene as indicated in the publication. The reason for this is that gene names can be continuously remapped to different regions of the genome over time, making persistence of the data impossible if we were to use only gene names to identify the target gene. By providing the actual sequence or clone used in the experiment, we can ensure that the appropriate gene is identified as the RNAi target gene regardless of which release of the database you are currently working in.

You may wish to submit more than one RNAi probe (e.g. when two or more genes are targeted by RNAi simultaneously). If this is the case, be sure to select "Yes" after the "Add another probe?" question at the bottom of the page, and then click on the "Submit" button.

Once all of your probes have been submitted, make sure that "No" is selected after the "Add another probe?" question, and click on the "Submit" button.

If all of the probes check out OK, you will be brought to the third page of the web form.

Web Form Page Three

The third page of the web form looks like this:


RNAi CGI 3.1.png

RNAi CGI 3.2.png


This page requests all of the experimental details about the RNAi experiment in question. This information includes Strain/Genotype, treatment conditions, life stage, dsRNA delivery method (e.g. bacterial feeding), Phenotype observed (or not_observed), Gene Regulation objects, Genetic Interaction objects, species, and general remarks about the experiment.

Strain/Genotype

Whereas the "Genotype" information for a strain used in an RNAi experiment may be written in as free-text, the "Strain" field will only accept strains that are officially recognized by WormBase. Be sure to check the database for the existence of your strain name before submitting (otherwise you may receive an error upon submission). If your strain name does not currently exist in the database, you may write in the genotype in the "Genotype" field, or request the addition of the strain into the database.

Treatment/Temperature

The "Treatment" field is a free-text field in which the curator may specify any particular experimental conditions that apply to this RNAi experiment. This may include details about growth conditions, dsRNA delivery, or specimen manipulation. The "Temperature" field must be filled in with an integer value indicating the temperature in degrees Celsius (e.g. "25" for 25 degrees Celsius).

Life Stage

The "Life Stage" field must be filled in with an official life stage name, as stored in ACeDB/WormBase. For an official list of Life Stage names in WormBase, see:

Life_Stage_System_in_WormBase

or

Definitions_for_Life_Stages

Delivered by

The "Delivered by" field provides a drop-down menu of four dsRNA delivery methods: Injection, Bacterial feeding, Soaking, and Transgene expression.

WormBase phenotype ID

This field needs to have at least one phenotype in the form of a WormBase phenotype ID, for example: WBPhenotype:0000050 (for embryonic lethality). Without at least one phenotype, the *.ace-generating script will throw an error.

If you need to add other information pertaining to the phenotype, such as penetrance, quantification, or a remark, click on the "Phenotype Info" button to the right. This will open up a separate window that looks like this:

RNAI Phenotype Info.png

This form provides fields for the curator to enter information regarding the penetrance, temperature sensitivity, quantification, molecule interactions, phenotypes-not-observed, and a general, free-text remark field.

The Penetrance section records the range of penetrance of the phenotype in question (indicated to the left). The curator can provide a lower limit ("from") and/or an upper limit ("to"); if there is only one penetrance value indicated in the publication, just fill it in the "from" field. This value is an integer value representing percent penetrance. Below the "Range" fields, the curator can toggle the check boxes next to "Incomplete", "Low", "High", or "Complete" indicating the general quantity of penetrance. Each field also provides a free-text "Description" field to provide a description of the penetrance.

The Temperature sensitive fields allow the curator to indicate whether or not the phenotype is dependent on temperature and free-text "Description" field to provide details.

The Quantity fields record the lower and/or upper limits of the phenotype quantification as integer values on a scale that can be described in the "Description" free-text field. An example of a phenotype quantity might be "average brood size" or "body bends per minute".

The Affected by field captures molecules/compounds/drugs that affect the phenotype as indicated by this RNAi experiment. The molecule must be entered as the official WormBase molecule ID, usually the MeSH ID (e.g. D003609).

The NOT field provides a toggle check box for indicating if the phenotype in question was NOT OBSERVED in the experiment. This captures negative results of RNAi experiments.

Finally, the Details section provides a free-text field to provide descriptions of the phenotype beyond which could be captured in the other fields.


Additional phenotype information...

OBSOLETE; If you cannot find an appropriate phenotype for your RNAi experiment in the Worm Phenotype Ontology (WPO; worm_phenotype.obo can be downloaded from http://www.obofoundry.org/), please visit (http://tazendra.caltech.edu/~postgres/cgi-bin/new_objects.cgi) and enter the phenotype term you want to request by clicking the “Update Phenotype!” button and filling out the fields under “Suggest data through this CGI”.

New Gene Regulation

If the RNAi experimental result suggests that the RNAi target gene is somehow functionally involved in the regulation of a gene (at the transcriptional, translational, or post-translational level) and no Gene Regulation object exists for this relationship in the database, check the box here to request that a Gene Regulation object ID be assigned to this RNAi experiment. If the curator checks the box, a separate browser window will open with a simple text field to enter in the details of the Gene Regulation event:

RNAi Gene Regulation Info.png

As the note suggests, don't forget to send this info to Xiaodong so she can generate a new Gene Regulation Object.

Gene Regulation Object

If a Gene Regulation Object ID has already been assigned to this experiment from a different curation pipeline, enter the Gene Regulation Object ID here (e.g. WBPaper00006370_lin-3, cgc6303_lag-2).

New Interaction

If the RNAi experiment indicates a genetic (or any other type) interaction and an Interaction Object ID has not yet been assigned to this interaction, select the number of genes involved in the interaction from the provided pull down menu. This will automatically open another browser window which will provide a form for entering details specific to that particular interaction (see image below):

RNAi Interaction Info.png

This form provides the curator with fields to enter information for each interacting gene as well as general information about the interaction. The gene-specific information includes the gene's WBGeneID (e.g. WBGene00002299), "Variation" information (e.g. e1370, mgDf47) if this gene was perturbed by an allele or other variation, "Transgene" for a transgene harboring the gene (or variation of the gene) that contributes to the interaction, "This RNAi" toggle box to indicate that the perturbation of this gene was via RNAi (as opposed to an allele), and "Direction" drop-down menu to indicate if the gene is the "Effector" or "Effected" gene (for directional interactions) or "Non-directional" (for interactions in which directionality is irrelevant).

Below the gene-specific fields is a drop-down menu for indicating the "Interaction Type" (e.g. Genetic, Suppression, Enhancement), a "Interaction-relevant Phenotype" field to indicate the phenotype about which the interaction is based, and a free-text "Remark" field to describe any details of the interaction not sufficiently explained by the other fields.

Interaction Object

If an Interaction Object ID has already been assigned to this experiment from a different curation pipeline, enter the Interaction Object ID here (e.g. WBInteraction0001295).

Picture

OBSOLETE; The current RNAi data model no longer supports Picture objects

Species

Enter the species in which the RNAi experiment is taking place. Caenorhabditis elegans is set by default.

Remark

The "Remark" field is a free-text field to enter in any remaining relevant information that is pertinent to the RNAi experiment, but has not yet been captured by any other field of the curation form. This is often where standard remarks regarding the ambiguity of the dsRNA identity (see below) are entered. Information relevant to a genetic interaction, gene regulation event, or complex phenotype may also be put here.

If you are done curating this particular RNAi experiment and have more from the current paper, select "Yes" next to the "Add another experiment from this paper?" question at the bottom of the page and then click "Submit". If you are done curating RNAi experiments from this paper (or are done for the current session) and would like to generate an *.ace file, select "No" next to "Add another experiment from this paper?". Finally, indicate whether or not you would like to receive an e-mail of the *.ace file (as text in the body of the e-mail) next to "Would you like to receive a copy of the ace file via e-mail?". Once you are set, click the "Submit" button to generate the ace file (to be displayed in the web browser). To generate the *.ace file, you may simply copy and paste the text from the screen into a plain text file and save it with the ".ace" extension.

Post-processing your *.ace file

The .ace files from the CGI form need to be concatenated and processed using the following script (rnai_ace_modify.pl). Talk to a current RNAi curator for a copy of this script. This script converts the “NOT” tag to a “phenotype_not_observed” tag to be compliant with the current model and it splits the molecule data and interaction data into separate files: Hence, this script will take the .ace file from the CGI form and split it into 3 parts:

(1) FILENAME.ace.new -- for RNAi data

(2) FILENAME.ace.mol -- for molecule data

(3) FILENAME.ace.interaction - for interaction data

To run this script, type ./rnai_ace_modify.pl

It will ask you for the input FILENAME from the .ace file that you want to convert into these 3 files.

The following files get uploaded to CitaceMinus:

(1) FILENAME.ace.new -- for RNAi data

(2) FILENAME.ace.mol -- for molecule data

and the interaction file “FILENAME.ace.interaction” needs to be parsed into the interaction OA. (see the interaction Wiki page for instructions).

(2) Batch Form

The second basic method for generating RNAi *.ace files involves filling out all relevant RNAi curation information on a spreadsheet with each row (more or less; explanation below) as an RNAi experiment and each column as an RNAi object field. An Excel-based template file may be found here:

File:RNAi Curation Template.xls

Much of what applies to the RNAi Curation CGI Form described above applies similarly (or exactly) to the Batch Form. There are a few spreadsheet-specific changes that we will describe here.

First, there are a number of columns that are highlighted in yellow and have the comment "remove column before submission" in parentheses. These are temporary aids to the curator that must be removed before submitting the spreadsheet to the script, as they will cause an error.


Filling out the Batch Form

Shown below are screenshots from a fictional RNAi experiment entry in the batch form.

Paper Identification Information

Curation Template


The first 6 columns (A-F) of the spreadsheet record WBRNAi ID, WBPaper ID (WBPaper000#####), Laboratory (2-letter code), Publication Acceptance Date (YYYY-MM-DD), Curator Name, and Curator E-Mail. The WBRNAi ID column is only used once the *.ace file has already been made and is removed before submission to the *.ace-file-generating script. Whereas the Laboratory code and Publication Acceptance Date are optional, the WBPaper ID and the Curator Name are required.


RNAi probe (dsRNA source) Information

RNAi Batch Form 2.png


The next 8 columns (G-N) record RNAi probe information (i.e. information about the dsRNA identity/sequence). The first column (G) is an exception. The "Same As Above" column either takes no entry or "YES" (all CAPS important), available from an Excel pull down menu; this column indicates whether the row/line of the form is to be included as part of the same RNAi experiment as the line(s) before it (the closest preceding line with paper information provided). This is necessary, for example, when submitting more than one RNAi probe, as each RNAi probe must go on its own line. It is important that information that is unique to an RNAi experiment (e.g. Strain/Genotype, Treatment, Temperature, etc.; indicated by RED text) is not also added to this line as it will overwrite any information provided in lines above it (belonging to the same RNAi experiment).

The "PCR Product", "Primer1", "Primer2", "Genomic Coordinates", "Clone", and "Sequence" columns record the dsRNA source as in the CGI Web Form (see above). The "Target ID" column (yellow, hence to be removed before submission) is free-text to indicate which gene the RNAi probe is targeting. This column is handy for keeping track of which RNAi experiments have already been curated, as there is no other easy way to see the gene's name in the form.


Experimental Information

RNAi Batch Form 3.png


The next 8 columns (O-V) record experimental details including "Strain", "Genotype", "Treatment", "Life Stage", "Temperature", "Delivered by", "Species", and "Remark". The "Life Stage" and "Delivered by" columns provide Excel pull down menus to select terms from a standard list (syntax is important). Note that these fields are the ones that are unique to an RNAi experiment; if two or more rows contain information for one of these columns, only the bottom-most row will be read. The formats for the remaining columns are as indicated in the CGI Web Form section above (i.e. "Genotype", "Treatment", and "Remark" are free-text, "Strain" must be a recognized WormBase strain name, "Temperature" must be an integer value for degrees Celsius, and "Species" must be a recognized Latin species name of a nematode species in WormBase).


Phenotype Information 1

RNAi Batch Form 4.png


Phenotype Information 2

RNAi Batch Form 5.png


The next 17 columns (W-AM) record phenotype information. The "Gene Regulation ID" records a Gene Regulation Object ID, if one has already been generated for this experiment. The "Phenotype ID" must be given as the WormBase Phenotype ID as recorded in the Worm Phenotype Ontology (WPO; worm_phenotype.obo is available at http://www.obofoundry.org/), for example, WBPhenotype:0000081 for "L1 arrest". The yellow "Phenotype Name" column records the human-readable name of the phenotype for curatorial convenience.

The following columns (Z-AE) record the penetrance of the phenotype: "Penetrance From" records a lower limit of penetrance (integer of percentage), "Penetrance To" records an upper limit of penetrance (integer of percentage), and "Penetrancs(sic) Incomplete", "Penetrance Low", "Penetrance High", and "Penetrance Complete" are record ("YES" or empty; pull down menu) whether or not the penetrance quantification falls under any of these categories. "Heat Sensitive" and "Cold Sensitive" indicate ("YES" or empty; pull down menu) whether or not the phenotype is heat or cold sensitive, respectively. "Quantity From" and "Quantity To" capture the lower and/or upper limits of the phenotype quantification (integer values), if one is provided. "Quantity Description" is a free-text field for describing the nature of the phenotype quantitation. The "NOT" field would be filled with a "YES" if the phenotype was not observed (negative result). "Phenotype Remark" is a free-text field for describing any aspects of the phenotype not adequately captured by the other fields. The yellow "Evidence" field (optional and must be removed before submission) provides the curator with a field to capture the source of the evidence within the paper (e.g. Figures, Tables, Text, etc.).


Molecule Information

RNAi Batch Form 6.png


The next two columns (AN and AO) capture molecule information, if applicable. This would apply in circumstances in which a compound/drug was used in the experiment and potentially had biological activity or an affect on the phenotype being reported. The "Molecule" column must be filled with the WormBase recognized ID for a molecule, the MeSH ID by default. The yellow "Molecule Common Name" field should indicate (for curator convenience) the common, human-readable name of the compound.


Interaction Information 1

RNAi Batch Form 7.png


Interaction Information 2

RNAi Batch Form 8.png


The last 10 columns (AP-AY) capture genetic interaction information. If a relevant Interaction Object ID has already been generated for this RNAi experiment, it may be entered in the "Interaction object ID" field, obviating the need to enter any further interaction information. If not, the following fields must be populated. The "Interaction Type" field provides a drop-down menu to select from a list of standard interaction type names (e.g. "Suppression"). The "Interaction-relevant Phenotype(s)" field records the phenotype ID of the phenotype about which the interaction is based, usually the same phenotype as was given under "Phenotype ID" (Column X). "Interaction Remark" is a free-text field describing the details of the interaction, for example, which gene suppressed which other gene with respect to what phenotype and under what conditions.

The "Gene" column reports the WormBase Gene ID (e.g. WBGene00000898) for the genes involved in the interaction. The yellow "Gene_name" column records the human-readable gene name of each gene involved in the interaction. The "Variation" records the allele/variation name of the gene involved in the genetic interaction, if applicable. This must be a recognized WormBase allele/variation name. If a transgene is involved in the interaction, the name of the transgene (official WormBase name) must be entered here. The "RNAi" column captures ("YES" or empty) whether or not the gene for that row was perturbed by RNAi in the interaction. "Direction" refers to the directionality of the interaction and the role that the gene (for that row) had in the interaction. This field provides a drop down list of flags: "Non_directional", "Effector", and "Effected". A suppressor/enhancer gene/mutation would be an "Effector" and the suppressed/enhanced gene/mutation would be the "Effected", whereas if the interaction is Non-directional (as in mutual enhancment, for example), each gene/mutation would be flagged as "Non_directional".


Submitting the Batch Form to Generate an *.ACE File

Once you have finished filling out the batch form and you are ready to submit it for processing into a *.ace file, you first need to convert it to a *.csv tab-delimited file. One option for doing this is to open the Excel document in Open Office, delete the yellow columns, and then "Save As..." and choose the "Text CSV (.csv)" option:

Open Office Save As CSV.png

Once you click "Save", the following window may come up:

Open Office Keep Current Format.png

Click on "Keep Current Format". You will then be brought to the Open Office Export dialog box:

Open Office Export Options.png

Non-Macintosh Export

If you are not working on a Mac, click on the "Field delimiter" field, select "{Tab}", and then click "OK":

Open Office Export Default 1.png


Open Office Export Default 2.png


Macintosh Export

Alternatively, if you work on a Mac, select the "Character set" field in the dialog box and select "Western Europe (Apple Macintosh)". Next, select the "Field delimiter" field and select "{Tab}". Then click "OK"

Open Office Mac Export 1.png


Open Office Mac Export 2.png


Open Office Mac Export 3.png


This will save your *.csv file in the current directory (where the *.xls file is) or wherever you have specified.


Next, you will have to secure-copy your *.csv file to curation@elbrus.caltech.edu. Open a terminal, cd to your directory, and at the prompt type:

$ scp *.csv curation@elbrus.caltech.edu:~/RNAi/YourDirectory

(OR just "$ scp *.csv curation@elbrus:~/RNAi/YourDirectory" if you are on Caltech campus)

Enter the password.

Note: you will need to establish "YourDirectory" under the "RNAi" directory and place the batch form processing script into that directory.

Next, acquire secure-shell access to curation@elbrus:

$ ssh curation@elbrus

Enter the password.

Move to your directory:

$ cd RNAi/YourDirectory

Now run the Perl script with the following syntax:

$./rnai_batch_mapping.pl -i *.csv -t -o YourOutputName_Test.ace -e YourErrorFile_Test.txt

The "-i" option indicates your input *.csv file, the "-t" option indicates that you are running a test process of the *.csv file that will use temporary WB RNAi IDs (in case there are errors you don't want to assign new WBRNAi IDs), the "-o" option indicates the output *.ace file, and the "-e" indicates the error text file that will capture and report any errors that occur during processing.

The script will need to run for a few minutes (maybe several minutes depending on the size of you *.csv file). Once the script is done, check the error.txt file for any errors that may have occurred. If errors did occur, you will need to track down the source of the errors, fix them, and re-run the test processing.

The script generates two *.ace files: YourOutputName_Test.ace and RNAi_interaction.ace

Once your processing is free of errors, you may wish to test both *.ace files by reading them into ACEDB and ensuring that they read into the database with no errors.

Once your processing is free of errors, you will run the "real" processing as follows:

$./rnai_batch_mapping.pl -i *.csv -r -o YourOutputName.ace -e YourErrorFile.txt

Note that the only change is from using the "-t" option to the "-r" option. The "-r" option assigns real WB RNAi IDs to the RNAi experiments in the *.ace file.

Once the real processing runs smoothly and there are no errors, test the two *.ace files by reading them into ACEDB and making sure they read in without any errors. If both files read in without error, both files are ready to be submitted for upload to the database. Secure-copy the *.ace files to your local directory and archive them as needed.


Uploading the RNAi interactions .ACE file to the Interaction OA (Ontology Annotator)

Once you've generated you're RNAi interactions .ACE file and tested it by reading it into ACEDB (with no errors), you can upload the file to the Interactions OA. To do this, follow the instructions on the Gene Interaction Wiki Page

RNAi Paper Flagging and Processing of SVM "Low" Results

We have been using SVM to flag papers that may contain RNAi data. These papers are given a probability score (High, Medium or Low) based on the SVM training set to reflect the possibility that they contain RNAi data to be curated. The High and Medium scoring papers are automatically added to the list of RNAi papers that are to be curated (on the Paper Editor) and the Lows are stored separately. The Low scoring papers must be manually checked by a curator and if the paper has "curatable" RNAi data, the curator must manually add that paper to the list. The Low scoring paper list is being checked approximately every 3 months (it is added to automatically every Monday at 2am based on SVM results). The following is the SOP for checking these papers and adding them to the list of RNAi papers to curate.

SVM Low's - SOP for checking papers

The file containing the papers with a low priority score (named "low") is stored on tazendra in the directory: /home/postgres/work/pgpopulation/svm/gary_rnai New papers are concatenated to this list. The curator checking the recently added papers must go back in the list to the most recent paper that has a "commented out" mark "//". This mark means that the list has been checked up to this point in the paper list....all papers after this mark need to be manually checked. When the curator is finished, they should "comment out" the last paper on the list so the next curator knows where to begin checking next time.

SVM Low's - SOP for adding papers to curation pipeline

After checking the "Low" papers, the ones that do conatin data must be added to the curation list. To do so go to the curator_first_pass.cgi on the web which is listed under "curation Forms" on the site map for tazendra. Add the appropriate number of the WBPaper id, for example just "00038308" without the WBPaper prefix, and click on the query button. Under "rnai" in the Gene Function section add the following comment in the curator box "SVM -LOW". When you hit "Flag" at the bottom of the page this paper will be added to the RNAi curation list and noted as an SVM with a low probability score.


RNAi OA

NOTE: Warning: RNAi objects may have multiple lines in the OA due to the nested phenotype data (Penetrance info, etc.)

TAB1

  • pgid - the postgres ID
  • RNAi ID - rna_name - WBRNAiID #Note: This should be automatically assigned 8-digit padding. Assigned automatically based on highest value in rna_name table, if data is created without the OA, make sure to create a new OA line and change the name of the RNAi ID directly through postgres
  • Paper - rna_paper - ?Paper (Ontology)
  • Laboratory - rna_laboratory - ?Laboratory (Multi-ontology)
  • Date - rna_date - Text (YYYY-MM-DD)
  • Curator - rna_curator - Curator (Dropdown)
  • PCR Product - rna_pcrproduct - ?PCR_product (Multi-ontology) #Note: EBI/Hinxton will map these
  • DNA Text - rna_dnatext - Big text #Note: multiple Sequences should be separated by pipe "|" (caution: pressing <ENTER> may cause problems)
  • Sequence - rna_sequence - Text #Note: multiple DNA texts should be separated by pipe "|" (caution: pressing <ENTER> may cause problems)
  • Strain - rna_strain - ?Strain (Ontology) - Already exists in Postgres; No files need to be made for this
  • Genotype - rna_genotype - Big text
  • Treatment - rna_treatment - Big text
  • Life Stage - rna_lifestage - ?Life_stage (Ontology) - Already exists in Postgres; No files need to be made for this
  • Temperature - rna_temperature - Text (Integer)
  • Delivery Method - rna_deliverymethod - (Multi-dropdown) four choices: "Bacterial_feeding", "Injection", "Soaking", "Transgene_expression"
  • Species - rna_species - ?Species (Dropdown) #Note: Add Caenorhabditis_elegans to dropdown list added to species type to top of list
  • Remark - rna_remark - Big text


TAB2

  • From Genereg - rna_fromgenereg - Toggle #Note: We would like a toggle here for Xiaodong to create new RNAi objects while performing Gene_regulation curation and flag them for an RNAi curator to complete (just like the "From RNAi" toggle in the Gene_regulation OA) Named field "From Gene Reg" to make it smaller, I'd prefer "From Genereg" or "From GeneReg", if you'd really rather have "From Gene Regulation" let me know and I'll change it. This is important before we start curation because the name of the field is the value that gets stored then the field is toggled on, so changing the field name later would mean changing the data in postgres as well -- J Sure, this sounds fine. We can keep it as small as you want; so, "From Genereg" is fine with me -- C
  • Phenotype Observed - rna_phenotype - ?Phenotype ID & name (Multi-ontology)
    • Penetrance From - To - rna_penfromto - Text (Integer-space-Integer)
    • Penetrance - rna_penetrance - dropdown like in phenotype OA
    • Heat Sensitive - rna_heatsens - Toggle
    • Cold Sensitive - rna_coldsens - Toggle
    • Quantity From - To - rna_quantfromto - Text (Integer-space-Integer)
    • Quantity Description - rna_quantdesc - Big text
    • Phenotype Remark - rna_phenremark - Big Text -- Populate from Phenotype lines that have a Remark tag in #Phenotype_info following in .ace file.
  • Phenotype Not Observed - rna_phenotypenot - ?Phenotype ID & name (Multi-ontology)
  • Molecule - rna_molecule - ?Molecule (Multi-ontology)


TAB3

  • Person Evidence - rna_person - multiontology - for ?RNAi object's #Evidence
  • History Name - rna_historyname - Text #Note: this field is to accommodate older RNAi objects
  • Movie - rna_movie - Text #Note: this field is to accommodate older RNAi objects
  • Database - rna_database - Text #Note: this field is to accommodate older RNAi objects I'm not sure how this will get displayed in this form. There are actually three inputs for this field in the RNAi model. There is 'Database', 'Database_field', and 'Accession_number'. It would be ideal to accommodate all three of these and display them in the OA. Does that make sense? -- C Enter data in this format, split multiple database lines with pipes if there are any (haven't looked at data yet) :
    • Phenobank2 Gene&RNAID GeneID=507328 | Phenobank3 Gene&RNAID GeneID=123456
  • Expression Profile - rna_exprprofile - Text #Note: this field is to accommodate older RNAi objects
  • Method - rna_method - Text #Note: this field is to accommodate older RNAi objects
  • NODUMP - rna_nodump - toggle - Would it be possible to add a 'NO DUMP' toggle here as well? I think we want to have Xiaodong turn that on when she creates a new object from Gene_regulation curation so that a partially curated object doesn't get dumped until after an RNAi curator gets to it -- C added, but you might want it next to the From Genereg field so that the curator doesn't need to switch to an extra tab just to click another toggle. I'm assuming that they won't be entering a lot of data, and that they'd do this fairly regularly.


NOT USING

  • Evidence - postgres table and field not created, don't know what to parse into this field - Text #Note: this field is to accommodate older RNAi objects So there are only 8 RNAi objects that use the Evidence field, and they do so with 'Person_evidence'. In the OA, I think we should include: Person_evidence, Author_evidence, Curator_confirmed, Laboratory_evidence -- C We're only keeping Person_evidence for the #Evidence hash associated with the whole ?RNAi object -- C+J
  • Penetrance Incomplete - rna_penincomplete - Toggle I think we talked about this, but in case we didn't phenotype OA uses a dropdown for the 4 penetrance types instead of separate toggle fields -- J OK, this sounds like a good idea. Let's just make it a dropdown list here as well -- C
  • Penetrance Low - rna_penlow - Toggle
  • Penetrance High - rna_penhigh - Toggle
  • Penetrance Complete - rna_pencomplete - Toggle

Notes

Fields for the tags "Predicted_gene", "Gene", "Transcript", "Pseudogene", "Homol_homol" and "Uniquely_mapped" will be omitted from the OA and populated in ACEDB after probe mapping to the genome at the EBI during the build process.

Parsing WS RNAi into OA

There papers are not getting parsed into postgres tables for RNAi : WBPaper00004402, WBPaper00004403, WBPaper00004651, WBPaper00005654, WBPaper00006395, WBPaper00024497, WBPaper00025054, WBPaper00029258

/home/postgres/work/pgpopulation/rna_rnai/makeRnaiCuratorMappings.pl generates a mapping of RNAi objects to WBPerson curators based on WS231RNAiwithTimeStamp.ace

Ignore all #Evidence in #Phenotype_info and in Remark tag of ?RNAi object.

Check if any data doesn't have Method "RNAi"

These tags are not getting parsed into the OA : Author Gene Gene_regulation Homol_homol Interaction Predicted_gene Uniquely_mapped

These Phenotype subtags are not getting parsed into the OA : Curator_confirmed Molecule Paper_evidence . To see the Molecules not populated look on the sandbox at /home/postgres/work/pgpopulation/rna_rnai/phen_subtags_not_populated

Is there only one Not phenotype in all the data ? (WBRNAi00085012) If so, it'd be a lot easier to ignore this, and after it's live add the phenotype manually through the OA. If not, let me know some example of other objects, because I can't find any to test for a parsing pattern. Is this the same as Phenotype_not_observed ?

We're not keeping remark nor any other #Phenotype_info for the field Phenotype_not_observed right ? There's no place to put it

DNA_text is a double text field. I'm assuming that the first part is always going to be a string of atcg so will never have a space. In that case you could store the data in this field like qq(<atcg><space><next_thing>) but if you think you'd ever want spaces the better divider would be qq(<atcg>" "<next_thing") I didn't see anything else in the model that stores two values in a row besides the From-To fields, if there are any, let me know and we'll talk about them. I need to figure this out to know how to parse stuff. We talked about Database which I'm treating as a triple text field (instead of ontology values) and I think you don't have spaces in those values either, but if you do we should talk about it.

The OAs that have lifestage store the WBls:####### ID instead of the name, but the .ace file has names, so either there was some kind of translation, or more likely the data in there is wrong (someone should have switched it when the ?Life_stage class changed from text to IDs). I'm converting from names to IDs to store in postgres, but some names don't match the list of valid values in postgres in obo_name_lifestage. The list of RNAi objects and their Life_stage text is at /home/postgres/work/pgpopulation/rna_rnai/bad_lifestage

There are a lot of pcr_products at /home/postgres/work/pgpopulation/rna_rnai/bad_pcrproduct but we might be able to get a lot more matches if we make it case insensitive. Does case matter ?

There are a couple of bad papers at /home/postgres/work/pgpopulation/rna_rnai/bad_papers One paper is invalid linked to a lot of RNAi, the other has a typo lowercase p

For all these problem objects if you want to fix them, we can edit the source .ace file, but if we ever switch the dataset to populate, we'd have to make all the changes in the new .ace file again. Probably easiest would be to tell me which objects to ignore and I can hardcode in not giving an error for those, not populating postgres with them, and keep track to populate through the OA when live. (if you want to keep track and change in the .ace file, that's easier for me, but in the past we've had to make dumps of .ace several times and used various sets so if it's the same with this datatype, it will be more of a pain for you

The .ace file has the species as "Caenorhabditis elegans", the value you wanted me to add to the OA (fits the other values's format) is "Caenorhabditis_elegans" so I'm converting from space in .ace to underscore for postgres.

To sync to tazendra

  • Create OA tables /home/postgres/work/pgpopulation/rna_rnai/create_rnai_tables.pl
  • Create obo tables for pcr product data /home/postgres/work/pgpopulation/obo_oa_ontologies/20120322_pcrproduct/create_obo_tables.pl
  • Populate obo tables for pcr product based on WS229_PCR_products.txt with /home/postgres/work/pgpopulation/obo_oa_ontologies/20120322_pcrproduct/populate_obo_pcrproduct.pl (takes 30 minutes, lots of lines)