Specifications for CCC Curation from Textpresso Search Page

Requirements for Using Textpresso Search Results in General CCC Curation

These specifications are for allowing a curator to search any Textpresso implementation using the CCC categories, submit the resulting sentences to a curation form, make annotations, and download the annotations in a simple three-column format, or the GO's gene_association file (GAF) format.

Searches

Curators would be able to search Textpresso using their chosen criteria and export the sentences to the CCC curation form. The XML mark-up of each returned sentence will be used to populate the curation boxes on the form and color-code the search results. Curators would need an option to send results to a curation form with a box to name the search so they can select it in the list of source files on the curation form.

Future options for filtering search results may include filtering papers that are in the corpus but not appropriate for curation (e.g., WB's 'functional annotation papers' or TAIR's black list of papers on other organisms) or restricting searches to a particular level of SVM classification (e.g., high-confidence SVM papers only). For the former, we would need a consistent tag? For the latter, we would need to integrate the SVM classification results into postgres?

From Searches to the Curation Form

This pipeline would make use of the XML format of returned sentences to construct a version of the current CCC curation form; a version for TAIR is shown here:

http://tazendra.caltech.edu/~azurebrd/cgi-bin/forms/tair/tair_ccc.cgi

1) Keep three action buttons on the top - Submit, Search for Sentence, Search for Paper

2) Display all information from within the <bibliography> tags.

This will display more information than is currently shown, but seemed easier than just picking a few tags from within the bibliography section. The additional information may also be helpful to curators.

3) Do not display information within any <field_references> tags.

Is this how scrambled sentences are typically identified?

4) Potentially curatable sentences are found within the <field_results> tags. Working from left to right on the curation form, here is how the information from the XML would translate to the curation form.

First box, Gene/Protein Name

This box lists all entities within the species-specific protein or gene tag. Right now, the category name for this will be different for each implementation, for example:

protein_celegans

genes_arabidopsis

dicty_genes

For TAIR we displayed both the name of the gene as presented in the sentence, as well as the name in the sentence mapped to a specific TAIR gene ID. This is because some Arabidopsis gene names are used for more than one locus and the TAIR curators wanted a way to ensure they'd be making the annotation to the correct locus. It would be fine to implement this universally, I think. This requires a mapping file from each group and a regular pipeline for updating the gene or protein names file.

Second box, Component Term in Sentence

This box lists the component term as identified by the component category.

For the various implementations there are currently several component categories.

For elegans and Arabidopsis:

localization_cell_components_082208 CCC_TAIR

For dicty:

localization_cell_components_050808 localization_cell_components_082208

If more than one cellular component category is used in the mark-up, we will need to figure out how to determine which one the curator wants to display on the curation form. Alternatively, we could restrict each implementation to only one component category at a time.

Currently, this box only displays the component terms in sentences that matched the category. It might be helpful, though, if in the future curators could enter new component terms displayed in the sentence but not in the category and have this entry somehow be added to the category for the next mark-up.

Third box, CC Term in GO

This box displays, if available, any GO terms that have already been curated from component terms in sentences, or allows curators to enter a new GO term, if needed.

The suggested GO terms come from the relationship index that has been built up over time from elegans curation. I think it'd be good to use one relationship index for all implementations to leverage community efforts.

Curation Actions and Sentence Display

The color-key displayed at the top of the sentence can stay.

The curator options are:

Curating

To make a new GO annotation, curators need to select, or create, an entry in each of the three boxes on the left-hand side of the form. Once a selection is made, the region will be highlighted in blue. For the last column, CC term in GO, curators can either select one of the suggested GO terms in the list (where is this relationship file located, i.e. its path --K), or enter a new one. Then select curate from the list of radio buttons above the sentence and, if you are ready to enter your annotations, click on Make connections:Submit at the top of the page.

If there is no list of GO terms in the third box, that means that a GO annotation using the term in the second box has not yet been made. In this case, the curator will need to enter the new GO term manually.

An autocomplete function for adding new GO terms would be good here. Also, we need a more efficient way to handle having to make duplicate annotations from one sentence. Right now I reload the sentence using the search sentence button, but it'd be good if curators could make more than one annotation from a sentence more easily.

The way the form currently works is that once a sentence has been annotated and the information submitted, the sentence is removed from the display. Curators may want to be able to see what they've annotated though, so we might need to come up with a different strategy. Also, since some of these source files can be quite large, curators may want to have a way to just pick up where they left off if we decide not to remove sentences after curation.

Marking Sentences Not Used for Curation

If an annotation cannot be made from a sentence, then curators may record the reason that an annotation was not made. Keeping track of these sentences will help build up a training set for improving search results. The different options for not curating are described below:

Already Curated/Already Done

If a curator does not wish to make another annotation for information previously curated, they can select the appropriate entity from each curation box and the Already Curated radio button.

The way this was supposed to work for elegans was that the information about previous curation would be entered by curators using this form, stored in a postgres table and then, when next presented, displayed at the bottom after the Already Done heading in red. I'd imagine that this feature is something groups may want to specify from the outset of their searches with options perhaps to include them in the curation boxes, show the information as Already Done but not include them in the curation boxes, or not show this information at all.

Scrambled Sentence

If, during the pdf-to-text conversion, a sentence has become scrambled, curators can mark it as such here. These are becoming much less frequent.

Should we label this Scrambled Sentence/Run-on Sentence to collect both, split these two or not worry about the run-on sentences? Having a run-on sentence does not necessarily preclude making an annotation, though, so these options might have to become check boxes instead of radio buttons.

False Positive

If a returned sentence has nothing to do with subcellular localization, then it is marked as a false positive. For example:

In contrast, PP2AA3 rescues root tip organization weakly even when expression is driven by the RCN1 promoter, demonstrating a more stringent requirement for A subunit function in the root apical meristem.

These sentences could be saved in a flat file possibly for future category refinement.

Not GO Curatable

This is intended to mark sentences that may describe subcellular localization, but the information contained in them would not normally be curated for GO. For example, the localization described is for a mutant protein, or the localization is for the wild-type protein in a mutant background. An example sentence:

No alteration in expression levels of soluble GFP or GFP::RAB-3 was observed in the synapses, cell body or axon in uba-1 animals (Figure 1A: d1-d6, f1-f6, 1C, 1D, Figure S4F, S4G, S4H).

Like the false positives, these sentences could be used to refine searches, e.g. to develop lists of terms or HMMs, that could be used to filter potentially uncuratable sentences from the search results.

Dumping Annotations

Annotations will be stored in postgres tables with two options available for dumping them out into tab-delimited text files. These options would be presented as buttons on the curation form.

Here are examples of what would be contained in the files using the sample sentence below:

SentenceID 9 -- S 7 P 43065 S s3 E The first enzyme, gamma-glutamate cysteine ligase (GSH1), responsible for synthesis of gamma-glutamylcysteine (gamma-EC), is, in Arabidopsis, exclusively located in the plastids, whereas the second enzyme, glutathione synthetase (GSH2), is located in both plastids and cytosol.

1) Three-column, tab-delimited format

1) 3-column user submission file

Locus Name	GO ID	Paper ID
Column 1 of ftp mapping file	GO:0009536	PMID:nnnnnnnn or TAIR:43065

AT4G23100 GO:0009536 TAIR:43065

AT5G27380 GO:0009536 TAIR:43065

AT5G27380 GO:0005829 TAIR:43065

For column 1, map the gene symbol (second column) to the locus name (first column) in the gene_aliases file:

ftp://ftp.arabidopsis.org/home/tair/Genes/gene_aliases.20101027

For column 2, use the geneontology.obo file to map the GO term to the GO ID:

http://www.geneontology.org/ontology/obo_format_1_2/gene_ontology_ext.obo

For column 3, use the PMID (preferred) or the TAIR paper ID, pipe separated, prefaced PMID: and TAIR: respectively.

This information could be taken from the bibliography XML.

2) 18-column, tab-delimited GAF 2.0

Column	Content	Required	Cardinality	TAIR Entry
1	DB	Required	1	TAIR
2	DB Object ID	Required	1	Column 1 of gene_aliases file
3	DB Object Symbol	Required	1	Column 2 gene_aliases file
4	Qualifier	Optional	0 or greater	NULL
5	GO ID	Required	1	GO:0005654
6	DB Reference	Required	1 or greater	PMID:21074051 or, if no PMID, TAIR:42184
7	Evidence Code	Required	1	IDA
8	With or From	Optional	0 or greater	NULL
9	Aspect	Required	1	C
10	DB Object Name	Optional	0 or 1	Column 3 of gene_aliases file
11	DB Object Synonym	Optional	0 or greater	ASK TANYA
12	DB Object Type	Required	1	protein
13	taxon)	Required	1 or 2	taxon:3702
14	Date	Required	1	Date annotation is made
15	Assigned By	Required	1	TAIR
16	Annotation Extension	Optional	0 or greater	NULL
17	Gene Product Form ID	Optional	0 or greater	NULL

Right now, TAIR only needs the simple, three-column format. Some elements of the Gene Association File Format, such as Taxon ID, will need to be hard-coded for each database, so we will have to store that information somewhere.

Back to Gene Ontology