https://wiki.wormbase.org/index.php?title=RSS_implementation&feed=atom&action=historyRSS implementation - Revision history2024-03-28T23:14:29ZRevision history for this page on the wikiMediaWiki 1.33.0https://wiki.wormbase.org/index.php?title=RSS_implementation&diff=6095&oldid=prevCgrove at 18:33, 16 August 20102010-08-16T18:33:05Z<p></p>
<table class="diff diff-contentalign-left" data-mw="interface">
<col class="diff-marker" />
<col class="diff-content" />
<col class="diff-marker" />
<col class="diff-content" />
<tr class="diff-title" lang="en">
<td colspan="2" style="background-color: #fff; color: #222; text-align: center;">← Older revision</td>
<td colspan="2" style="background-color: #fff; color: #222; text-align: center;">Revision as of 18:33, 16 August 2010</td>
</tr><tr><td colspan="2" class="diff-lineno" id="mw-diff-left-l79" >Line 79:</td>
<td colspan="2" class="diff-lineno">Line 79:</td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #222; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"></td><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #222; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #222; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div></pre></div></td><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #222; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div></pre></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="color: #222; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins style="font-weight: bold; text-decoration: none;"></ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="color: #222; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins style="font-weight: bold; text-decoration: none;"></ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="color: #222; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins style="font-weight: bold; text-decoration: none;"></ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="color: #222; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins style="font-weight: bold; text-decoration: none;">[[Category:Curation]]</ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="color: #222; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins style="font-weight: bold; text-decoration: none;">[[Category:Developer documentation]]</ins></div></td></tr>
</table>Cgrovehttps://wiki.wormbase.org/index.php?title=RSS_implementation&diff=447&oldid=prevTharris at 21:53, 11 November 20072007-11-11T21:53:20Z<p></p>
<table class="diff diff-contentalign-left" data-mw="interface">
<col class="diff-marker" />
<col class="diff-content" />
<col class="diff-marker" />
<col class="diff-content" />
<tr class="diff-title" lang="en">
<td colspan="2" style="background-color: #fff; color: #222; text-align: center;">← Older revision</td>
<td colspan="2" style="background-color: #fff; color: #222; text-align: center;">Revision as of 21:53, 11 November 2007</td>
</tr><tr><td colspan="2" class="diff-lineno" id="mw-diff-left-l79" >Line 79:</td>
<td colspan="2" class="diff-lineno">Line 79:</td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #222; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"></td><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #222; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #222; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div></pre></div></td><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #222; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div></pre></div></td></tr>
<tr><td class='diff-marker'>−</td><td style="color: #222; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div><del style="font-weight: bold; text-decoration: none;"></del></div></td><td colspan="2"> </td></tr>
<tr><td class='diff-marker'>−</td><td style="color: #222; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div><del style="font-weight: bold; text-decoration: none;">==TODO==</del></div></td><td colspan="2"> </td></tr>
<tr><td class='diff-marker'>−</td><td style="color: #222; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div><del style="font-weight: bold; text-decoration: none;"></del></div></td><td colspan="2"> </td></tr>
<tr><td class='diff-marker'>−</td><td style="color: #222; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div><del style="font-weight: bold; text-decoration: none;">Add in the correct RSS link into the page header. This should point at the feed in the header, populated with the appropriate object and class.</del></div></td><td colspan="2"> </td></tr>
</table>Tharrishttps://wiki.wormbase.org/index.php?title=RSS_implementation&diff=446&oldid=prevTharris: New page: ==OBJECTIVE== Create a simple RSS based notification system to easily track changes to any given object in the database. ==Approach== Since our databases are build-use-discard, the RSS ...2007-11-11T21:30:42Z<p>New page: ==OBJECTIVE== Create a simple RSS based notification system to easily track changes to any given object in the database. ==Approach== Since our databases are build-use-discard, the RSS ...</p>
<p><b>New page</b></p><div>==OBJECTIVE==<br />
<br />
Create a simple RSS based notification system to easily track changes to any given object in the database.<br />
<br />
==Approach==<br />
<br />
Since our databases are build-use-discard, the RSS system requires a database to track changes over time. Here's how it works.<br />
<br />
1. The first time a feed is requested, the appropriate object is fetched with timestamps on.<br />
<br />
2. This object is stored in a database with it's modification time set arbitrarily to 2007-01-01.<br />
<br />
3. The Tree of the object is parsed.<br />
<br />
4. Entries in the object that are *newer* than the arbitrary default modification date are saved, parsed, and linked as appropriate.<br />
<br />
5. Entries are concatenated and added to a second "history" table. The data schema formally has a simple one object-many history structure. Coupled with the parsed notes this lets us track many changes for a given object over time -- and even to pinpoint what database build these changes were made.<br />
<br />
6. Updated entries are found and stored in the history table, one row for each time anything is updated. Determining new entries and storing them in the database only needs to be done once. For subsequent requests, updated entries are culled directly from the database.<br />
<br />
===Ramifications===<br />
<br />
1. Feeds are created dynamically as needed.<br />
<br />
2. Database grows organically over time in response to user requests for feeds. No need to precalculate or maintain all RSS feeds as separate files.<br />
<br />
3. The RSS feed corresponds to the predominant object displayed on a given page only and not its full contents.<br />
<br />
== Determining if an object has changed ==<br />
<br />
I considered three options for determining if an object had changed from one release to another.<br />
<br />
1. Parsed Ace::Object tree<br />
<br />
The method I selected is the slowest but most effective approach. With this approach, it is easily possible to save the entire path to a modified item. This allows for a more meaningful RSS feed with hyperlinked objects.<br />
<br />
Alternative approaches:<br />
<br />
2. md5_sum of XML dump<br />
<br />
For a given class, visit each object. Dump out the data as XML and create an md5sum. Store the object ID, class, date, and the md5sum in a separate database.<br />
<br />
With each new release, repeat the procedure. If the md5sum is DIFFERENT, update the record in the database and create a new static RSS feed entry for that object. Otherwise ignore it. I also considered using the HTML content of the page itself but reasoned that small changes in the UI would trigger new entries in the RSS feeds.<br />
<br />
Using this approach, we have no way of indicating how the object has changed, only that it has.<br />
<br />
3. Parsed XML dump<br />
<br />
Parse the XML dump and look for time stamps newer than the last modification date of the object. This approach is rather tedious in terms of displaying useful information as one must climb up from the timestamp in a complicated (Perl) data structure.<br />
<br />
<br />
== Schema ==<br />
<br />
This is a quick hack. I have no idea how well this will scale.<br />
<br />
<pre><br />
<br />
DROP TABLE IF EXISTS `objects`;<br />
CREATE TABLE `objects` (<br />
`oid` int(11) NOT NULL auto_increment,<br />
`name` char(35) NOT NULL default '',<br />
`class` char(35) default '',<br />
`signature` char(32) NOT NULL default '',<br />
`last_modification_date` date default '2007-01-01',<br />
`ace_version` char(5) NOT NULL,<br />
PRIMARY KEY (`oid`)<br />
) ENGINE=MyISAM DEFAULT CHARSET=latin1;<br />
<br />
DROP TABLE IF EXISTS `history`;<br />
CREATE TABLE `history` (<br />
`sid` int(11) NOT NULL auto_increment,<br />
`oid` int(11) NOT NULL,<br />
`version` char(5) default '',<br />
`date` date,<br />
`signature` char(32) NOT NULL default '',<br />
`notes` text default '',<br />
PRIMARY KEY (`sid`)<br />
) ENGINE=MyISAM DEFAULT CHARSET=latin1;<br />
<br />
</pre><br />
<br />
==TODO==<br />
<br />
Add in the correct RSS link into the page header. This should point at the feed in the header, populated with the appropriate object and class.</div>Tharris