Difference between revisions of "WormBase-Caltech Weekly Calls"
From WormBaseWiki
Jump to navigationJump to searchm |
m (→March 3, 2011) |
||
Line 8: | Line 8: | ||
== March 3, 2011 == | == March 3, 2011 == | ||
+ | |||
+ | SUMMARY: | ||
+ | |||
+ | Every other month patch: | ||
+ | *We will try to generate an ACEDB patch every other month, starting with the Paper class data | ||
+ | *Need to coordinate with Jauncarlos and Todd | ||
+ | |||
+ | Need to check .ACE patch files before upload: | ||
+ | *Will need to be vigilant about checking for errors and inconsistencies before sending to Todd to put up on website | ||
+ | *Curators need to check their own data for problems | ||
+ | *Wen will check for consistency between the different data types | ||
+ | |||
+ | Think about connecting website to Postgres: | ||
+ | *Think about showing Postgres data "immediately" on site | ||
+ | |||
+ | |||
+ | DETAILS: | ||
Delayed release cycle: | Delayed release cycle: |
Revision as of 18:33, 3 March 2011
2011 Meetings
March 3, 2011
SUMMARY:
Every other month patch:
- We will try to generate an ACEDB patch every other month, starting with the Paper class data
- Need to coordinate with Jauncarlos and Todd
Need to check .ACE patch files before upload:
- Will need to be vigilant about checking for errors and inconsistencies before sending to Todd to put up on website
- Curators need to check their own data for problems
- Wen will check for consistency between the different data types
Think about connecting website to Postgres:
- Think about showing Postgres data "immediately" on site
DETAILS:
Delayed release cycle:
- Will require more work to prepare for more frequent release of certain data types
- Aside from Kimberly's data, most data types are not urgent (e.g. Expression pattern)
- What are the users feeling?
- Having data faster will help users; they don't ask, because they don't see it
- On-the-fly updating of website? Like Postgres?
- Since we use ACEDB, we have to patch WS with .ACE file, or rebuild whole thing
- Flat file Postgres database, replaced every night?
- Website calls Postgres directly for certain data types?
- Performing build without sequence is easy? Do everything without sequence?
- How to integrate sequence data with other data once they're decoupled through the patching process?
- We need .ACE patch files
- Concise description separate from most else (but connected to papers)
- Do papers first?
- Website can show anything
- If we have a lot of patches, will not have check for data inconsistency/confliction
- Trial patch .ace files for papers first
- Juancarlos: Scripts that check differences between data dumps; scripts are data type specific
- Curators need to talk to Juancarlos about the importance of different data tags
- Paper .ACE file: Would include bibliographic info, journals, authors, genes associated from abstract or added manually
- One reason for more frequent releases: because we have first pass author forms; show them we add it quickly
- what will be added through the forms: expression patterns? RNAi (difficult?)?
- We should check patch before we send to Todd!!! Don't want to crash database
- How frequently to patch? Weekly? Daily? Check with Todd, how often he can load them?
- Chron job to create patch ACE files, send to curators to check for problems, then send to Todd
- Interdependency of data types; curators rely on other curators?
- Postgres directly to website? Todd would have to work it out
- New information flag on website? Toggle visibility?
- How do we know that the data do not conflict with each other?
- What are common problems? Dumper script goes bad, makes broken lines, empty fields
- Error catching mechanisms? More checks on postgres? Dump files?
- Data merging problems? What are the cases that are conflicts? Prevent them? Know beforehand?
- If we don't know, as long as it doesn't crash the database or fail to load, then OK
- Don't do -D stuff, maybe? No deletions? Skip typos?
- Always have to check ACE files anyway, but have to do every week (2 weeks?)
- We can try a patch every other month
- What can we do without the patch?
- Did SAB talk about changing to relational databases?
- Get website going as is first, and see if it matters?
- If people don't want to change data models, we can switch over to relational
- Separate panel on website directly from Postgres?
- Wen can check the data integration every other month for patch