Difference between revisions of "Chado Schema Documentation HOWTO"

Revision as of 18:22, 24 November 2010

This page or section is under construction.

You bet it is.

The Chado schema documentation on this wiki is a mixture of generated content and material directly entered by GMOD users into this wiki. The generated part of the documentation consists of the table definitions that are included on the Chado module pages and on the Chado Tables page, listing all the tables in Chado.

All of the table descriptions on Chado module pages and the Chado Tables page are generated every time their is a new release of Chado. The column and table details, including comments, come from the PostgreSQL data dictionary.

Using Module and Table Documentation

This section describes how to use the Chado Schema Documentation when updating/creating content on this wiki.

Showing a Table Description

To show the information about any table, use the Chado Table Template for that table:

 {{ChadoTable_tablename}}

For example,

 {{ChadoTable_cvterm}}

will show the table description for cvterm:

Linking to Module Documentation

To link to a module page from another wiki page use:

 {{ChadoModuleLink|Module Name|text to show}}

For example:

 {{ChadoModuleLink|Publication|pub module}}

Which is shown as:

pub module

Linking to Table Documentation

To link to a specific table's description on a wiki page, use:

 {{ChadoTableLink|table_name}}

For example:

 {{ChadoTableLink|feature}}

will result in:

feature

Note that this links to the table description on the Chado Tables page, not to the description on the table's module page.

Updating Table Documentation Part I

Chado Table Templates are the building blocks of the Chado schema documentation. They are also auto-generated, and cannot be directly updated by editing them on the wiki. Preventing editing of the templates ensures that user updates are not just written over and lost the next time the templates are auto-generated.

However, it is possible to update table documentation. It is a two-step process and you must take the first step. Here's the recipe:

1. Enter your update on the wiki.
   1. Login to the wiki
   2. Go to the module page for the module the table is in.
   3. Click on the [edit] link to the right of the table you want to comment on.
   4. Go to the edit window towards the bottom of the page, and add your comments below the section "Additional Comments"
   5. Save the edits.
2. The next time Chado is released, all the module pages will be reviewed for any comments that have been added since the last update.
   1. New comments will be added to the Chado SQL table definitions (and probably dropped from the "Additional Comments" sections).
   2. The table documentation will be regenerated and reposted to the wiki as part of the Chado release.

See Updating Table Documentation Part II for more on how step #2 is done.

Updating Module and Table Documentation Part II

Table Templates are themselves a nest of smaller MediaWiki templates. This means it's hard to figure out how the wiki decides what to show. However, all this complexity doesn't really matter to the wiki user or even to the wiki editor. It is all auto-generated, and it is never directly updated using the wiki interface. The upside of this complexity is that it is easy to change the appearance of all Chado tables in the wiki. All you do is modify the appropriate template.

This section describes how to regenerate the Table Templates from a live Chado database as part of creating a new release of Chado. If you are not creating a Chado release, or not a GMOD web site manager, then you don't care about this section.

This step is itself a multistep process:

1. Integrate new comments on module pages with comments in the SQL schema.
2. Regenerate the wiki content
3. Push new wiki content to GMOD.org.

Integrate New Comments Into SQL DDL

This step involves walking through all the Chado Module pages, looking at any "Additional Comments" that have been added since the last Chado release, and then integrating them with the comments in the SQL DDL definitions of the tables. Integrated comments should then be removed from the module pages.

Regenerate Wiki Content

Next regenerate the wiki content after you have created a Chado instance using the newly updated SQL. This is done with scripts in the Chado source tree:

cd chado/doc/wiki

Edit generateChadoWikiTables.py and update any of these variables that you need to: <python>

1. UPDATE THESE 4 BEFORE RUNNING THE PROGRAM.

DB_NAME = "testdb" DB_USER = "gmodhack" MODULE_TABLE_PATH = "../../modules/module-tables.json" WIKI_DIR = "/tmp/ChadoWikiFiles" </python>

Before you can run this script, make sure that the postgresql_autodoc package is installed. The script won't run without it. Now run:

$ ./generateChadoWikiTables.py
Producing testdb.wiki from ./wiki.tmpl
$

This script generates all the wiki content you will need in the WIKI_DIR directory, which by default is:

Push Regenerated Wiki Text to GMOD.org

Now we need to update the wiki itself. First tar/compress the directory containing all the generated wiki files. Then copy it to the GMOD web server and unpack it. You'll need shell access to the GMOD web server to do this. Then update the three types of pages:

• Table Templates
• Module Pages
• Chado Tables page

Update Table Templates

On the GMOD web server create this script:

<python>

1. !/usr/bin/python
2. ===================
4. Upload all the table templates to the GMOD wiki.

IMPORT_TEXT_FILE_PATH = "/var/www/html/w/maintenance/importTextFile.php" TABLES_DIR = "Tables" MW_USERNAME = "Your gmod.org MediaWiki Username. e.g., 'Clements'" COMMENT = "Table definition for Chado Version a.b on yyyy/mm/dd"

import glob import os

for tablePath in glob.glob(TABLES_DIR + "/*.wiki"):

   tableFile = os.path.split(tablePath)[1]
   tableName = tableFile.split(".")[0]
   command = (
       "/usr/bin/php " + IMPORT_TEXT_FILE_PATH + " --title 'Template:ChadoTable_" + tableName +
       "' --user '" + MW_USERNAME + "' --comment '" + COMMENT + "' " + tablePath)
   print(command)
   os.system(command)

</python>

Set MW_USERNAME and COMMENT appropriately. Also set TABLES_DIR to be the relative path from the script to directory containing the Table Templates.

This script uses the Mediawiki maintenance script ImportTextFile.php to upload the Table Templates.

Update Module Pages

We don't yet have an automated way to synchronize the module pages with the update.

The first time this process is done (the process described on this page), you'll probably want to do a wholesale replacement of the "Tables" section of each page. This will replace the current hard-coded table defs, with the Table Templates. It will also update the table list to be current.

On subsequent updates, you will only need to touch the module pages if a table was dropped or added. The Table Templates will take care of the rest.

Update Chado Tables Page

The Chado Tables page lists every table defined in Chado. Replace this wholesale each time.

Internals

Documentation on the internals of how all this is done.

=== wiki.tmpl

wiki.tmpl defines the template used by postgresql_autodoc to generate the initial MediaWiki definitions of the tables. Since it generates MediaWiki markup and that markup is sensitive to newlines and blank lines, this template is unreadable. Therefore, a version is included below that is indented and has newlines. Note that this version won't work, but hopefully it will make the version that is used be more readable.