Difference between revisions of "GMOD Developer's Guide"

From GMOD
Jump to: navigation, search
m (Server Side Directory Structure)
m (DBMS)
 
(10 intermediate revisions by one other user not shown)
Line 2: Line 2:
  
  
This is the GMOD developer's guide
+
This is the GMOD Developer's Guide.
  
 
==Favored Platforms==
 
==Favored Platforms==
Line 8: Line 8:
 
===Server Side===
 
===Server Side===
  
<li>The following environment variables should be set:
+
The following environment variables should be set:
<ul>
+
 
<li> GMOD_ROOT
+
* GMOD_ROOT
<li> CHADO_DB_NAME
+
* CHADO_DB_NAME
<li> CHADO_DB_USERNAME
+
* CHADO_DB_USERNAME
<li> CHADO_DB_PASSWORD
+
* CHADO_DB_PASSWORD
<li> CHADO_DB_HOST
+
* CHADO_DB_HOST
<li> CHADO_DB_PORT
+
* CHADO_DB_PORT
            </ul>
+
 
<li>OS: any of Linux/BSD, Solaris, OSX.<br />
+
====OS====
        Windows: optional as cost/benefit less favorable.</li>
+
 
<li>Web Server: all of
+
Any of Linux/BSD, Solaris, OSX. Windows: optional as cost/benefit less favorable.
<ul>
+
 
<li>Apache
+
====Web Server====
<ul>
+
 
<li>With Modperl 1.3, temporarily; need to migrate to Modperl 2.0 (currently broken) or CGI soon.</li>
+
All of:
<li>Allen, Ken to investigate effort required to port their apps to ModPerl 2.0 or CGI. </li>
+
 
</ul>
+
* Apache
</li>
+
** With Modperl 1.3, temporarily; need to migrate to Modperl 2.0 (currently broken) or CGI soon.
<li>TomCat 4.1 (auxiliary Web server to support server-side Java/JSP)</li>
+
** Allen, Ken to investigate effort required to port their apps to ModPerl 2.0 or CGI.
</ul>
+
* TomCat 4.1 (auxiliary Web server to support server-side Java/JSP)
 
   
 
   
<li>Languages: all of
+
====Languages====
<ul>
+
 
<li>Perl 5.6.1 or higher </li>
+
All of:
<li>Java 1.4 </li>
+
 
</ul>
+
* Perl 5.6.1 or higher
+
* Java 1.4
<li>DBMSPostgreSQL 7.3</li>
+
 
<li>Compilers: gcc on Solaris</li>
+
====DBMS====
<li>Libraries: Perl bundle; requires some C libraries</li>
+
 
<li>Hardware:
+
[[PostgreSQL]] 7.3 or 8.0
<ul>
+
 
<li>Recommendations based on current MODs to be determined</li>
+
====Compilers====
<li>Minimum: 20GB disk, 512 MB ram, 1.8 GHz Pentium or equiv. Client or server. </li>
+
 
</ul>
+
gcc (on Solaris as well).
 +
 
 +
====Libraries====
 +
 
 +
Perl bundle; requires some C libraries
 +
 
 +
====Hardware====
 +
 
 +
Recommendations based on current MODs to be determined.
 +
Minimum: 20GB disk, 512 MB ram, 1.8 GHz Pentium or equiv. Client or server.
  
 
===Client Side===
 
===Client Side===
  
OS: Linux, Windows, Mac OSX
+
====OS====
Browser: any of Mac IE, Win IE, Netscape. Which versions within these?
+
 
Fly and worm will analyze server logs to determine largest market segments.
+
Linux, Windows, Mac OSX.
How to test?
+
 
 +
====Browser====
 +
 
 +
* Any of Mac IE, Win IE, Netscape. Which versions within these?
 +
* Fly and worm will analyze server logs to determine largest market segments.
 +
* How to test?
  
  
 
==Server Side Directory Structure==
 
==Server Side Directory Structure==
  
<li> $GMOD_HOME=<b>/usr/local/gmod/</b> (default -- set before installing)
+
* $GMOD_HOME=<b>/usr/local/gmod/</b> (default -- set before installing)
<p>  <dl>
+
** <b>bin/</b> -- all scripts
  <dt><b>bin/</b> -- all scripts &amp; executable<br />
+
Prefix all executables with 2-3 letter app prefix
    <dd>Prefix all executables with 2-3 letter app prefix<br />
+
** <b>sbin/</b> -- system binaries that may do dangerous things
  <dt><b>sbin/</b> -- system binaries that may do dangerous things<br />
+
** <b>web/</b>
  <dt><b>web/</b><br />
+
** <b>htdocs/</b> -- db
    <dd>
+
*** <b>cgi-bin/</b> -- appname
      <dl>
+
*** <b>webapps/</b> -- appname
    <dt><b>htdocs/</b><br />
+
** <b>conf/</b> -- all configuration files (attr/value format)
      <dd>db<br />
+
*** <b>gmod.conf</b>  always source this first; global site params
    <dt><b>cgi-bin/</b><br />
+
*** <b>db.conf</b> --  one for each datasource at the installation server, port, DB, login, password, banner/site-specific page config info optionally source one of these after gmod.conf according to "db" URL param (Perl: use CPAN INI parser to load)
      <dd>appname<br />
+
** <b>data/</b> -- inputs and outputs; operator writable -- appname
    <dt><b>webapps/</b><br />
+
** <b>lib/</b> -- appname or shared library, Perl plugins, jar files
      <dd>appname<br />
+
** <b>doc/</b> -- appname
    </dl></dd>
+
** <b>log/</b>
  <dt><b>conf/</b> -- all configuration files (attr/value format)<br />
+
** <b>examples/</b> -- appname
    <dd><b>gmod.conf</b>  always source this first; global site params<br />
+
** <b>src/</b> -- appname
    <dd><b>db.conf</b><br />
+
 
          one for each datasource at the installation<br />
+
 
          server, port, DB, login, password,<br />
+
* $CLASS_PATH := $GMOD_HOME/lib[/app]  
          banner/site-specific page config info<br />
+
Suggested CGI Parameter Names:
          optionally source one of these after gmod.conf<br />
+
** db=short symbolic name of datasource (MOD), used to look up conf file
          according to "db" URL param<br />
+
** class=object class within db to search
          Perl: use CPAN INI parser to load<br />
+
** id=unique primary key
  <dt><b>data/</b> -- inputs and outputs; operator writable<br />
+
** name=human readable, possibly ambiguous string
    <dd>appname<br />
+
 
  <dt><b>lib/</b><br />
+
 
    <dd>appname or shared library<br />
+
* What if multiple DB instances running on same site?
    Perl plugins, jar files.<br />
+
* Package manager?
  <dt><b>doc/</b><br />
+
* Testing process: is a goal. "Testing is good" -- LS
    <dd>appname<br />
+
** Developer side
  <dt><b>log/</b><br />
+
** Release testing: alpha, beta
  <dt><b>examples/</b><br />
+
** Installer side
    <dd>appname<br />
+
  <dt><b>src/</b><br />
+
    <dd>appname<br />
+
  </dl></p>
+
<li> $CLASS_PATH := $GMOD_HOME/lib[/app] </li>
+
<li> Suggested CGI Parameter Names:
+
<ul>
+
<li>db=short symbolic name of datasource (MOD), used to look up conf file</li>
+
<li>class=object class within db to search</li>
+
<li>id=unique primary key</li>
+
<li>name=human readable, possibly ambiguous string</li>
+
  </ul>
+
  
<li> What if multiple DB instances running on same site? </li>
 
<li> Package manager? </li>
 
<li> Testing process: is a goal. "Testing is good" -- LS
 
<ul>
 
<li>Developer side </li>
 
<li>Release testing: alpha, beta </li>
 
<li>Installer side</li>
 
</ul>
 
</li>
 
</ul>
 
  
 
==Interoperation==
 
==Interoperation==
  
<li>Support chado schema for genome features</li>
+
* Support chado schema for genome features
<li>Establish clear naming conventions for Chado extension to avoid intermodule conflicts:<br />
+
* Establish clear naming conventions for Chado extension to avoid intermodule conflicts:
        Use namespaces (using PostgreSQL "create schema" or table prefixes?) corresponding to Chado modules. Or will current conventions suffice?</li>
+
** Use namespaces (using PostgreSQL "create schema" or table prefixes?) corresponding to Chado modules. Or will current conventions suffice?
<li>Different apps should share portions of schema wherever that makes sense </li>
+
* Different apps should share portions of schema wherever that makes sense  
<li>Schema changes need extensive coordination, versioning and testing across all affected apps.</li>
+
* Schema changes need extensive coordination, versioning and testing across all affected apps.
<li>Apps are encouraged to be backward compatible with older versions of schema.</li>
+
* Apps are encouraged to be backward compatible with older versions of schema.
<li>Need to track which app versions work with which schema versions.</li>
+
* Need to track which app versions work with which schema versions.
<li>Releases:
+
* Releases:
<ul>
+
** Major (or "public") release is a coordinated release of all apps, with testing.
<li>Major (or "public") release is a coordinated release of all apps, with testing.</li>
+
** Minor releases / tagged versions as needed by development groups, no guarantee of installable tarball.
<li>Minor releases / tagged versions as needed by development groups, no guarantee of installable tarball. </li>
+
* Schema change coordination manners: notify gmod-schema list of all proposed changes. Is this sufficient (for now?)
</ul>
+
* Support a few standard external reps of data:
</li>
+
** Sequence feature apps should read/write GenBank &amp; EMBL formats (BioPerl and BioJava handle this). GFF? GTF? BED? GAME? BSML? TIGR add BSML adaptors to BioPerl?  
<li>Schema change coordination manners: notify gmod-schema list of all proposed changes. Is this sufficient (for now?)
+
** Citations: Medline flatfile format  
<li>Support a few standard external reps of data:
+
** Expression: MageML for microarrays  
<ul>
+
** Maps: use CMAP tab delimited format. (the new standard :-)  
<li>Sequence feature apps should read/write GenBank &amp; EMBL formats (BioPerl and BioJava handle this). GFF? GTF? BED? GAME? BSML? TIGR add BSML adaptors to BioPerl? </li>
+
** Adaptors for chado-DBI and/or chado-XML?  
<li>Citations: Medline flatfile format </li>
+
* AppDB communication:
<li>Expression: MageML for microarrays </li>
+
** SQL (DBI/JDBC)
<li>Maps: use CMAP tab delimited format. (the new standard :-) </li>
+
** TIGR developing Perl API for chado: get central dogma model, cv_term2genes, writes
<li>Adaptors for chado-DBI and/or chado-XML? </li>
+
** UCLA has developed a Perl middleware API (Class::DBI) for Chado
</ul>
+
** BioPerl2chado -- CJM readonly API -- out of date?
</li>
+
** Chado XML Dumper
<li>AppDB communication:
+
* Bulk dataloading
<ul>
+
** Colin's GAME2chado initializer
<li>SQL (DBI/JDBC)</li>
+
** Chado XML Loader
<li>TIGR developing Perl API for chado: get central dogma model, cv_term2genes, writes</li>
+
** GAME2chadx, chadx2game
<li>UCLA has developed a Perl middleware API (Class::DBI) for Chado</li>
+
** BSML2chado loader @ TIGR
<li>BioPerl2chado -- CJM readonly API -- out of date?</li>
+
<li>Chado XML Dumper </li>
+
</ul>
+
</li>
+
<li>Bulk dataloading
+
<ul>
+
<li>Colin's GAME2chado initializer</li>
+
<li>Chado XML Loader</li>
+
<li>GAME2chadx, chadx2game</li>
+
<li>BSML2chado loader @ TIGR </li>
+
</ul>
+
</li>
+
</ul>
+
  
  
 
==A First GMOD Release would Contain==
 
==A First GMOD Release would Contain==
  
<li>Sample data: </li>
+
* Sample data
<ul>
+
** Fasta files
<li>Fasta files</li>
+
** GenBank files
<li>GenBank files</li>
+
** BLAST output
<li>BLAST output</li>
+
* Loading tools for sample data
</ul>
+
* Stable chado release
<li>Loading tools for sample data</li>
+
* gmod-web with "basic" pages
<li>Stable chado release</li>
+
* Compatible Gbrowse release
<li>gmod-web with "basic" pages</li>
+
* Apollo
<li>Compatible Gbrowse release</li>
+
* Other support tools (chadx dumpers and loaders,xxx2chadx translators)
<li>Apollo (and possibly MANATEE)</li>
+
* DOCUMENTATION (README, INSTALL, CONFIGURATION-HOWTO)
<li>Other support tools (chadx dumpers and loaders,xxx2chadx translators)</li>
+
<li>DOCUMENTATION (README, INSTALL, CONFIGURATION-HOWTO)</li>
+
</ul>
+
  
  
Line 206: Line 182:
 
<li>Author contact information
 
<li>Author contact information
 
</ul>
 
</ul>
 +
 +
[[Category:GMOD Developers]]

Latest revision as of 05:18, 28 January 2008

This page or section needs to be edited. Please help by editing this page to add your revisions or additions.


This is the GMOD Developer's Guide.

Favored Platforms

Server Side

The following environment variables should be set:

  • GMOD_ROOT
  • CHADO_DB_NAME
  • CHADO_DB_USERNAME
  • CHADO_DB_PASSWORD
  • CHADO_DB_HOST
  • CHADO_DB_PORT

OS

Any of Linux/BSD, Solaris, OSX. Windows: optional as cost/benefit less favorable.

Web Server

All of:

  • Apache
    • With Modperl 1.3, temporarily; need to migrate to Modperl 2.0 (currently broken) or CGI soon.
    • Allen, Ken to investigate effort required to port their apps to ModPerl 2.0 or CGI.
  • TomCat 4.1 (auxiliary Web server to support server-side Java/JSP)

Languages

All of:

  • Perl 5.6.1 or higher
  • Java 1.4

DBMS

PostgreSQL 7.3 or 8.0

Compilers

gcc (on Solaris as well).

Libraries

Perl bundle; requires some C libraries

Hardware

Recommendations based on current MODs to be determined. Minimum: 20GB disk, 512 MB ram, 1.8 GHz Pentium or equiv. Client or server.

Client Side

OS

Linux, Windows, Mac OSX.

Browser

  • Any of Mac IE, Win IE, Netscape. Which versions within these?
  • Fly and worm will analyze server logs to determine largest market segments.
  • How to test?


Server Side Directory Structure

  • $GMOD_HOME=/usr/local/gmod/ (default -- set before installing)
    • bin/ -- all scripts

Prefix all executables with 2-3 letter app prefix

    • sbin/ -- system binaries that may do dangerous things
    • web/
    • htdocs/ -- db
      • cgi-bin/ -- appname
      • webapps/ -- appname
    • conf/ -- all configuration files (attr/value format)
      • gmod.conf always source this first; global site params
      • db.conf -- one for each datasource at the installation server, port, DB, login, password, banner/site-specific page config info optionally source one of these after gmod.conf according to "db" URL param (Perl: use CPAN INI parser to load)
    • data/ -- inputs and outputs; operator writable -- appname
    • lib/ -- appname or shared library, Perl plugins, jar files
    • doc/ -- appname
    • log/
    • examples/ -- appname
    • src/ -- appname


  • $CLASS_PATH := $GMOD_HOME/lib[/app]
  • Suggested CGI Parameter Names:
    • db=short symbolic name of datasource (MOD), used to look up conf file
    • class=object class within db to search
    • id=unique primary key
    • name=human readable, possibly ambiguous string


  • What if multiple DB instances running on same site?
  • Package manager?
  • Testing process: is a goal. "Testing is good" -- LS
    • Developer side
    • Release testing: alpha, beta
    • Installer side


Interoperation

  • Support chado schema for genome features
  • Establish clear naming conventions for Chado extension to avoid intermodule conflicts:
    • Use namespaces (using PostgreSQL "create schema" or table prefixes?) corresponding to Chado modules. Or will current conventions suffice?
  • Different apps should share portions of schema wherever that makes sense
  • Schema changes need extensive coordination, versioning and testing across all affected apps.
  • Apps are encouraged to be backward compatible with older versions of schema.
  • Need to track which app versions work with which schema versions.
  • Releases:
    • Major (or "public") release is a coordinated release of all apps, with testing.
    • Minor releases / tagged versions as needed by development groups, no guarantee of installable tarball.
  • Schema change coordination manners: notify gmod-schema list of all proposed changes. Is this sufficient (for now?)
  • Support a few standard external reps of data:
    • Sequence feature apps should read/write GenBank & EMBL formats (BioPerl and BioJava handle this). GFF? GTF? BED? GAME? BSML? TIGR add BSML adaptors to BioPerl?
    • Citations: Medline flatfile format
    • Expression: MageML for microarrays
    • Maps: use CMAP tab delimited format. (the new standard :-)
    • Adaptors for chado-DBI and/or chado-XML?
  • AppDB communication:
    • SQL (DBI/JDBC)
    • TIGR developing Perl API for chado: get central dogma model, cv_term2genes, writes
    • UCLA has developed a Perl middleware API (Class::DBI) for Chado
    • BioPerl2chado -- CJM readonly API -- out of date?
    • Chado XML Dumper
  • Bulk dataloading
    • Colin's GAME2chado initializer
    • Chado XML Loader
    • GAME2chadx, chadx2game
    • BSML2chado loader @ TIGR


A First GMOD Release would Contain

  • Sample data
    • Fasta files
    • GenBank files
    • BLAST output
  • Loading tools for sample data
  • Stable chado release
  • gmod-web with "basic" pages
  • Compatible Gbrowse release
  • Apollo
  • Other support tools (chadx dumpers and loaders,xxx2chadx translators)
  • DOCUMENTATION (README, INSTALL, CONFIGURATION-HOWTO)


Creating a software release

All software releases for GMOD should conform to a bare minimum standard. First, the package should be easily 'installable.' That is, it should be free of hard coded paths and other machine specific parameters. In addition, the package should contain:

  • A README document
  • A INSTALL document
  • A LICENSE file (containing an open source license; licenses used in GMOD projects include the GNU public license, BSD license, and the Perl artistic license)
  • The source code
  • An appropriate installation method:
    • For Perl modules, use Makefile.PL
    • Perl applications can be installed via Makefile.PL or an install script
    • For Java applications, use a suitable build tool, like Ant
    • For Java servelets that are distributed as war files, the war file should be in a tarball with the above mentioned documentation

Additionally, the following things are not absolutely required but are strongly advised for GMOD projects:

  • The application should contain a testing suite so that the user can verify proper builds.
  • Packages should contain sample data for testing and trying out the application
  • Including a tutorial for guiding the user on the application's use is a very good idea.
  • For applications that interact with a relational database, it should be able to interact with a Chado database via a database adaptor.

Finally, a page describing the appication and providing basic information about the application should be provide for www.gmod.org. This page should contain the following information:

  • A brief summary describing the application
  • Links to a demo or screenshot
  • Links to installation documentation
  • Links to downloads
  • Author contact information