Difference between revisions of "Running a GBrowse2 render farm"
| Line 1: | Line 1: | ||
| + | [[Image:renderfarm_1.png|thumb|Figure 1:Single server with multiple processors or cores]] | ||
| + | [[Image:renderfarm_2.png|thumb|Figure 2:Multiple servers sharing NFS-mounted file-based databases]] | ||
| + | [[Image:renderfarm_3.png|thumb|Figure 3:Multiple servers sharing the same relational databases]] | ||
| + | [[Image:renderfarm_4.png|thumb|Figure 4:Multiple servers with private databases]] | ||
| + | |||
GBrowse can be configured to use one or more "render slave" daemons. A render slave is a small Perl processes that runs in the background, processing requests to render GBrowse tracks. By distributing several render slaves across one or more computers, you can spread out the work of generating GBrowse pages, thereby achieving improved performance. A slave can be attached to a particular track, a particular set of tracks, or to all tracks. In addition, you may assign multiple slaves to tracks, in which case the load will be distributed across each slave in a round-robin fashion. | GBrowse can be configured to use one or more "render slave" daemons. A render slave is a small Perl processes that runs in the background, processing requests to render GBrowse tracks. By distributing several render slaves across one or more computers, you can spread out the work of generating GBrowse pages, thereby achieving improved performance. A slave can be attached to a particular track, a particular set of tracks, or to all tracks. In addition, you may assign multiple slaves to tracks, in which case the load will be distributed across each slave in a round-robin fashion. | ||
| Line 5: | Line 10: | ||
The main '''limitations''' on render slaves is that a slave needs access to the database underlying any track that it is asked to render. If you run slaves on multiple hosts, then each host must have access to the relevant databases. For relational database backends, such as MySQL, you will need to grant access permissions for each host. For file-based database backends, such as the BerkeleyDB and in-memory databases, the database files must be replicated among the hosts, or else mounted on a common filesystem such as NFS. | The main '''limitations''' on render slaves is that a slave needs access to the database underlying any track that it is asked to render. If you run slaves on multiple hosts, then each host must have access to the relevant databases. For relational database backends, such as MySQL, you will need to grant access permissions for each host. For file-based database backends, such as the BerkeleyDB and in-memory databases, the database files must be replicated among the hosts, or else mounted on a common filesystem such as NFS. | ||
| − | Figures 1 through | + | Figures 1 through 4 illustrate common render slave configurations. |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
Revision as of 21:48, 22 January 2009
GBrowse can be configured to use one or more "render slave" daemons. A render slave is a small Perl processes that runs in the background, processing requests to render GBrowse tracks. By distributing several render slaves across one or more computers, you can spread out the work of generating GBrowse pages, thereby achieving improved performance. A slave can be attached to a particular track, a particular set of tracks, or to all tracks. In addition, you may assign multiple slaves to tracks, in which case the load will be distributed across each slave in a round-robin fashion.
Configuration of a render slave is minimal. All the datasource-specific information is stored in the central GBrowse script (known as the "master" GBrowse) and sent across the wire to the render slave. Configuration of the master GBrowse is also quite simple, and requires just an extra line or two in its configuration files.
The main limitations on render slaves is that a slave needs access to the database underlying any track that it is asked to render. If you run slaves on multiple hosts, then each host must have access to the relevant databases. For relational database backends, such as MySQL, you will need to grant access permissions for each host. For file-based database backends, such as the BerkeleyDB and in-memory databases, the database files must be replicated among the hosts, or else mounted on a common filesystem such as NFS.
Figures 1 through 4 illustrate common render slave configurations.
