This JBrowse tutorial was presented by Scott Cain at the Programming for Biology at Cold Spring Harbor Laboratory in October 2016 using JBrowse 1.12.1.
This tutorial assumes an Ubuntu virtual machine at AWS (ami-47f1b050, in us-east-1), with the security group modified to allow http connections, and with the tutorial bundle zip file, also available on Amazon S3: Prog_for_Bio_2016_JBrowse.zip.
Prerequisites are installed by JBrowse automatically. A few things may fail to install (like legacy support for wiggle files), but that doesn’t matter.
Make sure you can copy/paste from the wiki.
It’s also very useful to know how to tab-complete in the shell.
It’s probably a good idea to use a browser like Chrome that has the ability to turn off caching while working on this tutorial. To do this in Chrome, with the browser open to the JBrowse page you’re working on, select Developer->Javascript Console from the View menu. In the console, click the “gear” icon (settings) and check the box labeled “Disable Cache”.
How and why JBrowse is different from most other web-based genome browsers, including GBrowse.
More detail: paper
Obtain the key file (something.pem) and machine IP address from the course website. Then use ssh from the terminal to access the machine:
ssh -i something.pem ubuntu@ip.add.ress
This will log you in as the ubuntu user, which as full, password-less sudo permissions on the machine.
cd /var/www/html
##curl -O https://s3.amazonaws.com/jbrowse-tutorials/Prog_for_Bio_2016_JBrowse.zip #that's a capital dash"O" not a zero/zed.
cp ~/Prog_for_Bio_2016_JBrowse.zip . ## if we don't need to update the zip file
unzip Prog_for_Bio_2016_JBrowse.zip
cd Prog_for_Bio_2016_JBrowse
unzip JBrowse-1.12.1.zip
mv JBrowse-1.12.1 jbrowse
setup.sh to configure this copy of JBrowsecd jbrowse
./setup.sh
Visit in web browser:
http://ip.add.re.ss/Prog_for_Bio_2016_JBrowse/jbrowse/index.html
You should see a “Congratulations” page.
There are four basic steps to setting up an instance of JBrowse:
GFF (Generic Feature Format) is a very commonly used text format for describing features that exist on sequences. We’ll head off to that page to talk about it a bit.
Here, we’ll use the Bio::DB::SeqFeature::Store adaptor in “memory” mode to read a directory of files. There are adaptors available for use with many other databases, such as Chado and Bio::DB::GFF.
Config file: pythium-1.conf
{
"description": "Programming for Biology 2015 P. ultima Example",
"db_adaptor": "Bio::DB::SeqFeature::Store",
"db_args" : {
"-adaptor" : "memory",
"-dir" : ".."
},
...
The first script to run is bin/prepare-refseqs.pl; that script is the
way you tell JBrowse about what your reference sequences are. Running
bin/prepare-refseqs.pl also sets up the “DNA” track.
Run this from within the jbrowse directory (you could run it
elsewhere, but you’d have to explicitly specify the location of the data
directory on the command line).
cd ~/Sites/jbrowse_demo/Prog_for_Bio_2015_JBrowse/jbrowse
bin/prepare-refseqs.pl --gff ../scf1117875582023.gff
Refresh it in your web browser, you should new see the JBrowse UI and a sequence track, which will show you the DNA base pairs if you zoom in far enough.
Next, we’ll use biodb-to-json.pl to get feature data out of the
database and turn it into JSON data that the
web browser can use.
In this case, we have specified all of our track configurations in
pythium-1.conf.
...
"TRACK DEFAULTS": {
"class": "feature"
},
"tracks": [
{
"track": "Genes",
"key": "Genes",
"feature": ["mRNA"],
"autocomplete": "all",
"class": "transcript",
"subfeature_classes" : {
"CDS" : "transcript-CDS",
"UTR" : "transcript-UTR"
}
},
...
]
track specifies the track identifier (a unique name for the track, for
the software to use). This should be just letters and numbers and - and
_ characters; using other characters makes things less convenient.
key specifies a human-friendly name for the track, which can use any
characters you want.
feature gives a list of feature types to include in the track.
autocomplete including this setting makes the features in the track
searchable.
urltemplate specifies a URL pattern that you can use to link genomic
features to specific web pages.
class specifies the CSS class that
describes how the feature should look.
For this particular track, I’ve specified the transcript feature
class.
Run the bin/biodb-to-json.pl script with this config file to format
this track, and the others in the file:
bin/biodb-to-json.pl --conf ../pythium-1.conf
Refresh JBrowse in your web browser. You should now see a bunch of annotation tracks.
When you generate JSON for a track, if you specify "autocomplete" then
a listing of all of the feature names from that track (along with
feature locations) will also be generated and used to provide feature
searching and autocompletion.
The bin/generate-names.pl script collects those lists of names from
all the tracks and combines them into one big tree that the client uses
to search.
bin/generate-names.pl -v
Visit in web browser, try typing a feature name, such as maker-scf1117875582023-snap-gene-0.26-mRNA-1. Notice that JBrowse tries to auto-complete what you type.
We’re going to add a couple more tracks that come from a flat file,
repeats.gff. To get feature data from flat files into JBrowse, we use
flatfile-to-json.pl.
bin/flatfile-to-json.pl --trackLabel "repeat masker" \
--trackType CanvasFeatures \
--type match:repeatmasker --key RepeatMasker \
--className generic_parent \
--subfeatureClasses '{"match_part" : "feature"}' --gff ../repeats.gff
bin/flatfile-to-json.pl --trackLabel "repeat runner" \
--trackType CanvasFeatures \
--type protein_match:repeatrunner \
--key RepeatRunner --className generic_parent \
--subfeatureClasses '{"match_part" : "feature"}' --gff ../repeats.gff
Visit in web browser; you should see the two new RepeatMasker and RepeatRunner tracks.
JBrowse can display alignments directly from a BAM file on your web server. Simply place the BAM file in a directory accessible to your web server, and add a snippet of configuration to JBrowse to add the track, similar to:
{
"label" : "bam_alignments",
"key" : "BAM alignments",
"storeClass" : "JBrowse/Store/SeqFeature/BAM",
"urlTemplate" : "../../simulated-sorted.bam",
"type" : "Alignments2"
}
This can be added by either editing the data/trackList.json file with
a text editor, or by running something like this at the command line to
inject the track configuration:
echo '{
"label" : "bam_alignments",
"key" : "BAM alignments",
"storeClass" : "JBrowse/Store/SeqFeature/BAM",
"urlTemplate" : "../../simulated-sorted.bam",
"type" : "Alignments2"
}' | bin/add-track-json.pl data/trackList.json
{
"label" : "bam_coverage",
"key" : "BAM Coverage",
"storeClass" : "JBrowse/Store/SeqFeature/BAM",
"urlTemplate" : "../../simulated-sorted.bam",
"type" : "SNPCoverage"
}
JBrowse can display quantitative data directly from a BigWig file on your web server. Simply place the BigWig file in a directory accessible to your web server, and add a snippet of configuration to JBrowse to add the track, similar to:
{
"label" : "bigwig_bam_coverage",
"key" : "BigWig - BAM coverage",
"storeClass" : "BigWig",
"urlTemplate" : "../../simulated-sorted.bam.coverage.bw",
"type" : "JBrowse/View/Track/Wiggle/XYPlot",
"variance_band" : true
}
This can be added by either editing the data/trackList.json file with
a text editor, or by running something like this at the command line to
inject the track configuration:
echo ' {
"label" : "bigwig_bam_coverage",
"key" : "BigWig - BAM coverage",
"storeClass" : "BigWig",
"urlTemplate" : "../../simulated-sorted.bam.coverage.bw",
"type" : "JBrowse/View/Track/Wiggle/XYPlot",
"variance_band" : true
} ' | bin/add-track-json.pl data/trackList.json
JBrowse can also display VCF variation data directly from a VCF file on your web server that has been compressed with Heng Li’s bgzip and tabix. Simply place the .vcf.gz and .vcf.gz.tbi files in a directory accessible to your web server, and add a snippet of configuration to JBrowse to add the track, similar to:
{
"label" : "bam_variation",
"key" : "VCF simulated variation",
"storeClass" : "JBrowse/Store/SeqFeature/VCFTabix",
"urlTemplate" : "../../simulated-sorted.vcf.gz",
"type" : "HTMLVariants"
}
This can be added by either editing the data/trackList.json file with
a text editor, or by running something like this at the command line to
inject the track configuration:
echo ' {
"label" : "bam_variation",
"key" : "VCF simulated variation",
"storeClass" : "JBrowse/Store/SeqFeature/VCFTabix",
"urlTemplate" : "../../simulated-sorted.vcf.gz",
"type" : "HTMLVariants"
} ' | bin/add-track-json.pl data/trackList.json
JBrowse has a very powerful faceted track selector that can be used to search for tracks using metadata associated with them.
The track metadata is kept in a CSV-format file, with any number of columns, and with a “label” column whose contents must correspond to the track labels in the JBrowse configuration.
The demo bundle contains an example trackMetadata.csv file, which can
be copied into the data directory for use with this configuration.
cp ../trackMetadata.csv data/
Then a simple faceted track selection configuration might look like:
"trackSelector": {
"type": 'Faceted',
},
"trackMetadata": {
"sources": [
{ "type": 'csv', "url": 'data/trackMetadata.csv' }
]
}
Copy the section above and put it in the empty curly braces in the
jbrowse_conf.json file in the jbrowse, save it, refresh your
browser, and you should now see the faceted track selector activated.
Tracks can be modified by changing several aspects of how the images are created. While this can be done be done both with HTML and Canvas tracks, this tutorial will focus on Canvas tracks only (the repeat tracks created above).
The configuration for the RepeatMasker track looks like this:
{
"key" : "RepeatMasker",
"trackType" : "CanvasFeatures",
"storeClass" : "JBrowse/Store/SeqFeature/NCList",
"urlTemplate" : "tracks/repeat masker/{refseq}/trackData.json",
"style" : {
"subfeatureClasses" : {
"match_part" : "feature"
},
"className" : "generic_parent"
},
"type" : "CanvasFeatures",
"compress" : 0,
"label" : "repeat masker"
}
Open the data/trackList.json file in your favorite editor and
Command-F will open a “find” window; search for “repeatmasker”. A simple
change we can make is to the color; in the line starting with “style”,
add:
"color" : "black",
save the changes and select the RepeatMasker track or reload the browser to see the change. Many attributes of the display can be modified in this way, see the JBrowse Configuration Guide for a list of options.
Much like GBrowse’s perl callbacks that can change the track display, in the JBrowse configuration file you can include JavaScript functions to change the way tracks look. For example, in the RepeatMasker track, we can change the color of the glyph depending on what kind of repeat it is (where we happen to know that the type of repeat is encoded in the name). In this example, we leave the glyph black, unless it is a low complexity repeat, where we’ll color it red. A function to do that would look like this:
"color" : "function(feature) { var name = feature.get('Name'); if (name.match('Low_complexity') ) { return 'red'; } return 'black'; }",
When editing the trackList.json file directly in this way, the function has to go all on one line, but if we create an “include file” (not covered here) the function can have carriage returns in it. Replace the simple “color : black” section we just created in the configuration file with the function above, save the file and reload the browser page to see the changes (you might have to mouse around to find a low complexity repeat).
The default action when you click on a glyph is to open a “floating” window that displays everything JBrowse knows about a feature. If you’d like something else to happen, you have several options (outlined here), including having a different floating window open or executing any JavaScript function you define. For this example, we’ll create a link that will Google the repeat’s name and open the result in a new window. In the RepeatMasker section of the JBrowse configuration, we’ll add a section that looks like this after the style section:
"onClick" : {
"iconClass" : "dijitIconDatabase",
"action" : "newWindow",
"url" : "http://www.google.com/search?q={name}",
"label" : "Search for {name} at Google",
"title" : "function(track,feature,div) { return 'Searching for '+feature.get('name')+' at Google'; }"
},
If you’re having difficulties, the RepeatMasker section of the configuration file should now look something like this:
{
"type" : "CanvasFeatures",
"trackType" : "CanvasFeatures",
"style" : {
"color" : "function(feature) { var name = feature.get('Name'); if (name.match('Low_complexity') ) { return 'red'; } return 'black'; }",
"subfeatureClasses" : {
"match_part" : "feature"
},
"className" : "generic_parent"
},
"onClick" : {
"iconClass" : "dijitIconDatabase",
"action" : "newWindow",
"url" : "http://www.google.com/search?q={name}",
"label" : "Search for {name} at Google",
"title" : "function(track,feature,div) { return 'Searching for '+feature.get('name')+' at Google'; }"
},
"compress" : 0,
"storeClass" : "JBrowse/Store/SeqFeature/NCList",
"label" : "repeatmasker",
"urlTemplate" : "tracks/repeatmasker/{refseq}/trackData.json",
"key" : "RepeatMasker"
},
JBrowse is built with a very flexible and powerful plugin system. The JBrowse developers are working on a plugin registry website, but for the time being, you can look at the source code for what will drive the website in the jbrowse-registry github repo, and in particular, the file that contains the info about the available plugins, plugins.yaml.
For this tutorial, we’ll look at a plugin that is shipped with JBrowse
but isn’t turned on by default. JBrowse plugins are typically stored in
the plugins directory, and in 1.12.1’s plugin directory there are 6
plugins:
CategoryUrl
DebugEvents
HideTrackLabels
NeatCanvasFeatures
NeatHTMLFeatures
RegexSequenceSearch
and the RegexSequenceSearch plugin is already activated (look under the “Track” menu for it). We will turn on the NeatHTMLFeatures plugin. Right now the Genes track has relatively “boring” glyphs composed of pink rectangles connected by black lines. To turn on NeatHTMLFeatures, just add
"plugins": [
'NeatHTMLFeatures'
],
to the top of data/trackList.json, just inside the first open curly
brace. Reload JBrowse and turn on the Genes track if it isn’t already
and see the wonder of the psuedo-3D gene glyphs.
Note that not all plugins are activated this way: typically, if the
plugin modifies the way tracks look, it will go here. If it modifies the
way JBrowse *works*, it will go in the jbrowse.conf file.
To highlight a region, you can either right-click on a feature and select ‘highlight this’, or you can set the highlight explicitly to a certain genomic region by clicking “View -> Set highlight” in the menu bar.
Beginning in JBrowse 1.10.0 you can also highlight a region with the mouse by clicking the highlighter tool (next to the Go button) and clicking and dragging to highlight a region.
JBrowse can display GFF3, BAM, BigWig, and VCF+Tabix files directly from your local machine without the need to transfer any data to the server. Just use the “File -> Open” tool from the menu bar to add tracks using local files.
Starting in version 1.10.0, users can define tracks that are combinations of the data in other tracks. The operations used to combine these tracks can be set operations (union, intersection, subtraction), arithmetic operations for quantitative tracks (addition, subtraction, multiplication, division), and/or masking operations to just highlight or mask some regions based on data in another track.
To add a combination track, select “File->Add combination track” from the menu bar, and drag existing tracks into the new combination track to start combining them.
If the old JBrowse is 1.3.0 or later, simply move the data directory
from the old JBrowse directory into the new JBrowse directory after
running the setup.sh script.
Facts about “JBrowse Tutorial Programming for Biology 2016”
| Has topic | JBrowse + |