GMOD

Java Testing - by Jon Slenk

Abstract

Good software cannot be written in the absence of testing. A range of testing methods exists, each with specific reasons for use. In this document we present arguments in favor of adopting ever more testing in the software development and maintenance lifecycles, and resources which may be utilized during such adoption.

Arguments in Favor of Testing There are many examples of computer system failures, from simple every-day desktop application issues to more physically dramatic failures [2]. Not all bugs cause the loss of billions of dollars yet one must admit the significance of a bug can be tremendous for the effected community; as bugs obstruct the goals of a system they become increasingly serious. Testing is widely accepted as a core tool used in the activity of finding bugs before they can do actual damage. Thus testing can be used to find failures which, if left undetected until later (let alone unresolved), would prevent the achievement of critical goals.

In the case of the Generic Model Organism Database (GMOD) project, a core goal is to produce reusable components which would allow a new Model Organism Database (MOD) to be assembled by mixing components from GMOD [4]. Clearly, to meet such goals the given components must be free from defects which would prevent new developers from using them. (There are, of course, several other criteria to be applied when deciding if given software meets one’s needs but those are outside the scope of this document.) Thus, we examine specific subjects and tools which may be used to achieve goals via testing.

Targets of Testing for GMOD

A simple description of the two most important questions regarding GMOD software is: “When will its development be completed?” and “Does it work?” These two issues are greatly intertwined, and all of the tools and techniques listed below may be employed in developing answers to those questions.

Documentation: If a system is opaque to those adopting it, they will face more roadblocks and will be more likely to abandon it. Thus, creating, maintaining and generating documentation are important activities in the software development process. Built into several tools are tests of coverage to help ensure at least a minimal documentation set. (Javadoc [5])

Program execution - Logging, Debuggers: When bugs do arise, one must analyze the system to determine how the bug was triggered. Logging and interactive debuggers give one information vital to the analysis. (JSwat Log4J [16]).

Program execution - Profiling: Bugs may be of many classifications. It may be the case that while a software system eventually gives correct results, it does not do so in a manner that is acceptable to end users i.e. it requires too many computing resources. Software profilers help investigate system behaviour to determine what bottlenecks exist. (Profiling [15])

Program behaviour - Unit tests: We must realize that software undergoes constant change. Thus, tests which were previously run quickly become invalidated. Additionally, software development is quite often a result of teamwork and thus changes made by one person or group can impact others. Having a framework for codifying and running tests easily is therefore of great benefit because it can be used to vet changes soon, reducing the window of opportunity for bugs to snowball into larger issues. (JUnit TestNG [8])

Program behaviour - Specifications: From an external perspective, if one can specify the goals of a system then one can automatically test to see if those goals have been met. From an internal perspective, specifications can be used to ensure that disparate software units can successfully interact. (JML [14]).

Program behaviour - Bug triggering: Tools exist to assist humans by automatically running through various testing methods in an attempt to trigger bugs. (Java PathFinder PMD [10], FindBugs [11])

Metrics - Code coverage: Software generally allows for choices which means that testing of one set of particular decisions might not have exercised another. Coverage tools assist one in determining how much of the system is actually being exercised. (Code coverage Clover [13]).

References

http://www.internetnews.com/ent-news/article.php/1450161

[2] http://www.esa.int/esaCP/Pr_33_1996_p_EN.html

http://en.wikipedia.org/wiki/Software_testing

[4] http://www.genome.org/cgi/content/full/12/10/1599

Resources Javadoc: http://java.sun.com/j2se/javadoc [6]

JSwat debugger: http://www.bluemarsh.com/java/jswat

JUnit unit testing framework: http://www.junit.org/index.htm [8]

TestNG expanded unit testing framework: http://www.beust.com/testng

Java PathFinder: http://javapathfinder.sourceforge.net [10]

PMD (Java ‘lint’): http://pmd.sourceforge.net

FindBugs: http://findbugs.sourceforge.net [12]