Managing the Ant Factory
by Sudhakar Ramakrishnan

Best Practices when Automating Your Builds with Ant

Over the course of the past few years a number of build automation tools have sprung up, each with its own advantages and disadvantages. Among them is Ant, a build tool that definitely stands out from the crowd for Java developers. In a recent OTN poll (see Sidebar 1 "OTN Open Source Poll"), 45.1% of the developers polled said Ant was their most-used open-source tool.

So what makes it so attractive? No doubt, a variety of things. It's a cross-platform build tool (unlike Make); its configuration files are XML-based (and hence, it has a low learning curve); it can be used with a variety of IDEs (including Oracle JDeveloper); and it can automate a large number of build tasks without any customization (though it's also extensible, using Java classes). (See Sidebar 2 "The Ant View of the World," for more details.)

More specifically, basic tasks that Ant can automate include creating and deleting directories, downloading and copying files using FTP, JARing and ZIPing files, and creating documentation. Ant also provides more advanced features, as Michel Casabianca notes elsewhere on OTN in his article "Starting with Ant" (Oracle Magazine, November/December 2002). These include the ability to read source code from source code control systems such as CVS, Clearcase, SourceSafe, and others; as well as the abilities to execute SQL queries and scripts, transform XML to human-readable HTML, and generate stub files for remote method invocation.

These features notwithstanding, the process of automating large builds with Ant requires forethought and planning. In my view, any build automation process scripted in Ant should work to eliminate user interaction completely. It should be architected for efficiency, to reduce time on redundant or duplicate tasks and to encourage reuse of scripts. And it should be extensible and easy to manage, with some attention to the readability of the build file. In this article, I propose to discuss these topics in some detail, highlighting best practices and things to keep in mind as you automate your builds, noting Ant's relevance whenever possible. I'll also provide a resource list at the end, with hyperlinks to articles I've referenced, so that readers interested in using Ant to automate their builds can find a range of advice and tools in one place to assist them in their efforts. Many of these articles are available right here on OTN.

Automating Builds: Eliminating User Interaction

By definition, automating your build means working to eliminate most, if not all, human interaction at runtime. The ideal is a "one-button" build that runs from start to finish without a hitch. The rationale for this should be obvious: build automation improves developer productivity, reducing release time. It can also improve the quality of your code, as builds and rebuilds are rationalized. In fact, the more complex your source code, the more automated your build process needs to be. Easy to say, but hard to do. Setting up such a build requires up-front time, insight, and planning. In fact, designing your build automation process means first, "getting the big view." This involves a number of issues, some key ones which are touched on below.

To begin properly, you need to collect the right tools, files, and libraries you'll use for your build. Though Ant provides the ability to execute basic source code control operations—such as checking in, checking out, and labeling code—it also currently supports a number of source control systems that can be added to your toolset, including Clearcase, Continuous/Synergy, CVS, Perforce, PVCS, StarTeam, and Visual SourceSafe. Support for all systems except CVS is available in the optional test package. Note that components dependent upon Ant might need to be updated or patched before you begin and you'll need to take care of integration issues when you're using multiple libraries.

Next, you'll need a good basic understanding of your directory structure, as well as the source files, targets, and dependencies that will govern your scripts. Here, there are accepted norms for building your directory and file hierarchy. Ant takes source files from directories, pulls components from libraries, and executes a process that results in a target. Along the way, the process may be dependent for its execution on the target (the result) of a previous process. Thus, the network of dependencies needs to be mapped. You certainly don't want to have the situation where you create cyclic dependencies, where your script returns to a process again and again.

You'll then begin to automate your steps, avoiding tools that mandate user intervention, while monitoring changes that can break a build process. Since Ant is a cross-platform tool written in Java (unlike Make) it can eliminate the user intervention requirements that results from having to build scripts for different platforms. In the same vein, you also need to avoid using tools that require entry into GUI input screens. Picture a build that's hung up while it waits for screen input from the builder. Thus, the build automation should use command line input whenever possible. Some tools also allow you to build reusable components that are modified through parameters, to reduce undue scripting. And you'll need to manage script changes according to standard rules.

You'll have to pay attention to localizing your build. Many products produced these days are built for local consumption, but many also are made with international editions. This insures that when the software is loaded, all the instructions for use, all the menus in the product, and all the help files and fonts are in the local language. Here there are NLS variables that need to be used and a number of Unicode formats that need to be taken care of. The build needs to be aware of these.

Some of you may also have to execute a grab-bag of tasks to maximize your build automation. Do you need to install optimized code, to increase efficiency? Do you need to use a new runtime? Do some services need to be started up before you can perform server operations? Good build automation also means understanding something about server-side development. Developers working on server-side Java applications are typically interested in the compilation of servlets, deployment of JSP files, and the deployment of HTML files, configuration files, and images.

OTN Open Source Poll Question: Which open-source tool do you use the most?
Ant: 2093 Votes	45.1%
CVS: 1147 Votes	24.7%
Xdoclet: 423 Votes	9.1%
Eclipse: 348 Votes	7.5%
Jalopy: 252 Votes	5.4%
Jtidy: 8 Votes	0.2%
Lomboz: 6 Votes	0.1%
Other: 362 Votes	7.8%
Total: 4639 votes	99.9%
Poll posting: Feb 17, 2004. The poll ran for approximately 2-3 weeks on the OTN Java Center.

No doubt you'll also have to add in secure sign-in features in your build. This pertains to code that provides security through registered vendor assurance services (e.g., as when you download a plug-in, a dialog box comes up "Do you trust this vendor?"), as well as to including public encryption keys for secure sign-in.

You may also have to apply obfuscation. This is the process by which you make your code secure, so it's not readable. The decision to apply some form of obfuscation at the end of your build needs to be made at the architecture and deployment levels whenever source code cannot be shipped as is and needs some security.

In most cases, you'll also have to produce documentation and notification of the release via email and the Web. This means generating some documentation that can label the build version, describe the bug fixes, after which you'll send out an email to fellow developers saying that the build is complete and available for downloading on such and such a Web site.

One big task is to synchronize and integrate the build with the product release process (PRP). This means working with the QA and Release teams beforehand, to minimize user interaction at the time of the build. Here, build designers should be aware of the work that's going on in extreme programming. Martin Fowler, Chief Scientist at ThoughtWorks, has done some great work here, following a methodology called "continuous integration," where build automation and multiple daily builds are coupled with building unit tests of your build scripts to reduce integration problems. Using the methodology, if somebody does something to change the build process illegitimately for that unit, the test unit can tell you if and where the build process is broken. See Martin Fowler and Matthew Foemmel's article "Continuous Integration" (see the link below) for more details.

Making Your Builds Efficient

While the build designer's primary goal when automating builds is to eliminate user interaction, there's an implicit secondary goal of efficiency that needs to be considered as well. Efficiency comes when you reduce the time spent in duplicate tasks (e.g. by eliminating redundancy whenever possible), and by reusing components, so as not to "reinvent the wheel" whenever you're assembling your build. Picture a robot accomplishing secondary tasks along the way as it does its primary tasks, rather than making it come back again and again to a starting point. Or if you're producing documentation for a particular module, why not do it at the same time for a number of modules? And why not make your build scripts more efficient by putting all the reusable steps in one place, so they execute at once, reducing build time?

The Ant View of the World What makes Ant different? Instead of a model (as with Make) where your build is extended with shell-based commands, which are machine-specific, the configuration files in Ant are XML-based, which is cross-platform. Each build project in Ant is a collection of targets. A target is a collection of tasks. Each task has a set of properties. Thus, each Ant task can be called programmatically in a relatively few lines of code and extended in the same way. As John Kirkley notes in "Ant Builds On," (see our Reference section below for KirkeyÕs articles, as well as many others), developers use Ant and its XML-based build files "to compile, run unit tests, deploy to a remote application server using FTP or Telnet, and run deployment tests with a single command," says Kirkley. "Additional Ant capabilities include the ability to issue SQL statements, zip a logfile copy, and email the copy as a MIME attachment. And if a developer really wants to add some OS-specific shell commands, Ant includes an exec task feature that allows for the execution of different OS-specific commands based on the OS on which Ant is running."

Efficiency can also be maximized by expanding the usual practice of nightly builds to include another tactic of extreme programming: continuous builds. This means, as and when build scripts are checked in to the build project you try to see if you can come up with a vendor view of what the project is going to look like as soon as you can. That means you're building all along the way, using idle times when your computers are not being used for other programming tasks, possibly at lunch time. You can also design build levels, i.e., Level 1 to Level n, moving from more simple to more complex builds, where Level 1 builds are done continuously (because they're more simple), while Level Six builds are done at night (because they're more complex and take more time.)

Efficiency also means parameterization of your build code. Here's where you can take advantage of the Ant 1.6 import facility that allows you to build reusable libraries, which you then "parameterize." Parameterization of Ant build macros and scripts means building a particular kind of task as a build module, but then sending new parameters to it for reuse. Picture a car body with a base white paint that's given different over-coats, depending upon whether it's a standard or custom model. (See Stefan Bodewig's articles on this process, referenced below.)

Finally, one way to make your build automation process more efficient is to manage it with a tool like Maven—for some, a competitor to Ant. According to the official site of the Apache Maven Project (see link below), Maven is a Java project management tool that relies heavily on the concept of a project object model (POM), in which all Maven products—such as documentation, source metrics, and reports—are controlled by the POM. By providing a well-defined project structure and development process, along with documentation to keep developers and clients up-to-date on the project's progress, the developer's life is made much easier. Maven reduces the everyday drudgery of build automation work and keeps developers focused on the real programming task at hand. It's particularly great where there aren't many people dedicated to documenting and propagating the key information about the project. To help you decide which tool is more suitable for your project—Ant or Maven—a good place to start is to read Julien Dubois' article "Master and Commander," right here on OTN. The link is given below.

Making Your Builds Easy to Manage and Extensible

In the end, your build files must be both easy to manage and extensible. Since the goal is to have a one button build, architecting a build when you have hundreds of source files is not an easy task. It requires thinking holistically and globally and taking into account Ant's XML format, one that makes it easier to get up to speed and at the same time make it potentially a nightmare to manage if you're not careful down the road. As your builds become more complex, the angle brackets you see all across the build file, based as they are on the hierarchical declarative structure of XML, can quickly become unwieldy and unreadable if you don't document your build files. The last thing you want is for your build to become the primary focus in your programming. (James Duncan Davidson talks about it in his recent article "Ant and XML," and it's also addressed to some extent in Stefan Bodewig's article "New Ant 1.6 Features for Big Projects," where the author discusses how such issues will be handled in the future. Both links are given below.)

Finally, you need to address what you should do if your build task cannot do what you want it to do. Here's where one of the virtues of Ant really comes into play with its ability to create custom tasks and to extend it if you want. Even if you don't have the common set of runtime libraries that support your build process, you can just go ahead and create them, utilizing a particular feature called the alter Apache Tools Ant task. What this does is give you a plug-in mechanism for you to take the bull by the horns and create custom pathways for your scripts. For more ideas about this, take a look at Stefan Bodewig's article "Ant 1.6 for Task Writers," where he talks about customization and extensibility.

And that's it. If you work to eliminate user interaction, architect your scripts for efficiency, and keep your build manageable and extensible, keeping in mind the strengths and the "gotchas" in Ant, you'll go a long way down the path to automating your builds.

Ant: References and Links The Apache Ant Project. Read all about Ant on the official site, including background on Ant, the current documentation, and the current version available for download (1.6.2, as of July 16, 2004), as well as contributions from the Ant community. Ant Builds On, by John Kirkley. Here's a behind-the-scenes look into the making of this cross-platform building tool. Ant and XML. x180: James Duncan Davidson's Journal, 2004/03/31. The creator of Ant looks back and talks about the pros and cons of choosing XML as the build file format for Ant. Hierarchy is good, but when builds files grow to thousands of lines long, the angle brackets of XML aren't as flexible as you'd like in a scripting language. Test Drive: Using Ant for Builds. This viewlet demonstrates how users with existing Ant projects can use these projects within JDeveloper. Starting with Ant (Part 1), by Michel Casabianca. Here's an article that will help you begin using this invaluable tool for building and deploying Java projects. It covers some basic Ant tasks that you would likely perform during the Java development process. More with Ant (Part 2), by Michel Casabianca. Here's the second part in Michel's series on using Ant for building and deploying Java projects. This article discusses some of Ant's more advanced features that are available in Ant's two task packages: the core task package and the optional task package. Jump-Start Your Java Development on Linux (Part 2: Starting Out), by Robert Clevenger. This article serves as a good hands-on guide to developing and deploying Java client applications on Linux using Ant. See the section "Getting and Installing Ant." Continuous Integration, by Martin Fowler and Matthew Foemmel. In this article the ThoughtWorks build experts talk about architecting a fully automated build and test process that allows a team to build and test their software many times a day. New Ant 1.6 Features for Big Projects (Part 1), by Stefan Bodewig. Learn new features of Ant 1.6 and see how they might impact the way you structure your build process. Ant 1.6 for Task Writers (Part 2), by Stefan Bodewig. Take advantage of the changes in Ant 1.6 internals to write a task or even a library of tasks. Ant Deployment to OC4J (Buttso's Blog). "Deploying to an OC4J standalone instance from an Ant script is quite easy to do, and I'm sure many people have got it nutted out. I always tend to forget the syntax and have to scrounge through old build files to grab the exact format of the command. So for ease-of-use, I am putting up here the Ant targets I use to perform deployment operations from Ant scripts." Using CVS in Oracle JDeveloper. This is a viewlet that shows you how to do a number of things including creating a CVS connection, importing a project, modifying code, comparing versions, monitoring version history, and checking-in changes. Oracle JDeveloper and Apache Ant. Ant integration within Oracle JDeveloper is done by adding an Ant build file to a JDeveloper project or by generating a new Ant build file from an existing JDeveloper project. Here's how you do it. In-Container Testing with JUnit, by Julien Dubois. In this article you learn how in-container testing with JUnit is superior to mock objects for integration testing, and how to apply that technique using Oracle JDeveloper. Apache Maven Project. Here's the official site of the Apache Maven Project, the Java project management and project comprehension tool that can work as a compliment to Ant. Master and Commander, by Julien Dubois. Maven and Ant are both open-source build tools from the Apache Software Foundation and choosing between them is one of the most important decisions a project leader can make. In this article, Dubois provides a high level comparison to help you choose.