Tutorial: Tasks using Properties, Filesets & Paths

After reading the tutorial about writingtasks [1] this tutorial explains how to get and set properties and how to usenested filesets and paths. Finally it explains how to contribute tasks to Apache Ant.

Content

• The goal
• Build environment
• Property access
• Using filesets
• Using nested paths
• Returning a list
• Documentation
• Contribute the new task
• Resources

The goal

The goal is to write a task, which searchs in a path for a file and saves thelocation of that file in a property.

Build environment

We can use the buildfile from the other tutorial and modify it a little bit.That's the advantage of using properties - we can reuse nearly the whole script. :-)

<?xml version="1.0" encoding="ISO-8859-1"?><project name="FindTask" basedir="." default="test"> ... <target name="use.init" description="Taskdef's the Find-Task" depends="jar"> <taskdef name="find" classname="Find" classpath="${ant.project.name}.jar"/> </target> <!-- the other use.* targets are deleted --> ...</project>

The buildfile is in the archive tutorial-tasks-filesets-properties.zip [2] in /build.xml.01-propertyaccess(future version saved as *.02..., final version as build.xml; same for sources).

Property access

Our first step is to set a property to a value and print the value of that property.So our scenario would be

 <find property="test" value="test-value"/> <find print="test"/>

 <property name="test" value="test-value"/> <echo message="${test}"/>

So what to do? Handling three attributes (property, value, print) and an execute method.Because this is only an introduction example I don't do much checking:

import org.apache.tools.ant.BuildException;public class Find extends Task { private String property; private String value; private String print; public void setProperty(String property) { this.property = property; } // setter for value and print public void execute() { if (print != null) { String propValue = getProject().getProperty(print); log(propValue); } else { if (property == null) throw new BuildException("property not set"); if (value == null) throw new BuildException("value not set"); getProject().setNewProperty(property, value); } }}

(by the way: a short word to ants "namespaces" (don'tbe confused with xml namespaces:an <antcall> creates a new space for property names. All properties from the callerare passed to the callee, but the callee can set its own properties without notice by thecaller.)

There are some other setter, too (but I haven't used them, so I can't say somethingto them, sorry :-)

After putting our two line example from above into a target names use.simplewe can call that from our testcase:

import org.apache.tools.ant.BuildFileTest;public class FindTest extends BuildFileTest { public FindTest(String name) { super(name); } public void setUp() { configureProject("build.xml"); } public void testSimple() { expectLog("use.simple", "test-value"); }}

Using filesets

Ant provides a common way of bundling files: the fileset. Because you are readingthis tutorial I think you know them and I don't have to spend more explanations abouttheir usage in buildfiles. Our goal is to search a file in path. And on this step thepath is simply a fileset (or more precise: a collection of filesets). So our usagewould be

 <find file="ant.jar" location="location.ant-jar"> <fileset dir="${ant.home}" includes="**/*.jar"/> </find>

What do we need? A task with two attributes (file, location) and nestedfilesets. Because we had attribute handling already explained in the example above and thehandling of nested elements is described in the other tutorial the code should be very easy:

public class Find extends Task { private String file; private String location; private Vector filesets = new Vector(); public void setFile(String file) { this.file = file; } public void setLocation(String location) { this.location = location; } public void addFileset(FileSet fileset) { filesets.add(fileset); } public void execute() { }}

In the other tutorial we have reused the already written targets of our buildfile.Now we will configure most of the testcases via java code (sometimes it's much easierto write a target than doing it via java coding). What can be tested?

• not valid configured task (missing file, missing location, missing fileset)
• don't find a present file
• behaviour if file can't be found

public class FindTest extends BuildFileTest { ... // constructor, setUp as above public void testMissingFile() { Find find = new Find(); try { find.execute(); fail("No 'no-file'-exception thrown."); } catch (Exception e) { // exception expected String expected = "file not set"; assertEquals("Wrong exception message.", expected, e.getMessage()); } } public void testMissingLocation() { Find find = new Find(); find.setFile("ant.jar"); try { find.execute(); fail("No 'no-location'-exception thrown."); } catch (Exception e) { ... // similar to testMissingFile() } } public void testMissingFileset() { Find find = new Find(); find.setFile("ant.jar"); find.setLocation("location.ant-jar"); try { find.execute(); fail("No 'no-fileset'-exception thrown."); } catch (Exception e) { ... // similar to testMissingFile() } } public void testFileNotPresent() { executeTarget("testFileNotPresent"); String result = getProject().getProperty("location.ant-jar"); assertNull("Property set to wrong value.", result); } public void testFilePresent() { executeTarget("testFilePresent"); String result = getProject().getProperty("location.ant-jar"); assertNotNull("Property not set.", result); assertTrue("Wrong file found.", result.endsWith("ant.jar")); }}

If we run this test class all test cases (except testFileNotPresent) fail. Now wecan implement our task, so that these test cases will pass.

 protected void validate() { if (file==null) throw new BuildException("file not set"); if (location==null) throw new BuildException("location not set"); if (filesets.size()<1) throw new BuildException("fileset not set"); } public void execute() { validate(); // 1 String foundLocation = null; for(Iterator itFSets = filesets.iterator(); itFSets.hasNext(); ) {  // 2 FileSet fs = (FileSet)itFSets.next(); DirectoryScanner ds = fs.getDirectoryScanner(getProject()); // 3 String[] includedFiles = ds.getIncludedFiles(); for(int i=0; i<includedFiles.length; i++) { String filename = includedFiles[i].replace('','/');   // 4 filename = filename.substring(filename.lastIndexOf("/")+1); if (foundLocation==null && file.equals(filename)) { File base  = ds.getBasedir();   // 5 File found = new File(base, includedFiles[i]); foundLocation = found.getAbsolutePath(); } } } if (foundLocation!=null) // 6 getProject().setNewProperty(location, foundLocation); }

On //1 we check the prerequisites for our task. Doing that in a validate-methodis a common way, because we separate the prerequisites from the real work. On //2 we iterateover all nested filesets. If we don't want to handle multiple filesets, the addFileset()method has to reject the further calls. We can get the result of a fileset via its DirectoryScannerlike done in //3. After that we create a plattform independend String representation ofthe file path (//4, can be done in other ways of course). We have to do the replace(),because we work with a simple string comparison. Ant itself is platform independant and cantherefore run on filesystems with slash (/, e.g. Linux) or backslash (, e.g. Windows) aspath separator. Therefore we have to unify that. If we found our file we create an absolutepath representation on //5, so that we can use that information without knowing the basedir.(This is very important on use with multiple filesets, because they can have different basedirsand the return value of the directory scanner is relative to its basedir.) Finally we store thelocation of the file as property, if we had found one (//6).

Ok, much more easier in this simple case would be to add the file as additionalinclude element to all filesets. But I wanted to show how to handle complex situationswhithout being complex :-)

The test case uses the ant property ant.home as reference. This property is set by theLauncher class which starts ant. We can use that property in our buildfiles as abuild-in property [3]. But if we create a new antenvironment we have to set that value for our own. And we use the <junit> task in fork-mode.Therefore we have do modify our buildfile:

 <target name="junit" description="Runs the unit tests" depends="jar"> <delete dir="${junit.out.dir.xml}"/> <mkdir  dir="${junit.out.dir.xml}"/> <junit printsummary="yes" haltonfailure="no"> <classpath refid="classpath.test"/> <sysproperty key="ant.home" value="${ant.home}"/> <formatter type="xml"/> <batchtest fork="yes" todir="${junit.out.dir.xml}"> <fileset dir="${src.dir}" includes="**/*Test.java"/> </batchtest> </junit> </target>

Using nested paths

A task providing support for filesets is a very comfortable one. But there is anotherpossibility of bundling files: the <path>. Fileset are easy if the files are all undera common base directory. But if this is not the case you have a problem. Another disadvantageis its speed: if you have only a few files in a huge directory structure, why not use a<filelist> instead? <path>s combines these datatypes in that way that a path containsother paths, filesets, dirsets and filelists. This is why Ant-Contribs [4] <foreach> task is modified to support paths instead of filesets. So we want that,too.

Changing from fileset to path support is very easy:

Change java code from: private Vector filesets = new Vector(); public void addFileset(FileSet fileset) { filesets.add(fileset); }to: private Vector paths = new Vector();  *1 public void addPath(Path path) {  *2 paths.add(path); }and build file from: <find file="ant.jar" location="location.ant-jar"> <fileset dir="${ant.home}" includes="**/*.jar"/> </find>to: <find file="ant.jar" location="location.ant-jar"> <path> *3 <fileset dir="${ant.home}" includes="**/*.jar"/> </path> </find>

On *1 we rename only the vector. It�s just for better reading the source. On *2we have to provide the right method: an addName(Type t). Therefore replace thefileset with path here. Finally we have to modify our buildfile on *3 because our taskdoesn�t support nested filesets any longer. So we wrap the fileset inside a path.

And now we modify the testcase. Oh, not very much to do :-) Renaming the testMissingFileset()(not really a must-be but better it�s named like the think it does) and update theexpected-String in that method (now a path not set message is expected). The more complextest cases base on the buildscript. So the targets testFileNotPresent and testFilePresent have to bemodified in the manner described above.

The test are finished. Now we have to adapt the task implementation. The easiest modification isin the validate() method where we change le last line to if (paths.size()<1) throw newBuildException("path not set");. In the execute() method we have a little more work.... mmmh ... in reality it's lesser work, because the Path class does the whole DirectoryScanner-handlingand creating-absolute-paths stuff for us. So the execute method is just:

 public void execute() { validate(); String foundLocation = null; for(Iterator itPaths = paths.iterator(); itPaths.hasNext(); ) { Path path = (Path)itPaths.next(); // 1 String[] includedFiles = path.list(); // 2 for(int i=0; i<includedFiles.length; i++) { String filename = includedFiles[i].replace('','/'); filename = filename.substring(filename.lastIndexOf("/")+1); if (foundLocation==null && file.equals(filename)) { foundLocation = includedFiles[i]; // 3 } } } if (foundLocation!=null) getProject().setNewProperty(location, foundLocation); }

Of course we have to do the typecase to Path on //1. On //2 and //3we see that the Path class does the work for us: no DirectoryScanner (was at 2) and nocreating of the absolute path (was at 3).

Returning a list

So far so good. But could a file be on more than one place in the path? - Of course.
And would it be good to get all of them? - It depends on ...

In this section we will extend that task to support returning a list of all files.Lists as property values are not supported by Ant natively. So we have to see how othertasks use lists. The most famous task using lists is Ant-Contribs <foreach>. All listelements are concatenated and separated with a customizable separator (default ',').

So we do the following:

 <find ... delimiter=""/> ... </find>

If the delimiter is set we will return all found files as list with that delimiter.

Therefore we have to

• provide a new attribute
• collect more than the first file
• delete duplicates
• create the list if necessary
• return that list

So we add as testcase:

in the buildfile: <target name="test.init"> <mkdir dir="test1/dir11/dir111"/> *1 <mkdir dir="test1/dir11/dir112"/> ... <touch file="test1/dir11/dir111/test"/> <touch file="test1/dir11/dir111/not"/> ... <touch file="test1/dir13/dir131/not2"/> <touch file="test1/dir13/dir132/test"/> <touch file="test1/dir13/dir132/not"/> <touch file="test1/dir13/dir132/not2"/> <mkdir dir="test2"/> <copy todir="test2">  *2 <fileset dir="test1"/> </copy> </target> <target name="testMultipleFiles" depends="use.init,test.init"> *3 <find file="test" location="location.test" delimiter=";"> <path> <fileset dir="test1"/> <fileset dir="test2"/> </path> </find> <delete>  *4 <fileset dir="test1"/> <fileset dir="test2"/> </delete> </target>in the test class: public void testMultipleFiles() { executeTarget("testMultipleFiles"); String result = getProject().getProperty("location.test"); assertNotNull("Property not set.", result); assertTrue("Only one file found.", result.indexOf(";") > -1); }

Now we need a directory structure where we CAN find files with the samename in different directories. Because we can't sure to have one we createone on *1 and *2. And of course we clean up that on *4. The creationcan be done inside our test target or in a separate one, which will be betterfor reuse later (*3).

The task implementation is modified as followed:

 private Vector foundFiles = new Vector(); ... private String delimiter = null; ... public void setDelimiter(String delim) { delimiter = delim; } ... public void execute() { validate(); // find all files for(Iterator itPaths = paths.iterator(); itPaths.hasNext(); ) { Path path = (Path)itPaths.next(); String[] includedFiles = path.list(); for(int i=0; i<includedFiles.length; i++) { String filename = includedFiles[i].replace('','/'); filename = filename.substring(filename.lastIndexOf("/")+1); if (file.equals(filename) && !foundFiles.contains(includedFiles[i])) {   // 1 foundFiles.add(includedFiles[i]); } } } // create the return value (list/single) String rv = null; if (foundFiles.size() > 0) { // 2 if (delimiter==null) { // only the first rv = (String)foundFiles.elementAt(0); } else { // create list StringBuffer list = new StringBuffer(); for(Iterator it=foundFiles.iterator(); it.hasNext(); ) { // 3 list.append(it.next()); if (it.hasNext()) list.append(delimiter);   // 4 } rv = list.toString(); } } // create the property if (rv!=null) getProject().setNewProperty(location, rv); }

The algorithm does: finding all files, creating the return value depending on the userswish, returning the value as property. On //1 we eliminates the duplicates. //2ensures that we create the return value only if we have found one file. On //3 weiterate over all found files and //4 ensures that the last entry has no trailingdelimiter.

Ok, first searching for all files and then returning only the first one ... You cantune the performance of your own :-)

Documentation

A task is useless if the only who is able to code the buildfile is the task developer(and he only the next few weeks :-). So documentation is also very important. In whichform you do that depends on your favourite. But inside Ant there is a common format andit has advantages if you use that: all task users know that form, this form is requested ifyou decide to contribute your task. So we will doc our task in that form.

If you have a look at the manual page of the Java task [5] you will see that it:

• is plain html
• starts with the name
• has sections: description, parameters, nested elements, (maybe return codes) and (mostimportant :-) examples
• parameters are listed in a table with columns for attribute name, its description and whether it's required (if you add a feature after an Ant release, provide a since Ant xx statement when it's introduced)
• describe the nested elements (since-statement if necessary)
• provide one or more useful examples; first code, then description.

<html><head><meta http-equiv="Content-Language" content="en-us"><title>Taskname Task</title></head><body><h2><a name="taskname">Taskname</a></h2><h3>Description</h3><p> Describe the task.</p><h3>Parameters</h3><table border="1" cellpadding="2" cellspacing="0">  <tr> <td valign="top"><b>Attribute</b></td> <td valign="top"><b>Description</b></td> <td align="center" valign="top"><b>Required</b></td>  </tr>  do this html row for each attribute (including inherited attributes)  <tr> <td valign="top">classname</td> <td valign="top">the Java class to execute.</td> <td align="center" valign="top">Either jar or classname</td>  </tr></table><h3>Parameters specified as nested elements</h3>Describe each nested element (including inherited)<h4>your nested element</h4><p>description</p><p><em>since Ant 1.6</em>.</p><h3>Examples</h3><pre> A code sample; don't forget to escape the < of the tags with &lt;</pre>What should that example do?</body></html>

Here is an example documentation page for our task:

<html><head><meta http-equiv="Content-Language" content="en-us"><title>Find Task</title></head><body><h2><a name="find">Find</a></h2><h3>Description</h3><p>Searchs in a given path for a file and returns the absolute to it as property.If delimiter is set this task returns all found locations.</p><h3>Parameters</h3><table border="1" cellpadding="2" cellspacing="0">  <tr> <td valign="top"><b>Attribute</b></td> <td valign="top"><b>Description</b></td> <td align="center" valign="top"><b>Required</b></td>  </tr>  <tr> <td valign="top">file</td> <td valign="top">The name of the file to search.</td> <td align="center" valign="top">yes</td>  </tr>  <tr> <td valign="top">location</td> <td valign="top">The name of the property where to store the location</td> <td align="center" valign="top">yes</td>  </tr>  <tr> <td valign="top">delimiter</td> <td valign="top">A delimiter to use when returning the list</td> <td align="center" valign="top">only if the list is required</td>  </tr></table><h3>Parameters specified as nested elements</h3><h4>path</h4><p>The path where to search the file.</p><h3>Examples</h3><pre> <find file="ant.jar" location="loc"> <path> <fileset dir="${ant.home}"/> <path> </find></pre>Searches in Ants home directory for a file <i>ant.jar</i> and stores its location inproperty <i>loc</i> (should be ANT_HOME/bin/ant.jar).<pre> <find file="ant.jar" location="loc" delimiter=";"> <path> <fileset dir="C:/"/> <path> </find> <echo>ant.jar found in: ${loc}</echo></pre>Searches in Windows C: drive for all <i>ant.jar</i> and stores their locations inproperty <i>loc</i> delimited with <i>''</i>. (should need a long time :-)After that it prints out the result (e.g. C:/ant-1.5.4/bin/ant.jar;C:/ant-1.6/bin/ant.jar).</body></html>

Contribute the new task

• is our task welcome? :-) Simply ask on the user list
• is the right package used?
• does the code conform to the styleguide?
• do all tests pass?
• does the code compile on JDK 1.2 (and passes all tests there)?
• code under Apache license
• create a patch file
• publishing that patch file

Now we will check the "Checklist before submitting a new task" described in that guideline.

• Java file begins with Apache license statement. must do that
• Task does not depend on GPL or LGPL code. ok
• Source code complies with style guidelines have to check (checkstyle)
• Code compiles and runs on Java1.2 have to try
• Member variables are private, and provide public accessor methods if access is actually needed. have to check (checkstyle)
• Maybe Task has failonerror attribute to control failure behaviour hasn't
• New test cases written and succeed passed on JDK 1.4, have to try on JDK 1.2
• Documentation page written ok
• Example task declarations in the documentation tested. ok (used in tests)
• Patch files generated using cvs diff -u to do
• patch files include a patch to defaults.properties to register thetasks to do
• patch files include a patch to tasklist.html to link to the new task page to do
• Message to dev contains [SUBMIT] and task name in subject to do
• Message body contains a rationale for the task to do
• Message attachments contain the required files -source, documentation,test and patches zipped up to escape the HTML filter. to do

Package / Directories

This task does not depend on any external library. Therefore we can use this asa core task. This task contains only one class. So we can use the standard packagefor core tasks: org.apache.tools.ant.taskdefs. Implementations are in thedirectory src/main, tests in src/testcases and buildfiles fortests in src/etc/testcases.

Now we integrate our work into Ants distribution. So first we do an update of ourcvs tree. If not done yet, you have to checkout the ant module from Apaches cvs serveras described in Access the Source Tree (AnonCVS)[7] (password is anoncvs):

cvs -d :pserver:[email protected]:/home/cvspublic login //1cvs -d :pserver:[email protected]:/home/cvspublic checkout ant  //2

cvs -d :pserver:[email protected]:/home/cvspublic logincd ant   //3cvs -d :pserver:[email protected]:/home/cvspublic update //4

We use the -d flag on //1 to specifiy the cvs directory. You canspecify the environment variable CVSROOT with that value and after that you haven�tto use that flag any more. On //2 we get the whole cvs tree of ant. (Sorry,but that uses a lot of time ... 10 up to 30 minutes are not unusual ... but this hasto be done only once :-). A cvs update doesn't use a modulename but you have to beinside the directory. Therefore we go into that on //3 and do the updateon //4.

Now we will build our Ant distribution and do a test. So we can see if thereare any tests failing on our machine. (We can ignore these failing tests on latersteps; windows syntax used here- translate to xNIX if needed):

ANTHOME> build // 1ANTHOME> set ANT_HOME=%CD%dist   // 2ANTHOME> ant test -Dtest.haltonfailure=false  // 3

Next we apply our work onto Ants sources. Because we haven't modified any, this isa relative simple step. (Because I have a local copy of Ant and usually contribute mywork, I work on the local copy just from the beginning. The advantage: this step isn'tnecessary and saves a lot of work if you modify existing source :-).

• move the Find.java to ANTHOME/src/main/org/apache/tools/ant/taskdefs/Find.java
• move the FindTest.java to ANTHOME/src/testcases/org/apache/tools/ant/taskdefs/FindTest.java
• move the build.xml to ANTHOME/src/etc/testcases/taskdefs/find.xml (!!! renamed !!!)
• add a package org.apache.tools.ant.taskdefs; at the beginning of the two java files
• delete all stuff from find.xml keeping the targets "testFileNotPresent", "testFilePresent", "test.init" and "testMultipleFiles"
• delete the dependency to "use.init" in the find.xml
• in FindTest.java change the line configureProject("build.xml"); to configureProject("src/etc/testcases/taskdefs/find.xml");
• move the find.html to ANTHOME/docs/manual/Tasks/find.html
• add a <a href="Tasks/find.html">Find</a><br> in the ANTHOME/docs/manual/tasklist.html

ANTHOME> buildANTHOME> ant run-single-test  // 1 -Dtestcase=org.apache.tools.ant.taskdefs.FindTest // 2 -Dtest.haltonfailure=false

And ... oh, all tests fail: Ant could not find the task or a class this task relies upon.

Ok: in the earlier steps we told Ant to use the Find class for the <find> task (remember the<taskdef> statement in the "use.init" target). But now we want to introduce that task asa core task. And nobody wants to taskdef the javac, echo, ... So what to do? The answer is thesrc/main/.../taskdefs/default.properties. Here is the mapping between taskname and implementingclass done. So we add a find=org.apache.tools.ant.taskdefs.Find as the last coretask (just before the # optional tasks line). Now a second try:

ANTHOME> build // 1ANTHOME> ant run-single-test -Dtestcase=org.apache.tools.ant.taskdefs.FindTest -Dtest.haltonfailure=false

ANTHOME> ant test -Dtest.haltonfailure=false

This test run should show us two things: our test will run and the number of failing testsis the same as directly after the cvs update (without our modifications).

Apache license statement

Simply copy the license text from one the other source from the Ant source tree.

Test on JDK 1.2

Until version 1.5 Ant must be able to run on a JDK 1.1. With version 1.6 this is not arequisite any more. But JDK 1.2 is a must-to-work-with. So we have to test that. You can download olderJDKs from Oracle [8].

Clean the ANT_HOME variable, delete the build, bootstrap and dist directoryand point JAVA_HOME to the JDK 1.2 home directory. Then do the build, set ANT_HOMEand run ant test (like above).

Our test should pass.

Checkstyle

There are many things we have to ensure. Indentation with 4 spaces, blanks here and there, ...(all described in the Ant Task Guidelines [6] whichincludes the Sun code style[9]). Because there are so many things we would be happy to have a tool for do the checks.There is one: checkstyle. Checkstyle is available at Sourceforge [10] and Ant provides with the check.xml a buildfile which will do the jobfor us.

Download it and put the checkstyle-*-all.jar into your %USERPROFILE%.antlib directory.All jar's stored there are available to Ant so you haven't to add it to you %ANT_HOME%libdirectory (this feature was added with Ant 1.6).

So we will run the tests with

ANTHOME> ant -f check.xml checkstyle htmlreport

Hint: start at the buttom of the file so the line numbers in the report will keepup to date and you will find the next error place much more easier without redoing the checkstyle.

After cleaning up the code according to the messages we delete the reports directory anddo a second checkstyle run. Now our task isn't listed. That's fine :-)

Publish the task

Finally we publish that archive. As described in the Ant Task Guidelines [7] we can post it on the developer mailinglist or we create a BugZillaentry. For both we need some information:

Sending an email with these information is very easy and I think I haven't to show that.The other way - BugZilla - is slightly more difficult. But it has the advantage that entrieswill not be forgotten (once per week a report is generated). So I will show this way.

You must have a BugZilla account for that. So open the BugZilla Main Page [11] and follow the linkOpen a new Bugzilla account [12]and the steps described there if you haven't one.

1. From the BugZilla main page choose Enter a new bug report [13]
2. Choose "Ant" as product
3. Version is the last "Alpha (nightly)" (at this time 1.7)
4. Component is "Core tasks"
5. Plattform and Severity are ok with "Other" and "Normal"
6. Initial State is ok with "New"
7. Same with the empy "Assigned to"
8. It is not required to add yourself as CC, because you are the reporter and therefore will be informed on changes
9. URL: no url required
10. Summary: add the subject from the table
11. Description: add the body from the table
12. Then press "Commit"
13. After redirecting to the new created bug entry click "Create a New Attachment"
14. Enter the path to your local path file into "File" or choose it via the "File"'s button.
15. Enter a short description into "Description", so that you could guess, what the path file includes. Here we could add "Initial Patch".
16. The "Content Type" is "auto-detect". You could use the "patch" type, if you only provide a single path file, but we want do upload more that one, included in our patch.zip.
17. Then press "Commit"

Resources