Friday, August 22, 2008

ANT and duplicate files in archives

by

ANT is one of the most common tools used by software developers today.  One of the reasons that is is so popular is because it can do anything.  I’m serious.  I bet there is an ant task somewhere that can compile my files, package them in jars, do my laundry, and pick the kids up form daycare.  OK, so maybe it isn’t quite that robust, but it is pretty close.  However, with great ability comes the increased likelihood of great failures in design, and I may have just found one in ANT. 

The tasks to create archive files (zip, jar, war, etc) allow multiple files of the same fully-qualified name to exist within a single archive - by default.

A little back story, we recently switch over to the Bamboo Continuous Integration Build Server, and in doing so had to rewrite some of the ANT build scripts.  The .ear files that were created were able to deploy out to our application servers just fine, but we noticed that they were larger than they were before.  Almost twice as large.  But when we would look on the application servers at the exploded directories, they were identical to previous versions.  So where was the extra size coming from?

We found it by accident.  I had a copy of one of the .ear files on my Windows Desktop, and I wanted to quickly check something in it.  Instead of taking the time to open up a command line and use java to un-jar the .ear file, I took a shortcut and renamed it to a .zip extension.  That allowed me to look at its contents through Windows Explorer without having to unarchive it.  And lo and behold, I saw that every file in my lib directory was duplicated.  Some quick Googling led me to Dan Moore’s blog, where he described the same basic problem.

In both Dan’s case and mine, we had (for a variety of reasons) a jar task that was including files from two different locations which had the potential to have files with the same name.  If this happens, then by default, the ANT jar task (or ear, or war, or any other archiving task) will include both files in the archive.  This isn’t a bug, it is the intended result.

Please note that the zip format allows multiple files of the same fully-qualified name to exist within a single archive. This has been documented as causing various problems for unsuspecting users. If you wish to avoid this behavior you must set the duplicate attribute to a value other than its default, “add”.

The archive tasks accept a duplicate attribute, which can have one of three values:

• add - the default, which adds the duplicate file to the archive
• fail - causes the task to fail if duplicate files are found
• preserve - preserves the first file, and disregards subsequent files

While I like the ability to specify what you want the task to do with duplicate files, why on earth would the default be to add them both?  I would think that preserve would be the most intuitive, with fail being the second best.  I honestly cannot think of a single time when you would actually want to add both.  When you unarchive the file, all file systems will have a conflict with the duplicate files anyway, so it isn’t like you can use both.

A quick change to add a duplicate attribute with the value “preserve” brought our .ear files back down to the normal size.  Problem solved.  Thanks go out to Dan Moore, whose well written blog helped us solve the problem quickly.