Text files, test data, and the Jakarta Commons BeanUtils

Earlier, I wrote about testing only one thing at a time. A little one-liner I tossed out in that was using text files to load object graphs in your test cases. I thought I’d elaborate on that a bit more.

The technique I use here is based heavily off the Jakarta Commons Bean Utils package. If you’ve never come across this, the Bean Utils provides utility methods to, well, use Java Beans. In particular, what I’m using here is the BeanUtilsBean.populate() method; this populates an object instance with values supplied in a Map of name-value pairs.

Loading the text file

The central class in this exercise is the BeanLoader. In this form, at least, it loads a Java Properties file from a resource, then uses it to populate a new object instance. Here’s the main code:

package net.twasink.beanloader;

import java.io.IOException;
import java.util.*

import org.apache.commons.beanutils.BeanUtilsBean;
import org.apache.commons.beanutils.ConvertUtilsBean;

/**
* Load Domain classes from files. Loaded instances are cached, to enable
* cross-references.
*
* The files are based on Java property files. Property values are converted using the various
* converters in this package. This is based on the Jakarta Commons BeanUtils toolset.
*/
public class BeanLoader {
  private BeanUtilsBean beanUtils = ...
  
  public  T loadClass(String key, Class clazz) throws IOException {
    ...
    String className = clazz.getSimpleName().toLowerCase();
    Properties properties = FileUtil.loadProperties("/" + className + "-" + key + ".txt");
    T object = ReflectUtil.createInstance(clazz);
      try {
      beanUtils.populate(object, properties);
      loadedClasses.put(classData, object);
      return object;
    } catch (Exception e) {
      throw new RuntimeException("Couldn't populate " + className + ": " + key, e);
    }
  }
  ...
}

(FileUtil.loadProperties simply creates a new java.util.Properties instance, and loads it with the resource listed, via Properties#loadProperties())

How does this work?

The Map or properties file can be thought of as a series of name-value pairs. The names are the name of Java Bean properties – one way to think of these is as “getter” and “setter” methods. Indeed, the default mechanism uses getters and setters. The populate method uses reflection to call the setters with the values from the Map. So, a property entry like address=1060 W Addison St is similar to the Java code setAddress("1060 W Addison St") (bonus wuffie points for the first person to get the reference)

Setters, of course, can take any type of object, and the BeanUtils make no restriction. However, we’ve loaded the data from Strings. Does this mean we are restricted to String properties? In a few words: of course not. That would, after all, be rather silly.

Built into the BeanUtils package are Converters which convert Strings to other types. By default, BeanUtils comes with converters for a few building block types: numbers, dates, files. In addition, you can register your own.

Custom Conversions

In the example, I snipped how the BeanUtilsBean was created. So let’s go back and fill that in:

public class BeanLoader {
  private BeanUtilsBean beanUtils = createBeanUtils();
  ...
  private BeanUtilsBean createBeanUtils() {
    PropertyUtils propertyUtils = ...
    ConvertUtilsBean converterUtils = new ConvertUtilsBean();
    BeanUtilsBean beanUtilsBean = new BeanUtilsBean(converterUtils, propertyUtils);
    List converters = createConverters(beanUtilsBean);
    registerConverters(converters, converterUtils);
    return beanUtilsBean;
  }
  
  private List createConverters(BeanUtilsBean beanUtils) {
    List converters = Arrays.asList(new DateConverter(),
    new CollectionsConverter(beanUtils),
    new DomainConverter(this));
    return converters;
  }
  
  private void registerConverters(List converters,
    ConvertUtilsBean converterUtil) {
    for (BeanConverter converter : converters) {
      converter.register(converterUtil);
    }
  }
  ....
}

/**
* Interface to assist registration of converters.
*/
public interface BeanConverter extends Converter {

  /** Register (possibly mutiple) classes to be converted via this converter instance.*/
  void register(ConvertUtilsBean converterUtilsBean);

}

Here we create our own BeanUtilsBean instance, and configure it with three custom converters: one for dates, another for collections, and finally one for “domain” objects. (Note that in older versions of BeanUtils, this configuration was only possible at a static level, not an instance level). I could, of course, have registered as many converters as I wanted, and I could easily have used some sort of IoC technique or discovery technique to populate the converters.

These converters then register themselves with the ConvertUtilsBean that our BeanUtilsBean instance will use; this lets the ConvertUtilsBean know what classes each converter supports. Then, whenever the BeanUtilsBean comes across a property of the right type, it will use the appropriate converter.

So, let’s look at these converters. Here’s the DateConverter:

import java.util.Date;

public class DateConverter implements BeanConverter {
  // Not thread-safe; who cares?
  private static final DateFormat DATE_PARSER = new SimpleDateFormat("dd/MMM/yyyy");
  private static final DateFormat TIME_PARSER = new SimpleDateFormat("dd/MMM/yyyy HH:mm");
  
  /** Convert String to Date. */
  public Object convert(Class clazz, Object value) {
    assert Date.class == clazz : "Not a date";
    try {
      DateFormat parser = (value.toString().length() == 11) ? DATE_PARSER : TIME_PARSER;
      return parser.parse(value.toString());
    } catch (ParseException e) {
      throw new ConversionException(e);
    }
  }
  
  /** @inheritDoc */
  public void register(ConvertUtilsBean converter) {
    converter.register(this, Date.class);
  }
}

The register() method allows this to indicate that it converts Strings to java.util.Date instances. To do this, the BeanUtilsBeancalls the convert() method, passing in the target type and the value to convert. The DateConverter uses a DateFormat to convert the string, and if it has a problem, throws ConversionException out.

Pretty much all of the converters work like this. The CollectionsConverter, for example, takes in a comma-seperated list and uses it to create a HashSet (at least in the current implementation); naturally, the object instances in the Set are populated via the BeanUtilsBean. The DomainConverter uses the loader to load new object instances from files, like so:

// Convert method from DomainConverter
public Object convert(Class type, Object value) {
  try {
    return beanLoader.loadClass(value.toString(), type);
  } catch (IOException e) {
    throw new ConversionException(e);
  }
}

This technique allows you to load object graphs by referring to other files, particularly with a small addition to the loadClass() method that caches previously-loaded instances; the “value” in the name-value pair would be the “key” argument to loadClass().

(You can find all the source files for this example in this zip file I warn you though… there’s no test cases (didn’t upload them), and the only doco is the JavaDoc. The syntax for specifying collections is particularly arcane)

Questions and some answers

I’ll just pre-empt some likely questions here:

Isn’t this like serialisation?

Actually, no. Serialisation writes out the instance variables of a class. Bean properties, however, are methods. Sure, they quite often fall back to instance variables, but they don’t have to.

In addition, the resulting text files are relatively human readable (and writeable!). Java serialisation comes in two basic forms: binary and XML. Neither is that readable.

Doesn’t this break encapsulation?

Um, maybe. 🙂 It depends.

Firstly, you don’t have to use public setter methods. The sample code above makes a reference to a PropertyUtils instance. The default implementation, as supplied by the BeanUtils package, does require public methods. But you can change it. The source zip file includes a FunkyPropertyUtils that basically ignores visibility constraints – you need a public getter, but you can have a private setter. It would only take a little bit of work to allow it to expose completely private setters.

Depending on what technologies you use, you may already have a bunch of getters and setters on your domain classes. For example, if you use Hibernate you probably already have the setters you want.

Secondly, it depends on how much detail you expose, and the level of granularity of your objects and converters. For example, on the project I built this for originally, we have Money objects. Money has a value and a currency. So, the converter I wrote for Money (not included) took String values like AUD 90.00; it didn’t need to have a set of name-value pairs like currencyCode=AUD,value=90.00. So encapsulation is violated only to the extent that you don’t provide suitable converters. Let me re-iterate this: the job of the Converter is to convert from Strings to your objects. How you do that is up to you.

It’s generally a good idea to have some way of creating your domain objects from Strings anyway; when you get down to it, the user interface is essentially a bunch of Strings, and that’s what users enter. The converters help here.

You also don’t have to use setters. This built up on Java Beans. Java Beans have a facility called “BeanInfo”, which allows you a lot of control over the properties. For example, you can assign different writer methods instead of setters. It’s just that the getter/setter model is easy. If you want to change how properties are located in the first place, you can overload the PropertyUtils.getPropertyDescripters() method; this is what FunkyPropertyUtils does to bypass visibility constraints.

So, to sum up: yes, getters and setters are evil But this doesn’t force you to break encapsulation if you don’t want to.

Is this really worth the effort?

Yes. On the project I built this for, creating test data programatically was taking anywhere between 15 minutes to an hour by hand. Doing it with text files took minutes. Also, by having a library for these files (we put them all in a special source folder, called ‘test/data’ unimaginatively enough), we were advertising what test fixtures were available; over time, this should promote re-use and prevent duplication.

How fast is it?

When writing unit tests, speed is important. Normally, it’s not advised to read a file when writing unit tests. This gets speed out of caching the instantiated objects – each file only needs to be read once.

What else can you use this for?

Well, Struts uses BeanUtils – in fact, they evolved out of Struts. So you could use the same converters for automatically converting user-entered data into your FormBeans, if you use Struts. I’m sure BeanUtils gets used by a bunch of other projects as well.

Anyway, that’s it. I hope this helps someone else, and that you get ideas of how use this technique for yourself.