Text files, test data, and the Jakarta Commons BeanUtils

Earlier, I wrote about testing only one thing at a time. A little one-liner I tossed out in that was using text files to load object graphs in your test cases. I thought I’d elaborate on that a bit more.


The technique I use here is based heavily off the Jakarta Commons Bean Utils package. If you’ve never come across this, the Bean Utils provides utility methods to, well, use Java Beans. In particular, what I’m using here is the BeanUtilsBean.populate() method; this populates an object instance with values supplied in a Map of name-value pairs.

Loading the text file

The central class in this exercise is the BeanLoader. In this form, at least, it loads a Java Properties file from a resource, then uses it to populate a new object instance. Here’s the main code:

package net.twasink.beanloader;

import java.io.IOException;
import java.util.*

import org.apache.commons.beanutils.BeanUtilsBean;
import org.apache.commons.beanutils.ConvertUtilsBean;

/**
* Load Domain classes from files. Loaded instances are cached, to enable
* cross-references.
*
* The files are based on Java property files. Property values are converted using the various
* converters in this package. This is based on the Jakarta Commons BeanUtils toolset.
*/
public class BeanLoader {
  private BeanUtilsBean beanUtils = ...
  
  public  T loadClass(String key, Class clazz) throws IOException {
    ...
    String className = clazz.getSimpleName().toLowerCase();
    Properties properties = FileUtil.loadProperties("/" + className + "-" + key + ".txt");
    T object = ReflectUtil.createInstance(clazz);
      try {
      beanUtils.populate(object, properties);
      loadedClasses.put(classData, object);
      return object;
    } catch (Exception e) {
      throw new RuntimeException("Couldn't populate " + className + ": " + key, e);
    }
  }
  ...
}

(FileUtil.loadProperties simply creates a new java.util.Properties instance, and loads it with the resource listed, via Properties#loadProperties())

How does this work?

The Map or properties file can be thought of as a series of name-value pairs. The names are the name of Java Bean properties – one way to think of these is as “getter” and “setter” methods. Indeed, the default mechanism uses getters and setters. The populate method uses reflection to call the setters with the values from the Map. So, a property entry like address=1060 W Addison St is similar to the Java code setAddress("1060 W Addison St") (bonus wuffie points for the first person to get the reference)

Setters, of course, can take any type of object, and the BeanUtils make no restriction. However, we’ve loaded the data from Strings. Does this mean we are restricted to String properties? In a few words: of course not. That would, after all, be rather silly.

Built into the BeanUtils package are Converters which convert Strings to other types. By default, BeanUtils comes with converters for a few building block types: numbers, dates, files. In addition, you can register your own.

Custom Conversions

In the example, I snipped how the BeanUtilsBean was created. So let’s go back and fill that in:

public class BeanLoader {
  private BeanUtilsBean beanUtils = createBeanUtils();
  ...
  private BeanUtilsBean createBeanUtils() {
    PropertyUtils propertyUtils = ...
    ConvertUtilsBean converterUtils = new ConvertUtilsBean();
    BeanUtilsBean beanUtilsBean = new BeanUtilsBean(converterUtils, propertyUtils);
    List converters = createConverters(beanUtilsBean);
    registerConverters(converters, converterUtils);
    return beanUtilsBean;
  }
  
  private List createConverters(BeanUtilsBean beanUtils) {
    List converters = Arrays.asList(new DateConverter(),
    new CollectionsConverter(beanUtils),
    new DomainConverter(this));
    return converters;
  }
  
  private void registerConverters(List converters,
    ConvertUtilsBean converterUtil) {
    for (BeanConverter converter : converters) {
      converter.register(converterUtil);
    }
  }
  ....
}

/**
* Interface to assist registration of converters.
*/
public interface BeanConverter extends Converter {

  /** Register (possibly mutiple) classes to be converted via this converter instance.*/
  void register(ConvertUtilsBean converterUtilsBean);

}

Here we create our own BeanUtilsBean instance, and configure it with three custom converters: one for dates, another for collections, and finally one for “domain” objects. (Note that in older versions of BeanUtils, this configuration was only possible at a static level, not an instance level). I could, of course, have registered as many converters as I wanted, and I could easily have used some sort of IoC technique or discovery technique to populate the converters.

These converters then register themselves with the ConvertUtilsBean that our BeanUtilsBean instance will use; this lets the ConvertUtilsBean know what classes each converter supports. Then, whenever the BeanUtilsBean comes across a property of the right type, it will use the appropriate converter.

So, let’s look at these converters. Here’s the DateConverter:

import java.util.Date;

public class DateConverter implements BeanConverter {
  // Not thread-safe; who cares?
  private static final DateFormat DATE_PARSER = new SimpleDateFormat("dd/MMM/yyyy");
  private static final DateFormat TIME_PARSER = new SimpleDateFormat("dd/MMM/yyyy HH:mm");
  
  /** Convert String to Date. */
  public Object convert(Class clazz, Object value) {
    assert Date.class == clazz : "Not a date";
    try {
      DateFormat parser = (value.toString().length() == 11) ? DATE_PARSER : TIME_PARSER;
      return parser.parse(value.toString());
    } catch (ParseException e) {
      throw new ConversionException(e);
    }
  }
  
  /** @inheritDoc */
  public void register(ConvertUtilsBean converter) {
    converter.register(this, Date.class);
  }
}

The register() method allows this to indicate that it converts Strings to java.util.Date instances. To do this, the BeanUtilsBeancalls the convert() method, passing in the target type and the value to convert. The DateConverter uses a DateFormat to convert the string, and if it has a problem, throws ConversionException out.

Pretty much all of the converters work like this. The CollectionsConverter, for example, takes in a comma-seperated list and uses it to create a HashSet (at least in the current implementation); naturally, the object instances in the Set are populated via the BeanUtilsBean. The DomainConverter uses the loader to load new object instances from files, like so:

// Convert method from DomainConverter
public Object convert(Class type, Object value) {
  try {
    return beanLoader.loadClass(value.toString(), type);
  } catch (IOException e) {
    throw new ConversionException(e);
  }
}

This technique allows you to load object graphs by referring to other files, particularly with a small addition to the loadClass() method that caches previously-loaded instances; the “value” in the name-value pair would be the “key” argument to loadClass().

(You can find all the source files for this example in this zip file I warn you though… there’s no test cases (didn’t upload them), and the only doco is the JavaDoc. The syntax for specifying collections is particularly arcane)

Questions and some answers

I’ll just pre-empt some likely questions here:

Isn’t this like serialisation?

Actually, no. Serialisation writes out the instance variables of a class. Bean properties, however, are methods. Sure, they quite often fall back to instance variables, but they don’t have to.

In addition, the resulting text files are relatively human readable (and writeable!). Java serialisation comes in two basic forms: binary and XML. Neither is that readable.

Doesn’t this break encapsulation?

Um, maybe. 🙂 It depends.

Firstly, you don’t have to use public setter methods. The sample code above makes a reference to a PropertyUtils instance. The default implementation, as supplied by the BeanUtils package, does require public methods. But you can change it. The source zip file includes a FunkyPropertyUtils that basically ignores visibility constraints – you need a public getter, but you can have a private setter. It would only take a little bit of work to allow it to expose completely private setters.

Depending on what technologies you use, you may already have a bunch of getters and setters on your domain classes. For example, if you use Hibernate you probably already have the setters you want.

Secondly, it depends on how much detail you expose, and the level of granularity of your objects and converters. For example, on the project I built this for originally, we have Money objects. Money has a value and a currency. So, the converter I wrote for Money (not included) took String values like AUD 90.00; it didn’t need to have a set of name-value pairs like currencyCode=AUD,value=90.00. So encapsulation is violated only to the extent that you don’t provide suitable converters. Let me re-iterate this: the job of the Converter is to convert from Strings to your objects. How you do that is up to you.

It’s generally a good idea to have some way of creating your domain objects from Strings anyway; when you get down to it, the user interface is essentially a bunch of Strings, and that’s what users enter. The converters help here.

You also don’t have to use setters. This built up on Java Beans. Java Beans have a facility called “BeanInfo”, which allows you a lot of control over the properties. For example, you can assign different writer methods instead of setters. It’s just that the getter/setter model is easy. If you want to change how properties are located in the first place, you can overload the PropertyUtils.getPropertyDescripters() method; this is what FunkyPropertyUtils does to bypass visibility constraints.

So, to sum up: yes, getters and setters are evil But this doesn’t force you to break encapsulation if you don’t want to.

Is this really worth the effort?

Yes. On the project I built this for, creating test data programatically was taking anywhere between 15 minutes to an hour by hand. Doing it with text files took minutes. Also, by having a library for these files (we put them all in a special source folder, called ‘test/data’ unimaginatively enough), we were advertising what test fixtures were available; over time, this should promote re-use and prevent duplication.

How fast is it?

When writing unit tests, speed is important. Normally, it’s not advised to read a file when writing unit tests. This gets speed out of caching the instantiated objects – each file only needs to be read once.

What else can you use this for?

Well, Struts uses BeanUtils – in fact, they evolved out of Struts. So you could use the same converters for automatically converting user-entered data into your FormBeans, if you use Struts. I’m sure BeanUtils gets used by a bunch of other projects as well.

Anyway, that’s it. I hope this helps someone else, and that you get ideas of how use this technique for yourself.

Advertisements

Author: Robert Watkins

My name is Robert Watkins. I am a software developer and have been for over 18 years now. I currently work for people, but my opinions here are in no way endorsed by them (which is cool; their opinions aren’t endorsed by me either). My main professional interests are in Java development, using Agile methods, with a historical focus on building web based applications. I’m also a Mac-fan and love my iPhone, which I’m currently learning how to code for. I live and work in Brisbane, Australia, but I grew up in the Northern Territory, and still find Brisbane too cold (after 16 years here). I’m married, with two children and one cat. My politics are socialist in tendency, my religious affiliation is atheist (aka “none of the above”), my attitude is condescending and my moral standing is lying down.

4 thoughts on “Text files, test data, and the Jakarta Commons BeanUtils”

  1. Making test data setup cheaper and yet still reliable is definitely something that I’ve been thinking about lately. I’ll add this as something to try. Please keep us up to date on how this experience works out or not.

  2. On my latest project I was using BeanUtils.copyProperties to move the data from a struts form into a domain object as you mention.

    It worked fine until someone else decided to refactor some of the property names on the bean.

    After this happened several times I gave up on using it in that part of the project. (insert rant here)

    I had to write a couple of really big movePropertiesToForm and movePropertiesToDomainObject methods – so that when they change them again the compiler will let them know they’re being a wally.

    I am also using BeanUtils.copyProperties down at the Hibernate layer – because our design mandates* that there has to be an object in between our domain object and the database. It can be a bit useful as an adapter for ‘one way properties’. Eg if I have a Date property which the domain object wants as a java.util.Date, but the database wants as a java.sql.Date. I simply write a second set of setters and getters, and then I tell hibernate about the ones it needs to know about – the getter for instance takes the java.sql.Date and converts it to a java.util.Date etc etc.

    This example would actually be more useful at the web layer, where everything tends to get passed around as strings, but as I said… some of the refactoring cowboys would need to be reigned in first.

    * personally I’d rather just save the domain object directly since I think that is what hibernate is set up to do. We have some other nasty restrictions where there are trivial workarounds that would give them the desired functionality they want, but for hystorical reasons they need their tables done strangely (eg we have to generate our own primary keys and cannot use surrogate keys).

  3. Hey, Rick. Nice to hear from you again. Sorry it took so long for me to reply.

    Firstly: Copying bean properties and refactoring. When using any reflection-based technique, you’re essentially using dynamic bindings. When using dynamic bindings, you really should have good tests around them to avoid bungles such as mistyped names. Good tests would then flag the errant refactoring. Naturally, your tests should use the BeanUtils as well.

    If they ignore the test failure, then I suggest educating with a 2×4 clue bat (foam or wood; your choice).

    Secondly: the Hibernate mapping may be better done using a User Type. Simply tell Hibernate to use your UtilDateToSqlDate user type, instead of doing its own stuff. This helps to avoid contaminating your domain code with this sort of detail.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s