CVS to Subversion Migration – a war story

One of the things that was annoying us at Wotif for some time has been how slow our CVS repository had been getting. Doing a tag after a release, for example, could take up to 15 minutes, CruiseControl was taking up to 45 minutes to do a scan, that sort of thing. Basically, the server we were using was almost literally having the hard disks melt… a 4GB CVS repository was being hammered almost continually. So, we finally bit the bullet a few weeks ago, and moved to Subversion, and I thought I’d share the experience.

We’d originally planned a slow transition to Subversion. However, after a necessary OS upgrade on the server, the CVS performance absolutely tanked – we’d lost some hard disk tuning settings. So it became a bit rushed. Quite literally, the migration got done over night.

h2. Creating the Subversion repositories

We used the “cvs2svn”:http://cvs2svn.tigris.org tool to do the actual migration. I was pleasantly surprised at how fast it was – creating the Subversion dumps took about 3 hours to do (which, considering it was 4 years of history, and 4GB in size, is damn good). We’d decided to create a few repositories, one for each important module in our CVS repository, so part of the process involved shuffling directories around. In the end, the largest repository came in with about 6300 revisions.

h2. Setting up the server

I’d decided to use the Apache mod_svn module as a front end, running over https. I followed the instructions in “Pragmatic Version Control With Subversion”:http://www.pragmaticprogrammer.com/titles/svn/ on this, and had it up and running within a few minutes.

We’d previously experimented with running ssh+svn, but this had a limitation in that you needed everyone to have a shell account and write access to the repository; our IT admins had objections to that (which was fair enough – they need to look after the box, after all). Using Apache meant that we could use one of the many Apache authentication options – that said, I went with the simplest and used the htpasswd file.

h2. Setting up FishEye

We use “FishEye”:http://cenqua.com/fisheye at Wotif, and it supports Subversion. Setting up Subversion access in FishEye was fairly easy – we just followed the instructions in the doco.

One thing we were not prepared for was how long FishEye took to “slurp” in the SVN repositories. The largest repository took nearly 30 hours – all up, it was a bit over 40 hours. The culprit was that we used the JavaSVN client, rather than the native JavaHL bindings; we also had FishEye set up on another box, rather than the server where Subversion was located. This made the calls a little slow… by contrast, I set up a FishEye instance on a box with a copy of the repositories, using JavaHL and the file:/// protocol (as recommended by Cenqua), and it took about 45 minutes to do the full index. Just a little faster… 😉 and we’ll look at relocating our FishEye instance soon.

h2. CruiseControl

Replacing the CVS setup in “CruiseControl”:http://cruisecontrol.sourceforge.net with Subversion was trivial, and we were very happy that the 45 minute “checking for modifications” period dropped to about a second. 🙂

One mildly annoying thing was that the Subversion revision number isn’t shown in the reporting web application; it is in the log file, but not shown in the app. We could have changed our stylesheet, but we went for something a little different – we wrote a Subversion-based label incrementer. Now the labels in CruiseControl have the repository number built in. This seems to be working well.

(Previously, we had a custom Publisher that did a CVS tag – this was no longer needed)

h2. IDE configurations

We use both IntelliJ and Eclipse in our environment. IntelliJ has Subversion built in, with about the same level of support as their CVS support – that is to say, not very much. Eclipse doesn’t have any native Subversion support, but there are two plugins – “Supclipse”:http://subclipse.tigris.org and “Subversive”:http://www.polarion.org/index.php?page=overview&project=subversive. I’m personally using Subclipse, but a couple of other people are trying out Subversive; we’ll see how it goes over time.

I can say that we’re not really happy with the Subversion plugins for Eclipse – they just aren’t as good as the CVS support. I really hope that the Eclipse project decides to make SVN support standard in the next major release of Eclipse, with the same level of support as CVS – particularly the preview merge support.

h2. Pain points so far

The biggest pain point we’ve had so far has been merges. We use parallel development of projects in a few different branches, with a “main line” that we regularly sync up with. With CVS, we’d got this down fairly pat, and the excellent support in Eclipse for CVS merges made the process fairly smooth. We’re still figuring out the best way to do this in Subversion – the process I’ve settled on for now is using the command line, saving the output, and then rectifying conflicts (using the “Edit Concflicts” feature in Subclipse) and fixing any “skipped” files. I then examine the result using the Synchronise view in Eclipse before committing. I’ve done two merges under Subversion now, and the second one wasn’t _too_ painful. We’ll see how this progresses over time.

We also had a nasty problem with a Java keystore file that was in our repository but NOT marked binary; this caused a couple of our branches to be unusuable until we fixed it.

h2. Overall results

Personally, I’m happy with the result of the move. We had the expected effect on the speeds of tagging, branch, logging, etc – Subversion is really good in this area. We aren’t yet as skilled wtih Subversion as we were with CVS, but that will come in time.

Subversion is proving to be a good successor to CVS – it certainly has picked up a lot of momentum over the last two years. It could use a few features (I’d really like merges to be recorded in the history, ala ClearCase), but overall, it’s pretty impressive.