I’m a big fan of automating anything that can be automated. Those of you that may know me, might even call me a bit of a fanatic. That’s the reason I got married, automatic dishwasher on the cheap. There’s a reason for my fanaticism, I believe that the more of me you remove from the loop, the less problems will occur. Well, not me in particular, I don’t think I’m that crap. Well, ok, I’d prefer it if you didn’t ask anyone’s opinion on that 😉 Especially the wife.
The largest source of problems in a project is us bumbling humans, and we’re not just talking about the bugs that we introduce!. Damn those pesky humans.
Automate Everything
Ok, so how do we help obsolete ourselves? Build automation is a really good start. As Steve McConnell once said “The build is the heartbeat of the project. If the hearts not beating the project is dead.” If that metaphor’s good enough for Steve, its good enough for me. Automated building is one of the best tools to keep that heart beating.
So, one of the keys to having a project run as smoothly as possible is automating the builds. There are many reasons for wanting to automate the build :
- Repeatability
You get the same output artefacts every time. No forgotten steps, miss-typed commands, forgotten copies. Trusting the process to give you the same outputs means you no longer have to worry about the end result working. - Create different builds automatically
even though its a good idea to run different configurations locally, you can perform pre-configured builds for a specific purpose without messing with your local setup. Debug,Release,Nightly etc, different build configurations can be run on demand, or on schedule. - Reduced Interruptions.
It’s subtle, but often overlooked, you get to reduce the number of people that interrupt you to get the latest version. Reducing the number of interruptions to your flow keeps up your development momentum, and its not very often you get to do that! - Tests can be run automatically.
One of the great things about modern development is that there are plenty of Unit test frameworks available. Unit tests , no matter how time consuming can be executed against the build. Really time consuming operations can be run overnight, on a scheduled build. - Build Farms.
Real men have build farms. More Pc’s projects a manly image. Ok, the wife might dispute that, she just seems to call it clutter. But having a good automated build system means that you can distribute the build amongst multiple PC’s, reducing time consuming builds and utilising those old 386’s you’ve got holding the doors open. Well, ok, it does actually have to qualify as a pc. - Time.
Another important factor. All the time you used to spend running the various builds & unit tests you can now spend on developing the product. Multiply that by the number of members in your team.
So, it all sounds good in practice, but how do we achieve these benefits in reality?
An Automated Build System is for the project, not just for Christmas
Firstly this is really an up-front decision. The more complex the project gets the more time consuming it will be to switch systems. So I recommend making the automated build tool one of the first choices in the project.
Choosing which system you’ll implement depends largely on your budget.
Microsoft Team Foundation Server (TFS)
TFS provides all the automation you’d need and more, but is pretty costly, and requires a reasonable amount of infrastructure and admin to get up and running smoothly. Its not something you’d consider implementing in small teams and projects, unless it’s maintained outside your project. You can read more about TFS on MSDN
Two others worth considering are CIFactory and Team City.
CIFactory
It’s quite amazing , but CIFactory is the work of one man – Jay Flowers. Ok, fair enough he did have a little help where the CIFactory community had submitted various add on packages, but the all of the CIFactory framework was done by Jay. When you look at what it does, that’s a phenomenal piece of work by one guy, who wasn’t doing it as his day job either!
I implemented CIFactory on the last project, which started about a year ago, and was really happy with how it went. The install was straight forward, and the packages it provided for things like Unit testing , and code coverage were all up to date and matched what I was already using day to day, so at the time it was a perfect fit. The range of pre-configured packages is pretty good too, including nCover, mbUnit Analytics, and even Simian!
The really nice thing about CIFactory is that once it’s installed the whole system builds itself as well as your projects. All of the CIFactory tools and scripts are held in the same source hierarchy as your code, and the beauty of this is it’s completely self contained. To recreate your build system, all you have to do is re-extract the entire source tree to a HDD and point IIS back at the folder where the report pages are. It really is a nice clean and ultra flexible system. That may not seem like a major benefit, but for small teams spending any more time than necessary on tooling is obviously a big negative.
Now for the bad news.
When starting a project for the new client this time round I naturally went back to the slick tool I used over a year ago, and found little had changed. This is absolutely NOT a dig at Jay! Repeat, NOT disrespecting ANY of Jay’s work. His body of work is amazing, and I still had CIFactory installed and running with few modifications (as in the image), but it did take me more than a day to sort out. And to be perfectly honest staring at pages of nAnt scripts and XML files isn’t particularly pleasurable. Not only that, but many of the automatically installed packages now have broken links, and the pre-installed packages are now out of date. After trawling the CIFactory pages, Jays blog, and the forums, it was kind of obvious that CIFactory has probably reached the end of its natural lifespan, activity seemed to be very low on all fronts. I expect that there will still be the hardcore users that like it for its pure configurability (and it’s probably still the best for that) but it just seems like too much effort when there are easier alternatives.
Team City
Yes, I’ve discovered Team City. If you’re a ReSharper users (if not why not? you have to be start raving BONKERS) then you’ll understand why I’m in love with TC. It has all of JetBrains slickness packaged up in an easy to use, and easy to setup automated build system. JetBrains really are the Dogs Danglies, and I’d be more than happy to bear children to them.
Ok, enough with the sycophantic JetBrains stalking, trust me, Team City is about the best thing you’ll come across for automated building that doesn’t require you to sacrifice your first born to get installed and working. Not only that, but there are Plug-ins for the most popular IDE’s and task tray notifications that you can download from the interface. To be fair, CIFactory (Built on Cruise Control) also had task tray notifications with cctray.
What’s even more amazing is that the standard version is FREE! yes. Read that again FREE!
Ok, there’s FREE and there’s FREE, so this version of FREE has licensing restrictions, the most limiting of which is you can only install 3 build agents, but the other restrictions aren’t really that restrictive. So, given that there are only 2 developers that need to build our new project, I installed the latest version of TeamCity on our build server. I then set up our projects with the minimum of fuss. A minor problem occurred because the Solution builder that’s included can’t build Deployment projects. This isn’t a problem with Team City per se, only devstudio can build web deployment projects. This was easily fixed by using Team City’s command line build runner and building the offending setup project directly with devenv.exe! Even though I hadn’t really used TC before I had it building the projects (command line builder included) in about 3 hours. That more like it!
A feature of both CIFactory and TC is that they release the build artefacts from at the web page – so anyone with a browser can download the build output.
CIFactory does this by zipping up the project using a nAnt file set that looks at the project’s output folder. This is a bit of a problem if you want to include other files in the output. No big deal, but you have to go in and get your hands dirty with the XML properties files for the deployment stage.
With TC, the artefacts are held inside your project, exactly where you left them!. Generally these are generated in your projects bin/release folder by default. TC Allows you to expose any path or file relative to your projects root as an artefact, so you can include anything you like by simply providing TC with a nAnt style wildcard spec.
Once the project is built the resulting artefacts are available for download.
Here we have the build output from the deployment project.
Now anyone with a browser can retrieve the latest version of your project without interrupting you.
One thing that TC does do that CIFactory doesn’t is distribute the build. I used all 3 licences for the build agents for our mini build farm. The first machine is the dedicated build machine where the TC server is running. This machine also functions as a build agent.
Notice that the Build machine CPU rating. It’s core2 duo E6600. Note my machine’s CPU rating. We just WONT go into that. Keep in mind the build machine is running inside a VM, with Oracle11, SQLServer 2008, and IIS running in it. <sigh>.
Once the build agents are set up, TC distributes a running build according to a set of rules you specify. Our TC server is set to distribute to the fastest build agent first. Another good choice would be the first Idle agent too. Since there are only 3 of us and the build doesn’t take that long at the moment, I don’t think the distribution strategy matters a great deal.
Here TC is distributing the build across the agents.
One nice feature of all of TC’s Ajax enabled interface is that you still manipulate controls whilst the server/agents are building. For example, as soon as the artefacts start becoming available you can begin download.
Another feature of TC is that you can see the build statistics on a per agent basis,
I guess this would be more useful in a large farm, but i guess its still useful to know which agents are successfully building the project. If they’re not it might point to environmental problems on that particular machine.
When TC is performing a GET to extract the sources, well, with subversion anyway, it actually does an export of the source tree, so you get an unpolluted source hierarchy from which you can extract your artefacts. You can also control who does the checking out, the agents or the server – checking out on the server reduces the load on the source control system, since only one machine is doing the full checkout – but then the sources are distributed to each agent as TC calls upon each agent to build.
All in all TC is pretty amazing, and the fact that you can install it for small dev teams for free makes it my current best friend !
Coming next, a step by step guide on how to set up a complete system from start to finish. Using Subversion as the source repository, and then installing TC on top of that, all with the canonical hello world project!