I am not writing about how to evaluate a CI Server, if one needs to evaluate them the best resource I can suggest is CI Feature Matrix just list out your requirements before going to this page, and see which one is the best fit for you. Just for the idea of current state of CIS see What Continuous Integration Server are you using in 2008?.
Evaluating CI Server is not the important part: Once we have decided that we will have a CI server in-place, personally I don't think evaluating the CI server is the important part , I have played with quite a few continues integration servers (just to name a few: cruise control, continum, luntbuild, Hudson and Bamboo), while playing with them I realize that every single one of them dose the same basic thing really well. All of them will check out source from a Source Control Management system, will build the code in one or another way and will run/publish results of your tests. ITs what you want to do with the CI server.
The Important Part, Evaluating Continuous Integration: Yeah we need to Evaluate Continuous Integration Itself. Continuous Integration is not just an activity or state but a Philosophy, a culture which is foster by agile methodology, to integrate and test the code more frequently to find out bugs when it’s less expensive to fix them. Are you and your team comfortable with it? Have you spend enough time with the team to let them know the importance and advantages of it? have you trained them how will they carry out there work from now onwards? are you keeping an eye on the state of builds? are people writing Unit Tests at the very first place? and so on.. make your self comfortable with Continuous Integration.
Open Source To rescue: Don't invest on any commercial tool straight away just because you think it will be cool, its not a tool you are looking for its the process. Use open source tools to introduce CI, to prepare your team for CI (you might like the open source alternatives so much that you might stick to them) make CI a part of development cycle and "Evaluate CI for your team and your team for CI". This will help you to refine requirements but more importantly you will know who needs the Improvement they way CI is being conducted or your team. This Evaluating CI can be done in two possible ways: 1. By using complete open source solution: (e.g Hudson with code analysis tools like cobertura, PMD and FindBugs) 2. By using Evolutionary copy of a commercial CI Server: (e.g Bamboo with code analysis tools like clover/ cobertura, PMD and FindBugs)
What are your waiting for Start Evaluating...
Need to know about Continuous Integration go to Martin Fowler Continuous Integration or buy Continuous Integration: Improving Software Quality and Reducing Risk (Addison-Wesley Signature Series)
