As I noted previously, the beta test process has been started for FogBugz 6.  Thanks to those of you who have already signed up! 

You may be wondering why you have not already been contacted, so I thought I'd go into the beta testing process in more detail.

We have been using FogBugz 6 internally for many months now in the well-known process of dogfooding.  If anything, we're too familiar with it because we have adapted to the way that it works.  Beta testers let us get fresh eyes on FogBugz 6 again, to remind us how it looks to a newcomer.  Beta testers give us feedback on mouse sequences we've long since internalized, and graphic design issues that we don't even see any more.

Beta testing also helps us with technical issues.  FogBugz 6 has been well tested on our installation of Windows 2003 Server with a SQL Server 2005 backend, but our customers have a variety of configurations that FogBugz has to work on.  We have several virtual machines (VMs) set up with alternate configurations for testing, but since we don't spend all day in FogBugz on those configurations, they aren't as well tested as our main FogBugz installation.  Beta testers give us the opportunity to have people use FogBugz 6 and let us know what doesn't work on their particular system.

So why don't we release the beta to everybody who signed up at once?  As Eric Raymond quipped, "Given enough eyeballs, all bugs are shallow", so the more eyes we get looking at this, the better, right? 

The problem with that approach is that most of the beta testers at any given time will find the same bugs over and over again.  Jakob Nielsen, guru of usability testing, has shown that the return on adding more users diminishes quickly after 5 users because the added users are just reporting the same problems.  If we started the beta test with everybody, we would get dozens of reports of obvious bugs, and then beta testers would stop looking further because they've already found a bug in our product. 

So we stage the beta. 

1. We release it to a few people, get their feedback, and fix all the bugs that we can. 
2. We release it to a few more, who let us know if we didn't fix those original bugs, and uncover new bugs that the first beta testers missed because they were blocked by the original bugs. 
3. We fix those new bugs and any original bugs that need re-fixing and repeat step 2.
4. And so it goes until we're confident enough to build a release candidate of FogBugz 6 and let more people try it. 

So if your name hasn't come up in the beta lottery yet, don't fret.  We've got plenty more testing to do, and we'll be needing more beta testers right up until we're ready to release.