Recently, on our FogBugz support forum, someone asked how to deal with interruptions in Evidence Based Scheduling. You can read the full discussion here.
I wanted to post my response here, because it highlights some best practices. The question was basically this: How do I account for my developers getting interrupted? Sometimes they get pulled away on support issues. Sometimes those support issues blow up into full-blown projects. I want to account for this in FogBugz, but I don't want to add more overhead. What do I do?
Our answer:
This is one of those areas where we have to decide whether we make the software as complicated as reality, thereby making the software less usable, or keep the software simple, thereby making it a less-than-perfect reflection of reality. In general, we lean toward the latter. Indulge me in a semi-related example and then I'll detail how this design philosophy applies to EBS and interruptions.
Often, people write us asking that only X type of user be allowed to set X field (e.g., only project admins can set priority). Rather than adding rules to the software, we recommend that this be addressed at the team level. This is because we've generally found that if you try to enforce your will/process on a person or class of people via software rules, they will either bridle at it, ignore you, or work around the system where possible. If you make the system impossible to work around, they will just stop using it and keep things on Post-it notes on their monitor. Everyone loses.
The best-case scenario is where everyone agrees with and follows the rule. In this case, the rule is then unnecessary, because everyone is already on board. All you need is minimal oversight to make sure that errors don't slip through. You can do this with a filter pretty easily. If we did this by disallowing a particular action in the interface, then that simply becomes a hindrance in the 3% of instances where you need to work around the rule. If one project admin is sick and the other one on vacation, you need to bother the site admin to make someone a temporary project admin and... well, you get the point. This makes the software less usable, which makes people use it less, which degrades the value of the software.
With respect to EBS, you mention two different kinds of interruption. In one case, it's a support issue, which might give rise to a couple hours of dev work troubleshooting, replicating, and filing a new bug. In another, the interruption blows up into a fully formed project. I'll address the latter first, because it's handled in EBS.
You can see the trend of your ship date with the Ship Date Over Time graph. If the line trends to the right, rather than upward, you are adding new work to the release faster than you can finish it. If interruptions blow up into legitimate sub-projects, you need to estimate those projects and figure out what gets dropped from the current release. This is a prime EBS use case, and sort of the thing it was designed to do.
As to the unplanned interruptions, it might be better to deal with these out-of-system, at least partially, rather than trying to track them explicitly. This will be greatly helped if you break up and estimate your regular development tasks in smaller chunks. This has several benefits. Not only will this tighten then ship date distributions for all developers, it will tighten the overall ship date confidence distribution.
Then, you just need to figure out the cases where things took much longer than they were estimated and figure out why. If your developer gets pulled away from a 1 hour bug into a 3 hour call they would charge 4 hours to a 1 hour case. This is what we would call a slow velocity. They're not super-easy to tease out of the system but you can review closed cases and see what the elapsed vs. estimated times are.
If a dev gets pulled away on one of these calls, she can simply make a note in the case (e.g., "pulled away on Dunder-Mifflin call") and move on with her work. Then, when the case shows up on your filter of recently closed cases, you have an explanation for why the case ran long. You can then use your managerial skills to figure out what to do about it. The danger here is forcing the devs to account for other cases in which they overran their estimates, but these can be helpful as well (e.g., "took much longer than expected because the Initech API is a mess."). Whether it's a tech call or a messy dependency, these cases highlight risks to your ship date. Reviewing these cases regularly can help you highlight and manage those risks. And, as we've seen recently, letting a computer tell you about risk can itself be risky. It's knowledge work, and demands a human touch.
So, to summarize, the cushioning effect of EBS should allow you to focus on shipping software, rather than counting hours. We would only recommend tracking interruptions if the tracking gives you clear, actionable information about how to avoid those interruptions and ship your software. If your top dev gets pulled away for 15 minutes to help a customer, thereby overrunning her 2 hour estimate by 30 minutes (when you figure in task-switching penalties), that's not really a big deal. She might've even gained something by the interaction. If you as a manager see that your devs are constantly overrunning their estimates, and that they're forever on the phone with Dunder-Mifflin, or forever chasing down silly PHP config issues, you might want to dedicate some brain time to figuring out how to resolve the larger issue, rather than spending your time figuring out just exactly how much time you're wasting.
