The FogBugz Plugin Architecture

Monday, August 10, 2009 by David Fullerton

Feature Bloat

Feature bloat is a familiar problem for any software project, especially one approaching a decade of continuous development.  For FogBugz, the sirens' call of bloat usually sounds something like this:

"Hi Fog Creek!  I love the new tagging feature in FogBugz 7!  My only problem is that I don't seem to be able to create a reverse-alphabetical, zebra-striped tag cloud.  In my organization we only use reverse-alphabetical, zebra-striped tag clouds and without them our entire production line has ground to a halt!  How soon will you be adding this feature?"

Now, 90% of our users have never even heard of reverse-alphabetical, zebra-striped tag clouds, and adding support for it to FogBugz will just add one more confusing feature and make FogBugz as bloated as the Michelin Man after Thanksgiving dinner.  But this is something that at least one customer really needs, and we don't want to just ignore them forever.

In the past we've always dealt with the threat of bloat by implementing the 90% solution really, really well.  The 90% solution means building the features that most people are asking for while we punt on the other 10%.  This lets us spend our energy on making sure the features for the 90% are both powerful and easy to use, and make our customers ridiculously happy.

Unfortunately, adding 90% solutions together doesn't mean that 90% of our users are 100% happy. That last 10% is never the same 10% for everyone, so we end up with an insane Venn diagram of competing desires, some of which may even be mutually exclusive!  With every release we address the biggest of these, but we also accumulate a few more customers who just wish we would add that one feature that would make FogBugz perfect for their organization.

One of our goals for FogBugz 7 was to provide a way of addressing all of these requests for features, without compromising the core simplicity and elegance of FogBugz out-of-the-box.

Sometimes we get carried away trying to make everyone happy

Enter Plugins

Plugins provide an elegant solution to this problem: create snap-on features that integrate seamlessly with FogBugz, but can be added and removed with the click of a button.  With a simple plugin, we can satisfy the 10% of customers dying for that reverse-alphabetical, zebra-striped tag cloud without the other 90% ever having to know that it even exists.

Even more exciting, plugins finally allow other people to modify FogBugz without having to dive into our scary Wasabi source code.  Anyone can write their own custom plugins, and then share them with the world in the FogBugz Plugin Gallery.

Installing plugins is dead simple: if you have FogBugz on your own server just download the zip file from the Plugin Gallery and click the upload link in FogBugz.  For FogBugz On Demand, just follow a link to the Plugin Gallery from your FogBugz account and it will be automatically installed when you click the install button.

Install plugins from the FogBugz Plugin Gallery with just a few clicks

The Plugin Architecture

So how did we actually do it?  First we had to port FogBugz from classic ASP to ASP.NET, but that's the subject of another post.  Once we did that, we were able to use some cool facilities built into .NET.

In FogBugz, plugins are simple .NET assemblies (DLLs) loaded into FogBugz.  As such, they can be written in any .NET language (C#, VB.NET, F#, etc.), and can leverage the powerful .NET Framework libraries.

These plugin assemblies communicate with FogBugz in two ways: by calling methods on the API, and by implementing special interfaces.

The interfaces that a plugin implements determine when and how FogBugz calls into the plugin.  Let's say a plugin wants to add a color-coded label to a case, and it should be displayed as a column in the case list.  All the plugin developer has to do is implement IPluginGridColumn, which specifies how to display the column and sort it, and FogBugz does the rest.  When FogBugz needs to display the column, it looks up the plugin and calls the appropriate methods (see an example).

Inside those methods the plugin will probably want to query FogBugz for some information.  It does this using special API methods that let a plugin do things like query the database, edit a case, or add a notification to the top of the page.  Since the plugin is running as an assembly on the server, there are no crazy hoops to jump through: for example, loading a bug is as simple as calling api.Bug.GetBug(caseId)!

Since these interfaces and API methods are the heart and soul of the plugin architecture, every single method and property has full Visual Studio IntelliSense documentation, and every interface has an example implementation in the Plugin Developers Wiki.

Full Visual Studio Intellisense

The FogBugz API includes complete Visual Studio Intellisense

What Plugins Can Do

So what exactly can Plugins do?  A lot, and we're actively expanding it.  Here are just a few quick examples:

  • Create new FogBugz pages
  • Add new column types to the case list grid
  • Create custom fields on cases
  • Extend search with custom search axes
  • Add customizable filter criteria
  • Load, modify, and save FogBugz entities like Cases, Users, Projects, Milestones, Wikis, and more
  • Create custom database tables
  • Query FogBugz database tables
  • Create FogBugz "Editable Tables" with AJAX popups
  • Write raw binary data and set content headers to allow custom file downloads
  • Run periodic tasks during FogBugz background maintenance

Much more information and loads of examples can be found in the Plugin Developers Wiki; many of the examples above have a complete annotated example to help developers get started!

The Kanban Board plugin adds a new column to the grid

Plugin Security and AppDomains

Being able to run directly on the server is great for plugin developers, but it could be a nightmare for system administrators, especially for plugins coming from unknown third parties. In fact, our sys admins nearly had a fit when we first started talking about how plugins would work.  The problem, however, is not nearly as bad as it seems at first blush thanks to the power of .NET AppDomains.

AppDomains in .NET are a way to run assemblies in specially managed sandboxes with fine-grained control of what they can and can't do.  For example, the .NET FileIOPermission specifies what directories an assembly can read or write, and the WebPermission controls what URLs an assembly can hit over the web.  .NET itself defines dozens of permissions, and has facilities to allow developers to define their own.

So what permissions do we deny plugins in FogBugz?  Almost everything!  They are forbidden from accessing the file system or the registry, or connecting to databases, or running unmanaged code, or...well it's easier to say that we basically start with a completely blank slate, and only give plugins the handful of permissions they need to run.

"Wait!", you say.  "Plugins can't access the database?  How do they do anything interesting?".  Plugins aren't allowed to talk to the database, but FogBugz sure can.  So if a plugin wants to do something that it doesn't normally have permission to do, the request has to go through FogBugz via an API.  FogBugz, in turn, is very restrictive about what it allows plugins to do: for example, every SQL query is run through a validator to ensure that it doesn't do anything dangerous.

"But what if I want my plugin to be able to talk to another database", you cry again.  Well, in the 7.0 release all plugins run in a single, very restricted AppDomain, but in future releases we plan to add support for FogBugz administrators to loosen the restrictions for certain plugins known to be safe.  This would allow, for example, a plugin that integrates FogBugz with a sales database, or lets the plugin access certain network resources.

We start with a blank slate and only allow necessary permissions

Enforcing FogBugz User Permissions

Of course, when loading information from the database, Plugins need to be aware of another level of security: FogBugz user permissions.  Not every user in FogBugz has access to every case, or is allowed to modify a project.  To make things simpler for plugin developers and to reduce the risk of data exposure, whenever possible the FogBugz plugin architecture adopts a "fail closed" policy.

A "fail closed" policy means that by default we enforce the permissions of the current user when the plugin runs.  For example, if a user visits a plugin page that displays information about a case, a permission check is automatically run when the plugin tries to access the case data.  If the current user does not have access to view the case, the plugin will receive an exception.

Automatically enforcing permissions means that much of the time plugin developers can write without having to worry about permissions.  In the worst case, the plugin will return an error rather than revealing potentially sensitive data.  In the special situations where a plugin needs to do something unusual with permissions, it can turn off the automatic permission handling on an object and manage the permissions itself.

FogBugz automatically enforces the user's permissions

The Future of the Plugin Architecture

We're all extremely excited about the first version of the plugin architecture in FogBugz 7.  We think it provides a powerful foundation for a wide variety of plugins.  Already we've implemented several long-requested features for FogBugz as plugins: Workflow, Custom Fields, Project Backlog, and CSV Export (built-in to FogBugz 7) are just the start!

We're also excited about the plugins under development that will start to push the architecture to its limits.  The Balsamiq Mockups plugin allows creating beautifully simple user interface mockups from inside FogBugz which can be attached to cases and wiki pages.  The Kanban Board plugin allows creating an Agile Kanban board with FogBugz cases, complete with AJAX drag-and-drop.  You can browse these plugins and many more at the FogBugz Plugin Gallery.

FogBugz is just getting started with plugins.  Stay tuned in the coming weeks and months for more and more feature-loaded plugins, along with expansions and additions to the plugin architecture that will allow developers to do more and more.  If you have an idea for a plugin that you'd like to see, you can even suggest it to the development community at the FogBugz Plugin Ideas uservoice site, and maybe somebody will take it and run with it!

Or, even better, stop waiting for  somebody else to add reverse-alphabetical, zebra-striped tag clouds and get started writing it yourself!  The FogBugz Developer Wiki has all the information you need to get started, and the Plugin Developer Discussion Group is a great place to get all of your questions answered.

A simple "Hello, World" plugin is only a few lines long


Categories: FogBugz

Tags:

Actions: E-mail, Permalink


EBS 2.0

Thursday, July 23, 2009 by Brett Kiefer

Death Marches

Figuring out when a software project will ship is a famously difficult problem, and nearly everyone who writes software for a living has to grapple with it. I remember asking an experienced developer at my first job how I should calculate my project ship date. He said (and please imagine him cheerily chirping in a Scottish accent): "Just go with your gut. Then double it, add twenty percent, and pray."

For a long time that was the best advice I got, and it appeared that no one else had a better idea. So I got to see lots of smart, dedicated, otherwise rational software developers work brutal hours and take unwise shortcuts on miserable death march projects because, dammit, they had committed to a date and they were going to DELIVER.

Iterating on EBS

In our last major release of FogBugz, we launched a new approach to the problem called Evidence-Based Scheduling (EBS). EBS takes your team's project milestones and tasks and uses a Monte Carlo method to generate a probability distribution of when you'll ship your product.

Since it's part of your project management system, you can get all of this without a lot of sweat: just estimate your tasks and tell FogBugz more or less when you're working on something; it does the rest. The best thing is that (like a genetically modified shark!) EBS gets smarter, because it constantly feeds your historical data back into the prediction algorithm. If this is the first you've heard of it, check out Joel's introduction here.

We've loved hearing from the customers who have found EBS useful. Thanks, y'all! At the same time, many users have come to us with great ideas for features we should add. And it's their thoughtful suggestions that made us confident that time spent adding to EBS would be well-spent. Today I'm happy to announce EBS 2.0, launching as part of FogBugz 7. EBS 2.0 introduces two new features that make it possible to model real-world projects: inter-project timelines and strict dependency modeling.

Inter-Project Timelines

In EBS 1.0, each project's timeline was considered individually, so it was difficult to model a dependency on a shared resource; for instance, if a system administrator had cases in multiple projects, FogBugz assumed that he would begin work on all of them full-time in parallel.

Instead, EBS 2.0 assumes that you will work on all of your milestones in order, but it allows you to take control of parallelization by assigning percent times to projects. So if you always spend 10% of your time recruiting, 20% of your time working on your Amazing FogBugz Plugin, 10% staring out the window, and the rest of your time working on your development tasks in all other projects, that's now easy to model.
 
This allows you to divide your FogBugz install into granular projects (per-client, for instance), and still get meaningful answers from EBS about when users will start and finish work. While EBS 1.0 could model the work of the traditional, project-monogamous software developer pretty well, EBS 2.0 allows, say, a visual designer or consultant - anyone who divides time between multiple projects - to express her schedule accurately.

Strict Dependency Modeling

EBS 1.0 didn't have strict dependencies. Each user was assumed to finish his tasks in the first milestone, then start working on the second immediately. And that's the way it was, and we liked it. Usually. FogBugz is optimized for tracking software projects, and in most kinds of software development, a strict dependency is the exception, not the rule. Even if I'm waiting for someone else to finish a component, I can look at its spec and write my software accordingly.

But! Sometimes you really can't start work on a milestone until something else happens first. For example, our sysadmins are stellar, but even they have a hell of a time racking hardware before it arrives, and we in turn have a hard time deploying and testing our software on servers with no operating system installed. In EBS 1.0, you just couldn't express this kind of strict dependency.

EBS 2.0 gives you the vocabulary of strict dependencies and start dates. You can now say "No one can travel back in time until the Flux Capacitor is complete and the duped terror cell steals the plutonium."
 


". . . AND we can't build the time machine until the DeLorean arrives on October 20th."

 
Though the interface is simple, these dependencies can be used to build up a much more realistic model of your work. Given your dependencies, EBS will now build a directed graph of all of your milestones. It combinines the simple EBS 1.0 timeline with any of the strict dependencies and start dates you specify in order to calculate probable ship dates and risk for all of your projects.

Bonus: New Graphs!

As the model for project completion became more complex, we realized we'd need a more expressive graph. So we've added a very data-rich graph called the User Timeline. It shows all of the users on whom a milestone depends, along with a graphical representation of all of the work they have to do to complete the milestone. You can see their work on other projects, idle time caused by unsatisfied dependencies, and estimated completion probabilities. The graph is interactive, allowing you to drill down to get more information about any user's remaining work, so that you can rebalance tasks and see what other projects are influencing your ship date. 


Here we can see that Marty will be unable to travel back in time until Doc has finished the Flux Capacitor (the light blue) and gets the plutonium from the duped terror cell (orange). Until all of that is done, Marty is idle (the red area), because the only thing he's supposed to do is blocked.

So if we need to get Marty time-travelling earlier than that, the graph tells us that we need to either break the dependency on the Flux Capacitor (not likely!) or speed up its development somehow. For instance, we can see that Marty is free right now, so if we can find work for him to do on the Flux Capacitor, we would speed up the whole project. If it sounds like EBS is helping us to find the probable critical path for the project, that's because it is!

The Future

There is still a lot more that can be done with Evidence-Based Scheduling, and we'll continue to invest in new features. Of course, with the release of the FogBugz Plugin Architecture in FogBugz 7, which provides hooks for interacting with EBS, we hope to see a lot of innovation coming from outside of Fog Creek. That'd be neat.

We'd be delighted to hear what you have to say about EBS 2.0, so create a free FogBugz account and try it out!

The FogBugz Developer Series

Next up, a look at the exciting FogBugz Plugin Architecture with FogBugz Plugins development lead David Fullerton.


Actions: E-mail, Permalink


Announcing FogBugz 7.0

Monday, July 20, 2009 by Dan Wilson

(Click here to skip straight to the "What's New" page.)

FogBugz 7.0 has launched, and the Fog Creek office is buzzing with activity.  We're answering customers' questions, helping them through the upgrade process, and making sure that nothing falls through the cracks (by using FogBugz, of course).

Focused Flexibility

We're proud to have created a new version of FogBugz that adds an enormous amount of flexibility, but stays true to our core design philosophy.  You can now add structure to your task list with subcases, and use tags to create your own taxonomy.  You can use Burn Down Charts to track your agile development sprint, or use new EBS 2.0 features to track a longer-term effort involving dependencies and part-time participants.

But the ease and simplicity of the FogBugz case workflow hasn't changed. Resolved cases are still automatically assigned back to case opener for verification.* The number of case fields is kept at a bare minimum,* and of course the number of required fields is still zero.

*FogBugz Plugins

Sorry for all the asterisks, but I've left out a major part of the FogBugz 7 story.  We've adhered to our design philosophy in core FogBugz, but that doesn't mean you can't bend the rules by installing a Plugin or two.  You can use the Custom Workflow Plugin to tweak the rules for case assignment (as well as create custom categories and statuses), or install the Custom Fields Plugin to add those extra few case fields your organization can't live without.  These are just two of the many Plugins available at launch in the FogBugz Plugin Gallery.

We've also opened up the FogBugz Plugin Platform, allowing any developer to create Plugins that can deeply integrate with nearly every piece of FogBugz functionality.  We're launching with a first batch of plugins created by the FogBugz development community including Balsamiq Mockups for FogBugz, Kanban Board, FB_Scratchout, and ClarkKent.  We look forward to seeing many more, and will follow up with blog posts about our favorites.

An Emphasis on Usability

FogBugz 7.0 isn't just about big new features and the introduction of Plugins.  It's also about the dozens of usability tweaks that will make life easier for our users.  Case number fields are now aided by auto-complete, so forget about having to remember or copy six-digit numbers.  A new context menu control in the case list allows you to perform actions on cases more directly.  An "Add Fields" control in the case view clears unused fields out of your way. These enhancements are really just the tip of the iceberg; if you're evaluating FogBugz 7.0 we recommend a nice, long test drive.

The Passion of the Kiwi

A lot of hard work went into fulfilling the team's product vision and making sure FogBugz 7.0 meets a wide range of customer needs.  For example, the Plugin architecture was born out of months of hard work by the FogBugz development team, who first ported our entire code base to the .NET platform and then created a massive API that touched nearly every part of the application.

The herculean task of creating the Plugin architecture was nearly complete when I arrived at Fog Creek last September.  Right off the bat, I was tasked with writing feature specifications, which I spent months churning out, each one a little bit less clueless than the last.  It's been quite an experience to work with the FogBugz team, to see (the good parts of) my mockups turn into living, breathing features that exceeded my expectations.

The FogBugz Developer Series

Stay tuned for a series of blog posts from the developers who crafted FogBugz 7.0.  You'll first hear from Brett Kiefer, designer and developer of Evidence-Based Scheduling (EBS) v2.0.


Categories: FogBugz

Actions: E-mail, Permalink


FogBugzMiddleware for Django

Wednesday, June 03, 2009 by Benjamin Pollack

Django is an increasingly popular Python web framework, being used by such companies as Google and the Washington Post. Django has a concept of “middleware,” which allows you to stash extra processors between your application and the low-level framework. Today, I took some time to write FogBugzMiddleware, a piece of Django middleware that allows you to automatically report 500 and internal 404 errors to your FogBugz install. Installing it takes just a few seconds, and will immediately allow you to start using FogBugz to manage your website’s crash reports.

You can download the software at http://bitbucket.org/bpollack/fogbugz-middleware/. The software is free for use and redistribution under a BSD license.


Categories: FogBugz

Actions: E-mail, Permalink


Removing Problematic Info

Tuesday, April 28, 2009 by Rich Armstrong

I wanted to provide a solid workaround for case deletion, a common customer request.   If you have access to your own FogBugz database, you can edit and delete cases directly. This applies primarily to people hosting their own install of FogBugz.

If you cannot access the database or if you're using FogBugz On Demand, the simplest way of approximating deletion is to sequester the case. Create a new project named "Sensitive Info", and add it to a new department called "Confidential". Set the department permissions such that only site admins have access, then move the case into the new project and close it.  No one other than site admins will be able to see the case or any reference to it (including case relations).  It will not show up in any searches except by site admins.

If you have access to the database and sequestration doesn't work for you, then the following is an example of one way to proceed.  It is offered without guarantee as to its safety or efficacy, but simply as an example of how one might go about moving data in this way. This example will use MySQL, but the process should be applicable to Access or SQL Server.

First, back up your database.

Let me say that again: back up your database.

Is your database backed up? Okay, then let's proceed.

Next, you'll need to find the ID of the offending BugEvent. Bug events are the entries on the case page.  They look like "Assigned to Joe Biden by Barack Obama" and a timestamp.  They can also take the form of an email, with text like "Replied by Rich" and a timestamp at the top of the email.  If you click on that timestamp, you'll see a #BugEvent value added to the URL.  This is the bug event's ID.

Log into your FogBugz instance of MySQL on the command line.  Run this command:

SHOW DATABASES;

You will get output like this:

+--------------------+
| Database           |
+--------------------+
| information_schema |
| FogBugz            |
| mysql              |
| test               |
+--------------------+
4 rows in set (0.00 sec)

In this case, your FogBugz database is named FogBugz.

Run this SQL:

SELECT * FROM FogBugz.BugEvent 
  WHERE ixBugEvent = 8675309;

The text in the "s" field should match up to the text you want to delete.  Note that if this is an encoded email, the text might be a block of gibberish.

Now, we want to create a table to hold the data we're going to remove.  This is not strictly necessary if you just want to blow away data, but you might come to regret deleting data outright, and this gives you some insurance.

First, create a new database:

CREATE DATABASE DumpingGround;

Now, we want to create an exact replica of the BugEvent table in the new database. To do this, we use a handy feature of MySQL called "SHOW CREATE TABLE". (The equivalent in SQL Server is "Script Table As".)

SHOW CREATE TABLE FogBugz.BugEvent;

You'll get something like this:

| BugEvent | CREATE TABLE `BugEvent` (
  `ixBugEvent` int(11) NOT NULL auto_increment,
  `ixBug` int(11) default '0',
  `sVerb` varchar(128) default NULL,
  `dt` datetime default NULL,
  `ixPerson` int(11) default '0',
  `s` longtext,
  `fEmail` smallint(6) default '0',
  `fExternal` smallint(6) default '0',
  `sChanges` longtext,
  PRIMARY KEY  (`ixBugEvent`),
  KEY `bugindex` (`ixBug`),
  KEY `dtindex` (`dt`),
  KEY `BugEventDtAsc` (`dt`,`ixBugEvent`),
  KEY `BugEventDtDesc` (`dt`,`ixBugEvent`)
) ENGINE=MyISAM AUTO_INCREMENT=18675309 DEFAULT CHARSET=latin1 |

We want to create a copy of that table in a new database to store information.  Take everything between the second set of pipes (||) and copy it off, adding a database name to the table name, so it goes from `BugEvent` to DumpingGround.BugEvent. 

CREATE TABLE DumpingGround.BugEvent (
 ... 
) ENGINE=MyISAM DEFAULT CHARSET=latin1;


Okay, now we have a place to put the data we're going to remove.  Is your database backed up?  Good.  Here's the command to run.

INSERT INTO DumpingGround.bugevent 
  SELECT * FROM FogBugz.bugevent
    WHERE ixBugEvent = 8675309;

You should get this output:
Query OK, 1 row affected (0.00 sec)
Records: 1  Duplicates: 0  Warnings: 0

Right now, these two queries should give you the same results:
SELECT * FROM FogBugz.BugEvent 
  WHERE ixBugEvent = 8675309;
SELECT * FROM DumpingGround.BugEvent 
  WHERE ixBugEvent = 8675309;

Now, we just need to remove the data from the bug event in the FogBugz database.  We don't want to delete the row entirely. It might cause problems with data integrity. So here's what we run:

UPDATE FogBugz.BugEvent 
  SET sVerb = 'Removed', ixPerson = -1, s = '' 
  WHERE ixBugEvent = 8675309;

This changes how the event shows in the interface.  It will now say "Removed by FogBugz". The timestamp and any other information, such as assignments or changes in priority, will be preserved, but the data is gone.  You could also experiment with placing text into the "s" column, such as "Please contact an administrator to retrieve this text: Event# 8675309".  That's up to you.

You'll need to secure the DumpingGround database to your satisfaction so that the data remains preserved but secure.





Actions: E-mail, Permalink


Interruptions and EBS

Wednesday, March 25, 2009 by Rich Armstrong

Recently, on our FogBugz support forum, someone asked how to deal with interruptions in Evidence Based Scheduling.  You can read the full discussion here.

I wanted to post my response here, because it highlights some best practices.  The question was basically this: How do I account for my developers getting interrupted?  Sometimes they get pulled away on support issues.  Sometimes those support issues blow up into full-blown projects.  I want to account for this in FogBugz, but I don't want to add more overhead.  What do I do?

Our answer: 

This is one of those areas where we have to decide whether we make the software as complicated as reality, thereby making the software less usable, or keep the software simple, thereby making it a less-than-perfect reflection of reality.  In general, we lean toward the latter. Indulge me in a semi-related example and then I'll detail how this design philosophy applies to EBS and interruptions.

Often, people write us asking that only X type of user be allowed to set X field (e.g., only project admins can set priority). Rather than adding rules to the software, we recommend that this be addressed at the team level. This is because we've generally found that if you try to enforce your will/process on a person or class of people via software rules, they will either bridle at it, ignore you, or work around the system where possible. If you make the system impossible to work around, they will just stop using it and keep things on Post-it notes on their monitor. Everyone loses. 

The best-case scenario is where everyone agrees with and follows the rule.  In this case, the rule is then unnecessary, because everyone is already on board. All you need is minimal oversight to make sure that errors don't slip through. You can do this with a filter pretty easily. If we did this by disallowing a particular action in the interface, then that simply becomes a hindrance in the 3% of instances where you need to work around the rule. If one project admin is sick and the other one on vacation, you need to bother the site admin to make someone a temporary project admin and... well, you get the point. This makes the software less usable, which makes people use it less, which degrades the value of the software.

With respect to EBS, you mention two different kinds of interruption. In one case, it's a support issue, which might give rise to a couple hours of dev work troubleshooting, replicating, and filing a new bug. In another, the interruption blows up into a fully formed project.  I'll address the latter first, because it's handled in EBS.

You can see the trend of your ship date with the Ship Date Over Time graph.  If the line trends to the right, rather than upward, you are adding new work to the release faster than you can finish it. If interruptions blow up into legitimate sub-projects, you need to estimate those projects and figure out what gets dropped from the current release. This is a prime EBS use case, and sort of the thing it was designed to do.

As to the unplanned interruptions, it might be better to deal with these out-of-system, at least partially, rather than trying to track them explicitly. This will be greatly helped if you break up and estimate your regular development tasks in smaller chunks.  This has several benefits.  Not only will this tighten then ship date distributions for all developers, it will tighten the overall ship date confidence distribution.

Then, you just need to figure out the cases where things took much longer than they were estimated and figure out why.  If your developer gets pulled away from a 1 hour bug into a 3 hour call they would charge 4 hours to a 1 hour case.  This is what we would call a slow velocity. They're not super-easy to tease out of the system but you can review closed cases and see what the elapsed vs. estimated times are.

If a dev gets pulled away on one of these calls, she can simply make a note in the case (e.g., "pulled away on Dunder-Mifflin call") and move on with her work.  Then, when the case shows up on your filter of recently closed cases, you have an explanation for why the case ran long. You can then use your managerial skills to figure out what to do about it. The danger here is forcing the devs to account for other cases in which they overran their estimates, but these can be helpful as well (e.g., "took much longer than expected because the Initech API is a mess."). Whether it's a tech call or a messy dependency, these cases highlight risks to your ship date. Reviewing these cases regularly can help you highlight and manage those risks.  And, as we've seen recently, letting a computer tell you about risk can itself be risky. It's knowledge work, and demands a human touch.


So, to summarize, the cushioning effect of EBS should allow you to focus on shipping software, rather than counting hours. We would only recommend tracking interruptions if the tracking gives you clear, actionable information about how to avoid those interruptions and ship your software. If your top dev gets pulled away for 15 minutes to help a customer, thereby overrunning her 2 hour estimate by 30 minutes (when you figure in task-switching penalties), that's not really a big deal. She might've even gained something by the interaction. If you as a manager see that your devs are constantly overrunning their estimates, and that they're forever on the phone with Dunder-Mifflin, or forever chasing down silly PHP config issues, you might want to dedicate some brain time to figuring out how to resolve the larger issue, rather than spending your time figuring out just exactly how much time you're wasting.


Categories: FogBugz

Tags:

Actions: E-mail, Permalink


Running FogBugz on IIS7? Read this!

Thursday, March 19, 2009 by Michael H. Pryor

Bug: The FogBugz installer by default would turn on server side debugging for FogBugz on a new install.  This means that the FogBugz application would run in single threaded mode and performance is greatly impacted. 

 

 

Fix: We will address this in our next update but if you are running 6.1.44 or anything less, go to the asp settings in IIS7 for FogBugz and expand the Debugging node.  Turn of server side debugging and save your changes.  FogBugz will resume normal speed after this.


Categories: FogBugz

Actions: E-mail, Permalink


An API Tutorial Plus Excel 2007 Export... Plus Zombies!

Wednesday, March 18, 2009 by Rich Armstrong

NOTE: FogBugz 7 has been released, and it includes a CSV export plug-in. This article is still a good starting place for people interested in exploring the API, or in extracting other types of data, like hours worked, into a spreadsheet. 

 

Sometimes, you just need an Excel spreadsheet.  Sometimes, you just need to be able to slice and dice your cases in the way that's right for you.  Sometimes, you need to drop the case report into a Powerpoint slide to show to the higher-ups.

Sometimes, you need to go out hunting radioactive zombies and you need a hard copy... because neither WiFi nor 3G will penetrate the lead lining of your specially adapted Semi-Autonomous Radioactive Zombie Eradication Vehicle (SARZEV).

Well, if this sounds like you, then I have good news.  I'm happy to report that because Excel 2007 supports XML, it works well with the FogBugz API.  There's a few steps here, but in the process you'll learn a little about Excel and a lot about our super-easy API.

The first step is to come up with a view into the data that suits your needs.  If you're going out to hunt the undead, information is your most powerful weapon. Well, maybe a rail gun is your most powerful weapon... but information is a close second.

First, get your data. Here's an example of the data you might want:

 

To get this from FogBugz to Excel, you just need to get the same data from the API, which sounds scary, but come on! Aren't you going off to destroy the undead?  Why are you scared of a little API action? Seriously, it's easy.

First, you'll need an API token.  This is a 30-character code that will allow you to act via the API as if you were signed in to FogBugz. I use Chrome almost exclusively, but when I am working with the API, I like to use FireFox. Its XML rendering is a bit friendlier.  You want to go to:

https://[your fogbugz]/api.asp?cmd=logon&email=[your email]&password=[your password] 

 

This token is valid until you hit Log Off in the FogBugz interface or log off in the API. Now, you're ready to build your query.

Remember that page with the original search on it?  If you go and look at the URL string, you'll see a searchFor parameter.  This is the encoded search string that you will use to replicate the exact same search in the API.  Here's what mine looked like:

 

Build that into the URL below and you'll start getting your first data back from the API

https://[your fogbugz]/api.asp?token=[your token]&cmd=search&q=[your search parameter]

So, mine is (broken out for clarity):

https://zombiewatch.fogbugz.com/api.asp?
token=k5ea5jecjsv1qjo7gg54mcngebb852
&cmd=search
&q=assignedto:%22lead-lined%22+status:active

This should give you the same list of cases, but with only the case number.  For our example, we need to add a cols parameter to tell FogBugz what case information to return.

&cols=ixBug,sTitle,sVersion,sComputer

Here, we are using the FogBugz built-in custom fields to denote the type of zombie and its decay level.  We call them "Type" and "Decay" in the interface, but the API needs to be consistent, so we use the original names.  The data is unaffected.

So here's what the whole URL looks like. 

Now, open Excel 2007 and open a new spreadsheet, then choose the Data tab and From Other Sources, then From XML Data Import.

 

Paste your URL into the Open dialog that comes up, then hit Open.

 

You might get one or more messages that look like this.  Just hit OK on each of them.

 

 Then you'll get a message like this:

 

Again, it's fine to just click OK here. And here's you're reward.

 

Okay, so it's not the prettiest thing in the world, but click that "Summarize with Pivot Table" button and you can get something pretty good:

 

From here, you can get charts and graphs and all sorts of other neat reports that will help you make your case that the undead will soon rule the earth.

Be careful out there. 

ps. 

I've created an Auditor read-only user on zombiewatch.fogbugz.com, with the password of "password".

If you want, you can get a new token for this user by going here. With that token, you should be able to play around with the API to your heart's desire.  If you want to mess around with your own site, though, you can do so by going to try.fogbugz.com.

OH, and, duh, here's the full documentation on the API.


Actions: E-mail, Permalink


How We Use FogBugz for Recruiting

Tuesday, March 10, 2009 by Dan Ostlund

FogBugz can be used for just about any process oriented activity.  Lots of our customers use it not just to track bugs, but also to do all of their customer support as well.  Problems or questions come in to your Inbox, and your sales or technical support team take care of them.  In conjunction with areas and due dates you get a really effective system with minimal setup effort.  That's one of the many ways we use FogBugz here at Fog Creek.

FogBugz is also used here to handle all of our job applications.  We get hundreds of them, especially during our recruiting for summer interns.  I can't even imagine trying to keep track of all of this in Outlook or something--it just sounds like a total nightmare.

What we do is simple, and it works extremely well.  Applications go to jobs@fogcreek.com, which feeds into the Inbox in a FogBugz install devoted specifically to recruiting.  If you already use FogBugz for project management or customer service, you could just make another mailbox for job applications. 

Our hiring process has been written about before and consists of the following steps: A resume review, a phone interview, a screen sharing interview, and several in-person interviews.  We use these steps to narrow the pool down to the best candidates, hopefully resulting in an offer from us to some scary good programmers.

Every application that comes into the jobs@fogcreek.com email gets an auto-reply, which is at least a little funny, and assures them that a real person will look at the email, asks them to make sure they have provided us with some important information, and also asks them to keep replying using this same email thread.  The last request is because the subject line number tells FogBugz to append any new emails to the existing case, and all of the relevant information is kept in one place.  This works without a hitch.

We take each phase of the interview outlined above and make a fixfor that maps onto it.  We've made, for example, Needs First Review, Needs Phone Interview, Needs Copilot Interview, Needs In Person Interview, and so on.

The case comes into jobs@ and we clean up the title so the case is easy to read (usually, position: person's name, in the case title), and then we move it through the process. The case is then passed off to developers to review.  Candidates who get high ratings are assigned to the Needs Phone Interview fixfor and we arrange a time to talk on the phone.

We even use the free-text fields in FogBugz to keep track of the resume ranking that we give, and we use the remaining field to keep track of the questions that developers have asked the candidates; this keeps us from asking a candidate the same questions in different interviews.

This describes the process for developer and intern candidates, but we hire other kinds of people here too, believe it or not. To keep all of these positions separate we've created Areas like QA, developer, intern (by year), sys admin, and so on.

Using the list view allows us to easily sort by area, or due date, or fixfor so it's easy to see if something is overdue, or if we got a new email from a candidate already in the system, or whether a developer is slacking on resume reviews because they've been too busy eating Fun Dip

Once we decide to bring someone in for a visit to our New York offices we assign the case over to our people who handle the travel arrangements and they pick up the thread.  We follow this all the way through to making offers, and helping with travel and moving arrangements.

We could collect resumes with a web form, but we like the low-barrier nature of just emailing to our jobs address.  Everyone has a resume, and they usually take the time to write a cover letter, so why make them paste all of that stuff into whatever form we'd come up with, which is different from whatever form everyone else comes up with?  It's just more overhead.

That's it.  Collect the resumes, and shepherd them through the process.  A case for each candidate, all the information you'll need right there in the case, and easy to sort and organize.  If you're using some other system for collecting and responding to resumes, give FogBugz a try for this purpose.  I predict you won't go back.

 


Categories: FogCreek, FogBugz

Tags: ,

Actions: E-mail, Permalink


GTD, FogBugz, and Customer Service

Thursday, February 19, 2009 by Dan Ostlund

FogFanz has an interesting article up about using FogBugz as a Getting Things Done (GTD) tool.

It's a great topic, and one that we talk about here a lot.

A growing number of Fog Creek employees use FogBugz as a GTD tool.  Rich, the fellow in charge of our customer support efforts, has designed his workday around a series of lists in FogBugz that give him the next things he needs to do.  Filters are the means of constructing those lists.

Since he's not in control of the volume of customer email he gets, our Inbox for customer service becomes his defining task list for the day.  Rich and the support team work through the Inbox first. He takes care of these cases in the order in which they arrive, and rigidly enforces the "no cherry-picking" rule because cherry picking makes the whole system break down.  You spend (waste) time thinking about what to do with a case several times over, avoid addressing your weak spots, and inevitably answer the customer who tells you how great you are and wait to deal with the one who has the malignant Mono mystery Unix install failure.

To Rich, every case that comes into the Inbox is an opportunity to make that thing never happen again, or at least reduce how often it happens.  He spawns off a case in the Fix It Twice area so that it ends up in a trusted system, one of the keys of GTD: You have to capture everything.  These are the cases that make those Inbox problems go away forever.  These are the cases that will reduce your support volume.  These are the cases that will make support staff and customers happier.

Joel has talked about using Five Whys and Fix it Twice on his blog, and we've even been inspired by these guys.  Here at Fog Creek Fix it Twice has become a bit of an OCD obsession.  Rich has helped us make this less of an ad hoc system.

Anytime there are spare cycles Fix it Twice cases get attention, so when the Inbox is done Rich digs into the Fix it Twice list.  This is where the real magic happens, and where Rich has brilliantly gotten support off the treadmill of trying to keep up with more and more support requests.  This is where he gets to think creatively about how to fix problems permanently, which also happens to be a nice break from the routine of daily support, and turns support into a strategic role--a thinking-person's job--and a role less prone to burn out.

To us, your GTD "inbox" is NOT the same as your FogBugz Inbox.  Your GTD inbox is the Fix it Twice area.  The cases in Fix it Twice are the things you will get done, the things that will reduce your support load.

The progression from processing the Inbox to processing Fix It Twice is as close as we've ever gotten to implementing GTD's idea of contexts in a real way.  Effectively, it boils down to Inbox is @reactive, and Fix it Twice is @strategic.

Rich has a few extra steps (that he may write about), and has put a lot of thought into this FogBugz/GTD system, making it simple to follow, but powerfully effective in reducing the volume of support problems we encounter.

The point is that he has lists in FogBugz that help him to organize his day's work, and that this melding of GTD thinking with strategic customer service thinking has helped us move away from running to catch up, to more effectively driving the direction of support efforts.  If you have a growing customer base and don't make Fix it Twice part of your life, you'll always be trying to run a little faster on a merciless treadmill.  That is until something breaks badly--your support staff gets burned out, or your customers become so sick of your inability to help them that they burn you in effigy (or in modern terms, say nasty things about you on Twitter).

There are lots of tools to help you follow GTD, but FogBugz can definitely work in that role.  It can work as a personal system, or you can make it work for strategic customer service.
 



How Would You Change the FogBugz Interface?

Thursday, February 05, 2009 by Dan Ostlund

We have a fan blog!  We didn't start it--I mean, we're fans of ourselves, but that might be a bit much.  Instead it's kept by a longtime user of FogBugz who has a background in usability testing, and in a recent post he gives FogBugz a theoretical makeover.

He proposes some interesting ideas that are worth a look.  I especially like that he appreciates some of the hard design decisions that go into keeping an interface simple and easy to use, and yet still providing all the functionality customers want.  For more in that vein, you might also take a look at his post on possible FogBugz 7 features (just remember those aren't our promises!).

Drop by the blog and let him know what you think of his FogBugz design ideas.


Tags: ,

Actions: E-mail, Permalink


Placeholders

Saturday, January 31, 2009 by Rich Armstrong

Snippets are a powerful feature of FogBugz, but many people don't know about placeholders in snippets.  These allow you to put a tag in your snippet text that will automatically be replaced with a value from the case.

The easiest way to see how these work is to see them in action. To do this, I created a test snippet which shows me all placeholders and the values they're replaced with.  Take the whole of the text below and copy it into a "test" snippet, then use it in a case (preferably one that's been emailed in).

A lot of folks ask us (as I asked when I first joined) why we don't have a snippet for extracting the first name out of the email's From address.  The simple answer is that this is not just a simple database value that we can pop into the text.  This will be one of my big wishes for future versions of FogBugz, but it won't be coming any time very soon.

Note that these placeholders work in outgoing autoreplies as well. Here's the snippet text: 

{case} (case) for the case ID
{sender} (sender) for the sender's email address
{subject} (subject) for the subject of the message
{ticket} (ticket) for the external ticket ID
{url} (url) for the URL of the FogBugz install
{ticketurl} (ticketurl) for an external link to the case
{ticket} (ticket) for the non-URL portion of the link.
{username} (username) for the name of the logged on user
{useremail} (useremail) for the email of the logged on user 

Categories: General, FogBugz

Tags:

Actions: E-mail, Permalink


Deleting a Wiki Article

Thursday, January 22, 2009 by Rich Armstrong

A lot of people are surprised we don't just have a button to delete a wiki article.  While that would be nice, it's not that simple.  Wiki articles are usually linked to other wiki articles, and these inbound links need to be dealt with before we can move forward with deletion.

Simply go to Wiki > Customize and choose your wiki to edit.  If on the Edit Wiki page you click the "List all articles in this Wiki" link, you'll be taken to a list of all the articles, each of them with a count of inbound links.  Click the number of inbound links and you'll get a pop-up listing the articles that link to the article you'd like to delete.  Once you've edited those articles to remove the links, you can go back to the Edit Wiki page and select "List all orphaned articles in this Wiki".  Your article should now appear in this list, along with a handy delete button.


Categories: FogBugz

Tags: , , ,

Actions: E-mail, Permalink


Change the Starting Case Number in FogBugz

Saturday, January 10, 2009 by Adam Wishneusky

Sometimes there arises the need to increase the starting or next case number in FogBugz. Say you have imported cases into FogBugz from another instance of FogBugz or another tool such as Bugzilla. To avoid case number conflicts, you can change the next case number with a simple API command which you can run right in your web browser.

The FogBugz API accepts both GET and POST HTTP requests, so you can do this by putting the following URLs into your favorite web browser. Note: these URLs are for windows self-hosted installations. PHP installs should use .php in place of .asp and On Demand accounts use https instead of http.

First, login using your email address and password:
http://yourfogbugz/api.asp?cmd=logon&email=youremail&password=yourpassword

Replace the italicized values with the appropriate ones. The API should give you an XML response with a token that looks something like this: 1ntkuxplkqjj6ldk325p512lar6vg2. Copy the token. Now change your staring case number by putting the following URL in your browser:

Update the starting case number:
http://yourfogbugz/api.asp?token=yourtoken&cmd=adminSetCaseNumber&ixBug=desirednumber

Replace the italicized values with the appropriate ones. Use a desirednumber that's higher than your highest imported case number and use the copied token from before for yourtoken.

On a related note, if you're clearing out the cases in your FogBugz database in order to start again without losing your projects and users, you'll want to use TRUNCATE TABLE on both your Bug and BugEvent tables in your database if you want the case number to start from 1 again.

Full FogBugz API Version 5 documentation


Categories: FogBugz

Tags:

Actions: E-mail, Permalink


FogBugz iPhone Spec

Wednesday, December 31, 2008 by Michael H. Pryor

We've had a number of customers contact us asking us if we were going to create an iPhone app for FogBugz, or if they could create one themselves.  For the last few months, we've had a spec sitting around but we're so busy working on FogBugz 7 that we haven't had time to implement it (and I don't see it happening anytime soon).

While we're still going to reserve the right to build this app ourselves, I figured I'd post the spec anyway in case someone wanted to beat us to the punch.  Maybe someone will write something so awesome, that we won't even bother competing -- we'll just point people to theirs.

 Read our first draft iPhone FogBugz application specification.

 

 


Categories: FogBugz

Tags:

Actions: E-mail, Permalink