Before coming to Fog Creek Software, I never had to think about who my customers were. When I was working as a consultant, my customer was the client that was paying the bills; if they wanted a feature and were willing to pay for the development, then that's what I did. When I was developing prototype software for a research group, my customers were the scientists; if they needed software to acquire data, analyze data, or control equipment, then I would build it for them. There was never any question about who I should be listening to, because they were real people that I interacted with daily.

Things are different in the software product world. FogBugz has thousands of existing customers and many more potential customers. And each of those customers uses FogBugz in a slightly different way. Some customers use FogBugz as:

• A simple bug tracker
• A help desk to manage customer support
• Project management software
• Support forum software
• All of the above and then some, as we do here

Because of the multitude of ways in which FogBugz is used by our customers, defining what the "customer" wants can be extremely difficult. When I was a consultant, I could just ask the client to find out what feature I should implement next. Because there was a single voice to listen to, my decisions were easy.

When there are thousands of voices, feature decisions become much harder to make.

• Do we ask for a majority vote on each feature?
• Do we pick features that will make many of our customers a little happier?
• Do we pick features that will make a minority of our customers a lot happier?
• Do we trust our own design judgment on which features will work well together?

In the end, we do what we think will make our customers in aggregate happiest and most productive. Unfortunately, because we have limited resources, we can't do everything. When we don't implement a customer's feature request in a given release, they are disappointed, as if we were a consultant that failed to deliver what was promised.

To some extent, that's a consequence of our customer service ideals. When you call us, a capable person picks up the phone and is able to help you do what you want. We respond to all emails personally within one business day, and regularly respond on our discussion boards. That level of personal interaction raises the expectations of our customers that we are serving them directly. So when our software fails to deliver what they personally need, we are failing to live up to our own lofty standards.

What our customers can't see is that we live up to those standards internally. When we are doing triage for a release, we have customer advocates arguing for each of the features, and we have long discussions to balance all of the factors discussed above. Unfortunately, some features will always have to get postponed (but still kept in FogBugz to be considered for the next release). 

Making a software product is hard. I had it much easier when I worked as a consultant with only one voice to satisfy, but the flip side was that I could only make one company better with the software I wrote. FogBugz makes thousands of companies better at once, which makes it worth sorting through all of those voices to decide what should be implemented next.

Things have been hopping here in the office (and correspondingly quiet on the blog) for the past couple months while we've been beta testing FogBugz 6. I have found it interesting to see how the beta test process has evolved. 

Early in the beta testing period, our beta customers were finding dozens of bugs, from major ones like not being able to install in certain configurations to minor ones like fixing the placement of buttons in the German version. After our intrepid developers tracked down those bugs and fixed them, we tested the fixes internally and sent out a new beta release with the collected fixes. 

As the beta testing period has continued, the problems being found are getting more obscure. A new feature doesn't have quite the functionality one customer desires. Another customer finds an error that we can't reproduce internally despite our best efforts. 

We still want to fix these problems, but every code change requires more testing to ensure that nothing else has been broken by the change. So throughout the beta process, we have had to become more selective as to what kinds of problems we will fix. At this point in the beta process, we are only fixing bugs that cause FogBugz to crash. There are minor bugs that we have decided to not fix for the FogBugz 6 release, and that decision kills us. We want to send out an absolutely perfect release with no known problems. But if we tried to fix every problem, we'd be changing code and re-testing for many more months, if not years. 

We think that FogBugz 6 is a big step forward for FogBugz with many exciting new features, and we're eager to get it into the hands of customers. So at some point, we have to make the choice that releasing FogBugz 6 is more important than fixing the remaining bugs, even though that means that we will already have dozens of open cases to be fixed in a future 6.1 release. We'll get back to closing those cases one by one after the release. 

In my last post, I discussed the qualities that made for a good tester.   And Joel explains why a company should hire testers.  But what motivates a good tester? 

When I asked "Jill the Very, Very Good Tester" about how she approached her job, she stated that her two main responsibilities were to the end-user and to the company.  Let's explore what she means.

End-User Responsibility

Look around a typical software company.  You'll see that it's a technology-centric environment.  Everybody there can configure their own computer, has a complicated cell phone, and plays video games with interfaces so complex that only a teenage boy could learn them.  When they interact with technology, they know what the designer was trying to do because technologists have similar mindsets.  Since they only interact with other technologists, it's easy for them to forget that not everybody thinks like they do.  For instance, an ex-coworker once uttered with a straight face, "That's not a bug!  We have a 15 step workaround for that, so we don't have to fix it!"

The tester's job in such situations is to be the user advocate.  They need to explain how the rest of the world will react to the technology by constructing reproduction cases with the structure of:

• 1,2,3) This is what I did. 
• Expected: This is the result I expected. 
• Actual: This is the result I got.

Such cases demonstrate for the developers what the expectations of the rest of the world will be.  A good tester will anticipate the reactions of the non-technologist end users and catch unexpected behaviors before they are released to paying customers.

Company Responsibility

The tester also has a responsibility to the company.  They manage the risk for the company by providing clear feedback on the current status of the software and its readiness for release.  We all know that when a developer says they're 90% done, they're about halfway there.  But we need testers to inform the developers and their managers of the true status.

A tester will document what works and what doesn't in the software.  They have test plans written, with a method for exercising every feature described in the specification.  They deliver the current status to make it abundantly clear how far the project is from being complete.  One of Jill's suggestions for reporting is in the form of a two-dimensional matrix, where each feature is tested on each of the different deployment platforms; once developers see all the empty spaces, they will be less convinced they're almost done.

There will be pressure from management to give convenient answers, but it's the tester's responsibility to deliver accurate information.  Managers can not make the right decision on whether the software is ready to release without the truth.  They may decide to release it anyway, but then they are making a fully informed decision and are less likely to be surprised later if their customers are outraged.

Sounds awesome, doesn't it?  The tester as the representative for the Common Man, and the standard-bearer for Truth within the organization.  Things rarely work out so nobly for the poor tester, but I thought Jill's perspective may explain why she's a "Very, Very Good Tester".

I'm currently in the Quality Assurance (QA) phase of the Software Management Training Program here at Fog Creek Software.  That's why I'm currently running the beta test of FogBugz 6 among other things.  To prepare for my stint in QA, I talked to "Jill the Very, Very Good Tester", immortalized in The Basics of Bug Tracking, for her thoughts on testing, and reflected on the qualities of the good testers I worked with in my previous career as a software developer. 

Good testers see the world differently than software developers.  Or from most people, really.  When they're in testing mode, they try things that no "sane" person would ever try.  They see the flaws, the interstices where systems grind against each other, and figure out ways to exploit them.

• What happens if I start doing one task, interrupt to do a second, go back to the first task, and then skip to a third task?
• What happens if I paste a ridiculously long piece of text including all sorts of non-standard international characters into every possible text field, including the zip code?
• What happens if I intentionally try to inject SQL code into the database?
• What happens if I don't understand any of the instructions and just start double-clicking randomly on everything I can see? 
• What happens if I just start hitting random keys on the keyboard?

I used to work with a guy named Henry who was incredibly good at this.  I'd spend hours testing my code, clicking on every button, trying every possible combination of inputs I could think of.  I'd give him the software, and he'd be back at my desk in five minutes, saying "Eric, I broke it!"  I'd ask him what he did, and he'd explain how he was doing something that nobody had ever requested or even mentioned, and because the software wasn't designed to do that, it broke.  Fortunately, Henry would also have figured out a way around the bug so that he could keep working while I was trying to fix it, cursing all the way.

At the same time as they are coming up with all these crazy ways to break the software, good testers also have to be incredibly methodical.  They map out every aspect of the software before starting, create a test plan to make sure they exercise all of the desired functionality from the specification, and systematically go through every corner of the software.  These are the kind of people you want running your FDA certification process because they will follow the process to the letter and nothing will fall through the cracks (a former coworker of mine is now a consultant in that field).

When they do find a place where the software doesn't work as expected, they reproduce the bug in a minimal number of steps and file a bug.  Joel describes the importance of this careful reproduction process in the The Basics of Bug Tracking, both in helping to track down the bug and making sure that it's well and truly fixed.   

Somebody who's good at the methodical part may lack the craziness necessary to invent truly dastardly ways of breaking things.  Somebody else who has the completely off-the-wall perspective to try absurd things may not have the patience to figure out exactly what they did that finally broke the software.  So finding testers who have both skills is hard. 

I'm not particularly good at testing yet.  I still think like a programmer, so I know what a feature is supposed to do and tend to do what the developer expected when using that feature.  I'm still working on developing that outsider's perspective, of trying things no programmer would ever try because it shouldn't work. 

I'm also learning to test in a more methodical fashion.  I'm improving at my coverage when doing a full test of a new feature.  And when I come across odd behavior in FogBugz, I can track down a decent set of reproduction steps, and often even isolate it to a section of code that's causing the problem. 

Being a tester has encouraged me to think outside the perspective of a software developer, and to develop aspects of myself that didn't get used as a programmer.  I'll be glad to have that different perspective as I move forward into a career in technology management. 

I’m no usability expert.  In fact, I own a phone with a camera that only ever snaps photos by mistake, usually the inside of my pocket. But I don’t need to be an expert to know that sites which require you fill out a huge form to get a price quote are poorly designed.  You’ve all seen these sites: Visit any auto insurance webpage to see what I mean. I hate this.  I have filled out exactly zero of these forms in my life, and I don’t expect the number to rise appreciably.

I know, auto insurance probably needs personal information to generate a quote, but anyone else is making themselves a problem.   These forms make me think there is something to hide, or I am going to have a sales person hounding me, or that my volume of spam is going to reach newly annoying highs.  Ultimately this is a trust problem, and not a problem with the form I think, but that’s another matter.  I would be really interested to see some figures about how many people bail out at the form filling page.  I suspect it’s a lot.

I’ve been thinking about this because we get a lot of people asking for price quotes.  I always happily give them out, but also think “hey, this is on our website,” and since they got our email from the website they should have seen it.  But they don’t.  So something is wrong, or at least wrongish.  I think our quote tool is hard to find.  I now suffer from over familiarity with the site, so I no longer see it, but I can recall having trouble finding it the first time, and the number of people who still ask tells me it’s too hard to find.

You get there this way: Go to our site, choose FogBugz from the list of products, select pricelist on the right side under the Learn About FogBugz area, and then inside there select the “Get an exact price quote now!” link which, adding to our trouble, is not easy to spot.   It’s worth it for us to think about how we could place this to make it super obvious to anyone who visits the site. 

The more interesting question though is, if we make it less hassle for a customer to find out how much an order will cost, are they more likely to make a purchase, or is this just too insignificant to matter?  And, moreover, can enough “too insignificant to matter” things that get improved lead to a kind of critical mass that, when taken together, lead to a much better experience and thus to more sales?

I had to make new button graphics for a web page, and they had to match the existing buttons.  I couldn't get/find the original graphics files with the layers that neatly separated the text from the background colors.  So I had to somehow change the text on this button:

which meant that I needed to remove the text and get a blank version of the button and write new text on it.

My first thought was basically "clone the background and paint/pencil over the text with matching background colors to create a blank button."  But my clone-and-paint dexterity wasn't good enough to match the original colors and shading, or I was just impatient.

So now I decided to create the button background anew, and then write text over it.  Using Paint.NET and existing tutorials on gradients, I could only make flat-looking buttons that didn't match the reflecting-light looks of the originals. 

 But then I remembered from the gradient tutorials that you can select certain pixels and then stretch them to cover more surface area.  Well, maybe I could use the rectangular Select tool, copy the gradient pixels from the old button, top-to-bottom

make a new image the same size (248 by 45), and paste the pixels and stretch them to fill the width of the new button.

So now my whole new button was the gradient.

But it didn't look right at the left and right edges, so I also had to find a way to copy the borders on those sides  I lassoed the text in the middle of the old button:

and then did a Reverse Lasso (Edit: Invert Selection) to get everything except the text, and copy it.

The new button only had a background layer.  I added a new layer and pasted the border on it.

Once I viewed both layers of the new image:

I had my blank button, ready for new text and/or saving as a PNG.

In retrospect, I could have done that lasso and paste of the border first, and then stretched the existing gradient pixels in the new file over the middle of the image.  That might have been easier.  In any case, I hope this is useful for other graphics newbies, or anyone who has to remove something from the middle of a background. 

Working in customer service, I am very interested in what people are saying about our products and services. In order to research what is being said along with who is saying it, I started to collect a list of sites where people commonly post about products in our category and search for posts there etc. Every time someone linked to an external blog I would bookmark that so that I could check on it periodically. Finally, I used Google to search for a few key terms, including the company name, and would browse through the results. The issue with all of this was that there was no way to immediately tell if I had visited a resource or not. Nor could I tell that I had forgotten to check one of my sources. I spent an inordinate amount of time reviewing old information only to realize 100 words in, that I had read it already.

Enter Google Alerts.

One day, I was clicking around in my Gmail account and I ran across the my services link. Right at the top of the list was the mysterious Google Alerts feature. Google describes them as:

… email updates of the latest relevant Google results (web, news, etc.) based on your choice of query or topic.

The configuration only requires you to plug in a search term and select how often you want Google to tell you about new things that it has found that match. It literally saved me a couple of hours a month. Not a lot of time in abstract, but keep in mind that those hours were previously completely wasted and totally annoying. Now I use it to keep track of a bunch of companies and people who I like to keep tabs on.

Every once in a while, it really helps me at work by letting me know about something really good ( or really bad ) that is going on with our customers. By getting and staying informed, I am armed in the battle for excellent customer service with the weapon that is timely relevant information.

For example, I have a Google Alert configured to tell me once a day about anything new that references Fog Creek Software. Recently, it informed me of a blog post (one that I would very likely have never seen otherwise) which detailed the poor experience of one of our Copilot users. I used this information to reach out to a customer who rightfully complained about a bad experience that was the result of a bug in our software. He not only responded positively to my effort to apologize but also made himself available to troubleshoot the problem he experienced. I helped him by recognizing the issue and compensating him for the trouble. He is helping us by providing us with a source of information, should we need it, about a bug in our software. In addition, I would hope that it may lead to some repeat business for us in the future. In my opinion, it represents a small but important victory in an effort to ensure a truly enjoyable customer experience.

Every day we get emails from our amazing customers about really useful stuff they are doing with our products.  The kind of stuff that you wouldn't necessarily want to post on the main Fog Creek pages because it usually requires a bit of monkeying around in the source code, or a deeper understanding of what is going on behind the scenes.  We never had a good place to talk about this stuff, until know; I've started this blog to do just that. 

This blog is for people who would be interested in being beta testers for our products (FogBugz beta coming out soon, sign up here).   I'll talk about some tricky bugs we've found in IIS, Norton, McAfee, and MySQL that can affect FogBugz.  Future posts will be about CaseDetective, translating FogBugz, a Gmail like notifier for new FogBugz cases, a proxy for Vault to make it think it is talking to its own built in tracker, and more ideas that I haven't thought up yet.

So stay tuned!