This is a repost of my article in Issue 19 of the MailChimp UX Newsletter. Our team publishes a new issue every couple weeks so if you enjoy this, please subscribe!
“It’s a bug, and they need to fix it!” my wife exclaimed last week after a rebate check for an online purchase was sent to an old address for the second time. While I was also pretty peeved, I couldn’t help but feel a bit of sympathy for the team that would need to troubleshoot the issue.
Perhaps I shouldn’t. When a user encounters a bug or even a confusing aspect of your application that costs them time, it also increases load on your support team and erodes the user’s trust in your product. Just like a leaky bucket, big bugs will drain trust quickly, and lots of small issues will kill it slowly over time. Once that trust is gone, it’s gone.
We’re not immune to this at MailChimp–no matter how hard we work to improve our app. And I’ll be the first to admit that our bucket has its share of both big and small issues.
Find the Holes, Protect Your Bucket
Image: CC by Andy
When you work on something every day, it’s very easy to overlook common shortcomings. This is why we rely heavily on interviews, surveys, analytics, and other research methods to reveal the weak spots in our app. Sometimes a seemingly innocuous oversight can have devastating consequences. Aarron Walter recently shared on A List Apart how we use account closing surveys to make decisions about feature additions, but they also help us find some of the biggest leaks in our trust bucket.
If you approach working on an application like playing a tower defense game, finding problems as users close accounts is definitely a last-ditch effort. We try to pay attention to every feedback channel our users might express opinions or frustrations from. Our Support team helps by tagging all of the chat and email correspondence to help us discover common problems. We also watch social outlets like Twitter, Facebook, and blog comments.
Prioritize and Patch
Image: CC by Guian Bolisay
Collecting feedback from many channels is easy, but filtering and determining what to act on is tough. If we made changes based on every comment, tweet, email, chat, and fax (we don’t really collect fax feedback), we’d have more tickets than our small team could address. Many companies just hire more people to address every bug. The problem is that many of the suggestions we get are contradictory or would help only a few people. That approach doesn’t scale, and can actually introduce usability issues.
Instead, we focus on tagging, filtering and prioritizing the feedback we get. That image above would be me if I didn’t have Gregg Bernstein and the rest of our Research team. Gregg keeps a pulse on a lot of the suggestions we receive and forwards the most repeated and interesting requests.
Some of filtered feedback in my inbox from Gregg.
If you were to look at the commit logs for a given 4 week release, a large number of the comments include words like fixed, tweaked, refined, adjusted and changed. This is where a lot of those suggestions are getting made and some bugs, typos or display issues that affect people’s workflows get hot-patched in between release cycles as well.
Turn Weakness Into Strength
Image: CC by Kevin Smith
While a lot of our changes to the app are based on feedback and suggestions, we’re always quietly working on secret new things as well. MailChimp gets a lot of attention when we introduce new, innovative features, so it’s easy to focus on that instead of making minor refinements. Often times though, a new feature is simply rethinking a weaker part of the app that hasn’t been updated in a while.
This is the kind of stuff that everyone—from UX research all the way to our awesome dev engineers—enjoys working on most. It’s exciting to build new things, even if that new thing is just a reimagined incarnation of an existing feature.
Make Dedicated Refinement Time
Image: CC by P J R
Sometimes it’s best to slow down, set aside ideas for new changes, and work solely on maintenance and refinement. Reducing the ticket queue isn’t a sexy task, and minor refinements don’t make for popular blog posts, but sometimes it’s necessary. That was actually the motivation behind this article—we’re just now wrapping up an entire release focused solely on cleanup and maintenance. We tend to do this at least once a year and the holidays are typically a good time for it.
During this release cycle, we’ve also been planning features for the next one and making big plans for the rest of 2014, but the code changes we made this time were all about fine-tuning the details.
Keep Doing Both
When working on small bugs and refinement it’s easy to get nearsighted, missing the big picture of how our changes affect the overall UX. Working on big, new features allows us to see that but we often miss small details, fail to predict how people will use them, or discover edge cases we didn’t consider. The most important thing is that we keep doing both so that our bucket keeps getting bigger and better with as few leaks as possible.