Blog

This is the first blog post in a series of posts about our quality assurance process for our JavaScript products. For us, Quality Assurance involves dealing with bugs, handling tickets and writing test cases. All of the text in this post applies to any SW development but is extra relevant for JS/CSS development since it is just so easy to fail. Here are a few reminders of a few easy ways to fail.

  • 1. Leave an extra comma and bye bye IE. Even has its own domain. http://trailingcomma.com/
  • 2. Forget a var statement and say Hello! to a global variable.
  • 3. Use more than 4096 CSS selectors in a CSS file. More info here

How bugs spread

Let’s take look at the big picture and see what the implications are of finding bugs at different stages in your product development cycle. We will start in the most zoomed in tier (you, the developer) and gradually “zoom out” to your surroundings. Let’s say your company makes a software product sold to other businesses that include your piece of SW in their product which they then sell to end users. A bug can then be detected in these different tiers:

  • 1. Your development machine
  • 2. Your team
  • 3. Your organization
  • 4. Your customer (developers just like you)
  • 5. Your customer’s customer – the end users

Tier 1. Your development machine

bug1

In this image, it’s all about you – this is your local universe, your bubble. In this picture it is cheap and quick to find bugs. Within your bubble, bugs will eventually appear and try to escape unnoticed out into the wild. If you test your commits either manually or using a test suite your bubble will have a nice kind of safety net around it (see below). The dashed line can be seen as a net with holes as it’s simply not possible to prevent bugs from escaping completely.

bug1

No better place to find them than here, before committing to your source repository. If found here, you are already in the right context and no time consuming “task switching” is necessary.

Cost if found: None, you found it – it’s your secret.

Tier 2. Your team

Assuming you work for an organisation with more than one team contributing to the final product, your team is the next bigger tier.

bug1

If a bug you caused escapes out of your bubble into the team bubble, one of the following things can happen:

  • 1. Your automated CI test suite finds it
  • 2. Your colleague is affected by it, has to open a ticket, possibly debug => time consumed
  • 3. It goes undetected

Cost if found: Low, working time spent for your team member(s). And, you may get hurt by a foam missile fired by your CI server.

Tier 3. Your organization

bug1

Now the bug has escaped your own bubble as well as your team bubble and likely enjoys life on a QA or Staging server somewhere within your company. Other teams may also have pulled and integrated the latest (buggy) source code from your team deliverable. If you are lucky, your company has a QA team as a last line of defense. For them to find the bugs is way more expensive than if it had been found while it only existed in your own bubble. Possible scenarios:

  • 1. Your QA testers find it. Open ticket, build test case => time wasted. If critical, build will be invalidated. Release possibly delayed.
  • 2. It goes undetected

Cost if found: Expensive, possibly multiple departments involved, administrative work.

Tier 4. Your client

bug1

If the bug isn’t caught before this point, it means it is part of one of your public releases and ultimately ends up in the hands of your client. Maybe the client does their own smoke testing, maybe not. And normally you’ll have many clients, so the more clients you have – the more heat you can expect in case a low quality build is released. If they don’t find the bug, then the next and final stop is… you guessed it: The real end user.

Cost if found: Very expensive, possibly affecting company reputation.

Tier 5. The end user

shutterstock_12332866

If the bug made all the way to the client, it made a home run and will celebrate all night long. The best case now is that end user doesn’t experience it at all. More likely scenarios include:

  • 1. User accepts the bug and just lives with it
  • 2. User gets mildly annoyed and complains to your customer (who forwards the complaint to your company)
  • 3. User gets really angry and stops using your customer’s product

Cost if found: Most expensive. Very long wait for end user to get a patch. Both your company and your client company reputations suffer.

Summing up

It should now be clear that the earlier a bug is found, the less resources, time and money have to be spent fixing it. Not to mention saving the embarrassment of phone calls and emails from frustrated users. In part 2 of this series, we’ll look at some of the methods we use at Bryntum to find and fight bugs as early as possible.


 
 
  1. Misiu 01/18/2014, 9:21 pm Reply

    Nice start, can’t wait for next part!

  2. Tom Coulton 02/03/2014, 2:36 am Reply

    Hi Mats, thanks for this! We’ve got the Japanese translation of this blog post on our site (日本語バージョンは次のurlへ) : http://www.xenophy.com/bryntum/10235

  3. accendo 02/21/2014, 6:57 am Reply

    the cost statistics was nicely explained with step by step process waiting for next part ..,

Leave a Comment

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>