Mats Bryntse
16 January 2014

JavaScript Quality Assurance pt.1

This is the first blog post in a series of posts about our quality assurance process for our JavaScript products. […]

This is the first blog post in a series of posts about our quality assurance process for our JavaScript products. For us, Quality Assurance involves dealing with bugs, handling tickets and writing test cases. All of the text in this post applies to any SW development but is extra relevant for JS/CSS development since it is just so easy to fail. Here are a few reminders of a few easy ways to fail.

How bugs spread

Let’s take look at the big picture and see what the implications are of finding bugs at different stages in your product development cycle. We will start in the most zoomed in tier (you, the developer) and gradually “zoom out” to your surroundings. Let’s say your company makes a software product sold to other businesses that include your piece of SW in their product which they then sell to end users. A bug can then be detected in these different tiers:

Tier 1. Your development machine

bug1

In this image, it’s all about you – this is your local universe, your bubble. In this picture it is cheap and quick to find bugs. Within your bubble, bugs will eventually appear and try to escape unnoticed out into the wild. If you test your commits either manually or using a test suite your bubble will have a nice kind of safety net around it (see below). The dashed line can be seen as a net with holes as it’s simply not possible to prevent bugs from escaping completely.

bug1

No better place to find them than here, before committing to your source repository. If found here, you are already in the right context and no time consuming “task switching” is necessary.

Cost if found: None, you found it – it’s your secret.

Tier 2. Your team

Assuming you work for an organisation with more than one team contributing to the final product, your team is the next bigger tier.

bug1

If a bug you caused escapes out of your bubble into the team bubble, one of the following things can happen:

Cost if found: Low, working time spent for your team member(s). And, you may get hurt by a foam missile fired by your CI server.

Tier 3. Your organization

bug1

Now the bug has escaped your own bubble as well as your team bubble and likely enjoys life on a QA or Staging server somewhere within your company. Other teams may also have pulled and integrated the latest (buggy) source code from your team deliverable. If you are lucky, your company has a QA team as a last line of defense. For them to find the bugs is way more expensive than if it had been found while it only existed in your own bubble. Possible scenarios:

Cost if found: Expensive, possibly multiple departments involved, administrative work.

Tier 4. Your client

bug1

If the bug isn’t caught before this point, it means it is part of one of your public releases and ultimately ends up in the hands of your client. Maybe the client does their own smoke testing, maybe not. And normally you’ll have many clients, so the more clients you have – the more heat you can expect in case a low quality build is released. If they don’t find the bug, then the next and final stop is… you guessed it: The real end user.

Cost if found: Very expensive, possibly affecting company reputation.

Tier 5. The end user

shutterstock_12332866

If the bug made all the way to the client, it made a home run and will celebrate all night long. The best case now is that end user doesn’t experience it at all. More likely scenarios include:

Cost if found: Most expensive. Very long wait for end user to get a patch. Both your company and your client company reputations suffer.

Summing up

It should now be clear that the earlier a bug is found, the less resources, time and money have to be spent fixing it. Not to mention saving the embarrassment of phone calls and emails from frustrated users. In part 2 of this series, we’ll look at some of the methods we use at Bryntum to find and fight bugs as early as possible.

Mats Bryntse

Siesta Testing