Making Good Software

A blog by Alberto G (Alberto Gutierrez)

Written by Alberto Gutierrez

December 16th, 2009 at 5:59 pm

Testing facts and principles

with 5 comments

What follows is a summary of my own high level approach for testing software, this strategy is based in facts and principles.

Facts

1. It is impossible to detect all the bugs from an application.

The closer you get to 100% coverage, the harder it is to find the remaining bugs.

TestCurve

2. The more important bugs are in the core layer and in the integration layer (backend).

That’s where the testing needs to be focus on. Core layer bugs and integration layer bugs are the most important because they create a cascade effect causing several parts of the application to fail.

3. Using UI automated tests makes harder the detection of bugs.

Even though they are still very popular, UI automated tests are not very effective finding bugs because they test the core layer and the integration layer indirectly. Testing indirectly the backend makes difficult to exercise it, and makes hard to tell where an error is coming from. It is also important to notice that UI automated tests are also slow and expensive to maintain.

4. Manual testing is still necessary.

There are some important bugs that can only be detected through manual testing, that’s the case of the bugs that can be found doing usability testing and exploratory testing.

5. Testing is worthless if is not executed in a continuous basis.

What’s been proven correct through testing now is going to change very soon so it will have to be proven right again. If this feedback is not fast enough, new changes won’t get proven and eventually new bugs will be entered into the system.

Principles

1. Prioritize what’s going to be tested.

Never have a test strategy that expects to cover 100% of the application.

2. Have as much automated tests as possible.

From your previous prioritization, automate as much as you can.

3. Schedule time for the necessary manual testing to be performed on the project.

Two of the main manual testing activities that are necessary to perform are usability testing and exploratory testing.

4. Use preferably backend automated tests instead of UI automated tests.

5 Responses to 'Testing facts and principles'

Subscribe to comments with RSS or TrackBack to 'Testing facts and principles'.

  1. […] This post was mentioned on Twitter by Lee Winder and OlivierBeauvais, Michael Kelly. Michael Kelly said: I disagree with "facts" 2 and 5, but the lists are thought provoking: http://bit.ly/7G37i0 […]

  2. […] Testing facts and principles | Making Good Software (tags: software test engineering) […]

  3. UI automated testing isn’t supposed to test the core and integration layers… It’s intended to test the UI (JavaScript, AJAX, links, etc.). It isn’t a replacement for unit testing, etc.

    Joseph

    18 Dec 09 at 10:28 am

  4. First of all. Thanks very much for your useful post.

    I just came across your blog and wanted to drop you a note telling you how impressed I was with the information you have posted here.

    Please let me introduce you some info related to this post and I hope that it is useful for software testing community.

    There is a good Software Testing resource site, Have alook

    http://SoftwareTestingNet.com

    simi

    simi

    29 Dec 09 at 9:11 am

  5. […] 永遠!問題只在於要發現它的困難與否。 […]

Leave a Reply