LEAN: Simple Rules

The LEAN simple rules:

1. Spend time only on what adds real customer value.
   
2. When you have tough problems, increase feedback.
   
3. Keep your options open as long as practical, but no longer.
   
4. Deliver value to customers as soon as they ask for it.
   
5. Let the people who add value use their full potential.
   
6. Don't try to tack on quality after the fact – build it in.
   
7. Beware of the temptation to optimize parts at the expense of the whole.
   

I love these rules. They apply to testing just as much as they apply to software development.

Testers Should Drive Software Testability

Software testability is a key aspect to allow the detection of difficult to uncover defects in software. Software testability supports the testing process and facilitates the creation of better quality software.


Software testability can be described as the probability that a piece of software will fail on its next execution during testing if the software includes a fault.


Testing and testability are complimentary: testing can reveal faults (testability cannot) but testability can suggest locations where faults can hide from testing (something testing cannot do alone).


Software testability must be designed into the software as it is developed. Therefore, it is an attribute of the software that requires close development cooperation with test.


Designing for testability requires that software is designed with a greater ability to fail when faults do exist.

James Bach proposes a set of Heuristics of software testability:

• 
  
  Controllability
  
  The better we can control it, the more the testing can be automated and optimized. 

• 
  
  Visibility
  
  What you see is what can be tested.

• 
  
  Availability
  
  To test it, we have to get at it.

• 
  
  Simplicity
  
  The simpler it is, the less there is to test.

• 
  
  Stability
  
  The fewer the changes, the fewer the disruptions to testing.

• 
  
  Information
  
  The more information we have, the smarter we will test.

A given piece of software will or will not hide a given fault from testing. The software testability of software will determine whether that fault is easily detectable by testing or not.


The more controllable the software, the more we can automate. The more we can automate, the less likely that human error will allow a defect to escape into the customer's hands.


Software testability is extremely valuable where functionality cannot be tested using a black box methodology. Software testability is tightly aligned with white box testing. Software testability must be designed into the software, so tester knowledge of any incorporated testability is required.


By including and improving the testability of algorithms in a software product, it will allow the testing organization to add automated and optimized white box tests to the automated test suite which will immediately fail and highlight a newly introduced bug.

 
For example, a particular algorithm when triggered in debug mode could print to STDOUT a specific message communicating that it is executing. If that STDOUT message is not detected by automation, the test will fail, thereby highlighting that the algorithm is no longer executing.


In black box and customer focused software testing, only those aspects of the software that are observable can be tested by the testing organization. The inclusion of debug information on specific software features will increase the ability and ease with which to test those software features which are not observable via black box techniques.


The simple inclusion of debug information will facilitate automated tests being developed to test whether an algorithm is executing or not. These automated tests will execute each and every build so if a regression is introduced it is detected immediately, a bug is filed, and development is made aware of the bug sooner rather than later.


The automation of these types of tests will also free up manual testing time so that a tester's time is spent on testing that cannot be carried out in an automated fashion, thus increasing test coverage of the software product.


By setting the tests up in automation, it also excludes the possibility of human error in not detecting the existence of a bug.


Software is said to have high testability if it tends to expose faults during exploratory black box testing, producing failures for most of the inputs that execute a fault. Defects can be identified and fixed quickly.


Software has low testability if it tends to project faults from detection during exploratory black box testing producing correct output for most inputs that execute a fault. These defects are extremely worrisome as customers using the software believe that everything is going as expected until down the line, suddenly the defect is exposed. This category of error can adversely affect the customer's business and thereby undermines the software's credibility and trustworthiness.


The goal of increasing the testability of software is not just to detect defects but more importantly, to detect defects as soon as they are introduced. Thus, reducing the cost and time to fix the bug and producing higher quality software each build of the release.


Reference: www.satisfice.com

Brain Rules

The best book I've read this year is "Brain Rules: 12 Principles for Surviving and Thriving at Work, Home, and School" by John Medina.

It is very enjoyable to read, not one of those books where you bribe yourself with a cup of tea and a biscuit if you just finish the chapter you're currently reading. It's easy to get engrossed in the text and not realize that hours have gone by.

The 12 principles are:

1. Exercise
   
2. Survival
   
3. Wiring
   
4. Attention
   
5. Short Term Memory
   
6. Long Term Memory
   
7. Sleep
   
8. Stress
   
9. Sensory Integration
   
10. Vision
    
11. Gender
    
12. Exploration
    

I'm not giving away. A lot of material is available for free at www.brainrules.net. However, you'll get a lot more out of the material if you read the book too.

So the question remains: What does this have to do with testing?

Well, testing is an intellectually demanding task where concentration, focus, being alert is essential if you are to be effective. Based on what I learnt in this book I made some changes.

For example, I am a lark! I am at my most productive in the morning. Why fight that? So I get into work before 8am every morning to absolute joyous silence (apparently larks are rare in engineering and testing – who knew?) and I get a good 3 or 4 hours of productive work done. If I test during this period, I am far more likely to uncover high priority bugs. To allow for this interrupted time, I've rescheduled all my manager-type meetings to the afternoon.

Another example is how I've integrated little walks into my day, even if it's just going all the way to the canteen (a good 10 minute return journey) to get coffee instead of making it on my floor. These little walks recharge my brain and boost my brain power. I come back to my desk and easily refocus and apply myself to my work.

Etc, etc, etc.

Read the book – it really will open your eyes to the physiology and evolutional biology behind how we think and work.

Books on Testing - Recommendations?

I've read the following books dedicated to the subject of testing:

• Testing Computer Software (Kaner, Falk, Nguyen)
• Lessons Learned in Software Testing (Kaner, Bach, Pettichord)
• Agile Testing: A Practical Guide for Testers and Agile Teams (Crispin, Gregory)
• Exploratory Software Testing: Tips, Trcks, Tours, and Techniques to Guide Test Design (Whittaker)
• Foundations of Software Testing (Graham, Van Veenendaal, Evans, Black)
• Software Testing (BCS)
• Automated Software Testing (Dustin, Rashka, Paul)
• Effective Software Testing: 50 Ways to Improve Your Testing (Dustin)
• How to Break Software: A Practical Guide to Testing (Whittaker)
• How to Break Web Software: Functional and Security Testing of Web Applications and Web Services (Andrews, Whittaker)
• The Art of Software Testing (Myers)
• Managing the Test People (McKay)
• Managing the Test Process (Black)
• TMAP Next for Result-Driven Testing (Koomen, Aalst, Broekman, Vroon)
• TPI NEXT, Business Drovem Test Process Improvement (Sogeti)

Obviously, if I search for "software testing" on Amazon, quite a number of books are returned.

Any recommendations?

Why Invest In Test Process Improvement?

Test Process Improvement is the continuous improvement of the quality and the efficiency of the testing process, in the context of the whole software development life cycle.

But, test process improvement is not testing. Why should you invest in it? Aren't you wasting time that you could be using to find bugs?

• Reduce overhead
  
• Increase test efficiency & effectiveness
  
• Allow test to embrace change (key to agile testing)
  
• Focus is on delivering results
  
• Improve Test's influence in order to deliver better quality
  
• Participating in a TPI program motivates & empowers test engineers
  

My organization invested in continuous TPI.

One major outcome was to increase the efficiency of the automation execution and analysis process. In many projects, this decreased from a 5 day duration to 1 day! This has freed up time for more manual and customer-like testing.

So yes, time was invested in TPI, but at the end of the pipeline, we now have more time to invest in more effective testing.

TPI is a cost but spending that cost in the right area will produce a return. The key is to pick the right test area to improve!

Rapid Software Testing, Dublin, September 2010 - places limited!

Rapid Software Testing is a three-day, hands-on class that teaches testing as a sophisticated thinking art.

The philosophy presented in this class is not like traditional approaches to testing, which ignore the thinking part of testing and instead advocate never-ending paperwork. Products have become too complex for that, time is too short, and testers are too expensive. Rapid testing uses a cyclic approach and heuristic methods to constantly re-optimize testing to fit the needs of your clients.

The Rapid approach isn't just testing with a speed or sense of urgency; it's mission-focused testing that eliminates unnecessary work, assures that everything necessary gets done, and constantly asks what testing can do to speed the project as a whole. This class presents an approach to testing that begins with developing personal skills and extends to the ultimate mission of software testing: lighting the way of the project by evaluating the product. One important tool of rapid testing is the discipline of exploratory testing-essentially a testing martial art. Exploratory testing combines test design, test execution, test result interpretation, and learning into a seamless process that finds a lot of problems quickly.

If you are an experienced tester, you'll find out how to articulate those intellectual processes of testing that you already practice intuitively. If you're a new tester, hands-on testing exercises help you gain critical experience.

 

MICHAEL BOLTON
Michael Bolton is an established and recognised leading-edge expert in the area of Software Testing. Michael has extensive experience in delivering workshops, tutorials, and conference presentations on the topic of Rapid Software Testing. He has over 20 years of experience in the computer industry testing, developing, managing, and writing about software.

Contact susan@isa-skillnet.com to book your place

Don’t Forget The Retrospective!

A retrospective is a valuable tool in any organization which values and expects continuous improvement.


A retrospective gives the opportunity to:

• look back and assess;

• consider;

• take everything into account;

• identify lessons learned;

• acknowledge success;

• try to set a better course for the upcoming project, or continuation of the current project;

A retrospective is a chance for the team to improve what they are doing and how they feel about what they are doing. By not holding regular retrospectives, teams are missing out on a valuable opportunity to grow and succeed.

Retrospectives enable whole-team learning, act as catalysts for change, and generate action.

Retrospectives empower team members and allow each and every member of a team to contribute and feel heard. Team members can see how they actively contribute to the increased success of the team.

Retrospectives focus on real problems that affect teams. During retrospectives, teams discover real solutions that they can implement immediately. Note, that it is the team who discover the real solutions. The solutions are not dictated to them from outside the team. This is where empowerment kicks in!

Organization is the key to an effective retrospective. Everyone must be made aware of the manner in which the retrospective will operate and what is expected of each and every participant.

Here are some suggested rules of engagement:

• We will try not to interrupt. (If the meeting is held over multiple sites, please verbally indicate when you are finished speaking – this will aid everyone as sometimes we can think the other person has finished speaking when they have not as we don’t have access to the visual clues.)

• We will accept everyone’s opinion without judgement.

• A reason should accompany each opinion.

• We will accept everyone’s supporting reason without judgement.

• We will talk from our own perspective. We do not try to imagine the perspective of others who may not be in attendance.

Set out the main objective of the meeting, for example, “The main objective of the retrospective is to seek out and understand the perspective and feedback from each member of the team. We will collate all feedback into positive and negative lists.”

Every opinion and perspective is valued!

The collated feedback from a retrospective can then be used to feed process improvement through the next stages of the project or into the next project.

Don’t forget: a retrospective is an opportunity! Don’t miss it!