Software Testing by Ron PattonSoftware Testing by Ron Patton

Software Testing

byRon Patton

Paperback | July 26, 2005

Pricing and Purchase Info

$46.90 online 
$51.99 list price save 9%
Earn 235 plum® points

Prices and offers may vary in store


In stock online

Ships free on orders over $25

Not available in stores


Software Testing, Second Edition provides practical insight into the world of software testing and quality assurance. Learn how to find problems in any computer program, how to plan an effective test approach and how to tell when software is ready for release. Updated from the previous edition in 2000 to include a chapter that specifically deals with testing software for security bugs, the processes and techniques used throughout the book are timeless. This book is an excellent investment if you want to better understand what your Software Test team does or you want to write better software.

Ron Patton is a software consultant living in Washington State. His software test experience is wide and varied from mission critical systems to painting programs for kids. In 1992 he joined Microsoft as a Software Test Lead in the Systems Group for Multimedia Viewer, the authoring tool and multimedia display engine used by Encarta, Ci...
Title:Software TestingFormat:PaperbackDimensions:416 pages, 9.15 × 7.35 × 1.1 inPublished:July 26, 2005Publisher:Pearson EducationLanguage:English

The following ISBNs are associated with this title:

ISBN - 10:0672327988

ISBN - 13:9780672327988


Read from the Book

Introduction It seems as though each day there's yet another news story about a computer software problem or security breach: a bank reporting incorrect account balances, a Mars lander lost in space, a grocery store scanner charging too much for bananas, or a hacker gaining access to millions of credit card numbers. Why does this happen? Can't computer programmers figure out ways to make software just plain work? Unfortunately, no. As software gets more complex, gains more features, and is more interconnected, it becomes more and more difficult—actually, mathematically impossible—to create a glitch-free program. Despite how competent the programmers are and how much care is taken, there will always be software problems. This is where software testing comes in. We've all found those little Inspector 12 tags in the pockets of our new clothes. Well, software has Inspector 12s, too. Most large software companies are so committed to quality they have one or more testers for each programmer. These jobs span the software spectrum from computer games to factory automation to business applications. This book, Software Testing, will introduce you to the basics of software testing, teaching you not just the fundamental technical skills but also the supporting skills necessary to become a successful software tester. You will learn how to immediately find problems in any computer program, how to plan an effective test approach, how to clearly report your findings, and how to tell when your software is ready for release. About the Second Edition When I wrote the first edition of Software Testing, software security issues were just beginning to make the headlines. Hackers and security problems had always been a problem, but with the interconnectivity explosion that was about to occur, few in the industry could predict the impact that security bugs would have on developers and users of computer software. In this second edition I've revisited every chapter to emphasize software security issues and point out how the basic testing techniques covered throughout the book can be used to prevent, find, and fix them. I've also added a chapter that specifically addresses how to test for software security bugs. If you're a reader of the first edition, you know that no matter what you do, your software will still be released with bugs. As you'll learn in the second edition, this axiom still holds true—even for security problems. However, by applying the lessons taught in this book you'll go a long way towards assuring that the most important bugs don't slip through and that your team will create the highest quality and most secure software possible. Who Should Use This Book? This book is written for three different groups of people: Students or computer hobbyists interested in software testing as a full-time job, internship, or co-op. Read this book before your interview or before your first day on the job to really impress your new boss. Career changers wanting to move from their field of expertise into the software industry. There are lots of opportunities for non-software experts to apply their knowledge to software testing. For example, a flight instructor could test a flight simulator game, an accountant could test tax preparation software, or a teacher could test a new child education program. Programmers, software project managers, and other people who make up a software development team who want to improve their knowledge and understanding of what software testing is all about. What This Book Will Do for You In this book you will learn something about nearly every aspect of software testing: How software testing fits into the software development process Basic and advanced software testing techniques Applying testing skills to common testing tasks Improving test efficiency with automation Planning and documenting your test effort Effectively reporting the problems you find Measuring your test effort and your product's progress Knowing the difference between testing and quality assurance Finding a job as a software tester Software Necessary to Use This Book The methods presented in this book are generic and can be applied to testing any type of computer software. But, to make the examples familiar and usable by most people, they are based on simple programs such as Calculator, Notepad, and WordPad included with Windows XP and Windows NT/2000. Even if you're using a Mac or a PC running Linux or another operating system, you will likely have similar programs available on your computer that you can easily adapt to the text. Be creative! Creativity is one trait of a good software tester. Note - The examples used throughout this book of various applications, software bugs, and software test tools are in no way intended as an endorsement or a disparagement of the software. They're simply used to demonstrate the concepts of software testing. How This Book Is Organized This book is designed to lead you through the essential knowledge and skills necessary to become a good software tester. Software testing is not about banging on the keyboard hoping you'll eventually crash the computer. A great deal of science and engineering is behind it, lots of discipline and planning, and there can be lots of fun, too—as you'll soon see. Part I: The Big Picture The chapters in Part I lay the foundation for this book by showing you how software products are developed and how software testing fits into the overall development process. You'll see the importance of software testing and gain an appreciation for the magnitude of the job. Chapter 1, "Software Testing Background," helps you understand exactly what a software bug is, how serious they can be, and why they occur. You'll learn what your ultimate goal is as a software tester and what traits will help make you a good one. Chapter 2, "The Software Development Process," gives you an overview of how a software product is created in the corporate world. You'll learn what components typically go into software, what types of people contribute to it, and the different process models that can be used. Chapter 3, "The Realities of Software Testing," brings a reality check to how software is developed. You'll see why no matter how hard you try, software can never be perfect. You'll also learn a few fundamental terms and concepts used throughout the rest of this book. Part II: Testing Fundamentals The chapters in Part II teach you the fundamental approaches to software testing. The work of testing software is divided into four basic areas, and you will see the techniques used for each one: Chapter 4, "Examining the Specification," teaches you how to find bugs by carefully inspecting the documentation that describes what the software is intended to do. Chapter 5, "Testing the Software with Blinders On," teaches you the techniques to use for testing software without having access to the code or even knowing how to program. This is the most common type of testing. Chapter 6, "Examining the Code," shows you how to perform detailed analysis of the program's source code to find bugs. You'll learn that you don't have to be an expert programmer to use these techniques. Chapter 7, "Testing the Software with X-Ray Glasses," teaches you how you can improve your testing by leveraging information you gain by reviewing the code or being able to see it execute while you run your tests. Part III: Applying Your Testing Skills The chapters in Part III take the techniques that you learned in Part II and apply them to some real-world scenarios that you'll encounter as a software tester: Chapter 8, "Configuration Testing," teaches you how to organize and perform software testing on different hardware configurations and platforms. Chapter 9, "Compatibility Testing," teaches you how to test for issues with different software applications and operating systems interacting with each other. Chapter 10, "Foreign-Language Testing," shows you that a whole world of software is out there and that it's important to test for the special problems that can arise when software is translated into other languages. Chapter 11, "Usability Testing," teaches you how to apply your testing skills when checking a software application's user interface and how to assure that your software is accessible to the disabled. Chapter 12, "Testing the Documentation," explains how to examine the software's documentation such as help files, user manuals, even the marketing material, for bugs. Chapter 13, "Testing for Software Security," shows you how to find bugs that allow hackers to gain access to (supposedly) secure computer systems and data. Chapter 14, "Website Testing," takes everything you've learned so far and applies it to a present-day situation. You'll see how something as simple as testing a website can encompass nearly all aspects of software testing. Part IV: Supplementing Your Testing The chapters in Part IV show you how to improve your test coverage and capability by leveraging both technology and people to perform your testing more efficiently and effectively: Chapter 15, "Automated Testing and Test Tools," explains how you can use computers and software to test other software. You'll learn several different methods for automating your tests and using tools. You'll also learn why using technology isn't foolproof. Chapter 16, "Bug Bashes and Beta Testing," shows you how to use other people to see the software differently and to find bugs that you completely overlooked. Part V: Working with Test Documentation The chapters in Part V cover how software testing is documented so that its plans, bugs, and results can be seen and understood by everyone on the project team: Chapter 17, "Planning Your Test Effort," shows you what goes into creating a test plan for your project. As a new software tester, you likely won't write a test plan from scratch, but it's important to know what's in one and why. Chapter 18, "Writing and Tracking Test Cases," teaches you how to properly document the test cases you develop so that you and other testers can use them. Chapter 19, "Reporting What You Find," teaches you how to tell the world when you find a bug, how to isolate the steps necessary to make it recur, and how to describe it so that others will understand and want to fix it. Chapter 20, "Measuring Your Success," describes various types of data, charts, and graphs used to gauge both your progress and success at testing and your software project's steps toward release. Part VI: The Future The chapters in Part VI explain where the future lies in software testing and set the stage for your career: Chapter 21, "Software Quality Assurance," teaches you the big difference between software testing and quality assurance. You'll learn about different software industry goals such as ISO 9000 and the Capabilities Maturity Model and what it takes to achieve them. Chapter 22, "Your Career as a Software Tester," gives you that kick in the behind to go out and be a software tester. You'll learn what types of jobs are available and where to look for them. You'll also find many pointers to more information. Appendix Each chapter in this book ends with a short quiz where you can try out the testing concepts that you learn. The answers appear in Appendix A, "Answers to Quiz Questions." Conventions Used in This Book This book uses several common conventions to help teach software testing topics. Here's a summary of those typographical conventions: New terms are emphasized in italics the first time they are used. Commands and computer output appear in a special monospaced font. Words you type appear in a monospaced bold font. In addition to typographical conventions, the following special elements are included to set off different types of information to make them easily recognizable. Note - Special notes augment the material you read in each chapter. These notes clarify concepts and procedures. Tip - You'll find various tips that offer shortcuts and solutions to common problems. Reminder - Reminders refer to concepts discussed in previous chapters to help refresh your memory and reinforce important concepts. © Copyright Pearson Education. All rights reserved.

Table of Contents


    About the Second Edition

    Who Should Use This Book?

    What This Book Will Do for You

    Software Necessary to Use This Book

    How This Book Is Organized

      Part I: The Big Picture

      Part II: Testing Fundamentals

      Part III: Applying Your Testing Skills

      Part IV: Supplementing Your Testing

      Part V: Working with Test Documentation

      Part VI: The Future


    Conventions Used in This Book


1. Software Testing Background.

    Infamous Software Error Case Studies

      Disney's Lion King, 1994-1995

      Intel Pentium Floating-Point Division Bug, 1994

      NASA Mars Polar Lander, 1999

      Patriot Missile Defense System, 1991

      The Y2K (Year 2000) Bug, circa 1974

      Dangerous Viewing Ahead, 2004

    What Is a Bug?

      Terms for Software Failures

      Software Bug: A Formal Definition

    Why Do Bugs Occur?

    The Cost of Bugs

    What Exactly Does a Software Tester Do?

    What Makes a Good Software Tester?



2. The Software Development Process.

    Product Components

      What Effort Goes Into a Software Product?

      What Parts Make Up a Software Product?

    Software Project Staff

    Software Development Lifecycle Models

      Big-Bang Model

      Code-and-Fix Model

      Waterfall Model

      Spiral Model



3. The Realities of Software Testing.

    Testing Axioms

      It's Impossible to Test a Program Completely

      Software Testing Is a Risk-Based Exercise

      Testing Can't Show That Bugs Don't Exist

      The More Bugs You Find, the More Bugs There Are

      The Pesticide Paradox

      Not All the Bugs You Find Will Be Fixed

      When a Bug's a Bug Is Difficult to Say

      Product Specifications Are Never Final

      Software Testers Aren't the Most Popular Members of a Project Team

      Software Testing Is a Disciplined Technical Profession

    Software Testing Terms and Definitions

      Precision and Accuracy

      Verification and Validation

      Quality and Reliability

      Testing and Quality Assurance (QA)




4. Examining the Specification.

    Getting Started

      Black-Box and White-Box Testing

      Static and Dynamic Testing

      Static Black-Box Testing: Testing the Specification

    Performing a High-Level Review of the Specification

      Pretend to Be the Customer

      Research Existing Standards and Guidelines

      Review and Test Similar Software

    Low-Level Specification Test Techniques

      Specification Attributes Checklist

      Specification Terminology Checklist



5. Testing the Software with Blinders On.

    Dynamic Black-Box Testing: Testing the Software While Blindfolded

    Test-to-Pass and Test-to-Fail

    Equivalence Partitioning

    Data Testing

      Boundary Conditions

      Sub-Boundary Conditions

      Default, Empty, Blank, Null, Zero, and None

      Invalid, Wrong, Incorrect, and Garbage Data

    State Testing

      Testing the Software's Logic Flow

      Testing States to Fail

    Other Black-Box Test Techniques

      Behave Like a Dumb User

      Look for Bugs Where You've Already Found Them

      Think like a Hacker

      Follow Experience, Intuition, and Hunches



6. Examining the Code.

    Static White-Box Testing: Examining the Design and Code

    Formal Reviews

      Peer Reviews



    Coding Standards and Guidelines

      Examples of Programming Standards and Guidelines

      Obtaining Standards

    Generic Code Review Checklist

      Data Reference Errors

      Data Declaration Errors

      Computation Errors

      Comparison Errors

      Control Flow Errors

      Subroutine Parameter Errors

      Input/Output Errors

      Other Checks



7. Testing the Software with X-Ray Glasses.

    Dynamic White-Box Testing

    Dynamic White-Box Testing Versus Debugging

    Testing the Pieces

      Unit and Integration Testing

      An Example of Module Testing

    Data Coverage

      Data Flow


      Formulas and Equations

      Error Forcing

    Code Coverage

      Program Statement and Line Coverage

      Branch Coverage

      Condition Coverage




8. Configuration Testing.

    An Overview of Configuration Testing

      Isolating Configuration Bugs

      Sizing Up the Job

    Approaching the Task

      Decide the Types of Hardware You'll Need

      Decide What Hardware Brands, Models, and Device Drivers Are Available

      Decide Which Hardware Features, Modes, and Options Are Possible

      Pare Down the Identified Hardware Configurations to a Manageable Set

      Identify Your Software's Unique Features That Work with the Hardware Configurations

      Design the Test Cases to Run on Each Configuration    136

      Execute the Tests on Each Configuration

      Rerun the Tests Until the Results Satisfy Your Team

    Obtaining the Hardware

    Identifying Hardware Standards

    Configuration Testing Other Hardware



9. Compatibility Testing.

    Compatibility Testing Overview

    Platform and Application Versions

      Backward and Forward Compatibility

      The Impact of Testing Multiple Versions

    Standards and Guidelines

      High-Level Standards and Guidelines

      Low-Level Standards and Guidelines

    Data Sharing Compatibility



10. Foreign-Language Testing.

    Making the Words and Pictures Make Sense

    Translation Issues

      Text Expansion

      ASCII, DBCS, and Unicode

      Hot Keys and Shortcuts

      Extended Characters

      Computations on Characters

      Reading Left to Right and Right to Left

      Text in Graphics

      Keep the Text out of the Code

    Localization Issues


      Data Formats

    Configuration and Compatibility Issues

      Foreign Platform Configurations

      Data Compatibility

    How Much Should You Test?



11. Usability Testing.

    User Interface Testing

    What Makes a Good UI?

      Follows Standards and Guidelines







    Testing for the Disabled: Accessibility Testing

      Legal Requirements

      Accessibility Features in Software



12. Testing the Documentation.

    Types of Software Documentation

    The Importance of Documentation Testing

    What to Look for When Reviewing Documentation

    The Realities of Documentation Testing



13. Testing for Software Security.

    WarGames-the Movie

    Understanding the Motivation

    Threat Modeling

    Is Software Security a Feature? Is Security Vulnerability a Bug?

    Understanding the Buffer Overrun

    Using Safe String Functions

    Computer Forensics



14. Website Testing.

    Web Page Fundamentals

    Black-Box Testin





      Objects and Other Simple Miscellaneous Functionality

    Gray-Box Testing

    White-Box Testing

    Configuration and Compatibility Testing

    Usability Testing

    Introducing Automation




15. Automated Testing and Test Tools.

    The Benefits of Automation and Tools

    Test Tools

      Viewers and Monitors



      Stress and Load Tools

      Interference Injectors and Noise Generators

      Analysis Tools

    Software Test Automation

      Macro Recording and Playback

      Programmed Macros

      Fully Programmable Automated Testing Tools

    Random Testing: Monkeys and Gorillas

      Dumb Monkeys

      Semi-Smart Monkeys

      Smart Monkeys

    Realities of Using Test Tools and Automation



16. Bug Bashes and Beta Testing.

    Having Other People Test Your Software

    Test Sharing

    Beta Testing

    Outsourcing Your Testing




17. Planning Your Test Effort.

    The Goal of Test Planning

    Test Planning Topics

      High-Level Expectations

      People, Places, and Things


      Inter-Group Responsibilities

      What Will and Won't Be Tested

      Test Phases

      Test Strategy

      Resource Requirements

      Tester Assignments

      Test Schedule

      Test Cases

      Bug Reporting

      Metrics and Statistics

      Risks and Issues



18. Writing and Tracking Test Cases.

    The Goals of Test Case Planning

    Test Case Planning Overview

      Test Design

      Test Cases

      Test Procedures

    Test Case Organization and Tracking



19. Reporting What You Find.

    Getting Your Bugs Fixed

    Isolating and Reproducing Bugs

    Not All Bugs Are Created Equal

    A Bug's Life Cycle

    Bug-Tracking Systems

      The Standard: The Test Incident Report

      Manual Bug Reporting and Tracking

      Automated Bug Reporting and Tracking



20. Measuring Your Success.

    Using the Information in the Bug Tracking Database

    Metrics That You'll Use in Your Daily Testing

    Common Project-Level Metrics




21. Software Quality Assurance.

    Quality Is Free

    Testing and Quality Assurance in the Workplace

      Software Testing

      Quality Assurance

      Other Names for Software Testing Groups

    Test Management and Organizational Structures

    Capability Maturity Model (CMM)

    ISO 9000



22. Your Career as a Software Tester.

    Your Job as a Software Tester

    Finding a Software Testing Position

    Gaining Hands-On Experience

    Formal Training Opportunities


    Professional Organizations Dedicated to Software or Software Quality

    Further Reading




A. Answers to Quiz Questions.

    Chapter 1

    Chapter 2

    Chapter 3

    Chapter 4

    Chapter 5

    Chapter 6

    Chapter 7

    Chapter 8

    Chapter 9

    Chapter 10

    Chapter 11

    Chapter 12

    Chapter 13

    Chapter 14

    Chapter 15

    Chapter 16

    Chapter 17

    Chapter 18

    Chapter 19

    Chapter 20

    Chapter 21

    Chapter 22