The ROI From Software Quality
By Botman, Pieter
The ROI from Software Quality Khaled El Emam. 2005. Auerbach Publications (http://www.crcpress.com). 296 pages. ISBN: 0-8493- 3298-2
GSQE Body of Knowledge areas: Project Management-Cost and Value Data; Project Management – Interpreting and Reporting Cost of Quality Data; Process and Product Measurement
Khaled El Emam opens this book by asserting that “there is a clear and hard-to-dispute benefit to improved software quality.” In a business environment, poor software quality costs the purchaser in many respects (he cites several pointed examples). Software producers and developers, not facing proportionate market or legal pressures, have been “getting away with” poor quality practices, poor economic and management practices, and issuing software products of unknown or uncontrolled quality.
This book is primarily aimed at managers wishing to perform ROI calculations, and to measure the economic factors associated with software quality in a meaningful way. Though the earlier sections discuss the plight of the software purchaser, the majority are written for the software producer, focusing on the management of development processes through measurement and economic analysis.
The opening chapters discuss software quality, its definition, and quality-related practices within the development life cycle. El Emam goes on to broaden the discussion to cover many ROI factors and practices beyond those related strictly to software quality assurance, such as project management, requirements, and process measurement. In subsequent chapters on cost of ownership and cost to developers, the discussion is raised to an economic level, explaining the underlying cost models associated with quality. Throughout these chapters, he cites underlying studies that have contributed hard evidence, and even brings out selected data values from these studies for use in example calculations.
The middle chapters contain explanations of economics concepts (such as “payback period” and “time value of money”), software and quality cost modeling, and the construction of the return on investment (ROI) calculation, and are the heart of this book. They are clear because they are kept separate from lengthy discussions of technical benchmarks and parametric data. Indeed, the material in these chapters will remain valid, while some material presented in earlier chapters will be affected as benchmarking data and software practices change. Readers having any prior exposure to economics and the mathematics of finance will not find these chapters overly complex.
One chapter is devoted to “The Quality of Open Source Software (OSS).” It begins with background material on the notable forces associated with OSS adoption, the open source development process, and large-scale review test cycles. Some studies of complexity and defect levels for certain OSS products are presented, followed by a section on OSS (and non-OSS) security vulnerabilities. The author never misleads the reader, and presents evidence to both support and refute some assertions relating to certain quality aspects of OSS products. His cautious and well-reasoned conclusion is that the adoption of OSS because of vaguely defined OSS process advantages has not yet been adequately justified. This view makes a great deal of sense considering that the range of OSS products is broad (from operating systems to simple character counting utilities), and that the technical factors relating to quality vary significantly over that range. This is a fascinating topic, but the reader may be left wondering whether this large topic would be better (that is, more fully) addressed elsewhere.
The book contains several useful appendices, containing more detailed aspects of models relating defects to effort (cost), and relating the cost of additional inspections to schedule reduction. The reader is referred to an associated Web site for additional material related to software quality ROI calculation.
This book presents a readable, pragmatic yet detailed view of the application of ROI techniques and economic cost models specifically to software development. It touches on many supporting topics related to quality theory, various software engineering practices, and benchmarking data from a wide range of sources, but cannot address them in great depth. All readers in the field of software engineering will appreciate and benefit from the disciplined economic analysis described here. Experienced managers who are already familiar with the fundamentals of software quality and measurement techniques should be able to apply these economic analyses within the context of their own process improvement programs. Readers new to software measurement will gain some insights into software quality factors, but should investigate the listed references in order to learn more about certain practices in detail (such as reliability modeling), and to understand the lack of uniformity across the many software projects contributing to the various benchmark data sets.
El Emam takes a pointed swipe at software engineering management practices within the software industry, saying that important decisions are frequently not evidence based. We have a long way to go in measuring products and processes. But by emphasizing the bottom line-the economic consequences of poor quality products and poor processes-he has given managers and senior engineers a push in the right direction.
Reviewer Pieter Botman
p.botman@ieee.org
Copyright American Society for Quality Mar 2006