Study of Adoption of CMM-based SPI

NICTA has published a technical report (PDF here) containing the details of a systematic review of reasons why organizations adopt CMM-based SPI, which I co-authored with Mahmood Niazi. At the recent NICTA Software Engineering breakfast seminars I spoke about some of the results from this systematic review.

Systematic review is a methodology for finding, collating, and analysing all published evidence relating to a particular research question. It’s much more than a typical literature review. It’s meant to avoid bias and to be replicable, by being planned, procedurally documented, open to inspection, and run by multiple researchers. Systematic reviews are among the most compelling kinds of research in evidence-based medicine. Systematic review in software engineering is less well established than in medicine, but over the last year or two it has been generating increasing interest.

Our searches gave us 591 papers which we whittled down to 43 primary studies that described why organizations had originally adopted CMM-based SPI. We grouped and counted responses and found the most common reasons were to improve product quality, performance (i.e. development cost/time, productivity), and for process reasons (process visibility, best practices, process measurement, etc). Companies less frequently mentioned customers or employees as motivating reasons. We also found some evidence to support a claim that there may be widespread bias within the SPI literature that favours process-related organizational motivations over product-related or performance-related motivations.

Comments 1

  • nice one! Though I have studied the systematic review guideline and seen a few semi-systematic reviews, the thoroughness of this one impresses me.

    However, the kind of stuff you can find in a literature seems to have limited the question you can answer…understandable. Iff we have a better reporting guideline on such SPI literature, maybe more interesting questions can be addressed.