Topic: How CMM Level 5 Improves ...

Critics that say reaching Level 5 of Carnegie Mellon's Software Engineering Institute (SEI) Capability Maturity Model (CMM) has little value in the real world overlook the very tangible benefits that can been achieved through consistent use of sound software engineering development practices throughout a software development organization. A continuous emphasis on process improvement and defect reduction avoids process stagnancy or degeneration and ensures continual improvement, translating into improved productivity, reduced defect leakage and greater timeliness. Tracing requirements across each development phase improves the completeness of software, reduces rework and simplifies maintenance. Verification and validation activities are planned and executed to reduce defect leakage. Customers have access to the project plan, receive regular status reports and their feedback is sought and used for process tuning.

The CMM is a framework that describes the key elements of an effective software process. It defines an evolutionary improvement path from an ad hoc, immature process to a mature, disciplined process. The CMM covers key practices for planning, engineering and managing software development and maintenance that improve the ability of organizations to meet goals for cost, schedule, functionality and product quality. At Level 5, the highest level defined by the CMM, the entire organization is focused on continuous process improvement. The root cause of defects is identified. Corrective and preventive actions are determined and carried out. Goals for the reduction in defect density are set, actions are planned to achieve the goal and the outcome is compared with the goal to determine the effectiveness of the actions. Risks are evaluated during the life of the project and actions are taken to mitigate them.

Example illustrates Level 5 advantages
Perhaps the best way to illustrate the advantages of the processes required for CMM Level 5 assessment is to describe how they positively impacted a recent project. The project, for a major United States bank, was designed to detect check kiting. Check kiting is a fraudulent practice where a customer obtains an unauthorized, non-interest paying loan by taking advantage of time delays between the time when the deposited funds are collected and when those funds are made available for withdrawal. Tasks for the project were allocated to the offshore development team members using the automated process tracking system. Senior analysts conducted reviews of specifications, coding and unit testing. Defects encountered during the review process were captured in a defect register. On a weekly basis, parameters such as weighted defects, productivity and timeliness were monitored and analyzed by the project manager and compared against the project goals and organization baselines.

In the first month of the project, the in-process weighted defect density was 2.25 per thousand lines of code (KLOC), well above project goals and organization baselines. The root cause of the defects was analyzed with a Pareto chart. The analysis led to the conclusion that most defects were validation errors or assignment errors. The project team decided on the corrective action of enhancing the coding standard checklist to tailor it to the client's standards. It was also decided to conduct business knowledge sessions to help the team learn more about the client's processes. The next month, the process weight defect density dropped to 0.79 per KLOC, a substantial improvement but still higher than the organization level baselines. Further corrective actions were implemented, including conducting self-review sessions to help improve coding clarity and additional reviews by senior staff members. The following month, the weighted defect density dropped to 0.38 per KLOC, well within the project goals and organization baselines. The defect level remained at this low level until the project was completed. In fact, defect leakage to the customer was zero throughout the entire project and the customer was extremely satisfied with the quality of the deliverables.

This project illustrates how CMM Level 5 methods can identify weaknesses and strengthen the process proactively. Customers gain numerous benefits by working with service providers that have been assessed at this level. The fit of the software to the requirements of the project is continually monitored to assure its completeness. The quality of the software, both in terms of correctness and its susceptibility to failure, is very high. The customer is assured of timely delivery of their project at a carefully controlled cost. The usability and maintainability of the code are considered from the earliest stages.

What to expect on the road to Level 5
Typically, achieving Level 5 assessment will require additional process improvements; enhancement or development of a project tracking system; establishment of business process capability baselines; quantitative control of processes; and implementation of a process improvement, change management and defects prevention systems.

For most organizations, one of the most challenging aspects of the process of preparing for the Level 5 assessment is getting everyone in the organization involved in the process improvement journey. You must work very hard to create an environment that encourages everyone to work on improving the process as well as the project at hand, as opposed to a single-minded focus on getting the job out the door. Another challenge is assembling complete process data, particularly defect data and time sheets. And, after the data is assembled, analyzing the metrics and weeding out abnormal data due to special causes presented significant obstacles.

But it is well worth the trip. Syntel regularly uses the lessons learned on successful projects to identify innovations and transfer them throughout the organization. Since its assessment, Syntel has reduced defect leakage by 26 percent, improved service level adherence by 15 percent, and improved schedule adherence by 11.5 percent compared to CMM Level 3 processes. Most important, its performance has consistently exceeded the expectations of customers.