By Earl Soukup and Dick Carlson


Introduction.

This case study details how a large enterprise-level program used Agile methods, including Scrum, and implemented the capability maturity model integration (CMMI) at the same time. The purpose of the program was to modify a large commercial software application to replace an existing and obsolete enterprise-level data management system. The application of the CMMI to level 3 was a company mandate.

The CMMI is a process improvement training and appraisal program administered by the CMMI Institute, a subsidiary of ISACA, an international professional association focused on IT governance. Often the CMMI is required by many DoD and U.S. government contracts, especially in software development.

The lead author, Dick Carlson, was hired as the program’s Agile coach to transition the program from a failing, document-laden system with high overhead activity to Agile methods, using Scrum. He would also guide the program in meeting CMMI requirements.

The effort proved to be a challenge for Scrum, as it had successfully managed small projects but was not yet known for managing larger programs. The program was expected to continue for another seven years to modify, design and develop major modifications before software improvements were deployed.

The company’s leadership team drove implementation of the large program. Its IT organization used “Macroscope,” an integrated, evolving and adaptable body of knowledge designed to help reduce risks and maximize the value of business transformation and information technology initiatives. However, the misuse of the “Macroscope” model’s implementation led to extensive cost overruns, schedule slippages and large-scale program planning. As a result, it took more than a year to prepare and execute.

Senior management insisted that the program comply with the CMMI, and that its execution activities would align with the organization’s overarching process improvement goals. Management hired Mr. Carlson to guide the program in adopting a hybrid process improvement approach, using Agile methods and aligning with specific CMMI practices.

AUTHOR’S NOTE: The company’s name is protected by a strict code of ethics regarding specific business practices and customer identification. Therefore, references to the company, its customers and the name of any program cannot be disclosed.

The Process

Due to the magnitude of the program, seven discrete project teams were formed. Each team was chartered to a distinct engineering domain, where its relationship with other teams of same or similar domains was critical to developing and implementing the application.

[NOTE: The application’s out-of-the-box functionality did not meet the enterprise’s operational needs. By company mandate, the vendor agreed to modify many of the application’s functions to comply with the enterprise’s operational requirements.]

Project team activities were monitored closely using the Scrum of Scrums technique in scaling multiple teams and large teams. They created a foundational team responsible for establishing critical work areas; fabricating test benches; building initial architectures critical to new and modified application components; establishing, updating and regulating a variety of standards across all teams; and coordinating certification events.

Each project team was assigned a Scrum master and product owner. This was necessary to:

  • Ensure teams remained focused.
  • Make sure requirements were well-defined and developed when they were needed.
  • Help team members remain engaged and productive.
  • Maintain product backlogs, schedules and release plans.

The program had one program manager, three project managers and three program and project engineers. Mr. Carlson was required to report directly to the program director and coordinate closely with the program manager, all project managers, and other key stakeholders.

The program’s product development plans were required to identify the program’s quality requirements, which included:

  • Using Scrum framework for new feature development and a hybrid Agile and waterfall approach for product sustainment.
  • Resolving errors during development sprints.
  • Recording escaped errors (defects), analyzing them for root cause and tracking them to closure.
  • Establishing and maintaining CMMI maturity level 3 practices, as appropriate.
  • Maintaining quality standards for multiple government accreditations.
  • Documenting applicable procedures for implementing Agile methods and verifying through quality checklists, collaborative peer reviews and supplier quality activities.
  • Modifying current processes significantly to accommodate the hybrid methods.
  • Participating in rigorous risk management.
  • Participating in configuration and data management.

Benefits

Mapping the CMMI and Scrum

CMMI practices are designed to provide value across a range of different situations and thus are stated in general terms. Because CMMI does not endorse any particular approach to development, little approach-specific information is provided. Therefore, those who don’t have prior experience implementing CMMI in situations similar to their current situation may find it difficult to interpret the practices.

The following tables identify relevant CMMI practices for complying with maturity level 2 (using text taken from the model). They also show how Scrum practices were implemented in each CMMI specific practice (SP) on the large project. To be appraised at maturity level 2, the Scrum implementation had to show evidence of CMMI practices being performed. A discussion of the program’s CMMI maturity level 3 processes follows later.

CMMI Maturity Level 2 Process Areas

Requirements Management

The purpose of requirements management (REQM) is to manage the project’s products and product components requirements. It also helps to identify inconsistencies between those requirements and the project’s plans and work products. (see Table 1.)

Project Planning

The purpose of project planning (PP) is to establish and maintain plans that define the project’s activities. (See Table 2.)

Project Monitoring and Control

The purpose of project monitoring and control (PMC) is to understand the project’s progress so that appropriate corrective actions can be taken when the project’s performance deviates significantly from the plan. (See Table 3).

Other Level 2 Process Areas

Configuration Management (CM)

CM is not specifically called out in Scrum. However, in an Agile environment it is easy to add a layer of CM to protect work products. Even for groups that like to use white boards, you can be creative and at least establish some basic protection by labeling items (e.g., “V1.1” or “Story dated YYYY/MM/DD”), by taking a photo, or by using other documenting techniques. The CM process area requires more than just versioning, but versioning is a good and fundamental start.

Product and Process Quality Assurance (PPQA)

Some basic PPQA activities are being accomplished naturally when the Scrum master checks that the Scrum process is being followed. Other PPQA activities are completed when a team performs code reviews, document reviews and validation testing. The Scrum master also removes process barriers and inefficiencies. However, Scrum does not specifically call out a level of objective process and product check, nor does it say that particular standards or processes should be defined and used. Therefore, Scrum does not automatically implement PPQA. However, refinements can be implemented to support a PPQA claim.

Supplier Agreement Management

There are no practices in Scrum that deal with selecting and managing suppliers.

Measurement and Analysis

The purpose of measurement and analysis (MA) is to develop and sustain a measurement capability to support management information needs. There are no practices in Scrum that establish a measurement program similar to the expectations of MA. However, the measures in Scrum can be used to implement MA (see Table 4).

Generic Practices

The table below shows the practices that have the clearest relationship to Scrum measurements. Approximately half of the level 2 generic practices of requirements management, project planning, and project monitoring and control are implemented by Scrum (see Table 5 on page 20).

CMMI Maturity Level 3 Compliance Requirements

There are two main areas where Scrum has gaps for level 3 compliance. One area is in the CMMI expectation that project data and lessons learned are shared among projects via a common process asset library (or repository). Another is the expectation that the engineering phases of requirements, design, implementation, verification, integration and validation are well-defined and implement level 3 engineering practices.

These CMMI concepts can be implemented in a Scrum environment, but they do not come with the common Scrum definition. The Scrum Alliance suggests implementing “communities of practice” to reach across teams to share lessons learned and to conduct “retrospectives” within a team. These activities can populate an asset library and thereby record best practices and tailoring guidelines.

Despite the upside potential for a community of practice, there are challenges that have not received sufficient attention from either the business or academic community. If formal, rational structures result when organizations attempt to conform to their understanding (correct or incorrect) of social reality, industries with less formal, less rational structures may have had less time to implement structural conformance. This will likely correlate to industry age.

Communities of practice may benefit from a flatter, horizontally linked organization. For an organization to seriously consider moving from a more vertical to a more horizontal structure, the firm will likely need to reassess its core competencies, capabilities and culture. It should also consider the challenges that may threaten the present continuity in its relationship with all stakeholders in such an undertaking.

Therefore, the following level 3 components are not readily implemented by Scrum without additional work:

  • Organizational process focus (OPF).
  • Organizational process definition (OPD).
  • Organizational training (OT).
  • Product integration (PI).
  • Risk management (RSKM).
  • Requirements development (RD).
  • Technical solution (TS).
  • Some engineering-specific practices (e.g., requirements validation and verification data analysis).
  • Generic goal 3 (i.e., using an organization-wide and tailored process with measurements).

Although program teams used the Scrum of Scrums practice, leaders got waivers against CMMI level 3 requirements. Fulfilling the requirements could compromise the enterprise’s involvement in enterprise-wide appraisals of its activities.

Scrum Implementation Results

Using the Scrum approach to manage all program activities resulted in the following outcomes:

  • High business value was delivered faster (as much as 50 percent improvement in productivity compared to previous Macroscope work efforts).
  • Applying Lean principles helped the company to better understand and implement Scrum practices.
  • Scrum masters documented each team member’s stand-up responses (helpful for root-cause analyses) to the following questions:
    • What have you completed or implemented since the previous stand-up meeting?
    • What are you planning to complete or implement next?
    • Do you have impediments preventing you from completing what you plan to do next?
    • Do you have any issues?
  • Impediments were turned into action plans. Core Scrum team members reviewed and determined appropriate actions by either assigning someone to carry out the action or by escalating the action to management.
  • Scrum masters conducted mini retrospectives at the end of each sprint week to capture potentially good ideas before they were forgotten.
  • Relationships with key stakeholders improved, which helped to build trust and grow knowledge.
  • Team members created a Scrum dashboard with all vital information about the team, their agreements, and their commitments. It also included sprint activities and goals, definitions of “done,” tasks, sprint metrics, announcements, action items, parking lot items and retrospective inputs for activities that needed corrective action or improvement.
  • Team members developed a guide for Scrum masters that provided definitive guidance while they performed their duties.
  • Scrum teams developed a sprint collaboration peer review process, used by all teams to document specific actions and comments on reviewed documents, design objects, code and other work products.

The Results

The modified application’s operational success was confirmed in early 2013, and the application was successfully implemented in late 2014. Significant follow-up work continues as the application owner improves the product and organizational changes necessitate revisions.

Case Study Summary

It will take time for an Agile-CMMI hybrid implementation to work effectively across the enterprise, as the CMMI does not prescribe implementation details. A crucial change causes a significant cultural shift. Combining the CMMI and Scrum is not easy, as two cultures must be brought together. Both sides must learn to understand each other and look for all potential connection points. As teams overcome the different cultural aspects, the quest for process improvement and software quality will be combined.

While the team met essentially all of the CMMI level 2 requirements for this program, the analysis of the effort presented in this case study confirmed that several of the CMMI level 3 project procedural requirements were met. To meet these requirements, a company must have defined processes as standard and repeatable business practices. CMMI levels as high as 4 and 5 are only achievable with a consistent, stable and collaborative effort from all stakeholders. They also occur when all levels of management impart their commitment and loyal support.

Figures and Tables:

Table 1. Requirements Management ( Click to view image )

Table 2. Project Planning ( Click to view image )

Table 3. Project Monitoring and Control. ( Click to view image )

Table 4. Measurement and Analysis ( Click to view image )

Table 5. Generic Practices ( Click to view image )


References and Notes

Agile (https://www.agilealliance.org).

CMMI (https://www.sei.cmu.edu/cmmi/).

Macroscope (http://macroscope.ca.fujitsu.com/Products/the-macroscope-methodology.html).

Scrum (https://www.scrumalliance.org).

SUGGESTED READING

Dick Carlson

Click to view image

Dick Carlson has an extensive engineering background that includes many years of practical knowledge and hands-on experience in the implementation and deployment of Agile, Lean, and Scrum values and principles in communications-electronics, and software and systems engineering within the aerospace, DoD, IT, and industry domains. He has developed and actively conducted comprehensive training courses for Scrum Teams, Scrum Masters, Product Owners, project/program managers, customers, executives, organizational leaders, and others interested in learning how to implement and deploy Agile, Lean, and Scrum. Mr. Carlson has a Bachelor of Science degree in Business and Management from the University of Maryland, and is a Certified Scrum Professional, Certified Scrum Master, and Certified Scrum Product Owner.

Agile & Lean Education Associates (ALEA)

http://www.a2zalea.com

info@a2zalea.com

Phone: 714-350-9946

E-mail: dcarlson@iascar.us

Earl Soukup

Click to view image

Mr. Soukup holds a Bachelor’s and Master’s degree in Electrical Engineering, a Juris Doctor in Law, certificates for software development, project management, functional management, systems engineering, and Agile and Lean including being a Certified Scrum Master. He was a development and test engineer, manager, and project manager for both hardware and software. Also he developed and taught courses in mathematics, electronics, ethics, Lean, and Agile. He is an accomplished analyst.

Earl Soukup

Click to view image

Mr. Soukup holds a Bachelor’s and Master’s degree in Electrical Engineering, a Juris Doctor in Law, certificates for software development, project management, functional management, systems engineering, and Agile and Lean including being a Certified Scrum Master. He was a development and test engineer, manager, and project manager for both hardware and software. Also he developed and taught courses in mathematics, electronics, ethics, Lean, and Agile. He is an accomplished analyst.


« Previous Next »