10 CRITERIA FOR SELECTING AGILE (WITH RISKS)
 

1  Customer Participation

An Agile Life Cycle relies on frequent requirements clarification, priority and cost discussions with the customer and is best applied when one or more customer representatives are available for frequent (preferably daily, but at least weekly) interaction with the program teams. The high tempo of an Agile project works best with on-site and involved customer representatives who are knowledgeable in their domain and empowered to make decisions so that team progress is not impeded.   For programs that are developing new hardware or that face substantial engineering challenges, it is often the case that System Engineering are also primary customers that should be included in iteration activities. 

Risks and Mitigation:

Customer representatives not collocated:  Mitigate by daily use of electronic meetings, telecons, shared access to design tools, etc.  Supplement with periodic face-to-face meetings.

Customer unwilling or unable to work closely with program team except at major, formal reviews:  Use Systems Engineering or Business Development as customer surrogates.  Provide demonstrations at major reviews.  Continue to encourage participation when appropriate.  Consider a traditional approach through PDR.

Customer not clear on expectations for participation:  Provide Agile training.  Include customer in retrospective after early iterations.  Identify specific customer points of contact for participation with Agile teams.

Large degree of customer turn-over:  Educate new customers on Agile methods and expected participation.  Monitor for evidence of churn based on personalities versus legitimate changes based on mission needs.

If you are unsure if Agile is for you consider hiring CMMI Ltd or a similar external consultants who can bring experience from many different companies to provide alternative perspectives that may be beneficial.  Whilst it may be prudent not to adopt all of the Agile Methodology picking and mixing techniques almost certainly will add value to any software development environment. 

Dynamism (% Requirements Change)

An Agile Life Cycle is intended to accommodate significant volatility in requirements addition, deletion, modification and prioritization. While an Agile Life Cycle can handle a more stable requirements environment, it offers limited advantage over other more traditional life cycles in very stable requirements environments.  However, the Agile practices such as Test-Driven Development, Re-factoring, Continuous Integration, etc. that lead to higher quality software should be considered.

Risks and Mitigation:

Very little requirements change:  If there is little change in requirements due to lack of customer input, the Agile practices of short iterations, with demonstrations at the end of each iteration, might encourage earlier customer input and avoid changes late in the life cycle.  If the limited change in requirements is because the domain is so well known, an Agile Life Cycle may not be appropriate, although selected Agile practices such as test-driven development may be helpful.

Requirements thrashing:  While requirements change is usually based on evolving understanding of customer needs or changes in the environment, there may be cases where problems with customer participation may drive thrashing.  Programs need to have measures in place to identify changes that are not associated with progress. Ideally requirements thrashing is not taking place within the iteration. Thrashing may indicate a need to reduce the iteration cycle time or could indicate a lack of understanding by customers or team members as to their role in the process and the impact that thrashing has on an iteration.

Culture

An Agile Life Cycle and many of the Agile practices involve an adaptive approach to development and are best matched to teams that are comfortable with change in many forms – especially changing priorities, requirements and designs as the program evolves.

Risks and Mitigation:

Training and experience: Resistance to adopting Agile practices may be indicative of a lack of Agile training or experience among the technical leadership.  Introducing an experienced Agile team member or acquiring Agile training can go a long ways towards improving Agile adoption.  Also consider hiring external consultants who bring experience from many different companies to provide alternative perspectives that may be beneficial in reducing resistance.

Culture does not accept change easily:  Mitigate by focusing first on early adopters and others who are less resistant to change.  Leverage small successes to reduce resistance.  Provide training, coaching and consulting to relieve fears that cause resistance.  See Fearless Change by Manns and Rising for strategies to use to introduce change into an organization.

Culture of blame:  Adopting Agile, or anything new, in a culture of blame usually results in all problems being attributed to the change.  Gradual adoption of Agile practices may work better in this kind of culture, so that no change is big enough to be the root of all problems.

Individuals do not adapt to change:  Individuals who cannot deal with the change associated with an Agile program might need to be assigned to a more traditional program.  Sometimes resistance to change is an inherent personality trait, while at other times it may derive from an individual’s lack of skills in the technology or domain.

Number of Long Lead Items

While long lead items add complexity to any program, they can cause additional problems for Agile programs if they impede the ability to start development and test early in the life cycle. 

Risks and Mitigation:

Long-lead components:  Involve supply chain management early in the planning cycle, so they understand when specific components are needed.  Use of simulators will add time and money to the effort, but it can minimize the risks introduced by late availability of hardware or software components.  It may be necessary to limit use of Agile to areas where issues associated with long-lead items are not so restricting.

Concurrent hardware and software development:  Use of simulators, as noted above, can mitigate risk.  The teams should also investigate whether iterative development of both hardware and software might be a win-win situation if it enables early software to drive early versions of hardware, and vice versa.  This is one example of a case where an LM engineer may be the primary customer of iteration capabilities.

Number of Personnel/Size of Team

Agile practices substitute close communication for larger amounts of formalized documentation, resulting in a practical limit on individual team size.  Therefore, an Agile Life Cycle and Agile practices are best suited to individual teams of about 10 people. With larger teams, informal communication methods begin to break down and must be supplemented by more formal mechanisms, including the use of multiple teams. 

Risks and Mitigation:

Size of individual team:  Use multiple teams, with each individual team using Agile practices or life cycle.  Team leads should meet regularly as a Team of Teams.  Integration between teams should occur at iteration or cycle boundaries. 

Size of program team:  Use written interim documents to supplement face-to-face communication.  Focus on establishing clear boundaries and interfaces between independent teams. 

Team composition derived from architecture:  Determine team composition based on the architectural partitioning to reduce the need for cross-team communication.

6  Agile Experience

Since Agile is relatively new compared to other methodologies and life cycles, there are relatively few individuals and teams with Agile experience.  This limited experience, combined with the Agile shift in focus from other life cycles and practices, increases risk to the program.

Risks and Mitigation:

Lack of team experience with Agile:  Seed the team with members who have employed Agile practices or life cycle previously. Where that is not possible, take a very aggressive/intensive approach to Agile training and mentoring and allow for more time for the team to become fully productive.  Start with a small number of teams, and expand the scope of Agile over time.

Lack of management experience with Agile:  Provide training and mentoring in Agile management.  Reassign managers who cannot let go of the ‘plan and command’ style of management. Full Spectrum Leadership training as well as training in Scrum Master and Product Owner roles are recommended.

7  Number of Subcontractors

Agile depends on close communication and flexible iteration scheduling to maintain team velocity (momentum) when developing and delivering system features. This is easiest when the team is collocated and when interdependencies of program and subcontractor schedules are minimized.

Risks and Mitigation:

Multiple subcontractors using different methodologies:  In general, risk increases with the number of subcontractors involved, especially if those subcontractors are inflexible by nature of their approach or contract.  Projects that involve significant numbers of subcontractors should consider applying Agile methods on a selective basis and only include subcontractor elements that have a proven ability to apply Agile methods.  On large programs with subcontractors, employing an experienced system architect would also be an effective risk mitigation strategy.

Significant subcontractor dependencies:   When there are significant dependencies on subcontractor components, extra care in synchronizing the sub-contractor and the Agile team Iteration schedules becomes crucial in meeting release dates and contents.  Isolating subcontractors to entire Integrated Product Team(s) may also help the program scheduling and coordination.  There is little additional risk when subcontractors are fully integrated into the iteration teams.

8  Contract

An Agile Life Cycle is geared towards systems with evolving or uncertain/dynamic requirements. Its focus on convergence on a solution is best applied on Level of Effort and Cost Plus types of contracts. 

Risk and Mitigation:

Fixed Price or inflexible contracts that consume significant resources on contract changes:  Mitigate by negotiating contract terms that ease the change process.  Revise contract to clearly distinguish “shalls” and “shoulds”.  If possible, use LOE and Cost Plus contracts that proved more flexibility in accommodating changes.

Contract does not specify customer involvement:  Work with customer to clarify expectations and/or use Systems Engineering as a customer surrogate.

Contract does not specify mechanism for changing requirements:  Negotiate mechanisms for evolving the baseline that welcome change rather than merely control change.  One way to do this is to establish a continuous review board for reviewing in-scope changes.

Contracts requiring traditional life cycle gates such as PDR, CDR:  One option is to adopt Agile later in the lifecycle, and focus on agility in software development rather than agility for the entire product.  With customer approval, Agile development for software could also be done early in the lifecycle in conjunction with more traditional approaches for the entire product.

Criticality

The commercial Agile principles of simple design and minimal documentation are applicable where criticality is constrained to minor loss of funds. Since LM projects generally involve considerable criticality with regard to national interests and have significant security constraints, the LM Agile projects should include additional emphasis on establishing an adequate architecture up front, and should allow additional time prior to release for any safety or security certifications that might be needed.  

Risks and Mitigation:

Critical project, but team is inexperienced with Agile:  Generally it is not a good idea to try any new method or tool on a critical project, and Agile is no exception.  If you are going to use Agile on a critical project, try to do it with a team experienced with using the Agile practices planned for the program.

Significant degree of criticality or certification requirements:  Use a combination of Agile and traditional methods.  Focus on formality and extra documentation where required for life-critical or certifiable portions of the system.  Supplement traditional methods with practices like Test-Driven Development and pair programming to increase reliability of key program features.  Ensure that requirements related to safety and certification are assigned high priorities, and that they are addressed each iteration as appropriate.

10  Personnel Experience

While all project life cycles would benefit from an increased staffing with experienced domain experts, it is more critical to an Agile Life Cycle which compensates for streamlined, minimized documentation by staffing the project with more seasoned individuals. Agile teams should also strive to maintain cross-functional discipline, and expose teams to the entire problem domain, which serves to minimize the risk of loss of an individual team member.

Risks and Mitigation:

Program team lacks domain experience:  For short projects, less than 6-9 months in duration, don’t adopt an Agile approach unless the team members have significant domain experience.  For projects that are longer in duration, use the early iterations to gain domain experience.

Program team lacks experience with the technology:  Use early iterations or prototypes to get the needed experience.  Expect the potential for significant changes once the technology is understood.

References:

Agile Project Management, by Jim Highsmith

Agile Retrospectives:  Making Good Teams Great, by Esther Derby, Diana Larsen and Ken Schwaber