By Bohdan Oppenheim


Abstract.

This paper presents arguments for why defense programs creating physical systems should clearly separate three developmental phases from each other: research, development and design. Research is to be performed first by small teams of scientists addressing the “unknown unknowns” and maturing fundamental science from TRL of 1 to about 3. Next, development of physical modules is to be performed by small and highly specialized engineers. Finally, the system-level design should focus on efficient trading off the module locations, sizes and shapes versus system performance, mass, power requirements, etc. The design with all modules mature and available is equivalent to a car design: to be performed by competent engineers but quite well established. A small cohesive and co-located Program Management team with excellent Systems Engineers and Architects, led by a permanent Program Manager/ Chief Engineer should manage all program phases, assuring smooth transitions between the expert teams and phases. The small weight penalty which may result from the above approach is compensated by orders of magnitude larger savings due to shorter program schedule and optimized engineering effort. Examples are cited.

1. Introduction

This paper addresses one of the most fundamental aspects of waste in many large defense programs creating physical systems: the massive waste of engineering labor and time.

It is useful to first review how efficient modern car development is. A typical new car program rigidly adheres to the following phases: 1) First develop all needed components and subsystems (engines, gearboxes, radios, seats, etc.) based on the latest competitive technology and marketing need, and test and validate them thoroughly, preferably in several combinations of sizes, shapes and features, to the level of maturity such that they will be ready for use in new cars. Once all modules are ready to be integrated, and only then: 2) Perform the car design which is a relatively routine problem of trading off the physical module locations, sizes and shapes to fit the styling envelope, performance requirements, vehicle mass and size, powering tradeoffs, etc. Such a design effort has no unknown unknowns, thus no big risks. While requiring towering engineering competence and experience, it remains a fundamentally engineering design: trade-offs and selections of parameters within finite trade space until all requirements are satisfied and some desired performance optimum is reached. Using this approach Toyota completed the Prius car design with new hybrid modules, in nine months from the end of styling to the beginning of error free production—a feat unmatched by any competitor, faster by a factor of 2 to 3 then the next best in class [1].

In contrast, many large complex defense programs in the last decades are contracted “for the entire job” including concept development; co-mingled research, development and design; starting with numerous low-Technology Readiness level (TRL)1 items. This is usually driven by the perception that cutting-edge technology is more appealing to stakeholders; and the rarely-justified hope that system and technology development can be accomplished in parallel [2].

Starting a large program with very low TRLs and then pursuing research, development and design mixed together under a massive contract is a major source of waste, if not the only one [3]. In effect we pay for a standing army of expensive engineers trying to look busy while small groups of “developmental” engineers frantically try to mature the TRLs. This is illustrated symbolically in Figure 1 A, with the large shaded box symbolizing the entire program effort (or cost) and the four boxes inside it denoting the various inefficient R&D efforts. It is only after the R&D tasks are completed, that the design increases in intensity. A number of aerospace programs notorious for terrible performance followed this pattern, starting with minimal TRL’s of one or two, e.g., NPOES [4] and JSF [2]. Numerous other examples are available on the Government Accountability Office webpages.

What is worse, the work on the low TRLs is often performed by engineers rather than scientists, using brute-force approach of endless and costly iterations rather than elegant and advanced science methods of rapid trade space exploration and set based design (see Section 2). Starting a large system development with low TRLs causes excessive schedules lasting 10-15-20 years—several times too long when compared to equivalent commercial programs, and costing tens or hundreds of billions of dollars—an order of magnitude too much. The real victim is the war fighter who cannot use the system when needed. In addition to huge original budgets, many programs suffer from major cost growth, and some have to be terminated. Overall, the total cost growth of recent poorly performing defense programs was $295 billion. This practice is in violation of the intent of the Defense Acquisition Logistics: [5] which clearly states that system design should start only after Milestone B, that is after all needed TRLs are quite mature and ready for integration. And this is precisely how commercial companies handle the development at a small fraction of the average defense program cost.

The history of defense and NASA programs offers plenty of examples of successful programs that reinforce the proposed approach, as follows. The Manhattan project which was one of the most difficult programs in human civilization had an efficient research phase during which mathematicians performing hand calculations (before computers!) proved that the nuclear chain reaction would not burn the earth’s atmosphere. Once the research was completed, the weapon development and design were completed in weeks. The nuclear submarine project [6] started not with the submarine design but with research on compact nuclear reactors. Once solved efficiently, the development of the nuclear plant and the submarine vessel proceeded predictably and efficiently. The early U.S. space program demonstrated similar advantages [7]. Iridium, one of the technically most successful space programs, is forever a prominent example of technical (if not marketing) efficiency [8].

This article submits that we can adopt a lot of commercial development practices to aerospace programs without sacrificing anything of value, and vastly reduce program schedule and cost, bringing weapons to the war fighter faster and more affordable. The recommended good-sense process is described in Section 2. In Section 3 we discuss the desired management of the entire program, and in Section 4 we identify potential weaknesses and strengths of the approach in the defense environment.

The present approach has been based on several Lean Enablers described in [9, 10] and also listed on the web [11].

2. Ideal Sequence: Research-Development-Design

Occasionally, a set of common words evolve into an idiom which, with frequent use, becomes a paradigm and can be very difficult to eliminate. The words “research and development” seem to be an inseparable pair in this category. This may have been justified in earlier decades of simpler systems. Now when the system complexity is vastly higher, and the research phase needs a dedicated scientific approach, the term has become destructive, costing billions of dollars in inefficient programs. Our task is to clearly untangle three development phases from each other: research using fundamental science, engineering development of modules, and engineering design, as follows (see Fig. 1 B):

The role of research teams is to develop each immature technology to 3 from TRL of 0-1, ending with a demonstration of technical feasibility and validation of the technology. This work phase is driven by global competition: “we need to develop better products, with better technologies all the time”. If the technology is challenging, involving significant unknown unknowns, a cost-plus contract may be justified for this phase. But it is critical that the work be done by a very small team (a few individuals is usually sufficient) of highly competent researchers with doctorates in sciences, the love of learning from scholarly journals, and the inner drive to succeed. Each small team should be contracted independently of others, because their areas of expertise do not usually overlap. These folks are rarely engineers. Aerospace design engineers are not needed on these teams therefore large defense programs cannot be justified for this phase; in fact such programs are the opposite of what is needed here. The teams should be protected from defense bureaucracy that would only slow the progress. Even though this phase may be open-ended and contracted cost-plus, the small size of the team(s) assures a modest budget and good progress. Modern science offers a rich body of knowledge on how to make such open-ended challenges efficient and even predictable, using set-based studies [12] trade space exploration [13], and optimized iterations [14]. Since the expenditures are small, a vast bureaucratic oversight should not be needed. If the teams are properly selected for their towering scientific competence, and not sabotaged by bureaucracy, rapid progress can be achieved in schedules lasting from months to a few years. For example, the research phase of the Manhattan project, one of the most difficult programs ever undertaken, took only one year [15].

Development. For each module under development, if and only if the research phase is successful (having achieved TRL of at least 3), a new contract should then be issued to a small focused team of developmental engineers. These engineers are different from scientists and from design engineers and must not be confused with them. The task for a team of developmental engineers is to mature the given TRL from 3 to the mature validated module of hardware, software or a combination, ready to be integrated into a later design. Since this phase has no unknown unknowns, there is no justification for any cost plus work, and the work should be predictable and plannable, with a fixed price and reasonable schedule. Ideally, each module should be packaged into several shape and size combinations, to make subsequent design(s) easier and to promote reusability. The added cost of multiple packaging is a small fraction of the module development effort but has big payoff due to module reusability. The software should also be created with long-term general reusability in mind. This phase calls for solid skills and specialized experience in designing the given module(s). An expert in physical system design may be needed on each team to formulate requirements for the module, which would be consistent with subsequent system design. The requirements should address environmental constraints, use scenarios, top-level interfaces with other subsystems/modules, top-level tradeoffs, and best applicable standards.

System Design. At this time all mature modules should be available for integration. The remaining system design phase involves “routine” tradeoffs between system performance, mass, strength, size, shape, power, years in service, reliability, etc. This is where we need a broad spectrum of system-level engineers and a “systems engineering factory”. The system design engineers should efficiently tradeoff the above parameters and select and move the modules around until all constraints are satisfied. This work, even though calling for a high caliber engineering competence, is fairly standard; this is what system design engineers do for a living. There should be no unknown unknowns left at this phase. All high-level technical risks should have been handled in the prior research or development phases. As such, a system-level contract must be contracted as fixed price and reasonably priced and scheduled, based more on commercial program estimates than the bloated defense programs of recent years. Any bidding company who says that they cannot bid a reasonable price in this situation, when all modules are already available, and the top-level requirements are stable should be excluded from consideration for incompetence.

Practically all carmakers follow the described research-development-design sequence, with the best in class demonstrating an amazing overall efficiency.

3. Systems Engineering and Architecting, and Program Management

Ideally, the three phases: research, development, and design should be contracted separately, each to the most qualified teams available for the given phase. Yet, there must be an overall management of the program from the beginning to the end. The following approach is recommended, following [9, 10].

From the program inception, there should be a single and small integrated program management team performing technical management (concept development and systems engineering and architecting), as well as business management (project management, risk management, acquisition, contract monitoring, program monitoring, and supporting functions). This should be a small cohesive co-located team handling the entire program from concept development to Milestone A. Next, the management team should contract and manage first the research phase, then the development phase to Milestone B, followed by the design phase and system integration to Milestone C, including system level verification and validation. The program management should also continue into the operational program phases of transition, operations and logistics, and disposal. This management team should be characterized by the following:

• Co-located minimum-size team. All people should be highly experienced in the system domain. The team must have total responsibility, authority, and accountability for both technical and business success of the entire program.

• The contract should call for managing the entire program during the entire lifecycle.

• There must be a single leader (called “Program Manager” or “Chief Engineer”) who is not subject to military rotations, who is the person dedicated to unconditional program success, and who has personal stake in the success (accountability for failure and high reward for success). This excellent leader should be competent in program management, systems engineering, domain engineering.

• Effective team approach: single, co-located, cohesive, and well-integrated team2.

The management team should manage the following phases of the program:

1. Capture stable system-level customer-need requirements and scenarios of operations (the fewer requirements the better). If these top-level requirements are not stable the program must not be allowed to proceed under any circumstances as this will guarantee budget raptures and risk total failure.

2. Perform enough concept development and system architecting to identify all low TRL (high risk) items, and the overall concept configuration. One year is regarded as plenty of time for a competent team to perform a comprehensive concept development and architecting in response to stable and wise top-level requirements. Modern approaches such as Model Based Systems Engineering [16], or Vienna Development Method [17] may be used in this phase, although the actual approach should be left up to the team and the contract should not be too prescriptive; otherwise it may slow the progress and introduce unnecessary bureaucracy.

3. Research contracts: Then, for each low TRL item, issue a Request For Proposal (RFP) and source select a small team, paying attention primarily to the past scholarly successes and credibility of the teams (illustrated in Fig. 1 B symbolically by four small “research boxes” denoting, say, four needed research topics). All such small contracts for maturation of TRLs should be issued in parallel. Large defense contractors are badly suited for this phase as they tend to activate a large “standing army”—precisely what we are trying to avoid. Monitor all projects in this research phase and wait until all low TRLs reach the level of at least 3. If even one research team fails to achieve success do not proceed to the next phase, as this will introduce unacceptable risk to the overall program. Depending on the case, this phase should not last more than one to a few years maximum. The guiding environment should be maximally patterned after best available research studies, e.g. a federal research laboratory, a research university, an FFRDC, DARPA, etc.

4. Development contracts: Once all research teams achieve success (TRL of 3), issue the next phase RFPs in parallel to seek proposals from small expert development teams who can demonstrate past success and current readiness to perform the development of each needed module. Typically, the different modules will use completely different teams as the modules have little in common (the four boxes denoted “development” in Fig. 1 A). Since these teams will not have any unknown unknowns, these contracts must be fixed price, and the price should be guided by best commercial programs, with some reasonable overhead for handling military security and external management.

5. Design phase: when all modules have been developed, verified and validated, and are totally ready for system integration, issue an RFP for a larger contract to perform system design and integration, (denoted as “Design” in Fig. 1 A). This program will need engineers from all domain subsystems, as well as competent system-level engineers representing all relevant branches of engineering. This single contract should have a reasonably short schedule and fixed budget because all modules have been already created. (This is like a car design to use available engines, gearboxes, seats, radios, etc.) This phase should perform formal system-level systems engineering and program management, including integration, verification and validation. This phase is essentially a routine engineering system level design even for a new weapon or space systems, and must be treated as such, rather than as a bloated multi-year full R&D program. There should be practically no development but plenty of best design activities. Contractually, passing the buck between the different parties involved in phases 1-5 must be avoided, demanding that a green light into the next phase is contingent upon the acceptance of the previous phase. Coordination and communication opportunities throughout the program stakeholders and life cycle should be maximized.

6. Keep the contractor in phase (5) and the program manager fully accountable for the entire program technical and business success.

The above approach offers the following significant advantages:

a. Each project in each phase is manned in an optimized way, assigning only the experts and managers needed. We eliminate the “standing army” of thousands of highly paid engineers and managers for many years of “looking busy” while only a few individuals are truly needed. The cost of issuing one massive contract that mixes research, development and design is symbolically illustrated by the shaded area in Fig. 1 A. In contrast, separated and optimized research, development and design are like the small shaded areas in Fig. 1 B. Clearly the cost and time of the latter are significantly smaller than the former.

b. The folks best suited for each phase are used: systems engineers and architects for the concept phase, scholars for the research phase, developmental engineers for the development of modules, and design engineers for the remaining low-risk design phase. We eliminate the present practice of asking engineers to address scholarly challenges for which they are poorly suited and which they attack by massive and costly iterations.

c. Lower risk: the program split into these phases automatically assures healthy milestones. If even one phase fails to deliver, the program can be stopped and the phase re-bid with minimum waste in overall schedule and treasure.

d. The approach is much closer to the well-proven commercial practice, which costs one to two orders of magnitude less than the recent defense programs.

e. The shorter overall schedule is conducive to more stable requirements and the absence of technology changes during the program, the two aspects that have destroyed many a massive long defense program. Of course, the stability of customer-level need and use scenario requirements should be pursued by all means, as unstable requirements can destroy any long program.

4. The Mass Penalty

A careful reader no doubt noticed one technical deficiency of the above approach: namely that the modules predesigned for the design phase have to be used “as is”, even if each is available in several size and shape combinations. The typical argument for contracting the entire program and all of its phases to a single contractor is based on the hypothesis that the contractor can then develop and optimize each module for minimum mass and best system layout. Theoretically there might be a merit in this argument. However, economics destroys it immediately, as follows: engineering labor rather than system weight is the most expensive item in large complex programs. Using pre-designed modules may carry a small weight penalty (which should be small indeed if the teams developing the modules understand the module use in the system of interest – not an unreasonable expectation), perhaps at worst requiring the system to be lifted into space by a slightly larger vehicle than what might be needed otherwise. For example, having to use a larger-size lift vehicle into space may cost an extra $50-$100 million dollars (a generous estimate), while the proposed approach will save billions if not tens of billions of dollars in much shorter program schedules. In addition, the proposed approach delivers the capability to the warfighter years ahead of traditional multi-year programs. It is simply common sense that this is a vastly better approach. Commercial programs understand it very well. Time for defense programs to do the same.

5. Summary

The proposed approach to complex weapon system development is based on clear separation of the program into research, then development, and finally design phases. Each phase should be performed using separate optimum-size teams of specialized experts, all coordinated by an efficient co-located small management team. The approach offers vast improvements over the current practice of one huge all-inclusive program lasting 10 to 20 years, costing a treasure, and wasting up to 90% of the cost or more because most engineers have really little to do most of the time, while a few are frantically trying to mature the TRL of selected modules using brute force iterations. Examples of poorly performing programs that started with low TRL have been cited. Examples have also been provided of successful programs that clearly separated research from development and from design.

The proposed approach has been practiced in the commercial world for tens of years. Thanks to it, we can buy a car for $20,000 rather than the billions it would cost to develop the car using the current defense contracting paradigm. The possible small added cost due to larger weight is compensated by orders of magnitude lower cost of engineering labor. The approach will yield higher affordability and faster availability to the warfighter. The approach is totally consistent with the Integrated Defense Acquisition Technology and Logistics Lifecycle Management Framework. Nothing in the present defense acquisition policy precludes the approach. Even the Program Objective Memorandum budget formulation [18] for defense programs which requires that military services perform program acquisition planning several years in advance could be adopted to handle the proposed program organization. A pilot program is recommended to follow the proposed approach. It has the potential to significantly cut the budget, schedule and bring the needed system into operations in a fraction of the current programs.

Tables and Figures:

Figure 1: Typical Program (A) versus Proposed Program (B) ( Click to view image )


References and Notes

References: 1. M. J. Morgan, and J. K. Liker, Toyota Product Development System, Productivity Press, 2006. 2. GAO, Assessments of Selected Weapon Programs, GAO - 08 - 467SP, 2008 3. A. B. Carter, The Under Secretary of Defense, Acquisition, Technology and Logistics, Memorandum for Acquisition Professionals, June 28, 2010. 4. T. Hall, NPOESS Lessons Evaluation, ATR-2011(5558)-1, The Aerospace Corporation, El Segundo, CA, December 2010 5. Integrated Defense Acquisition Technology and Logistics Lifecycle Management Framework, DAU, , accessed 09-25-2012 6. T. Rockwell, The Rickover Effect: The Inside Story of How Adm. Hyman Rickover Built the Nuclear Navy, John Wiley & Sons; 1995 7. S.B. Johnson, The Secret of Apollo, Systems Management in American and European Space Programs, John Hopkins, New Series in NASA History, 2002. 8. R. Leopold, The Iridium Story: An Engineer’s Eclectic Journey, Minta Martin Lecture, MIT Department of Aeronautics and Astronautics, Apr. 23, 2004. 9. J. Oehmen, Ed., The Guide to Lean Enablers for Managing Engineering Programs, PMI- INCOSE-MIT LAI, 2012 10. B.W. Oppenheim, Lean for Systems Engineering with Lean Enablers for Systems Engineering, John Wiley & Sons, 2011 11. INCOSE LSE WG, Lean Systems Engineering Working Group website, 2012, . 12. D. K. Sobek II, A. C. Ward, and J. K. Liker, Toyota’s Principles of Set - Based Concurrent Engineering, Sloan Management Review, Vol. 40, No. 2, Winter, 1999; pp. 67 - 83. 13. E. M. Murman, Lean Aerospace Engineering, Littlewood Lecture AIAA - 2008 - 4, Jan. 2008. 14. J. Warmkessel, Lean Engineering, Lean Aerospace Initiative, MIT, , 2002. 15. R. Rhodes, The Making of the Atomic Bomb, Simon & Schuster , 1986 16. MBSE, INCOSE, 2012, 17. VDM, 2012, 18. POM: , last accessed Oct. 22, 2012. Notes: 1. For a description of TRLs see , last accessed 9-23-2012. 2. Dividing this effort to more than one company (not unusual in mindless contracting focused on “spreading the wealth”) is just as effective as cutting a person’s brain into pieces, distributing the pieces and asking them to coordinate together- a suicidal proposition for program efficiency. The earlier GPS program suffered from it, burning budget and schedule on Interface Control Documents written by the 45 or so disjointed teams.

Bohdan Oppenheim

Click to view image

Bohdan “Bo” W. Oppenheim is a Professor of Systems Engineering at LMU. He is the founder and Co-Chair of the Lean Systems Engineering Working Group of INCOSE, co-leader of the effort developing Lean Enablers for Systems Engineering, author of Lean for Systems Engineering with Lean Enablers for Systems Engineering (Wiley, 2011) and the second author of the The Guide to Lean Enablers for Managing Engineering Programs (INCOSE, PMI, MIT LAI, 2012) . His engineering degrees include Ph.D., Southampton, U.K.; Naval Architect, MIT; MS, Stevens Institute of Technology; and B.S. (equiv.) from Warsaw University of Technology in Aeronautics. His credits include five books, 20 journal publications, $2.5 million in externally funded grants, and a 30-year industrial and consulting experience spanning naval, space, software and mechanical engineering. He is the recipient of 2011 Shingo Award, 2013 Shingo Award, 2010 INCOSE Best Product Award, 2011 Fulbright Award, and 2008 LACES Best Teacher Award.

Office: 310-338-2825
Home: 310-450-5713
E-mail: bohdan.oppenheim@lmu.edu

Bohdan Oppenheim

Click to view image

Bohdan “Bo” W. Oppenheim is a Professor of Systems Engineering at LMU. He is the founder and Co-Chair of the Lean Systems Engineering Working Group of INCOSE, co-leader of the effort developing Lean Enablers for Systems Engineering, author of Lean for Systems Engineering with Lean Enablers for Systems Engineering (Wiley, 2011) and the second author of the The Guide to Lean Enablers for Managing Engineering Programs (INCOSE, PMI, MIT LAI, 2012) . His engineering degrees include Ph.D., Southampton, U.K.; Naval Architect, MIT; MS, Stevens Institute of Technology; and B.S. (equiv.) from Warsaw University of Technology in Aeronautics. His credits include five books, 20 journal publications, $2.5 million in externally funded grants, and a 30-year industrial and consulting experience spanning naval, space, software and mechanical engineering. He is the recipient of 2011 Shingo Award, 2013 Shingo Award, 2010 INCOSE Best Product Award, 2011 Fulbright Award, and 2008 LACES Best Teacher Award.

Office: 310-338-2825
Home: 310-450-5713
E-mail: bohdan.oppenheim@lmu.edu


« Previous Next »