By Daniel Shoemaker and Carol Woody


Abstract.

Non-technical decisions made in policy, acquisition, governance, resources, processes, and every other aspect of managing software have a direct impact on the resulting operational security. However, these relationships are hidden because the structures we use to govern and organize software do not highlight the security decisions made throughout the life cycle and connect them to the ultimate results. As a result of this obscurity, seemingly appropriate choices result in unacceptable operational security risks because none of the participants recognize the cause and effect linkages.

The Importance of Contextual Factors

Software security is a critical topic. That is because a single key flaw in a major component can bring down the entire infrastructure. But when we think of software and its security we generally think about it in terms of the software engineering technology, people and processes that are involved in its production and sustainment. We rarely think of the development and maintenance of a system or software artifact within the larger context of how it was acquired, resourced, evolved, or managed. Nonetheless, those larger issues have significant real impacts on the security of every system.

It is reasonable to view software security issues strictly through the lens of the software engineering process. The discrete activities of the development and maintenance phases are well-understood and represent the orthodox understanding of the software industry. They are also the places where flaws are actually created. The activities of global product acquisition, strategic planning, resourcing, and overall business process alignment all take place in the realm of the business outside of the traditional lifecycle. As a result, the relationship between those activities and defects in code are less clear.

Nonetheless, the specific context in which a system is resourced, overseen, managed and assured will have a lot to do with how successfully it performs in actual practice. Software is only as good as the people, processes and tools that produce it. The criticality of the development context as introduced by Watts Humphrey in his 1989 publication [1] and further described as a key aspect of the software engineering discipline in his later publication [2] should be considered a fundamental tenet of the software engineering profession. That necessary coordination of context requires a perspective that is more in the realm of business strategy than it is technological. And in that regard, if there are disconnects between the technical processes and the relevant elements of the business operation there is a potential for the injection of serious exploitable vulnerabilities in the actual product.

It is well documented that coherent, well-defined and effective strategic processes are a factor in the production of software and systems. The Software Engineering Institute (SEI) created model after model in the 1990s to underwrite capability [3,4,5] as a critical element of effective software production. Likewise, SEI currently utilizes three different large-scale approaches; two which characterize the capability maturity of the overall process [6,7]; as well as another to describe the strategic maturity of services [8]. That is not to mention the various models of process capability that have been produced by the people at ISO [9,10].

This body of evidence leads us back to the initial premise, which is that the larger context is going to directly impact the correctness, and by implication the security, of every system, or software product. Greater attention has to be paid to contextual factors, which have been largely ignored in the debate about “why Johnny can’t code.”

Security Impacts of Contextual Factors

Attention has to be given to the impact of business strategy, organizational controls, business process alignment and strategic resourcing decisions on the resulting software and its security.

Business Strategy

The elements of business strategy that impact software and system security include such items as acquisition outsourcing decisions. But they also comprise decisions about organizational development strategy, staffing and training policy.

From a security perspective, the most important choice that an organization is likely to make is the buy-versus-make decision. Many business advantages are associated with commercial-off-the-shelf (COTS) products [11]. Nevertheless, the General Accounting Office lists five areas of high risk in a COTS strategy [12]. Those are: 1) malware, 2) counterfeits, 3) supply chain breakdowns, 4) supplier incapability, and 5) software defects. All of those areas can introduce major security concerns into the organization’s electronic infrastructure. Consequently, an intelligent supply chain risk management strategy is an essential component of good system and software security practice.

In addition to supply chains there are the risks associated with the organization’s acquisition process itself including specification failures, changing requirements, time and cost over-runs due to lack of control of the process, and insecure product selection [13]. All of these potential issues have to be factored into the outcome of the acquisition process. And very few of these difference-making factors are seen as being a consideration of the acquisition strategy itself. The acquisition strategy chosen to divide system components among several vendors, mandate a reliance on small businesses, rely on commercial off the shelf products (COTS), include open source (or not), purchase from foreign vendors, rely on legacy, and leverage existing hardware infrastructure just to mention a few of the options each contribute to the resulting attack surface, interfaces, and operational security capabilities of the resulting implementation.

These all have real consequences with profound security implications. And thus they are all valid areas of long-term security concern. An organization that allows promiscuous, or unmanaged outsourcing activity is literally playing Russian roulette with the likelihood of system, or software exploitation. Accordingly the points of failure associated with system and software acquisition all have to be thought about and dealt with as part of the overall process of ensuring the integrity and trustworthiness of an organization’s electronic infrastructure [14].

The actual technical work also requires a larger perspective. Organizational culture is shaped by strategic policies, which are explicitly defined by the organization in order to ensure the proper level of employee motivation, discipline and training. The Software Engineering Institute issued a model focused entirely on the importance of people to the software development effort [4].

The requisite motivation, discipline and level of skill have to be explicitly fitted to the overall organizational mission and maintained as such. That includes documenting the performance expectations for the work to be done in order to ensure clarity, the promulgation of those expectations to all members of the organization in order to ensure common understanding, and the monitoring of employee performance in order to ensure effectiveness. Anybody who thinks that the culture of the organization doesn’t impact the performance of the work has not been keeping up with the literature [15, 19, 20 to cite a few].

The large-scale effort that is required to acquire, implement, and sustain organizational technology requires deliberate and well-coordinated strategic planning and implementation tasks in order to ensure effectiveness. However, both the technical and the business side of the organization often fail to understand the need to actively tailor those tasks to each organizational application. That lack of understanding is unfortunate since, without such a sustainable strategy it is highly unlikely that the employees of the organization will be properly motivated to produce artifacts at the requisite level of effectiveness and security.

Controls

The concept of business management controls might seem far removed from the issue of security flaws in software. But most of the activities that go into the production of a software system have to be overseen and controlled in some formal fashion [16]. Otherwise the organization runs the risk of important decisions being made at the lowest and most inappropriate levels of the organization [3].

Some form of formal governance control is necessary in order to ensure proper and adequate system assurance. Because of the high degree of skill and specialization required, details about software and systems are largely invisible to anybody above the basic technical level. The organization uses governance control mechanisms to influence and direct the way in which code is produced, maintained, and applied. While this has been the case since the beginning of the profession, the increased impact of technology within the functioning of the organization has greatly expanded the importance of these controls.

The problem is that, without specifically designed management controls, which are designed to enforce visibility, the evolution and even use of the system will take place without the direct involvement of most of the organization, specifically all levels of management [15]. The inability to directly oversee technical work forces managers and the organization in general to rely on the capabilities, and even the good will, of employees who have no ability, or reason, to understand the overall strategic goals of the organization, including the desired level of software and system security.

Security controls are built into technical work in the same way that financial controls are built into the accounting process [16]. They must be intentionally designed discrete, systematic behaviors that can measure performance and then move the necessary information to the right decision maker at the right time. Controls allow the organization to both understand, as well as control the present status and long-term evolution of their systems. In the realm of software engineering the primary example of such controls would be the formally planned unit and integration tests and reviews that take place during development. Another example would be all of the formal steps taken to ensure proper configuration management [17].

The development and maintenance of systems as a whole has to be carefully coordinated in order to assure against the types of faults that are the basis for most of the exploits listed in the Common Weakness Enumeration (CWE). However, an effective governance system is also necessary to ensure that corrective action for all of identified defects is systematically initiated, overseen and then signed-off on. Thus, it can be shown that a rationally planned, implemented and executed governance control system is an important aspect of secure code.

Nevertheless the design and implementation of those controls is often left in the hands of business managers, who are no more knowledgeable about the software engineering process than software engineers are about financial accounting. To counteract this separation of knowledge, the organization as a whole has to make a concerted team effort to come up with meaningful and effective controls. This process does not take place by accident. It has to be planned and implemented as part of an overall software security effort. In that respect, control design and implementation is as much a part of the overall assurance process as static tests [21].

Reliance on incremental development places an even greater burden on these management decisions that directly impact operational security. Who will be making the determination as to when the operational security is sufficient to justify operational deployment and on what basis will they make their decisions? Planning for operational security must be included from the start [29].

Alignment

Proper business process to system alignment is a function of broad scale strategic management [21]. In essence, proper alignment ensures that all software and systems interact optimally with each other and all of the communities of interest that use them. The aim is to produce optimum performance and value for the organization [21, 22].

The aim of strategic alignment is to find the best top-down fit of all of the well-defined lifecycle primary, supporting, ancillary and management processes that are involved in the production and sustainment of software. Alignment is accomplished using explicit process engineering methods best characterized by the ISO 12207-2008 model [26]. This is a very high level and concept based design exercise, with a concrete outcome in the form of a lifecycle infrastructure fitted to the specific environment of that particular organization.

Nevertheless, there are distinct payoffs for proper alignment in the system and software assurance universe. The need to maintain clear and robust linkages between systems ensures against attacks on the interface between systems and users. As a result, strategic alignment becomes a crucial issue in the maintenance of suitable software security. Consideration of alignment on the user interface can also ensure against social engineering scams. Those types of attacks are common methods for exploitation of gaps and misalignments in system operation [23].

The activities that ensure proper alignment are a function of two large software engineering processes, software quality assurance, which in this era also implies security assurance, and classic configuration management [24]. Those processes are well-defined in the Software Engineering Body of Knowledge (SWEBOK) and can be customized to any application aimed at maintaining monolithic alignment between the systems and software assets of any organization [25]. The ability to align all system and software assets in optimum operational harmony produces real outcomes. Those outcomes include attack surface reduction, gap assurance, and protection against the injection and over-run exploits that comprise the SANS top 25 list [27].

NIST in the recent release of the special publication NIST SP 800-160 Systems Security Engineering [30] defined alignment of security engineering with systems engineering and directly related the tasks of security to the engineering tasks described in ISO/IEC 15288 [31].

The problem comes from the fact that alignment is a strategic activity that can only be enforced at the top of the organization. The primary concern is that this process is rarely carried out in most businesses simply because C-Level executives see system and software alignment as “technical” work. Nonetheless, anything less than total alignment introduces the prospect of security vulnerabilities and is therefore likely to allow breaches that impact the overall mission of the organization. Thus upper level managers have to specifically authorize and delegate the responsibility for alignment in the same manner as the other major functions of the organization.

This is usually in the form of high-level technical managers with the authority to make strategic decisions about system development and deployment across the organization as a whole. It is necessary to focus that authority into a single coordinating entity in order to ensure uniform development of the larger system. It is also important to centralize authority for alignment into a single place for the purposes of oversight and enforcement.

Resourcing

The strategic business processes that most directly impact the security of systems and software are the resourcing decisions. A product is no better than the sum of the people who make it, the tools they utilize and the environment within which the work is performed. Accordingly, it is in the decisions that provide those resources that risks can be directly weighed and evaluated and eventually either accepted, or mitigated [28].

The primary decision factors in resourcing far predate software and computers. Those are the classic elements of time and money. Decisions like schedule and delivery date impact the time available for assurance as well as the level and degree of inspection and testing. In the larger software engineering sense they also impact the amount of time that can be devoted to getting the specification and design right. Money dictates the number and types of people who can be devoted to a project. Funding impacts the tools available to verify designs and identify and remove defects. It also requires time to learn and apply tools effectively.

Staff capability impacts practically every aspect of the quality and security of the system and software portfolio. Nonetheless, individual staff capability is tied to resource decisions made in the larger sense. Those include such standard resource items as salary and staffing levels, which determines the type and level of talent available to a project. It also includes the general environment and the sophistication of the equipment that is utilized to do the actual work. Poorly staffed and supported software engineering teams are more likely to produce inferior and thus more insecure products.

But resourcing also embraces indirect factors such as whether to outsource. If outsourcing is the approach of choice, then resources determine how rigorously to monitor and control the contractors doing the work. Given the issues discussed earlier that are associated with supply chains, the level and degree of oversight and management of the outsourcing process can determine whether the products that are then integrated into the business operation are secure, or insecure.

Resourcing is often considered in the making of project plans. However it is not clear that resources are ever directly tied to assurance goals in those plans. That is because staff is often described in terms of the roles they fulfill rather than the criticality of the tasks. And the time allotted for project phases is most frequently dictated by the contract with the customer. Therefore it is important to also consider the sensitivity of the tasks themselves when considering how much money to devote to staffing the project. More importantly, it is critical to factor the assurance case into the business plan. That case is what should be a determinant for software engineering factors such as testing time, test sampling methods and most importantly of all the actual period of time that will be devoted to assurance.

Conclusions

More strategic, non-technical factors can have as great an impact on the security of the code as good programming practice. The decision makers involved in business strategy, organizational controls, business process alignment and strategic resourcing decisions must recognize the impact they have on operational security, understand the importance of coordinating their decisions with technology assurance needs, and accept responsibility for their choices. The strategic decisions affecting the processes, people, and tools should be thought about just as carefully and in as detailed a fashion as the specific software engineering tasks. That is not to suggest that we need to ignore secure coding advice and concentrate solely on strategy, alignment and resourcing. What this suggests is that the problem is a complex of small and large factors, all of which have to be considered as a systematic entity in the assurance of systems and software.

Every one of the factors we discussed has concrete consequences in the real-world and therefore it is impractical to expect secure products without effective planning. Organizational context must be included when considering how to create a secure software engineering production and maintenance environment in order to achieve satisfactory assurance.

Disclaimers:

Copyright 2014 Carnegie Mellon University

This material is based upon work funded and supported by the Department of Defense under Contract No. FA8721-05-C-0003 with Carnegie Mellon University for the operation of the Software Engineering Institute, a federally funded research and development center.Any opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the United States Department of Defense.NO WARRANTY. THIS CARNEGIE MELLON UNIVERSITY AND SOFTWARE ENGINEERING INSTITUTE MATERIAL IS FURNISHED ON AN “AS-IS” BASIS. CARNEGIE MELLON UNIVERSITY MAKES NO WARRANTIES OF ANY KIND, EITHER EXPRESSED OR IMPLIED, AS TO ANY MATTER INCLUDING, BUT NOT LIMITED TO, WARRANTY OF FITNESS FOR PURPOSE OR MERCHANTABILITY, EXCLUSIVITY, OR RESULTS OBTAINED FROM USE OF THE MATERIAL. CARNEGIE MELLON UNIVERSITY DOES NOT MAKE ANY WARRANTY OF ANY KIND WITH RESPECT TO FREEDOM FROM PATENT, TRADEMARK, OR COPYRIGHT INFRINGEMENT.


Daniel Shoemaker

Click to view image

Daniel P Shoemaker, Ph.D., is Principal Investigator and Senior Research Scientist at UDM’s Center for Cyber Security and Intelligence Studies. He is also a full time Professor and former Department Chair at University of Detroit Mercy. As the Co-Chair for the, National Workforce Training and Education Initiative he is one of the authors of the DHS Software Assurance Common Body of Knowledge (CBK). He also helped author the DHS IA Essential Body of Knowledge and he serves as a SME for the NIST-NICE workforce framework. Dan’s doctorate is from the University of Michigan and within the State of Michigan he leads the International Cyber-Security Education Coalition. This Coalition covers a five state region with research partners as far away as the United Kingdom. Dan also spends his free time authoring some of the leading books in Cyber Security. His book “Cyber Security: The Essential Body of Knowledge,” is Cengage publishing’s flagship book in the field. His first book, “Information Assurance for the Enterprise,” is McGraw-Hill’s primary textbook in IA and is in use all over the globe. His next book, “Engineering a More Secure Software Organization,” which is also published by Cengage, will be out soon.

E-mail: dan.shoemaker@att.net

Carol Woody

Click to view image

Dr. Carol Woody has been a senior member of the technical staff at the Software Engineering Institute, Carnegie Mellon University since 2001. Currently she is the technical lead of the cyber security engineering team whose research focuses on building capabilities in defining, acquiring, developing, measuring, managing, and sustaining secure software for highly complex networked systems as well as systems of systems.

Carnegie Mellon University

Software Engineering Institute

4500 Fifth Avenue

Pittsburgh, PA 15213
Phone: 412-268-9137
E-mail: cwoody@cert.org

Carol Woody

Click to view image

Dr. Carol Woody has been a senior member of the technical staff at the Software Engineering Institute, Carnegie Mellon University since 2001. Currently she is the technical lead of the cyber security engineering team whose research focuses on building capabilities in defining, acquiring, developing, measuring, managing, and sustaining secure software for highly complex networked systems as well as systems of systems.

Carnegie Mellon University

Software Engineering Institute

4500 Fifth Avenue

Pittsburgh, PA 15213
Phone: 412-268-9137
E-mail: cwoody@cert.org

Carol Woody

Click to view image

Dr. Carol Woody has been a senior member of the technical staff at the Software Engineering Institute, Carnegie Mellon University since 2001. Currently she is the technical lead of the cyber security engineering team whose research focuses on building capabilities in defining, acquiring, developing, measuring, managing, and sustaining secure software for highly complex networked systems as well as systems of systems.

Carnegie Mellon University

Software Engineering Institute

4500 Fifth Avenue

Pittsburgh, PA 15213
Phone: 412-268-9137
E-mail: cwoody@cert.org


« Previous Next »