By Claude Laporte and Rory O'Connor


Software Process Improvement Standards and Guides for Very Small Organizations An Overview of Eight Implementations

Abstract.

Very small entities (VSEs) — organizations with up to 25 people — are very important to the worldwide economy. The products they develop are often integrated into products made by larger enterprises. However, it has been established that such entities often do not utilize existing best practice standards and frameworks such as ISO/IEC/IEEE 12207 software life cycle processes standard. In addition, small organizations do not usually have the expertise to search for and adapt process improvement best practices from many frameworks to their needs. Finally, these organizations are usually also looking for low-cost evaluation or certification schemes that would provide them with visibility. To address their needs, ISO/IEC 29110 software and systems engineering standards and guides have been developed using elements of published standards. A four-stage road map has been developed to support process improvement activities of VSEs. In this paper, we present eight implementations of ISO/IEC 29110 as an exemplar of the potential benefits from the use of this standard.

Introduction

Software development is a highly complex undertaking. [1] For many small and very small software organizations, implementing controls and structures to properly manage their software development activity is a major challenge. [2] All software companies are not the same and vary according to factors including size, market sector, time in business, management style, product range and geographical location. [3] The fact that all organizations are not the same raises important questions for those who develop software process and process improvement models. To be widely adopted by the software industry, any process or process improvement model should be capable of handling the differences in the operational contexts of the companies making up that industry. But process improvement models, though highly publicized and marketed, are far from being extensively deployed. Their influence in the software industry, particularly for small and very small software companies, therefore remains at more a theoretical than practical level.

Industry, both military and civilian, recognizes the value of VSEs — i.e., enterprises, organizations (e.g., government agencies or not-for-profit organizations), departments or projects with up to 25 people — in contributing valuable products and services. A large majority of enterprises worldwide are VSEs. In addition, a large number of small projects conducted in large organizations are developed with ad hoc processes. More than ever, integrators of military systems depend on their numerous suppliers to deliver subsystems meeting evolving requirements correctly, predictably, rapidly, and cost-effectively. A supply chain of large systems often has a pyramidal structure. If an undetected defect is left in a low-level component, it may remain undetected once this component is integrated in a higher-level component.

International Standards and Guides for VSEs

The recently published set of ISO/IEC 29110 international standards (IS) and technical reports (TR) is aimed at addressing these issues as well as the specific needs of VSEs. The engineering standards and guides developed by an ISO working group, Working Group 24 (WG24), [4] are targeted at VSEs that do not have experience or expertise in selecting, for a specific project, the appropriate processes from life cycle standards such as ISO/IEC/IEEE 12207 [5] or ISO/IEC/IEEE 15288 [6] and tailoring them to the project’s needs.

A core concept at the heart of ISO 29110 is that of “profile groups” [7] that are a set of profiles. The “generic profile group” has been defined as applicable to VSEs that do not develop critical systems or critical software. The Generic Profile Group is a four-stage road map, called profiles, providing a progressive approach to satisfying a vast majority of VSEs. VSEs targeted by the “Entry profile” are VSEs working on small projects (projects that take no more than six person-months’ effort) and startups. The “Basic profile” targets VSEs developing a single application with a single work team. The “Intermediate profile” is targeted at VSEs developing more than one project in parallel with more than one work team. The “Advanced profile” is targeted to VSEs that want to sustain and grow as an independent competitive system and/or software development business. ISO 29110 is intended to be used with any life cycle, such as waterfall, iterative, incremental, evolutionary or Agile.

Table 1 lists the number of processes, tasks, work products and roles of each profile of the ISO 29110 four-stage software engineering road map. A conditional process is a process that may be mandatory under some specified conditions, may be optional under other specified conditions, and may be out of scope or not applicable under other specified conditions. These are to be observed if the specified conditions apply. The “advanced” profile is not yet finalized. Therefore, the information in Table 1 is subject to change.

The software engineering “Basic” profile [8] will be used to illustrate one of the four-stage ISO 29110 road maps.

The Basic Profile for VSEs Developing Software

The software engineering Basic profile [8] defines a VSE-appropriate software implementation (SI) process and a VSE-appropriate project management (PM) process. The main reason for including project management is that the core business of VSEs is software development, and their financial success depends on successful project completion within schedule and on budget (as well as on making a profit). The high-level view and the relationships between the SI processes and the PM processes are illustrated in Figure 1.

Due to space limitation, ISO/IEC 29110 is not described in detail here. ISO/IEC 29110 has been described in more detail in a 2013 “Crosstalk” article, [9] and we suggest readers consult this. The mappings between ISO/IEC 29110 and the best practice standards (e.g., ISO 15288) are listed in an ISO 29110 standard titled “Profile Specifications.” Also, for illustration purposes, each ISO 29110 management and engineering guide lists the elements of best practice standards used.

Assistance with Deploying ISO 29110

A set of deployment packages (DPs) [10] has been developed to define guidelines and explain in more detail the processes defined in the ISO 29110 profiles. This set of DPs has been designed to provide additional guidance to VSEs in their process improvement activities. The elements of a typical DP are: description of processes, activities, tasks, steps, roles, products, templates, checklists, examples, references and mapping to standards and models, and a list of tools. Hence, implementing a DP can help a VSE see its concrete step toward achieving full coverage of a profile.

These DPs mark a significant departure from existing software standards and are specifically designed to ease many of the issues and problems VSEs have with implementing standards on a day-to-day basis, as outlined earlier. These DPs are freely available from [11].

Implementations of ISO 29110

In the following subsections we will present eight implementations of ISO 29110 to illustrate the usage of this standard in a spectrum of organizational settings. Since most VSEs are using ad hoc processes, they either do not collect process measures or their process measures are not reliable. One interesting measure for the management of VSEs is the percentage of the total project effort spent in rework (e.g., correction of defects). Charette [12] reported that software specialists spend about 40 to 50 percent of their time on avoidable rework. Krasner [13] published data showing that CMM level two organizations spent 25 to 50 percent of their time on rework, while CMM level three organizations spend 15 to 25 percent. In the following implementation, we describe in detail the measures collected by one VSE, and we briefly present rework measures of two other VSEs.

Implementation 1: An IT Startup

An implementation project has been conducted in an IT startup VSE by a team of two developers. [14] Their web application allows users to collaborate, share and plan their trips simply. The use of the Basic profile of ISO 29110 has guided the startup to develop an application of high quality while using proven practices of ISO 29110. The total effort of this project was nearly 1,000 hours.

The IT startup has recorded the effort, in person-hours, spent on project tasks. Table 2 shows, for each major task, the effort needed to execute the task, the effort required to detect errors afterward, and the effort required to correct the errors (i.e., the rework).

Only 12.6 percent of total effort has been spent on rework (i.e., 125 hours/990.5 hours). This indicates that the use of appropriate standards can guide all the phases of the development of a product such that the wasted effort (i.e., rework) is about the same in a startup as in a more mature organization.

Implementation 2: A Canadian/Tunisian IT Startup

Metam is a company founded in 2013. The company has one site in Canada and one site in Tunisia. Its business domains are software development services, web solutions, mobile applications, and consulting services to implement ERP solutions. The Basic profile of ISO 29110 was used as the framework for the company’s software processes. [15] It was also used as a foundation to implement CMMI-DEV level two practices because it was requested by some military contracts. In Fall 2016, the 20-employee VSE was successfully audited by a third-party auditor against the ISO 29110 Basic profile.

Implementation 3: A Peruvian IT Startup

An ISO 29110 process improvement project was conducted in a Peruvian IT startup of four people. [16] After completing the implementation of the two processes of the Basic profile using an Agile approach, these processes were executed in a project with the second-largest insurance companies in Peru. Managing the project and developing the software took about 900 hours. The IT startup expended only 18 percent of the total project effort on rework. This startup became the first Peruvian VSE to obtain an ISO 29110 certification of conformity. The ISO 29110 certification greatly facilitated access to new clients and larger projects. In 2016, this VSE had 23 employees.

Implementation 4: A Large Canadian Financial Institution

The IT division of a large financial institution has over 3,000 employees who develop new applications and maintain more than 1,250 applications. The Cash Management IT department, which has six developers, is responsible for the development and maintenance of software tools used by traders. [17] Each year, the department is faced with more requests to add, correct or modify features related to supported applications. Before the implementation of the ISO 29110 Agile process, customers had the following complaints:

  • It is very difficult to know the status of specific requests.
  • Very often, there is an incident when a change is put in production.
  • There are a large number of faults detected by the quality assurance department.
  • The development process is painful and the documentation produced is not very useful.

In response to these problems, processes were evaluated by comparing the activities of the actual maintenance process to those of the Basic profile. Some shortcomings were found in the actual project management process and in the software implementation process. Figure 2 illustrates the coverage of the software implementation tasks to the Basic profile before the process improvement project.

The new project management process has been adapted to the context of the division by injecting a few tasks from the Scrum methodology. The new Agile process, using the Basic profile of the ISO 29110, has been tested on three pilot projects. Recently, a five-person team was added to the TSD department to carry out all non-urgent maintenance projects using the ISO 29110 Agile process.

Implementation 5: An Enterprise in the Automotive Field

TM4 is a Canadian company of 140 people (including 14 software engineers) that designs and sells electric powertrain systems in the automotive field. Their products are embedded software that controls the operation of engines in real time and software that controls the interactions between the components of a vehicle.

The company planned to increase its production systems in the coming years. Before this increase in production, and for the sake of improvement and compliance with standards, the company wanted to review and improve its software development processes. ISO 29110 was used in this effort to improve its processes. A compliance study was conducted to establish the difference between the processes in place and those proposed by the ISO 29110. A pilot project was successfully completed in May 2015. New projects are using the ISO 29110-based processes.

Implementation 6: A Large Utility Provider

The IT division of a large Canadian utility provider has 1,950 employees that support more than 2,100 software applications. The organization had already implemented 12 level two and three process areas of the CMMI-DEV. Traditional life cycles were used for the development of this division.

A small department within the IT division, the Mobility and Georeferenced Solutions Department, is composed of six developers, three analysts, one architect and one manager. Typical projects of the department are requests from internal customers to improve a few applications. The department was required to develop applications more quickly, and with very different technologies. Increasingly, the department had to develop proof of concepts. The problem was that the deliverables requested by the current methodology for typical projects of the IT division were so numerous that the level of documentation required was not suitable for small projects and small teams.

A project was launched within the small department to tailor ISO 29110 to their needs and to adapt it to a Scrum approach. [18] A pilot project was conducted that involved the creation of a web application for property management. This application greatly facilitated geographic data consultation. The total effort of this project was about 1,500 hours. About 8.5 percent of the effort was invested in prevention tasks, while only 9.6 percent was spent in rework tasks. The ISO 29110 process improvement project allowed the small department to shine within the IT division as it became a model for future small IT projects.

Implementation 7: Implementation in a Medical R&D VSE

A project has been conducted to develop and implement a quality management system for a medical R&D company of 15 employees [19]. The VSE manufactures a family of neuronavigation products that are used in over 400 laboratories around the world in the fields of cognitive neuroscience, rehabilitation research and veterinary sciences.

This project improved the business processes and implemented a quality management system in accordance with the ISO 13485 medical standard.

This project used the ISO 29110 systems engineering Basic profile [20] to facilitate the implementation of ISO 13485. ISO 29110 has guided the VSE in the development of tools, guides and templates. During this project, totaling more than 1,000 hours of effort, the implementation of the quality system was planned; processes, guides and templates were defined in collaboration with key resources of the company. A pilot project was conducted to validate the adequacy of the established process.

The use of ISO 29110 systems engineering Basic profile facilitated the implementation and the adaptation of a standard such as ISO 13485 for the VSE.

Implementation 8: A Small Canadian Company in Public Transportation

This project was created to define and implement project management and systems engineering processes at CSinTrans Inc. (CSiT), a Canadian company created in 2011. [21] The company specializes in the integration of interactive systems, communication, and security in the field of public transportation. The recently published ISO 29110 management and engineering guide for systems engineering [20] has been used as the main reference for the development of their processes.

ISO 29110 was a good starting point to align its processes with the CMMI-DEV since conformity to the CMMI-DEV is a requirement from some customers in the transit industry. To better respond to different types of projects, CSiT developed three process groups (i.e., light, standard, and full), each being adapted to meet certain attributes of projects, such as size and type (e.g., prototype, typical project).

ISO 29110 standard has helped raise the maturity of the young organization as the organization has implemented proven practices and developed uniform work products. ISO 29110 has also helped in developing lightweight processes, allowing the young company to remain flexible and maintain its ability to react quickly to its customers. In mid-2016, the systems engineering Basic Profile of the ISO 29110 was successfully audited by a third-party audit team composed of two independent auditors. One member of the audit team was a systems engineering domain expert.

Conclusion and future work

The eight implementations presented in this paper have demonstrated that using ISO/IEC 29110 made it possible to properly plan and execute projects, develop products, and conduct projects using proven system or software engineering practices. This disproves the perception that a process standard interferes with the creativity of software developers. Rework data collected from three of the VSEs (two of which were startups) was presented. This provided some evidence that the implementation of proven practices documented in ISO/IEC 29110 allowed them to execute their projects with a percentage of rework similar to CMMI® level 3 organizations. The relationship between the success of a software company and the software process it utilized has been investigated, showing the need for all organizations — not just VSEs — to pay attention to software process practices such as ISO standards.

The eight implementations described are a small subset of implementations around the world. For instance, in Thailand over 320 private organizations and 15 public organizations have been ISO 29110 certified against the Basic profile, in Mexico 33 organizations have been certified against the Basic profile. [22] Recently, a government agency launched a first program to certify 110 VSEs against the Basic profile.

Since a few ISO 29110 documents, such as the management and engineering (M&E) guides are freely available and have been translated in Czech, French, Portuguese, Spanish and adopted as national standards, the number of implementations should increase. Also, since the ISO 29110 M&E guides are freely available and easily understandable, more than 15 countries are teaching ISO 29110. For instance, in Thailand, 10 universities are teaching ISO 29110.

The software Intermediate profile has been recently published. [23] The Advanced profile should be published in 2018. The development of the systems engineering Intermediate and Advanced M&E guides has started in mid 2017. WG24 has also been mandated to develop an ISO 29110 service delivery guide as well as a standard to help VSEs in providing services delivered to internal or external customers.

Finally, since many VSEs developing systems are also involved in the development of critical software or systems, WG24 will conduct an analysis to determine if a set of ISO 29110 systems/software M&E guides and standards, for VSEs developing critical products, should be developed.

Additional Information

The following website provides more information as well as articles and deployment packages for software and systems engineering: http://profs.logti.etsmtl.ca/claporte/English/VSE/index.html

Figures and Tables:

Table 1: Processes, tasks, work products and roles of each software profile ( Click to view image )

Figure 1. Activities of Two Processes of the Software Engineering Basic Profile [3] ( Click to view image )

Table 2: Effort to execute a task, then detect and correct errors by the two-member team (from [14]) ( Click to view image )

Figure 2. Coverage of the Initial Software Tasks to the Basic Profile ( Click to view image )


References and Notes

  1. Clarke, P.; O’Connor, R.V.; & Leavy, B. (2016.) “A complexity theory viewpoint on the software development process and situational context.” In Proceedings of the International Conference on Software and Systems Process (ICSSP ‘16). ACM, New York, N.Y., USA, 86-90. DOI: http://dx.doi.org/10.1145/2904354.2904369
  2. Larrucea, X.; O’Connor, R.V.; Colomo-Palacios, R.; & Laporte, C.Y. (2016.) “Software process improvement in very small organizations.” IEEE Softw. 33(2), 85–89.
  3. Laporte, C.Y. & O’Connor, R.V. (2016.) “Systems and Software Engineering Standards for Very Small Entities: Accomplishments and Overview.” Computer, 49(8), 84-87.
  4. O’Connor, R.V. & Laporte, C.Y. (2014.) “An Innovative Approach to the Development of an International Software Process Lifecycle Standard for Very Small Entities.” International Journal Information Technology and the Systems Approach, 7 (1), 1-22.
  5. ISO/IEC/IEEE 12207, “Systems and software engineering – Software life cycle processes.” International Organization for Standardization/International Electrotechnical Commission: Geneva, Switzerland, 2008.
  6. ISO/IEC/IEEE 15288, “Systems and software engineering - System life cycle processes.” International Organization for Standardization/International Electrotechnical Commission: Geneva, Switzerland, 2015.
  7. International Organization for Standardization (ISO). (2016.) ISO/IEC TR 29110-1:2016 “Systems and software engineering - Lifecycle profiles for Very Small Entities (VSEs) - Part 1: Overview.” Geneva. Available at no cost from ISO at: http://standards.iso.org/ittf/PubliclyAvailableStandards/index.html
  8. International Organization for Standardization (ISO): ISO/IEC TR 29110-5-1-2 “Software Engineering - Lifecycle Profiles for Very Small Entities (VSEs)” Part 5-1-2: Management and engineering guide: Generic profile group: Basic profile, Geneva (2011). Available at no cost from ISO at: http://standards.iso.org/ittf/PubliclyAvailableStandards/index.html
  9. Laporte, C.Y.; O’Connor, R. & Fanmuy, G. (May/June 2013.) “International Systems and Software Engineering Standards for Very Small Entities.” CrossTalk: The Journal of Defense Software Engineering. Vol. 26, No. 3, 28-33.
  10. Laporte, C. & O’Connor, R. (23-26 September 2014.) “Systems and software engineering standards for very small entities: implementation and initial results.” In: 9th International Conference on the Quality of Information and Communications Technology (QUATIC 2014), Guimaraes, Portugal.
  11. ISO/IEC JCT1/SC7 Working Group 24 Deployment Packages repository, available online at http://profs.logti.etsmtl.ca/claporte/English/VSE/index.html
  12. Charette, R. (September 2005.) “Why software fails.” IEEE Spectrum, 42-49.
  13. Krasner, H. (November 1998.) “Using the cost of quality approach for software.” Crosstalk: The Journal of Defense Software Engineering, Vol. 11, No. 11.
  14. Laporte, C.Y.; Hébert, C. & Mineau, C. (2014.) “Development of a Social Network Website Using the New ISO/IEC 29110 Standard Developed Specifically for Very Small Entities.” Software Quality Professional Journal, ASQ, Vol. 16, No. 4, 4-25.
  15. Jeljeli, H. & Laporte, C.Y. (June 2016.) « Mise en oeuvre de processus logiciels de l’ISO/CEI 29110 dans une entreprise en démarrage, » Génie logiciel, No. 117 (in French).
  16. Garcia, L.; Laporte, C.Y.; Arteaga, J. & Bruggmann, M. (2015.) “Implementation and Certification of ISO/IEC 29110 in an IT Startup in Peru.” Software Quality Professional Journal, ASQ, 17(2) 16-29.
  17. Plante, F., Laporte, C.Y., Développement et mise en oeuvre d’un processus de type agile-ISO 29110 au sein d’une grande institution finançière Canadienne, Génie logiciel, No 120, March 2017 (in French).
  18. Lebel, K. & Laporte, C.Y. (September 2016.) « Développement, en mode Agile, d’une application à l’aide de la norme ISO/CEI 29110 au sein du département Solutions Mobilité et Géoréférencées d’une grande société d’État québécoise.» Génie logiciel, No. 118, (in French).
  19. Assoumane, M., Élaboration et implantation d’un système de gestion de la qualité pour l’entreprise Rogue Research Inc. en conformité avec la norme médicale ISO 13485. Rapport de projet de maitrise, École de technologie supérieure, August 2014 (in French).
  20. ISO/IEC TR 29110-5-6-2:2014. “Systems Engineering – Life cycle Profiles for Very Small Entities (VSEs) – Part 5-6-2: Systems engineering.” Management and engineering guide. Generic profile group: Basic profile, International Organization for Standardization/International Electrotechnical Commission. Geneva, Switzerland. Available at no cost from ISO at: http://standards.iso.org/ittf/PubliclyAvailableStandards/index.html
  21. Laporte, C. Y.; Tremblay, N.; Menaceur, J.; Poliquin, D. & Houde, R. (16-21 July 2016.) “Developing and implementing systems engineering and project management processes at CSIT - A small Canadian company in public transportation.” 26th Annual International Symposium of INCOSE (International Council on Systems Engineering). Edinburgh, U.K.
  22. Laporte, C. Y., Munoz, M., Gerançon, B., The Education of Students About Software Engineering Standards and Their Implementations in Very Small Entities. Submitted to the IEEE Canada- International Humanitarian Tech. Conference, July 20-22, 2017, Toronto, Ontario, Canada.

Claude Laporte

Click to view image

Dr. Claude Y Laporte has worked in defense and transportation organizations for more than 20 years. He is a professor, since 2000, at the École de technologie supérieure, a 6,000-student engineer­ing school in Montréal, where he teaches software engineering. He is the Project Editor of ISO/IEC 29110 set of systems and software engineering standards and Technical Reports. He is the co-author of a textbook about software quality assurance which should be published by John Wiley & Sons in 2013. Website address: .
École de technologie supérieure
Department of Software and IT Engineering
1100, Notre-Dame Street West,
Montréal, Québec, Canada, H3C 1K3
E-mail: Claude.Y.Laporte@etsmtl.ca

Rory O'Connor

Click to view image

Dr. Rory V. O'Connor is a Senior Lecturer in Software Engineering at Dublin City University (Ireland) and a Senior Researcher with Lero, The Irish Software Engineering Research Centre. In addition he is Ireland’s Head of delegation to ISO/IEC JCT1/SC7. His research interests are centered on the processes whereby software intensive systems are designed, implemented and managed. Website address: .

Lero, Irish Software Engineering
Research Centre,
Dublin City University, Dublin, Ireland
E-mail: roconnor@computing.dcu.ie


« Previous Next »