This chapter provides techniques for evaluating and quantifying an organization's maturity in enterprise architecture.

Overview

Organizations that can manage change effectively are generally more successful than those that cannot. Many organizations know that they need to improve their IT-related development processes in order to successfully manage change, but don't know how. Such organizations typically either spend very little on process improvement, because they are unsure how best to proceed; or spend a lot, on a number of parallel and unfocussed efforts, to little or no avail.

Capability Maturity Models (CMMs) address this problem by providing an effective and proven method for an organization to gradually gain control over and improve its IT-related development processes. Such models provide the following benefits:

• They describe the practices that any organization must perform in order to improve its processes.
• They provide a yardstick against which to periodically measure improvement.
• They constitute a proven framework within which to manage the improvement efforts.

The various practices are typically organized into five levels, each level representing an increased ability to control and manage the development environment.

An evaluation of the organization's practices against the model - called an assessment - determines the level at which the organization currently stands. It indicates the organization's maturity in the area concerned, and the practices on which the organization needs to focus in order to see the greatest improvement and the highest return on investment.

The benefits of capability maturity models are well documented for software and systems engineering. Their application to enterprise architecture has been a recent development, stimulated by the increasing interest in enterprise architecture in recent years, combined with the lack of maturity in this discipline.

This section introduces into TOGAF the topic of capability maturity models and their associated methods and techniques, as a widely used industry standard that is mature enough to consider for use in relation to enterprise architecture.

Background

The Software Engineering Institute (SEI),¹ a federally funded research and development center sponsored by the US Department of Defense and operated by Carnegie Mellon University, developed the original capability maturity model - SW-CMM, Capability Maturity Model for Software - in the early 1990s, which is still widely used today.

Capability maturity models have gained widescale acceptance over the last decade. These models and their associated methods were originally applied to IT solutions, particularly software solutions, but a number of IT-related disciplines have developed capability maturity models to support process improvement in areas such as:

• People - the P-CMM (People Capability Maturity Model), and the IDEAL Life Cycle Model for Improvement
• Systems Engineering - the SE-CMM (Systems Engineering Capability Maturity Model)
• Software Acquisition - the SA-CMM (Software Acquisition Capability Maturity Model)
• CMMI (Capability Maturity Model Integration)

The models have been adopted by large organizations, including the US Department of Commerce, the US DoD, the UK Government, and a number of large services organizations, to assess competencies.

The increasing interest in applying these techniques to the IT architecture and enterprise architecture fields has resulted in a series of template tools which assess:

• The state of the IT architecture process
• The IT architecture
• The organization's buy-in to both

The main issues addressed by US and UK Government use of these models include:

• eCommerce maturity
• Process implementation and audit
• Quality measurements
• People competencies
• Investment management

They involve use of a multiplicity of models, and focus in particular on measuring business benefits and return on investment.

Another key driver is the increasing use of outsourcing. Recent analyst projections indicate that around 75 percent of IS organizations are refocusing their role on brokering resources and facilitating business-driven demands, rather than on being direct providers of IT services. The capability maturity model is increasingly the standard by which outsourcers are being evaluated.

This section reviews the state of development in such techniques.

A closely related topic is that of the Architecture Skills Framework (see Architecture Skills Framework), which can be used to plan the target skills and capabilities required by an organization to successfully deliver an enterprise architecture, and to determine the training and development needs of individuals.

The US DoC ACMM Framework

Overview

As an example of the trend towards increased interest in applying capability maturity model techniques to IT architecture, all US federal agencies are now expected to provide maturity models and ratings as part of their IT investment management and audit requirements.

In particular, the US Department of Commerce (DoC) has developed an IT Architecture Capability Maturity Model (ACMM)² to aid in conducting internal assessments. The ACMM provides a framework that represents the key components of a productive IT architecture process. The goal is to enhance the overall odds for success of IT architecture by identifying weak areas and providing a defined evolutionary path to improving the overall architecture process.

The ACMM comprises three sections:

1. The IT architecture maturity model
2. IT architecture characteristics of operating units' processes at different maturity levels
3. The IT architecture capability maturity model scorecard

The first two sections explain the architecture capability maturity levels and the corresponding IT architecture characteristics for each maturity level to be used as measures in the assessment process. The third section is used to derive the architecture capability maturity level that is to be reported to the DoC Chief Information Officer (CIO).

Elements of the ACMM

The DoC ACMM consists of six levels and nine architecture characteristics. The six levels are:

0

None

1

Initial

2

Under development

3

Defined

4

Managed

5

Measured

The nine IT architecture characteristics are:

• IT architecture process
• IT architecture development
• Business linkage
• Senior management involvement
• Operating unit participation
• Architecture communication
• IT security
• Architecture governance
• IT investment and acquisition strategy

Two complementary methods are used in the ACMM to calculate a maturity rating. The first method obtains a weighted mean IT architecture maturity level. The second method shows the percent achieved at each maturity level for the nine architecture characteristics.

Example: IT Architecture Process Maturity Levels

The following example shows the detail of the IT architecture maturity levels as applied to the first of the nine characteristics, IT architecture process.

Level 0: None

No IT architecture program. No IT architecture to speak of.

Level 1: Initial

Informal IT architecture process underway.

1. Processes are ad hoc and localized. Some IT architecture processes are defined. There is no unified architecture process across technologies or business processes. Success depends on individual efforts.
2. IT architecture processes, documentation, and standards are established by a variety of ad hoc means and are localized or informal.
3. Minimal, or implicit linkage to business strategies or business drivers.
4. Limited management team awareness or involvement in the architecture process.
5. Limited operating unit acceptance of the IT architecture process.
6. The latest version of the operating unit's IT architecture documentation is on the web. Little communication exists about the IT architecture process and possible process improvements.
7. IT security considerations are ad hoc and localized.
8. No explicit governance of architectural standards.
9. Little or no involvement of strategic planning and acquisition personnel in the enterprise architecture process. Little or no adherence to existing standards.

Level 2: Under Development

IT architecture process is under development.

1. Basic IT architecture process is documented based on OMB Circular A-130 and Department of Commerce IT Architecture Guidance. The architecture process has developed clear roles and responsibilities.
2. IT vision, principles, business linkages, baseline, and Target Architecture are identified. Architecture standards exist, but not necessarily linked to Target Architecture. Technical Reference Model (TRM) and Standards Profile framework established.
3. Explicit linkage to business strategies.
4. Management awareness of architecture effort.
5. Responsibilities are assigned and work is underway.
6. The DoC and operating unit IT architecture web pages are updated periodically and are used to document architecture deliverables.
7. IT security architecture has defined clear roles and responsibilities.
8. Governance of a few architectural standards and some adherence to existing Standards Profile.
9. Little or no formal governance of IT investment and acquisition strategy. Operating unit demonstrates some adherence to existing Standards Profile.

Level 3: Defined

Defined IT architecture including detailed written procedures and TRM.

1. The architecture is well defined and communicated to IT staff and business management with operating unit IT responsibilities. The process is largely followed.
2. Gap analysis and migration plan are completed. Fully developed TRM and Standards Profile. IT goals and methods are identified.
3. IT architecture is integrated with capital planning and investment control.
4. Senior management team aware of and supportive of the enterprise-wide architecture process. Management actively supports architectural standards.
5. Most elements of operating unit show acceptance of or are actively participating in the IT architecture process.
6. Architecture documents updated regularly on DoC IT architecture web page.
7. IT security architecture Standards Profile is fully developed and is integrated with IT architecture.
8. Explicit documented governance of majority of IT investments.
9. IT acquisition strategy exists and includes compliance measures to IT enterprise architecture. Cost benefits are considered in identifying projects.

Level 4: Managed

Managed and measured IT architecture process.

1. IT architecture process is part of the culture. Quality metrics associated with the architecture process are captured.
2. IT architecture documentation is updated on a regular cycle to reflect the updated IT architecture. Business, Data, Applications, and Technology Architectures defined by appropriate de jure and de facto standards.
3. Capital planning and investment control are adjusted based on the feedback received and lessons learned from updated IT architecture. Periodic re-examination of business drivers.
4. Senior management team directly involved in the architecture review process.
5. The entire operating unit accepts and actively participates in the IT architecture process.
6. Architecture documents are updated regularly, and frequently reviewed for latest architecture developments/standards.
7. Performance metrics associated with IT security architecture are captured.
8. Explicit governance of all IT investments. Formal processes for managing variances feed back into IT architecture.
9. All planned IT acquisitions and purchases are guided and governed by the IT architecture.

Level 5: Optimizing

Continuous improvement of IT architecture process.

1. Concerted efforts to optimize and continuously improve architecture process.
2. A standards and waivers process is used to improve architecture development process.
3. Architecture process metrics are used to optimize and drive business linkages. Business involved in the continuous process improvements of IT architecture.
4. Senior management involvement in optimizing process improvements in architecture development and governance.
5. Feedback on architecture process from all operating unit elements is used to drive architecture process improvements.
6. Architecture documents are used by every decision-maker in the organization for every IT-related business decision.
7. Feedback from IT security architecture metrics are used to drive architecture process improvements.
8. Explicit governance of all IT investments. A standards and waivers process is used to make governance-process improvements.
9. No unplanned IT investment or acquisition activity.

Capability Maturity Models Integration (CMMI)

Introduction

The capability models that the SEI is currently involved in developing, expanding, or maintaining include the following:

• CMMI (Capability Maturity Model Integration)
• IPD-CMM (Integrated Product Development Capability Maturity Model)
• P-CMM (People Capability Maturity Model)
• SA-CMM (Software Acquisition Capability Maturity Model)
• SE-CMM (Systems Engineering Capability Maturity Model)
• SW-CMM (Capability Maturity Model for Software)

As explained in Architecture Maturity Models , in recent years the industry has witnessed significant growth in the area of maturity models. The multiplicity of models available has led to problems of its own, in terms of how to integrate all the different models to produce a meaningful metric for overall process maturity.

In response to this need, the SEI has developed a Framework called Capability Maturity Model Integration (CMMI), to provide a means of managing the complexity.

According to the SEI, the use of the CMMI models improves on the best practices of previous models in many important ways, in particular enabling organizations to:

• More explicitly link management and engineering activities to business objectives
• Expand the scope of and visibility into the product lifecycle and engineering activities to ensure that the product or service meets customer expectations
• Incorporate lessons learned from additional areas of best practice (e.g., measurement, risk management, and supplier management)
• Implement more robust high-maturity practices
• Address additional organizational functions critical to its products and services
• More fully comply with relevant ISO standards

CMMI is being adopted worldwide.

The SCAMPI Method

The Standard CMMI Appraisal Method for Process Improvement (SCAMPI) is the appraisal method associated with CMMI. The SCAMPI appraisal method is used to identify strengths, weaknesses, and ratings relative to CMMI reference models. It incorporates best practices found successful in the appraisal community, and is based on the features of several legacy appraisal methods. It is applicable to a wide range of appraisal usage modes, including both internal process improvement and external capability determinations.

The SCAMPI method definition document³ describes the requirements, activities, and practices associated with each of the processes that compose the SCAMPI method.

Conclusions

This section has sought to introduce into TOGAF the topic of capability maturity model-based methods and techniques, as a widely used industry standard that is mature enough to consider for use in relation to enterprise architecture.

The benefits of capability maturity models are well documented for software and systems engineering. Their application to enterprise architecture has been a more recent development, stimulated by the increasing interest in enterprise architecture, combined with the lack of maturity in the discipline of enterprise architecture.

Future versions of TOGAF will seek to build on this base, as more experience is gained on the use of these methods and techniques specifically relating to enterprise architecture.

Footnotes

1.

Refer to www.sei.cmu.edu/sei-home.html.

2.

Refer to secure.cio.noaa.gov/hpcc/docita/files/.

3.

Available at www.sei.cmu.edu/publications/documents/01.reports/01hb001.html.

return to top of page

The TOGAF document set is designed for use with frames. To navigate around the document:

• In the main Contents frame at the top of the page, click the relevant hyperlink (Part I, Part II, etc.) to load the Contents List for that Part of the TOGAF document into the Secondary Index frame in the left margin.
  
  
• Then click in that Contents List to load a page into this main frame.
  
  

Downloads of the TOGAF documentation, are available under license from the TOGAF information web site. The license is free to any organization wishing to use TOGAF entirely for internal purposes (for example, to develop an information system architecture for use within that organization). A hardcopy book is also available from The Open Group Bookstore as document G063.