The Open Group is a global consortium that enables the achievement of business objectives through technology standards. Our diverse membership of more than 800 organizations includes customers, systems and solutions suppliers, tools vendors, integrators, academics, and consultants across multiple industries.

The mission of The Open Group is to drive the creation of Boundaryless Information Flow™ achieved by:

Further information on The Open Group is available at www.opengroup.org.

The Open Group publishes a wide range of technical documentation, most of which is focused on development of Standards and Guides, but which also includes white papers, technical studies, certification and testing documentation, and business titles. Full details and a catalog are available at www.opengroup.org/library.

This document is The Open Group Open Agile Architecture™ Standard, also known as The Open Group O-AA™ Standard. It has been developed and approved by The Open Group.

This document follows a modular structure and is organized in the following parts:

Examples and case studies are provided as illustrations to foster understanding of the standard. Examples and case studies are not a normative part of the standard and therefore do not include requirements.

The target audience for this document includes:

This document does not contain any normative references at the time of publication. These may be added in a future release.

This document is The Open Group Open Agile Architecture™ standard. The objective of this document is to cover both the Digital Transformation of the enterprise, together with the Agile Transformation of the enterprise.

This documents covers both the Digital Transformation and the Agile Transformation of the enterprise. It is divided into two parts:

Refer to The Open Group website for conformance requirements for this document.

None.

For the purposes of this document, the following terminology definitions apply:

It is expected that this document will need to be revised from time to time to remain current with both practice and technology.

The obligation to demonstrate task achievement and take responsibility for performance in accordance with agreed expectations; the obligation to answer for an action. (Source: Vanderhaegen 2019)

Any map, diagram, or visualization that reveals both sides of value creation in a single overview. It is a category of diagram that illustrates the interaction between people and organizations. (Source: Kalbach 2016)

The time available between starting a product development initiative or process and finishing it in order to satisfy customers.

Consists of the existing code, components, and technical infrastructure needed to implement near-term features without excessive redesign and delay. (Source: Scaled Agile, Inc.)

A qualitative statement of intent that should be met by the architecture. (Source: The TOGAF^® Standard 2018)

A coordinated set of architectural constraints that restricts the roles/features of architectural elements and the allowed relationships among those elements within any architecture that conforms to that style. (Source: Fielding 2000)

A dialog between senior managers and project teams about the resources and time both available and needed to achieve the targets.

(Source: LEI)

An architecture with no end-state that is designed to evolve to support the changing needs of the digital enterprise.

The sum-totality of how customers engage with your company and brand, not just in a snapshot in time, but throughout the entire arc of being a customer. (Source: Richardson 2010)

The series of interactions between a customer and a company that occur as the customer pursues a specific goal. (Source: Forrester^®)

A methodology for creative problem solving that begins with understanding unmet customer needs. (Sources: Stanford 2010 and MIT)

A software system composed of application and infrastructure components that can be rapidly reconfigured using DevOps and cloud-native computing.

A synthesis of methods and guidance from a wide variety of practitioners and professional communities active in digital technology (Lean, Agile, DevOps, etc.) designed to create and manage products with an increasing digital component, or lead their organization through Digital Transformation.

A powerful, accessible, and potentially game-changing technology (social, mobile, cloud, analytics, Internet of Things (IoT), cognitive computing, and biometrics) often used in combination and usually characterized by its ability to positively impact an enterprise’s business model, customer experience, product, or operating system to enable innovation and growth.

The use of digital practices supported by digital technologies to achieve a shift in the business model, value proposition, operating system, or distribution system to radically improve customer relationships, profitability, internal processes, performance, accessibility, and market reach of an enterprise.

The representation of a selected abstraction of domain knowledge that is rigorously organized.

(Source: Evans 2003)

The complex community of organisms and their environment, functioning as an ecological unit.

(Source: Wind 2015)

(Source: Agile Alliance^®)

The identification of domain events, commands, persona, or entities to facilitate a structured conversation about the domain.

An architecture that supports guided, incremental change across multiple dimensions. (Source: Ford™ 2017)

A meta-non-functional requirement that aims to prevent other architecture requirements, in particular the non-functional ones, from degrading over time.

The functional characteristic of a product (goods or services).

Tools, machines, wiring, and other physical components of a system.

The protection of information and information systems from unauthorized access, use, disclosure, disruption, modification, or destruction in order to provide integrity, confidentiality, and availability.

The system’s property of being made up of elements that behave consistently as a whole.

(Source: Miraglia 2014)

A purposeful set of statements, models, and decisions that represent some future architectural state.

What the customer hopes to accomplish.

(Source: Christensen 2016)

Laying out the entire end-to-end customer experience.

The time between the initiation and completion of a process.

All of the actions, both value-creating and non-value-creating, required to bring a product from concept to launch (also known as the development value stream) and from order to delivery (also known as the operational value stream).

(Source: LEI)

The system’s property of being made up of elements that present a high independence of other elements. (Source: Miraglia 2014)

Design decisions which must be made before the work on independent modules can begin.

(Source: Parnas 1972)

The combination of assets and processes required to deliver a product or a service.

(Source: Van Mieghem 2015)

The result of an activity conducted by a provider and experienced by a consumer.

A fictional character which is created based upon research in order to represent the different user types that might use a service, product, site, or brand in a similar way. (Source: Friis Dam 2020)

Business model that is based on the two-sided market theory.

Any activity or group of activities that takes an input, adds value to it, and provides an output to an internal or external customer.

(Source: Harrington 1991)

A bundle of services and/or goods offered to customers.

A list of the new features, changes to existing features, bug fixes, infrastructure changes, or other activities that a team may deliver in order to achieve a specific outcome. (Source: Agile Alliance)

An organization structured around permanent teams by opposition to temporary teams or projects.

The process of changing a software system in a way that does not alter the external behavior of the code yet improves its internal structure.

(Source: Fowler 2019)

The obligation to carry forward a task to its successful conclusion.

(Source: Vanderhaegen 2019)

People, their behavior, cultural beliefs, skills, and expertise, and how work teams are forming and interacting, as well as organizational leadership, strategy, structure, policy, and procedures.

A set of entities and their relationships, whose functionality is greater than the sum of the individual entities. (Source: Crawley 2016)

A brief statement of intent that describes something the system needs to do for the user.

(Source: Patton 2014)

End-to-end collection of value-added and non-value-added activities that create an overall result for a customer, stakeholder, or end user.

A seemingly random or disconnected piece of information that at first appears to be background noise but can be recognized as part of a significant pattern by viewing it through a different frame or connecting it with other pieces of information. (Source: Schoemaker 2009)

Human participants and/or machines perform processes and activities using software, hardware, and other resources to deliver products or experiences.

Mike Sutcliff and his co-authors conducted a survey among 1,350 executives to discover why Digital Transformations fail [Sutcliff 2019]. The number one reason given for failure is the unspoken disagreements between top managers about goals. The recommendation is to “define and articulate not only the opportunity but also the problem it solves, and how the company will build the organization around the desired solution before investing”.

Tim Fountaine and his co-authors show that technology is not the biggest challenge when building the AI-powered organization [Fountaine 2019]: “Cutting-edge technology and talent are not enough. Companies must break down organizational and cultural barriers that stand in AI’s way”.

These two articles and surveys show that:

In order to become Agile, the organization must:

The enterprise culture needs to shift from experience-based to fact-based decision-making. Amazon™ epitomizes this shift. Ram Charan reports that "everything that matters can be tracked, measured, and analyzed, with insights generated and routine decisions automated …​ It liberates Bezos, executives, and frontline employees at Amazon from managing routine daily chores and the inevitable bureaucracy associated" [Charan 2019].

The digital enterprise needs to deliver a consistent user (customer, employee, or partner) experience across all touchpoints. The authors of Beyond Advertising recommend thinking about brands as touchpoint orchestrators: “the brand’s multiple touchpoints and how they interact with each other, from a digital out-of-home experience to a tablet, from mobile to the store” [Wind 2016].

Every interaction with a brand, from the first time you become aware that it exists to every touchpoint you encounter along the way in your daily life, has an impact: “From the customer perspective, touchpoints with a brand or product are not differentiated: it is the seamless experience that matters.”

Figure 3 illustrates the variety of touchpoints that a brand has to orchestrate to deliver a positive customer experience.

It can be predicted that: “Touchpoints will continue to multiply in an era where every object has the potential to become connected and interactive.”

This evolution impacts the enterprise as a whole: “New structures and processes will need to allow for agility and reaping the benefits both from decentralization and, when needed, the power that leveraging through centralization facilitates.”

New operating systems are required to create real-time, personalized experiences. One of their key functions is to enable touchpoint orchestration. The operating system is the combination of assets and processes required to deliver a product or a service; for a more detailed definition of operating system, see Section 2.35.

Business agility is the ability to quickly identify and respond to market and environment changes. Enterprises develop business agility when they sense changes in their environment early and act upon them decisively and rapidly.

“Employees deep down in the organization may be closer to weak signals at the edge of the organization and must believe they will receive an open hearing when raising concerns or suggesting ideas”. [Day 2019]

When Agile teams are vigilant and empowered to act, they are better at navigating digital turbulence than those at head office. When weak signals reach head office, they are filtered and cascaded down the organization. In this context, weak signals are the first indicator of a change or an emerging issue. This increases the likelihood that weak signals (represented by the purple parallelograms on Figure 4) will not be seen, or that no one will respond to them. This explains why head office can be slow to identify and act upon potential threats and opportunities.

Figure 4 illustrates that Agile teams close to the field have enough autonomy to modify products or journeys in response to weak signals. Agile teams do not have to seek permission from head office before experimenting with responses to weak signals.

Business agility is facilitated by organizational agility because:

Organizational agility was introduced in Section 3.1 and is covered by Chapter 12.

Digital Transformation does not have to be disruptive [Sutcliff 2019]. Business agility is about organizations learning to be capable of evolving rapidly.

Martin Fowler wrote: “There is no such thing as a universally accepted definition of architecture” [Fowler]. His view of architecture is aligned with that of Ralph Johnson, who defines architecture as “the important stuff (whatever that is)” [Johnson 2008]. The lack of a universally accepted architecture definition justifies why this document borrows from three different sources to analyze what architecture means.

Most architecture definitions, such as these examples, adopt a systems thinking view that models the enterprise as a set of interrelated elements:

Both of these definitions focus on “what the enterprise is”, its elements of form, rather than “what the enterprise does”, its functions; thus positioning architecture as a discipline which guides the design and evolution of the enterprise modeled as a system.

More detail on how the TOGAF Standard, Version 9.2 guides the delivery of architecture can be found in Sections 2.3, 2.4, and 2.5 of the TOGAF Standard; see [TOGAF Standard 2018].

To paraphrase Alfred Korzybski’s famous sentence [Korzybski 1958]: “a map is not the territory it represents, but if correct, it has a similar structure to the territory, which accounts for its usefulness”; the architecture model of the enterprise is not the enterprise. The enterprise makes architecture decisions that are reflected in the way it does business and operates. Architecture decisions may be implicit and are not necessarily documented or modeled.

Crawley [Crawley 2016] notes that “architecture is the embodiment of concept, and the allocation of physical/informational functions (processes) to elements of form (objects) and definition of structural interfaces among the objects”, and goes on to state that the systems engineering discipline adds two important ideas:

As shown in Figure 6, the O-AA development is structured along two axes that represent:

The “Experience Perspective” is in the problem space, while the “Work System Perspective” is in the solution space, as well as the “Technical System Perspective”.
The O-AA building blocks are positioned along the two axes, with the exception of: “Strategy”, “Value”, and “Data/Information & AI”.
The “Corporate Brand Identity” and “Corporate Culture” building blocks influence what the enterprise does.

The digital enterprise collects and processes vast amounts of data. Data describes a portion of “reality” that is of interest to the enterprise. Data becomes information when it is interpreted, when it has meaning. Data can be interpreted by humans or algorithms. Artificial Intelligence (AI) transforms data into predictions, prescriptions, and automation capabilities.

The combination of data and AI is having a transformational impact on industries. For example, predictive maintenance can improve service quality while lowering costs. Vast numbers of images can teach machines how to recognize faces or interpret chest x-rays.

Architecting systems that handle data at scale is critical. Non-functional requirements, such as compliance (e.g., privacy) or security requirements, influence Data Architecture decisions. The move from batch processing to real-time data flows is having a profound impact on Data Architecture.

“Software is eating the world.”

Marc Andreessen explained in a famous Wall Street Journal article why software is eating the world [Andreessen 2011]. The rapid evolution of software technology has fueled the growth of digital business. Following the lead of Internet giants, some enterprises from the old economy are framing themselves as tech companies; for example, Banco Bilbao Vizcaya Argentaria (BBVA): “If you want to be a leading bank, you have to be a technology company”.

In 2002, Amazon™ was facing a complexity barrier. The size of its home page reached 800MB and it took 8 to 12 hours to compile. Jeff Bezos issued a mandate that profoundly changed the way software is created and how the enterprise is organized.

By shifting toward modularity and APIs, Amazon became well-positioned to open its distribution and logistics capabilities to third-party vendors. The self-service nature of the platform made it easy for vendors to sell and distribute their products in a frictionless manner. This helped Amazon to compete against eBay^®, leveraging a business model which is different.

Software and hardware represents an increasing part of products and their supporting operations. Software architecture, product architecture, and operations architecture must be developed in a concurrent manner.

Except when they are in their creation phase (e.g., startups), enterprises have an “as-is” architecture, which may be explicit or implicit, documented or in the mind of employees.

This document distinguishes an architecture from its model; the map is not the territory. Architecture evolves due to emergence, which refers to what appears when the parts that compose a system come together and operate. An architecture model does not emerge; it is a representation of an “as-is” architecture or it is designed.

Security is a crucial element in any Agile Architecture, and enterprises must get it right. Businesses are in a world of digital enablement and iterative development, with Minimum Security Architecture (MSA) as a crucial element of Agile development. MSA is a continually evolving security architecture that fits into the cadence of a DevOps cycle, and is sufficient to guide developers toward the Minimum Viable Product (MVP) and to set architectural direction for future releases.

However, with this comes a danger: if an MVP is released with a security gap it is difficult to apply a solution once it has gained a user community. This means that products must correctly integrate security into the architecture from the beginning or risk the difficult – if not impossible – task of trying to remediate security gaps later on.

DevOps includes security as a fundamental aspect; it is not an add-on to the process. This prevents the silo thinking prevalent in literature, which attempts to add additional teams to DevOps leading to infinite variations of DevSecOps, ArchDevOps, etc.

The DevOps process (while technical in execution) is owned by the business. As such, the goal of DevOps is to help balance both business enablement and risk management (in the context of security). This can take the form of compliance and resiliency metrics that provide much needed assurance to the business in relation to system-related cybersecurity.

In addition to building security into the architecture from the beginning, security in an Agile Architecture must respond quickly through regular feedback loops that do not require extensive upfront planning. Because of these rapid feedback loops, there is a need for extra attention in order to balance security with other aspects of the architecture. The security team must consider the future state well beyond the current architecture to ensure that downstream teams also remain Agile. This underscores the purpose of security in an Agile Architecture – to enable the business to move rapidly in a world of defined and managed risk.

To this end, there are three core philosophies:

For more information, see the O-AA Security Playbook 2021.

Sometimes, and depending on the approach used, Enterprise Architecture frameworks distinguish Enterprise Architecture from solution architecture. Enterprise Architecture is at the entity level and has a medium to long time horizon. In contrast, the scope of solution architecture is sometimes limited to a project. For example, the TOGAF Standard, Version 9.2 describes solution architecture as typically applying to “a single project or project release, assisting in the translation of requirements into a solution vision, high-level business and/or IT system specifications, and a portfolio of implementation tasks”. The time horizon of solution architecture is therefore contingent to the time horizon of projects. This does not imply, however, that the approach could not be adapted to cover a solution that does not depend on, or is not tied to, a specific project.

To echo this, Murat Erder and Pierre Pureur observe that “solution architects focus on providing project-level architectures, but Enterprise Architects attempt to create strategies, architecture plans, architecture standards, and architecture frameworks for the whole enterprise” [Erder 2016].

When the definition of solution architecture refers to a project or an application, the points of attention below need to be addressed:

As Murat Erder and Pierre Pureur recommend, “architecting products, not just solutions for projects because architecting products is more efficient than just designing point solutions to projects and focuses the team on its customers”.

Architecturally significant decisions fall into the categories below. They define or modify:

They are classified according to:

Think about decisions as one-way or two-way doors [Rossman 2019]. When you walk through a one-way door, if you do not like what you see on the other side, you cannot get back to the initial state. This type of decision, also referred to as a Type 1 decision, is difficult to reverse. For example, once you have settled for the asynchronous architecture style, it is difficult to revert to a synchronous architecture style.

When you walk through a two-way door, if you do not like what you see on the other side, you can walk right back through the door and return to the previous state. This type of decision, also referred to as a Type 2 decision, is easy to reverse. Type 2 decisions should be delegated to the lowest reasonable organizational level.

The scope of a decision is the surface of its impacts along the Level and Perspective axes of Figure 7.

Table 1 formulates recommendations depending on the type and scope of decisions.

When combining intentional and emergent architecture, decisions that come from intentional architecture may change. New ones are added and past decisions can be reversed. Therefore, it is important to document the motivations behind past decisions. An Architecture Decision Record (ADR) provides an Agile and lightweight way of doing this.

Michael Nygard [Nygard 2011] wrote a blog that explains why and how to document architecture decisions. When someone does not understand the rationale of a decision the choice must be made to either accept it, though it should be changed because the context has changed, or change it without understanding the consequences.

Michael Nygard has developed an ADR template that is accessible on GitHub [Nygard] and outlined below:

ADR Title

Intentional architecture is illustrated here using a Car Sharing Platform (CSP) example that began as a startup to connect drivers to passengers, where drivers publish their available seats and passengers reserve them online.

At the beginning, the connection between users was free-of-charge. The CSP needed to add a profit formula to its business model and has experimented with several economic models:

The transactional model is now the most frequent; the driver sets a price per passenger based on a recommendation from the platform of around €0.065 per kilometer. The price must not exceed the cost of the journey so that the driver cannot make a profit from it. The CSP adds a commission that changes depending on the country of operation. In France, it is around 18% to 21%.

In order to fund its international development, the CSP conducted a few massive fundraising events. Because the economic situation and culture of each country is different, the economic model was customized by each country. For example, in some countries, the CSP does not charge a commission. In Poland, the CSP has switched to subscription. The net is that few countries are profitable.

The rather unprofitable international development combined with increased competition forced the CSP to change. The number of employees was significantly reduced and platform algorithms were improved; for example:

On the organizational side, the CSP was initially divided into large teams. Then, little by little, the CSP broke down its organization into smaller teams aligned with outcomes. It effectively created small startups of 6 to 12 people, regrouped into tribes.

On the software side, the CSP adopted DevOps. This enabled the enterprise to make more than ten updates of its services a day.

Figure 8 is a simplified representation of the CSP development stages; and see the corresponding points (1) to (6) in the text that follows. Architecturally significant decisions must be made at some development stages. Not all “Continuous Product Improvement” iterations require “Product Architecture Refactoring”.

“User Research” (1) helps to clarify driver and passenger needs. It provides inputs to segment the market. Passengers need to travel cheaply from Point A to Point B at some date and time. Drivers who have seats available can offer them. Passengers may work for enterprises that need to facilitate employee commuting.

The first architecturally important decision is about segmenting the market. Passengers and drivers are two obvious segments with complementary needs. Enterprises are a potential market segment, though less obvious to define.

The “Initial Product Architecture” (2) is simple. It is based on a two-sided market supported by a platform that connects passengers and drivers. The platform offers minimum community features to help better match passengers and drivers; for example, using social reputation. The key design objective is to balance the growth in demand for transportation with the growth of transportation offers. If the two are unbalanced, passengers or drivers will become frustrated.

The next stage is to develop a “Minimum Viable Product” (3), which consists of:

Because the MVP objective is to validate the product’s market viability, the CSP opts for a sacrificial architecture for its initial website.

Martin Fowler coined the term “sacrificial architecture” [Fowler 2014] to designate situations where a team deliberately chooses to throw away a codebase. He lists some examples of companies that have done this, such as eBay or Google^®.

When fewer resources are allocated to developing the website, more can be spent:

In this example, a sacrificial architecture is risky should rapid traffic growth exceed the capacity of the website. This is one example of the trade-offs that must be analyzed. The sacrificial architecture decision should be documented using an ADR.

Architecting the support function is important too. One of the goals is to minimize fixed costs until the CSP can prove that the economic model works. Support primarily relies on self-care, which is completed by a call center to handle cases that require human intervention.

The more sophisticated the self-care, the higher the software development costs. The more cases not solved by self-care, the higher the support costs. This is an example of an operating model decision that impacts architecture (4), (5). Architecturally important operating model decisions must be documented using ADRs.

Architecture evolution is also triggered by new ideas that emerge from the bottom up. For example, a new car rider’s localization feature was invented during a “coding night”. One hour before the scheduled departure time, the driver is asked to join Ride Map, and to indicate the exact starting point on that map. Passengers can then easily find their driver and are also invited to share their location, to facilitate the meeting.

This new feature requires updating the driver and passenger journeys and interfacing the platform with mapping software in real time. This impacts the existing platform architecture.

Two CSP business development strategies require intentional thinking:

The fast international expansion requires a well-thought out operating model and platform architecture. Examples of architecture questions include:

The shift toward a multi-modal transportation model requires a different approach. Instead of an iterative process, shifting to a concurrent development process is recommended. Though the CSP “as-is” architecture is modular, unavoidable dependencies remain. For example, a new “multi-modal reservation” sub-system needs to interface with the “car-pooling seats reservation” sub-system. A new “multi-modal pricing” module aggregates the prices quoted by car-pooling drivers with prices provided by third-party reservation systems.

Because seats offered by third-party reservation systems cannot be blocked by a distributed transaction monitor, an event-driven architecture that implements a Saga transactional model is mandated. Section 21.2.2 introduces this architecture style and the concept of Saga.

Waterfall and iterative processes often tend to identify the incompatibilities of architectural decisions too late, which is a major cause of rework. By contrast, SBCE delays architecture decisions until “the last responsible moment” [Atwood 2006]. It is counterintuitive that delaying decisions actually shortens development time while improving quality.

When the solution space is constrained too early, it leads to suboptimal architecture decisions. In contrast, SBCE works like this:

The set of design alternatives shrinks as they are eliminated, knowledge accumulates, and the remaining alternatives are developed to increasing levels of fidelity. Simultaneously, target ranges narrow in corresponding stages, converging on values that provide the best set of trade-offs.

The starting point is the creation of a context map to help decompose the CSP platform into sub-systems. Chapter 20 introduces context maps.

Figure 9 represents a context map for the multi-modal reservation domain.

Applying the Inverse Conway Maneuver, the CSP segments the organization into teams that:

Agile teams work in parallel and follow a responsibility-based management style. Figure 10 illustrates the big picture and describes how all the pieces fit together.

Each team of teams (tribe) and team (squad) mirrors some piece of the intentional architecture model. Integration events are scheduled on a regular basis; for example, for integrating the “Multi-Modal Reservation” sub-system with the “Third-Party Reservation” sub-system. The Third-Party Reservation sub-system encapsulates access to third-party reservation sites; for example, Ouibus.

Design and architecture conversations between teams happen continuously. An Obeya room can be set up to facilitate coordination and cooperation. This is a room that contains visually engaging charts and graphs where teams meet to engage in meaningful design and architecture conversations.

Integration events are experiments that aim to verify how the pieces of the larger system work together. The status is based on results, not on progress reports processed by a project manager who does not understand design and architecture issues.

This document does not impose adopting a “process-oriented” Agile framework such as the Scaled Agile Framework^® (SAFe^®) or Large-Scale Scrum (LeSS). Its purpose is to complement them. A word of caution though; the Agile community is increasingly criticizing these frameworks because they promote practices that are sometimes inconsistent with Lean and Agile tenets. In this chapter the concept of continuous architecture is introduced in the context of intentional architecture. Chapter 6 will drill down further into continuous architecture in a software context.

In this chapter this document discusses the topic of continuous architectural refactoring, a topic of increasing importance in today’s Agile, DevOps-oriented software landscape. Here, the focus is on software architecture, as this is a particularly important focus for continuous architectural refactoring.

To properly discuss the concept, it must first be adequately defined, and every word in the term is important. Each will be briefly discussed, each in turn (but not in order).

The remainder of this chapter will discuss the key considerations in planning for the continuous architectural refactoring of a software system; it answers the question of how the organization can be set up to be in a position to continuously evolve their architectures in response to changing requirements, architectural debt, and other headwinds. This document details these under three headings:

Each sub-section covers a different aspect of the necessary prerequisites for continuous architectural refactoring. Taken together, they offer a complete view of the enablers for successful continuous architectural refactoring.

Before deciding what technical and organizational mechanisms to put in place to facilitate continuous refactoring, it is important to first understand the conditions under which the organizationisation is operating. Once the business and technical constraints relevant to the system have been identified, the structures that will allow it to evolve within those constraints can be put in place. Fitness functions will allow for actual testing of the architecture to ensure it is fit-for-purpose and that guardrails will contribute to the guidance referred to in the definition of architecture in Ford [Ford 2017], keeping the development teams from going astray in their system designs.

Successful continuous architectural refactoring needs the development team to be empowered to iteratively make architectural changes. There are a number of key technical enablers for this, which are discussed here:

In addition, Agile development practices are a key enabler for continuous architectural refactoring. As described in Chapter 7, there are a number of practices which are promoted by Agile working. These practices allow continuous architectural refactoring to be successfully implemented; in particular the rapid iteration and experimentation, which allows architectural evolution to be readily incorporated into ongoing development activities.

Technical mechanisms such as continuous delivery and feature toggles are powerful enablers of continuous architectural refactoring, but they are certainly not the only ones. For example, what if there has been no buy-in from senior management to do any refactoring? (Hint: architectural refactoring gets continuously prioritized behind functional evolution.) Even if there is such buy-in, to paraphrase the definition of architecture in Ford [Ford 2017], continuous refactoring needs to be guided and incremental. The guidance comes in the form of an architectural roadmap, a best-guess hypothesis of how the architecture needs to evolve. Finally, organizations need to balance the tensions between these forces; sometimes there should refactoring, sometimes new functionality should be built.

BIt is worth noting that the development team structure is also a key enabler for continuous architectural refactoring, in particular the Inverse Conway Maneuver. This technique has been described separately in Chapter 4.

In an Agile organization employees are accountable to their peers, their manager, and their clients. Managers are accountable to their teams, the board of directors, and society. The management system cascades goals at all levels of the organization and promotes a constructive dialog to help set up accountability relationships between employees and managers. The reward system recognizes individual performance while promoting collaboration.

The organizational structure is flattened. Autonomous cross-functional teams, often named “feature teams” or “squads”, are formed. Cross-functional roles emerge to help construct robust communities of practice, often named “chapters” or “guilds”. Resource allocation is flexible and driven by the evolution of demand or activity level.

The left part of Figure 11 represents the three transformation dimensions this document has already introduced, with the addition of the “Enterprise Culture” dimension. Culture evolution results from changes in the three dimensions. For a culture change to take hold, people have to experience success operating in the new Agile organization.

The middle part of the figure lists a few important questions that the enterprise needs to address. These questions are key to deciding the ordering of change. Given the interdependencies that link the transformation dimensions, it is tempting to conduct change on the three dimensions in parallel, which leads to either a big bang scenario or incremental deployment transformation scenarios.

The right part of Figure 11 represents the three approaches to transformation. For example, the waterfall scenario is likely to create intermediary stages that are suboptimal. New ways of working may conflict with the existing management system. The enterprise which deploys a new management system on top of an existing organizational structure will have to redeploy it at a later time. This leads to either a big bang scenario or an incremental deployment. In the case of an incremental deployment, the challenge is to define the scope of each increment.

Figure 12 shows that each transformation increment should cover the entire hierarchical line. Why? Because if you change the way one level of the organization operates while the hierarchical line continues to operate the old way, the enterprise runs the risk of submitting its employees and managers to a double bind; i.e., employees and managers becoming at risk of being confronted with two irreconcilable demands or a choice between two undesirable courses of action. For example, re-prioritize your backlog but stick to established budgets and product plans.

Incremental transformation deployment is easier when two conditions are met:

When those conditions are not met, it is better to change the organizational structure before deploying new ways of working and a new management system. The new organizational structure reflects an architectural intent, which is either implicit or explicit.

This document illustrates the need to re-architect the organization with the example of an IT organization that evolved to become more Agile.

The legacy IT organization was structured by process/function (front-office, middle-office, finance, risk) rather than by product families (plain vanilla swaps, equity derivative, commodities, etc.). Two powerful central functions controlled operational teams:

Pools of software development resources were organized by technologies; for example, server-side Java^® development or mobile front-end development. Software development projects would on-board required resources for the duration of the project and release them when done. Last but not least, IT operations and support was separate from application development.

Though the specialization of the legacy organization was supposed to help the IT organization to capture economies of skill and scale, several problems surfaced.

The level of inter-team dependency is high because of the multiplication of organizational entities that have to intervene on a project. Time-to-market is increasing due to the high level of inter-silo coordination required; for example, between development and IT operations teams. Alignment between business and IT is complicated because of the lack of simple mapping between organizational structures.

The IT re-organization was inspired by the Spotify™ model. Small cross-functional teams named “squads” replaced projects. The size of a squad does not exceed 10 to 15 people. Unlike a project, a squad is a stable team composed of dedicated resources that develops an end-to-end perspective. Squads are product-centric, meaning they develop, deploy, and operate a service. Squads adopt a DevOps culture which translates into the mantra “you build it, you run it”.

Squads are grouped into tribes which are not bigger, on average, than 150 people. In order to maintain and develop functional or technical competencies, chapters and guilds are created; for example, chapters that regroup mobile developers or NoSQL Database Management System (DBMS) experts.

As the number of Agile teams grows to a few hundred squads running in parallel, it is important to define a taxonomy of Agile teams that clearly specifies the scope of each one and minimizes inter-team dependencies. True Enterprise Architecture thinking is required to discover an effective way to decompose the organization, and to draw boundaries that minimize inter-team dependencies.

Figure 13 represents a simplified taxonomy.

The primary goal of an Agile team’s taxonomy is to minimize redundancy and duplication [Rigby 2018]. Because the taxonomy of Agile teams may be different from the formal Profit and Loss (P&L) structure of the enterprise, it is necessary to map it to the division, business unit, and P&L structure of the enterprise.

The Agile Transformation described in this chapter must be initiated by executives. Because Agile Transformation moves the organization from a siloed model to cross-functional cooperation, executives form coalitions to drive change across the enterprise.

Executives who drive change must be driven by an intrapreneurship mindset. Figure 14 shows how they propel change throughout the organization.

Coalitions of executive intrapreneurs adopt an outside-in perspective to identify threats and opportunities that justify change. Middle management pioneers play several major roles:

Middle management pioneers consolidate and analyze feedback coming from the field. They report problems to the executives. When change issues jeopardize successful implementation, middle managers initiate conversations that challenge executives when necessary. The change journey is incremental, inclusive, and experiment-driven. The change model moves away from classical command and control to engage all organizational levels.

Most classical IT governance models:

Classical IT governance does not prevent large projects or programs from failing. Large enterprises have experienced project or program failures resulting in several hundred million dollar write-offs. For example, a large financial services firm had to “kill” a new securities handling system that was supposed to be deployed in several European countries, or a bank that failed to consolidate two core banking systems derailing the Mergers & Acquisitions (M&A) business case.

Among the root causes of these failures, governance issues represent a fair share; for example:

A governance model alone cannot fix problems that are rooted in the social system or culture of the enterprise; for example, when:

Governance deployment will not be effective if it does not address social system or organizational culture issues. The adoption of Agile ways of working at scale tends to challenge classical IT governance even further.

Enterprises which deploy Agile ways of working report that the first visible impact on IT governance is a reduction in the number of projects that get reviewed by IT governance committees. Because the shift to Agile tends to split large projects into smaller pieces no longer managed as a whole, fewer initiatives reach the minimum size to meet the threshold of IT governance. These smaller pieces are no longer projects, but products delivered by Agile teams; they move under the radar of classical IT governance.

Agile organizations are composed of teams of teams that operate in rapid learning and decision-making cycles. The Agile way of working is not compatible with static and siloed organizations that allocate decision rights along reporting lines. Classical IT governance body members are not close enough to the field. They often base their decisions on “second hand” information that may not reflect current reality. In contrast, Agile organizations “instill a common purpose and use new data to give decision rights to the teams closest to the information” [Brosseau 2019].

Agile organizations shift from project to product, which is about replacing temporary teams with stable teams led by product owners. Most Agile teams are cross-functional and stream-aligned. Product owners are responsible and accountable for delivering product features or sub-journeys, which include software components. Because Agile teams are cross-functional and led by product owners who have business and technology acumen, the project manager role is becoming less relevant. Classical IT governance is structured around projects and programs that are reviewed by committees. This is incongruent with new Agile ways of working, that are not project-centric.

In Agile organizations there are fewer standards and processes. Those that remain are defined differently. Instead of being specified as a set of tasks or activities to accomplish, standards are now used as guardrails, which express policy intent. For example, instead of mandating software engineers to select software products from a catalog of approved technologies, a policy could specify a set of operational and security requirements expressed as guardrails. In this example, the purpose is to develop software systems that can be operated reliably and securely. Candidate software products are assessed against guardrails by Agile teams in a bottom-up manner versus being defined and imposed top-down by an architecture office.

Agile governance should not be conceived in isolation. It is part of a larger whole which includes the enterprise social system and its organization. Agile governance is not compatible with command and control thinking.

When authority flows from top to bottom, management is at risk of making decisions that are not anchored in reality. Insights into real problems and opportunities become obscured by the simplification and abstraction of reporting information.

When bottom-up communication is reduced to project status reports discussed during governance committees, it reduces the number of interactions creating even more distance between reality and the few “in command”. Command and control thinking is not an effective way of aligning autonomous teams, because top-down flawed decisions are likely to clash with autonomous teams. Therefore, Agile governance shall not replace a true shift of the enterprise toward Agile, as illustrated in Figure 16.

Agile organizations promote both autonomy and alignment [Ross 2019] by:

Agile governance uses the four levers below to align autonomous teams:

As Agile teams become responsible for delivering and deploying software that impacts the quality of customer experience, the scope and size of the central IT organization decreases. What remains central mainly covers:

In addition to infrastructure platforms, the digital enterprise develops and operates business platforms. Platforms need governance as a set of rules concerning who gets to participate, how to divide cost and value, and how to resolve conflicts. For example, a large financial services organization has created a service marketplace governed by a Design and Operations Authority (DOA).

The missions of the DOA are to:

As more and more IT capabilities are moving under the responsibility of cross-functional Agile teams, business and IT governance are merged into an integrated model; as illustrated in Figure 17 and described in points (1) to (4) in the text below.

The integrated governance model does not isolate IT investment decisions anymore. IT investments are a subset of the investments made by the business to develop new products or journeys. Investment decisions are evaluated against business outcome metrics.

The Agile governance features described in this chapter need to be adapted to fit the context, culture, and maturity level of each enterprise.

Customer experience focus is a way of doing business that meets the expectations of customers. By analyzing the interactions between the enterprise and its customers, Agile Architecture shall enable the delivery of a pleasant and differentiated customer experience.

In today’s digital world, the quality of customer experience is becoming a critical success factor. Consumers and Business-to-Business (B2B) customers have expectations that are shaped by Internet giants. For example, a prime delivery service that includes free shipping, preset ordering options (preset address, preset credit card), and return options with prepaid or printed return labels is becoming the new norm of the retail industry.

A very large car manufacturer recently looked at cars from the perspective of “how it is used by a customer rather than how it is created or delivered”. Does this mean these companies no longer produce cars? Absolutely not. It illustrates the shift toward an outside-in stance that could lead to reinventing the automotive industry.

Just asking clients what they need is not enough to create a differentiated customer experience. Discovering the hidden or untold customer needs is the key to success. Agile Architecture shall use marketing and design methods to discover how customers are likely to use products and services. Job-to-be-done analysis, customer journey mapping, and design thinking are examples of methods used by Agile enterprises. Design thinking, which is a human-centered approach, incorporates human cognition and emotion as key aspects of the value definition.

Research seeks to understand the needs and desires, both explicit and latent, of target customers and users. Prototyping helps to experiment with research assumptions before a product or service is released. It can be done at any stage and helps to architect better solutions. Agile Architecture shall seek rapid feedback loops to verify customer and user assumptions. Examples of feedback loop frameworks include OODA (Observe, Orient, Decide, Act) and PDCA (Plan, Do, Check, Act).

Don Norman [Norman 2013] states that “feedback is critical to managing expectations, and good design provides this. Feedback – knowledge of results – is how expectations are resolved and is critical to learning and the development of skilled behavior”.

Agile Architecture shall enable a holistic orchestration of every single touchpoint of an enterprise and its ecosystem: “the boundaries between external and internal touchpoints are blurring and will continue to do so” [Wind 2015].

Clients and users are empowered to take control of their relationship with the enterprise and its media. Touchpoint orchestration delivers relevant utility or enjoyment at the right time, place, and social or physical context to the right people.

Agile Architecture shall identify the enterprise value streams. Value shall be specified from the standpoint of the customer. A value stream shall be identified for each product or service family – from concept to launch and from order to delivery [Rother 2003].

The analysis of value streams helps make the work flow by giving visibility to problems such as delay, rework, excessive handoff, or interruption. Identifying common steps across several value streams is a key input to operating model design.

Agile Architecture shall segment the organization into autonomous cross-functional teams. Team autonomy is a prerequisite to speed. Why? If teams spend too much time coordinating with other teams, it increases lead time.

The way teams are structured has a major influence on productivity, employee satisfaction, and agility. William Pasmore [Pasmore 2019] reports the discoveries of Eric Trist and his colleague who have analyzed why the introduction of new technology succeeds or fails. They reveal that the way work and teams are organized is the explaining factor: “teams of multi-skilled workers formed spontaneously and took on responsibility for the entire cycle of mining operations … These teams were capable of self-direction, reducing the dependence on supervisors to provide constant direction to individual workers. Problems were solved by the team as they were encountered, resulting in operations running more smoothly and safely.”

Experience quality and operational excellence require effective coordination and cooperation between autonomous teams. Agile Architecture shall balance freedom with responsibility and accountability. Why? Accountability and responsibility improve predictability, which is a prerequisite to:

Agile Architecture shall distribute authority, responsibility, and accountability at each layer of the organization: “it is essential to understand how authority, responsibility, and accountability are designed and broadcast within a socio-technical system and how these concepts can be managed during an engineering process of these systems” [Berdjag 2019].

High-performing teams are more likely to have loosely-coupled architectures than medium and low-performing teams [DevOps 2015 & 2017]. When dependencies are minimized, teams can develop, test, and deploy new functions and features without depending on other teams for additional work, resources, or approvals, and with less back-and-forth communication [Forsgren 2018].

Agile Architecture shall create new modular systems or decompose monolithic legacy ones into loosely-coupled sub-systems and components. Modularity matters because:

The centralized and monolithic data platform is an anti-pattern [Dehghani 2019]. Agile Architecture shall create modular data platforms using domain decomposition logic: “instead of flowing the data from domains into a centrally owned data lake or platform, domains need to host and serve their domain datasets in an easily consumable way”.

Source domain datasets represent closely the raw data at the point of creation and are not fitted or modeled for a particular consumer. Data consumers can always go back to the business facts and create new aggregations or projections. They use data platform self-service capabilities to retrieve the data they need, formatted the way they want.

Agile Architecture shall use a set of simple mechanisms that all elements and connections will use. For example, in the software domain all components or services will expose functionality using standard APIs and all inter-services communication will use these APIs.

The layered architecture pattern tends to lend itself toward monolithic applications, even if presentation and business layers are split into separate deployable units [Richards 2015]. On the business side, functional organizational units such as marketing, finance, or operations are equivalent to layers. Layering tends to create silos, which reduces agility and scalability.

To counter these drawbacks, Agile Architecture shall partition the enterprise and its systems. Partitioning shall be market-driven at the business level, capability-driven at the operating model level, and domain-driven at the software level.

Agile Architecture shall structure Agile teams in a way that maps the system’s intentional architecture.

Conway’s law states: “Any organization that designs a system (defined broadly) will produce a design whose structure is a copy of the organization’s communication structure.” The Inverse Conway Maneuver is about shaping the enterprise’s organization in a way that mirrors its intentional product and software architectures.

The interested reader can find a description of the Inverse Conway Maneuver in Chapter 1: “The Problem with Org Charts” of Team Topologies: Organizing Business and Technology Teams for Fast Flow [Skelton 2019].

It is also important to note that neither the enterprise’s organization, nor its product and software architectures remain static, and success in implementing this axiom relies on the co-evolution of both over time.

Organizations shall be described at different granularity levels:

Agile Architecture shall welcome changing requirements, even late in development. The architecture model is a living artifact that evolves and is continuously refined as requirements evolve and the organization learns.

Though BDUF is an Agile anti-pattern, does it mean architecture should solely be a product from emergence? As James Coplien argues [Coplien 2010], some intentional architecture saves waste and accelerates the decision process.

Agile Architecture shall seek a balance between intentional and emerging. Intentional architecture provides value if it is done differently. Intentional architecture represents a set of assumptions that must be verified. It should not slow down the integration of new requirements.

A true Agile team does not deliver a project but a product. Product-centricity shall drive the Agile organization: “Moving from project-oriented management to product-oriented management is critical for connecting software delivery to the business” [Kersten 2018]. The product-based focus helps to create stable Agile teams that have an end-to-end perspective and are staffed with stable resources.

In an Agile context, product-centricity refers to the shift from temporary organizational structures (projects) to permanent ones. A product-centric organization is composed of cross-functional Agile teams which are responsible for developing products and operating them. The DevOps principle “you build it, you run it” is core to product-centricity.

Agile Architecture shall create an environment that helps developers to build security into their design and codebase from the start. The Agile enterprise will shift from DevOps to DevSecOps, where security is embedded at every stage of the software delivery lifecycle.

A full discussion of this topic is available in the O-AA Security Playbook 2021.

Figure 18 also highlights a fifth concern, Organization, which belongs to the “Work System Perspective” and is related to the other concerns via the Inverse Conway Maneuver; see Section 9.12.

The Experience concern provides an outside-in view that informs product development. “Value-Driven” implies that unlike some Agile methods that prioritize features, this document prioritizes business objectives and outcomes. Features must solve business objectives, measured by Key Performance Indicators (KPIs).

In alignment with Section 12.1, this document recommends developing concurrently the technical and social systems of the enterprise. This document is not about aligning the “Information System” layer with the “Business” layer. Business decisions impact all concerns, including software and hardware concerns. For example, when Jeff Bezos issued his mandate, he made a software architecture decision and a business decision to impact Amazon’s organization and strategy:

By prohibiting any form of inter-process communication other than a service call, Jeff Bezos laid the foundation of what would become “microservices” (Technical System Perspective). He also created the conditions for increased team autonomy by reducing inter-team dependencies (Work System Perspective). By requesting that all service interfaces be externalizable, Jeff Bezos enabled the creation of a third-party vendors market. This was instrumental in helping Amazon compete against eBay based on a two-sided market business model.

Now it is clear that the building blocks do not represent layers, the relationships that link them can be reviewed; see Figure 19.

The arrows in the diagram represent preconditions. For example, the arrow that links "Customer Research" to "Product Architecture" represents that the outcome of "Customer Research" is a precondition to starting "Product Architecture".

The building blocks are generic. They are applied at different levels of granularity.

How to decompose an enterprise is a key architecture decision.

“Hierarchy, I shall argue, is one of the central structural schemes that the architect of complexity uses.” (Herbert Simon)

Simon observes that the word “hierarchy” usually refers to situations in which each of the sub-systems is subordinated by an authority in relation to the system it belongs to: “We shall want to consider systems in which the relations among sub-systems are more complex than the formal organizational hierarchy just described” [Simon 1962].

This is illustrated here using a banking example. At the top level, this universal bank is composed of three business lines: retail banking, wholesale banking, and global markets.

At the second level, the retail bank could be decomposed along a customer segment logic: consumer, small business, and enterprise.

Figure 20 represents a third level of decomposition that mixes customer segments and product logic. This example illustrates that levels should not be confused with layers.

How retail banking is decomposed matters. Each bubble represents a modular part of the business which can operate in relative autonomy vis-à-vis other parts of the business. The fewer dependencies that remain are identified, standardized, and managed. For example, “Consumer Prospect On-Boarding” will order credit cards for new customers.

Figure 21 illustrates the O-AA segmentation approach, which is described in more detail below.

The highlighted rectangles represent building blocks, and the arrows represent direct links between the building blocks and work products.

“Agile Strategy” influences the definition of the enterprise’s “Top-Level Organization” (a). Business lines and corporate functions are defined or adjusted to support the enterprise’s strategic direction.

“Experience Design” discovers the products that meet the needs of client segments. “Product Architecture” explores the modularity–integrality trade-off, defines how to decompose products into “Modular Components” (b), and identifies product platforms.

“Operations Architecture” specifies the “Value Streams & Processes” that deliver products and design the target “Operating Model” (c).

“Domain-Driven Design” drives the decomposition of the domain into bounded contexts which are organized into “Context Maps” (d). Context maps (see Section 20.3) define the patterns that link bounded contexts; see Figure 49, Figure 50, and Figure 51.

The “Modular Components” that compose products, the “Operating Model”, and the “Context Maps” are inputs to “Agile Organization”, which decompose the top-level organization into a “Teams of Teams” (e); guided by:

Figure 22 lists the shifts that Agile Architecture requires.

For supporting media, the Navigation Table contains hyperlinks to the other chapters of this document, which provide additional content to the materials covered in this chapter. The Part 2 hyperlinks follow the structure of O-AA Building Blocks.

The law of unintended consequences states that decisions to intervene in complex situations create unanticipated and often undesirable outcomes.

In her article, Why Hypotheses Beat Goals [Ross 2019], Jeanne Ross recommends the development of a culture of evidence-based decision-making by promoting hypothesis generation: "A hypothesis emerges from a set of underlying assumptions. It is an articulation of how those assumptions are expected to play out in a given context. In short, a hypothesis is an intelligent, articulated guess that is the basis for taking action and assessing outcomes".

In a complex situation, hypothesizing simple causal relationships may not be sufficient. It is also not practical to experiment with “hard-to-reverse” decisions. The Agile strategist needs new methods to frame critical decisions and prevent unanticipated and undesirable outcomes. Richard Adler proposes a well-articulated and comprehensive method to help bend the law of unintended consequences [Adler 2020].

One of the central ideas of this method is to test drive decisions, much like consumers road test vehicles to get a better sense of how they satisfy wants and needs before choosing which one to buy. Test driving a decision provides insight into its consequences for a business and other parties of interest. This is accomplished by projecting the likely outcomes of implementing that decision over time under a range of plausible future conditions. Test drives for decisions, much like cars, are admittedly limited in scope and offer insights into the future that are imperfect. That said, they offer far better chances for identifying (and improving) a Type 1 decision than the alternative of blindly choosing a Type 1 option without testing it.

Richard Adler’s method combines:

Its key differentiators are:

The mechanical details of modeling and simulating dynamic scenarios are fairly complicated. Richard Adler indicates that: "Fortunately, these details are left to programming and simulation experts; the test drive method insulates analysts and decision-makers from most of the underlying technology".

Simulations must be instrumented with relevant business and technical performance metrics to drive comparative analytics of outcomes. The decision rule identifies an option that is robust in the face of the law of unintended consequences, namely the one that avoids “train wrecks” and which performs better than the alternatives across a spectrum of plausible futures.

Simulations can also be used during execution to periodically re-project outcomes of the chosen decision given the results to date, and scenarios updated to reflect the current situation. In this mode, the method detects emerging problems, prompting and allowing for mid-course corrections to ensure success.

This iterative process drives decision options experimentation – testing and validation, or refinement – in a virtual world. This is similar to conducting Agile experiments, without the high cost associated with a real experiment, such as an MVP.

In summary, this method does not replace Agile experiments, but adds a new way of evaluating – and iteratively refining – decision options, which is well suited to Type 1 decisions.

To improve the odds for the success of strategy implementation, the Agile enterprise shall define an effective system of objectives.

In alignment with the strategic vision, the deployment process starts with the definition of the few top-level objectives which will be further decomposed recursively into sub-objectives. The objectives tree shall contain objectives:

An example is used here to illustrate a policy deployment anti-pattern. An enterprise wishes to change its positioning toward the high-end market. In order to satisfy high-end customers, it must dramatically improve service reliability.

It is not good enough to assign the same objective down through an organization, as illustrated in Figure 23. Why? Because when the objective reaches a lower level of the organization:

The Agile strategist will decompose objectives using the approach below:

Figure 24 illustrates how the “Improve the Reliability of Our Services” objective can be decomposed.

Some of the sub-objectives can be assigned to one team. For example, “Monitor Distributed Systems Effectively” could be assigned to “IT Operations”. Being responsible and accountable for an objective may require a team to manage dependencies with other teams. In this example, “IT Operations” needs inputs and buy-in from “Engineering”.

Other objectives, such as “Shift Toward an Open and Blameless Culture”, are too broad to be assigned to a single team. In this example, the objective will be shared by several teams.

When the objectives decomposition tree is stabilized, two things need to happen:

An OKR spells out the company’s priorities in terms of specific accomplishments and performance improvements. The Objective consists of a clearly defined qualitative change, while the Key Result is a specific, often quantitative performance target that must be met. Agile organizations should set OKRs annually and assess progress against them on a quarterly basis [McKinsey 2019].

KPIs can help the enterprise measure progress toward key results.

Strategy deployment is both top-down and bottom-up. When the vision of leaders is translated into concrete OKRs throughout the enterprise, the “catchball” process entails frank, reality-based discussions between and within management levels. Leaders normally define required results; the teams the means [Dennis 2006].

Eric Trist and Ken Bamforth studied the consequences of introducing new technology, coal-cutters, and mechanical conveyors in English coal mines [Trist 1951].

Autonomous teams are capable of self-organization. They are more resilient when they face adverse conditions, and can adapt when circumstances change.

Self-organization refers to the emergence of an order that results from the collective interactions of the system’s components. Self-organization is central to the formation of numerous structures, including biological ones. For example, there is no centralized control in a neural network. All the nodes are connected directly or indirectly to each other, but none is in control. Yet, together, they learn from complex patterns of inputs.

Self-organization cannot happen in an enterprise that:

Team autonomy, which is a precondition to agility, is hard to achieve at scale. First, the enterprise needs to trust its people, otherwise they will not enjoy the freedom to act. Second, effective alignment mechanisms are necessary to avoid chaos; see the four alignment levers in Section 8.3. Third, the work system should be designed to regroup activities that together make a functioning whole, otherwise the work will lack meaning.

Agile teams can be classified into three categories: stream-aligned, platform, and competency teams.

In technology companies, Marty Cagan reports that a product team is generally composed of a product manager, a product designer, and somewhere between 2 and about 10 to 12 engineers [Cagan 2018]. Product teams fall into the category of stream-aligned teams. Product teams are stream-aligned.

A product team is composed of highly skilled people who work together for an extended period. True collaboration between product, design, and engineering roles resembles the type of collaboration described in the experience report of Section 12.1. A product team is not a hierarchy. Product team members typically continue to report to their business unit; for example, engineers may report to an engineering manager.

Product teams have the responsibility for all product-related work, including the full customer experience. When the scope of the product is too large, a product team can be composed of sub-teams. Each sub-team is then responsible for a smaller but meaningful piece of that experience, and for the features that deliver it.

A capital management example is used here to compare and contrast a siloed command and control organization with an Agile one. This section shows how the wholeness and multi-skilling principles from socio-technical systems characterize the new Agile capital management organization.

Regulators issue minimum capital requirement regulations. Banks are mandated to compute the amount of liquid capital they need to cover for credit, market, and operational risks. The capital requirement numbers matter because they set a cap on growth and impact returns on equity. Because the bank was not managing its capital to the satisfaction of regulators and shareholders, the executive committee decided to launch a program to fix it; see Figure 27.

This is a classical program structure run by a program director assisted by a project manager. The co-sponsors of the program are business-line heads and their Chief Financial Officers (CFOs). The scope is huge; it covers all businesses such as retail banking, Corporate & Investment Banking (CIB), global markets, and consumer finance in all geographies.

Work streams are functionally aligned; for example, capital management models to improve the Risk-Weighted Asset (RWA) calculation, or the data quality to generate more accurate data. Each work stream regroups experts in a domain. For example, portfolio securitization experts are knowledgeable enough to decide which securitization structure is better to reduce the RWA.

After six months of operation, the program delivered few if any results. Why?

Since the bank had been successful at deploying agility on the IT side, business executives decided to radically change the capital management setup; see Figure 28.

Though the topic was regulatory, program sponsors identified two client constituencies; regulators and shareholders. Two expected outcomes were defined:

Capital management shifted from a temporary program structure to a set of permanent and cross-functional teams. The scope of each team was bounded by their business line and their products.

New capital management teams were now stream-aligned. The front line became responsible and accountable for the quality of capital management numbers. For example, the “Consumer Loans Capital Management” team took ownership of the regulatory capital consumed by the consumer lending business.

The cognitive load of stream-aligned teams is reduced compared to the former program streams. Each capital management team only needs to know their own business and products; not all of the bank’s businesses and products. Team members can remain capital management generalists because they get the support of the experts who staff competency teams.

Functional experts, such as “Capital Management Modeling” specialists or “Securitization” specialists, were regrouped into competency teams to provide support to stream-aligned capital management teams.

How this model works is illustrated here using the securitization example. Securitization is about creating composite financial instruments, real or synthetic, by pooling various financial assets. The issuer sells this financial instrument to investors. Securitization offers the bank an opportunity to free up capital.

The CFO defines the overall securitization strategy. However, it does not have enough operational knowledge to study the feasibility of securitizing enterprise loans. In this example, the “Enterprise Loans Capital Management” team is responsible for defining the specific enterprise loans securitization strategy and is accountable for the outcomes: compliance with regulations and reduction of regulatory capital consumption.

The new Agile organization helped improve the quality of regulatory capital management KPIs, accelerate RWA production lead time, and reduce operating costs. Far fewer issues are reported and regulatory capital consumption decreased by 15% on a like-for-like basis.

This approach drives experience design from an outside-in thinking stance. It iterates design thinking cycles, as described in Figure 30.

Each iteration tests the hypotheses formulated in ideation using prototypes. As progress is made in the product development lifecycle, new hypothesis testing practices are used.

For example, companies like Netflix^® and Facebook^® have pioneered techniques like “canary deployments” and “A/B testing”. Two different features are tried out simultaneously, and the business results are measured. For example, are customers more likely to click on a green button or a yellow one? Testing such questions in the era of packaged software would have required lab-based usability engineering approaches, which risked being invalid because of their small sample size. Testing against larger numbers is possible now that software is increasingly delivered as a service [DPBoK 2020].

Products are at the crossroad of strategy and marketing, design science, operations, and engineering. Understanding products requires the borrowing of practices and vocabularies from these disciplines. They jointly contribute to discover, architect, and develop products.

The term “product” is widely used in the Agile community. Though numerous articles and books discuss the shift from project to product, at the time of writing, few, if any, give a definition of product. This document believes the Agile and Enterprise Architecture communities need a shared understanding of what product means.

Theodore Levitt states that a truly marketing-minded firm tries to create value-satisfying goods and services that consumers will want to buy [Levitt 1960]. This implies that product includes both goods and services. In his book Innovation in Marketing [Levitt 1962], Theodore Levitt answers the question, what is a product?

He observes that designing a better and less expensive piano would have less impact on sales than providing a simpler and more enjoyable beginner’s piano method. He concludes that it is necessary to expand the product the organization wants to sell. It must be defined much more broadly to include non-core elements that contribute to a user’s satisfaction. The conclusion is that product refers to both goods and services.

Following Theodore Levitt’s path, Marty Cagan [Cagan 2018] provides a holistic definition of product:

“A product includes the functionality – the features. But it also includes the technology that enables this functionality. It also includes the user experience design that presents this functionality. And it includes how we monetize this functionality. It includes how we attract and acquire users and customers. And it can also include offline experiences as well that are essential to delivering the product’s value.”

When using the terms product and service, ambiguity often remains:

To avoid terminology and meaning ambiguity, the definition of product can be found in Section 2.40.

Products deliver outcomes which are defined by the customer/user experience and depend on circumstances. In the piano example, the outcomes could be:

Customer research combines traditional market research with customer/user experience research.

“A lot of times, people don’t know what they want until you show it to them.” (attributed to Steve Jobs)

Product discovery is about defining products from an outside-in viewpoint. Experience mapping helps to articulate the product’s value proposition and discover its features, outcomes, and benefits. It combines customer research and product discovery.

Ride-Hailing Company provides the features below:

This list of features does not say much about the benefits that riders and drivers experience. The major benefit for riders is to get a ride with minimum waiting time at a reasonable price. The major benefit for drivers is to maximize revenue while minimizing idle time. The relationships that link features to benefits are indirect and require an understanding of the system as a whole.

The business model of Ride-Hailing Company is based on a two-sided market platform. One side of the market caters to riders who need transportation. The other side of the market caters to drivers who need to earn money.

The value to each of its participants grows the more people use the service [Parker 2016]. Ride-Hailing Company’s network effect is captured in Figure 33.

Ride-Hailing Company has to balance rider demand with driver capacity, and needs to motivate drivers so enough of them show up at pickup time, otherwise riders have to wait too long.

The interest of the drivers is not aligned with that of the rider. The more drivers, the less waiting time for riders but the more idle time for drivers.

Behavioral science has shown how drivers can be subtly influenced to act against their own interests through the employment of social scientists and data scientists who experiment “with video game techniques, graphics, and non-cash rewards of little value that can prod drivers into working longer and harder – and sometimes at hours and locations that are less lucrative for them” [Scheiber 2017].

This example epitomizes the complex relations that can link features to outcomes. Just adding new features or managing a feature backlog does not guarantee success. Instead of prioritizing features, a competent product manager should prioritize outcomes. This implies shifting from a product owner role toward a product manager role, as described in Section 12.4.1.

Of course, the experience delivered by features can be enhanced by “killer” User Experience (UX) ideas; for example, see Figure 34. However, what used to be a killer UX is no longer differentiating. For example, several online services now include an equivalent UX to track delivery. Enjoying a lasting competitive advantage requires much more than adding features and flashy user interfaces to a service that is undergoing continuous evolution.

Finally, without also considering the “ilities” of the product, drivers and riders may quickly tire of the service due to poor performance and/or availability. One mechanism to ensure these are always being considered is through the use of Fitness Functions, as described in Section 6.3.2.

In a manufacturing context, Karl Ulrich gives a definition of product architecture [Ulrich 1993]. Several years later, Edward Crawley gives a systems architecture definition that is quite similar [Crawley 2016].

"Product architecture is: (1) the arrangement of functional elements; (2) the mapping from functional elements to physical components; and (3) the specification of the interfaces among interacting physical components." (Karl Ulrich)

The quality of customer experience, with its emotional dimension, is not part of Karl Ulrich’s definition. Experience design identifies the functional elements or features that contribute to the quality of customer experience.

In a manufacturing context, product refers to goods and not services. This document will now describe how this definition can be interpreted in a service context.

Product features are the equivalent of functional elements. In a service context, the physical elements are mostly the physical evidences and devices used by clients, employees, and other stakeholders. Therefore, it is necessary to extend the definition to cover non-physical components.

Fabrice Seite and his co-authors describe a service product at three levels [Seite 2010]:

At the second level, defining the modularity of the enabling process or value stream is a major product architecture decision. The mapping of product features to process steps is specified in alignment diagrams; see Figure 35.

At a higher granularity level, the theory of interdependence and modularity addresses architecture questions.

Product architecture breaks down products into components and specifies the interfaces that link components.

“Products and services have multiple constituent components, and they go through several value-added processes before reaching consumers. The place where any two components fit together is called an interface. Interfaces exist not just within products but also between stages in the value-added chain” [George 2018].

Clayton Christensen defines a product in a way that focuses on how components interact, fit, and work together [Christensen 2013]:

"A product’s architecture determines its constituent components and sub-systems and defines how they must interact – fit and work together – in order to achieve the targeted functionality." (Clayton Christensen and Michael Raynor)

According to Clayton’s theory, architecture is interdependent at an interface if one part cannot be created independently of the other part. He claims that interdependent architectures:

In contrast, a modular architecture specifies the fit and function of all elements so completely that it does not matter who makes the components or sub-systems, as long as they meet the specifications.

This theory formulates the hypothesis that flexibility comes at the cost of performance because a modular architecture gives engineers less freedom. When the basis for competition is functionality and reliability, a modular architecture is at a disadvantage.

However, Clayton Christensen observes that, over time, an inferior product targeting an underserved market will improve to such a point that its performance will challenge established products. When that happens, any further product improvement by incumbents is likely to overshoot customer expectations. This implies that customers will no longer be willing to pay a premium price, which is disruptive for incumbents.

This document illustrates the trade-offs that an integrated enterprise must make between integration and modularization. In an Agile at scale organization, team autonomy is important because a large number of Agile teams work in parallel. Autonomy minimizes inter-team dependencies. A modular architecture reduces dependencies but could degrade the overall system’s performance.

Vente-privée.com offers a complete service to sell unsold goods from well-known brands. The value proposition for these brands is to make money from an unsold inventory without hurting the brand. It organizes, exclusively for its members, event-driven sales of products with strong brand names and big discounts (-30% to -70%).

The service to brands includes the shooting of products, the selection of photos, the original trailer, the creation of a catalog, putting it online, issuing invitations to members, sales, online payment, and delivery to the members.

The value proposition for consumers is to be able to buy brand name products at very attractive prices. Although the quality of service for the consumer was not always there, the value proposition was so strong that it did not impact the triple-digit sales growth. However, the expectations of quality of service and personalization have continued to rise under the influence of tech giants.

For example, Amazon has created the expectation of fast delivery. Although Vente-privée’s business model is different to that of Amazon, it becomes difficult to ignore these new expectations. As the business model is based on building inventory upstream of the sale, it becomes necessary to develop predictive algorithms to anticipate demand and thus be able to deliver quickly.

Because of this lack of predictive analytics, the logistics system is not capable of estimating delivery times in real time. The company is also not able to put items acquired from different sales in the same basket. Vente-privée must upgrade its delivery service in a way that is well-integrated with sales. This makes it difficult to outsource a logistics system that has such specific requirements.

Vente-privée has developed a studio capable of creating high-quality videos to present brand products. It only takes a few days to produce a new video clip whereas a communication agency would take a month and cost much more. Vente-privée’s studio creates its own music and has a network of models that it mobilizes to shoot scenes featuring models wearing clothes. The video studio service can be easily modularized. Vente-privée could decide to make it a product that it would sell to other enterprises, emulating Amazon’s creation of AWS.

The Vente-privée example illustrates that the modularity–integrality trade-off is more subtle than Christensen’s theory says. Christensen’s theory does not account for modular systems that at the same time show or require high degrees of integrality. Increasing modularity does not necessarily entail decreasing integrality, and high levels of modularity are not always a sufficient condition to facilitate system reconfiguration, or architectural innovation [Miraglia 2014].

The delivery service can be loosely-coupled with the sales service via a standard set of APIs. However, the specificity of the Vente-privée’s business model directly impacts the logistics system design. This creates indirect coupling. On the other hand, the services the studio provides are similar to those provided by communication agencies. Therefore, they could be productized and sold to an outside market, though these services are part of the Vente-privée’s integrated product.

This document adopts the modularity and integrality definitions proposed by Stefano Miraglia:

These definitions recognize that modularity and integrality must be analyzed along several dimensions. Modularity and integrality are separate and concurring properties of complex systems.

"Dimensions of modularity reflect the different traits of independence among modules and describe component-level interrelationships. Dimensions of integrality encompass factors that affect the relationship between the component level and the system level, and describe cross-level interrelationships. The interplay between modularity and integrality is crucial to the dynamics and evolution of systems." (Stefano Miraglia)

Products built around modular product architectures can be more easily varied without adding tremendous complexity to the manufacturing system. For example, Swatch^® produces hundreds of different watch models, but can achieve this variety at a relatively low cost by assembling the variants from different combinations of standard components.

Product platforms support the development of highly differentiated products that share a substantial fraction of their components. Product platforms are also referred to as innovation platforms [Cusumano 2020]; their main benefits are:

The challenge is to resolve the tension between the desire to differentiate products and the desire for these products to share a substantial fraction of their components. Different methods in different industries emerged to create and manage product platforms; for example:

The classical integration-oriented platform approach in software product lines should evolve from a layered and hierarchical model toward a truly Agile model. Splitting the product teams into a platform team and several product teams suffers from limitations:

Christian Prehofer and his co-authors recommend evolving toward a compositional product family approach [Prehofer 2007]. Instead of being a fully integrated software solution, the software platform becomes a set of components, architecture guidelines, and documentation as code. Powerful digital platforms can be created by combining the compositional product family approach and the modern software architecture patterns as described in Chapter 21.

Now that software in a car costs more than the engine, and anticipating that this trend will continue with autonomous driving and electric engines [Kersten 2018], vehicle and software platforms could converge.

Product platform decisions are architecturally important decisions that impact product performance and product development cost. It impacts positively or negatively the ability of the enterprise to innovate. It also impacts customer experience.

For example, the Toyota™ New Global Architecture (TNGA) created a powerful competitive advantage in both its product development system and its products [Morgan 2019]. TNGA enabled a substantial boost in driving performance by creating a lower center of gravity while also allowing for more freedom in design. This illustrates the impact of platform architecture decisions on customer experience: better driving performance and increased ability to create styles that customers like.

The two major types of product architecture decisions are modularity-integrality trade-off and product platform decisions. Product architecture decisions affect the competitiveness of the enterprise. They combine intentional and continuous architecture. The higher the up-front investment to create the platform, the more front-loading design is required, as is the case in the automotive industry.

In such circumstances, SBCE is the way to go, rather than the iterative MVP approach. Once the platform is created, using an MVP approach is more realistic because of the reuse and flexibility the product platform provides.

When products are mostly made of software, it is easier to develop product platforms incrementally. As a general rule, this document recommends combining set-based innovation with Agile iterations, as described in Chapter 5.

McKinsey defines moments of truth as:

How well or how poorly the enterprise handles moments of truth has a decisive impact on the quality of the customer experience. Moments of truth test the quality of operations because:

This document recommends using moments of truth to prioritize operational improvements [Patton 2014].

The goal should be to maximize outcomes while minimizing costs by minimizing new features development. The focus on moments of truth gives the opportunity for cross-functional teams to develop their level of understanding of what really matters and to explore the space of solutions to discover the best way to deliver an outstanding experience.

This is illustrated with the example of a housing rental enterprise. The first moment of truth for new prospects is to be rapidly informed when available properties that meet their needs become available.

What does it mean to be informed rapidly? Is it to receive a choice of properties immediately? The next business day? How can the enterprise ensure the choice of property meets client needs? Answering these questions requires a deep understanding of client expectations and also the ability to determine which operating model improvements are required.

Generating a short list of available houses or apartments that meet needs most of the time is critical to the moment of truth. If clients rapidly receive a list containing mostly properties that do not match their needs, they will be rapidly disappointed. In order to prevent this, the enterprise needs a very large inventory. This may require the enterprise to develop new sources of rental properties that it can add to its inventory.

In addition to the large inventory requirement, new features are needed:

Mistakes and errors are inevitable, but dissatisfied customers are not. Paradoxically, when a service defect such as a late delivery or a lost parcel is fixed, frustrated customers can become more loyal than they would if no problem had occurred.

The difficulty is to grant employees the freedom and the responsibility not to follow “standard operating procedures” when solving urgent or important customer problems. The enterprise needs to:

Digital technologies, in particular AI, dramatically expand the scope of what can be automated. Does this mean the automation potential should be pushed to its limits? Not necessarily; for example:

Automating requires a careful assessment of benefits and drawbacks. Instead of automating in a big bang manner, the Agile enterprise will conduct experiments and automate using an incremental approach when possible.

Automation failures come in two forms: misses and false alarms. For example, the system has failed to detect a harmful situation, or the system acted when it should not have.

A failure to detect can harm the customer; for example, a fraudulent payment not being prevented by an automated fraud detection algorithm. On the other hand, if the fraud detection algorithm disables a credit card when the client is abroad late at night and during a weekend, it becomes a service failure in the case of a false-positive.

Computers and humans have distinct abilities. Therefore, augmenting the human with the power of automation and AI could be a legitimate goal if the man-computer symbiosis is achieved.

When automation is used as an augmentative tool, it leaves decisions to humans. Now that deep learning scores better than humans in the accomplishment of many tasks, should there be a switch to fully-automated decisioning? What happens when an unforeseen event occurs that the machine is not capable of handling?

Journey mapping is a great tool to help align experience mapping with the operating system that is supposed to deliver it. When the mismatch between experience and target operating model cannot be solved locally, it is necessary to look at the operating system as a whole. Improving the human side and automating will often require a holistic approach.

James Martin observes that “a value stream is often referred to as a process” [Martin 1995]. Unfortunately, the word process is used in many ways by different professionals; for example, systems analysts would call accounts receivable a process. “Accounts receivable is not a value stream; it is not an end-to-end stream of activities designed to satisfy a customer.”

This section describes the evolution of the process and value stream concepts. It will describe them in detail.

Value stream mapping is about applying the scientific method to improve the whole. The value stream improvement steps are:

Value stream mapping starts with the whole value stream to initiate an improvement cycle at that level. This helps to identify inter-process dependencies. Equivalent improvement cycles can then be applied to each of the processes that compose the value stream; see Figure 38. In summary, value stream mapping iterates PDCA cycles, and improvement cycles can be conducted concurrently on different parts of the value stream.

Improvement cycles follow PDCA:

Figure 39 illustrates this.

This document will now describe the current conditions of client on-boarding.

A future state map provides a set of challenges and target conditions. A target condition is a hypothesized work pattern that will move a step closer to the desired goals [Liker 2017]: “A good understanding of the current condition relative to the challenge will helps identify whether this target condition, or some others, makes sense to test”.

Remember, Table 3 helped to identify three specialized client on-boarding value streams. This document will now illustrate how to develop a future state map using the simple on-boarding value stream example.

Value stream mapping can be used to improve product development value streams and production value streams. Product development includes designing the value stream that delivers the product. The value stream mapping tool mostly helps to identify challenges, target conditions, countermeasures, and ideas for experimentation. Value stream mapping provides guidelines to stimulate thinking; it does not provide standard solutions to be applied mechanically.

The capability can have several meanings, some sample definitions are shown below:

In the first three definitions, capabilities provide a representation of the enterprise. To differentiate capability modeling from other enterprise’s representations, these definitions use qualifiers; for example:

The definition from Constantine Samaras and Henry H. Willis relates capability to achieving a desired effect. A capability becomes a result that must be achieved, rather than a representation of the enterprise.

The definition from Jan Van Mieghem and Gad Allon adds a criterion: the operating system should be treated as a black box. This helps further filter out capabilities that are just activities carried out by the operating system.

This document asserts that operational capabilities should refer to the outcomes the operating system delivers; to what it does well or should do well.

The examples below illustrate the concept of operational capability:

When capabilities are defined in this way, the focus is on the what and why questions. When capabilities remain in the problem space, it opens the range of possible solutions. When the term capability is used to mean an activity, even at an abstract or self-contained level, it raises two issues:

OKRs must be defined for each operational capability. The objective defines the “to-be” operational capability. Key results should measure operating system outcomes, not outputs or tasks. In applying the OKR discipline when defining operational capabilities, this document aims to:

Operations architecture starts with an outward view that specifies the operational capabilities that products need. The analysis of operational capabilities is conducted for each targeted customer segment and is prioritized around cost, lead time, quality, flexibility, and agility.

Product architecture specifies what operations architecture must do particularly well: the required operational capabilities. As there is more than one way to architect operations, the solution space shall be explored by developing scenarios that combine several levers:

One of the major challenges of operations architecture is that trial and error is not an option when the magnitude of investments and time required to build new operational capabilities are too high. When operational decisions are difficult to reverse (Type 1), this document recommends following the method defined in Section 11.2 to test drive the most critical decisions.

Many operations architecture decisions can be experimented and reversed if necessary (Type 2 decisions); for example:

Instead of applying these levers in isolation and in a piecemeal manner, this document recommends combining them to drive operational transformation guided by customer/user experience outcomes.

Combining Lean redesign, advanced automation, and analytics provides a multiplier effect. This is illustrated with the Figure 32 from Chapter 13.

Providing a great experience is challenging for insurers because they must balance the experience quality while collecting the most accurate information at a time when the customer is going though an unpleasant situation; for example, dealing with a traffic collision. Digital technology can help collect needed information without degrading experience quality.

Mobile technology combined with open APIs can reduce the burden of collecting the required information while improving its quality. The insurer can dispatch the vehicle’s location to emergency services. The driver can upload pictures of the damaged vehicles and file a claim with their smartphone.

The insurer can use advanced analytics to identify suspected fraud up-front and expedite the claim handling process for the majority of claims. It can also triage the claims that require special investigation and streamline the process for others.

The manual and time-consuming tasks that are done by customer service agents such as verifying insurance coverage can be automated. This not only reduces costs, it improves customer experience by reducing the lead time required to settle claims.

By combining these levers, the insurer can at the same time dramatically improve operational efficiency and the quality of customer experience. This is an example of operations innovation that enables experience innovation. Operations and experience design are not conducted in sequence, but concurrently by a cross-functional team.

This is illustrated with the two types of operations decisions with operations at AR, a fictitious apparel retailer.

When AR invested €100 million to open a logistics center for children’s clothes to reduce transportation costs, this decision could not follow a trial and error approach.

When AR introduced the automatic replenishment of basic products (10-15% of all products) it did not require a big up-front investment. With the help of academics in operations management, AR developed an algorithm for allocating inventory at logistics centers to stores.

The decision to introduce automatic replenishment did not change the essence of the decision system; it only automated non-essential decisions to reduce the store manager’s workload. This type of operational improvement can be conducted in an iterative manner using prototyping and experimentation.

The success of AR shows that operational excellence requires a great operational design and great people to carry it out. Neither can make up for the lack of the other.

The culture at AR encourages fast decision-making and autonomy, but not at the expense of operational excellence or brand consistency. Improvement ideas often come from the field. They are first tested and experimented locally.

When improvements are proven and ready for prime time, they are deployed incrementally on a global scale. For example, AR generalized the Japanese practice of having a five-minute staff meeting before opening time to discuss the day’s objectives.

This practice has become known elsewhere as “the Japanese meeting”. An AR executive commented, “You’ll get great ideas from the stores if you have the ability to hear them”.

Continuous improvement is ingrained into the culture at AR, which has similarities with the Toyota way. A standard practice is a best practice at some point in time. It can be challenged and improved at any time. When the new practice is proven, it is applied with discipline and consistency across the board. As Toyota does, AR pays attention to disseminating its culture.

Regional directors, HR directors, and country managers constantly visited stores to explain the culture directly to the staff and to monitor store performance.

AR illustrates how operations architecture and product architecture complement each other and must evolve concurrently.

Determining what should be the right level of product variety is a critical decision which requires proper interlock between product and operations strategies. In Fast Innovation, Michael George and his co-authors report that a formidable impediment to achieving operational excellence is the burgeoning complexity in a portfolio of offerings [George 2005].

In itself, complexity is neither good or bad; it depends on how complexity is perceived and valued by customers. Too often, complexity is driven by product owners who prioritize features rather than outcomes; see Section 13.4.4.

Higher product variety increases costs throughout the supply chain, production, distribution, and customer support.

More product variety also increases the likelihood of errors and operational problems. A European bank tried to understand the patterns that would drive demand for banking help desk support. It found a strong correlation between new product introduction and the rise in volume of help desk calls. Further investigation demonstrated that additional calls were due to either:

In the retail industry, the more product variety in a category, the more inventory you need to carry. The more inventory, the more working capital to fund that inventory, which hurts profitability.

Before offering too many products or variants of the same product, customer research should verify that:

In case of doubt, the opportunity to simplify the enterprise’s product portfolio should be analyzed and experimented.

The shift to an architecture capable of handling large amounts of data coming at high speed, processed continuously, and acted upon in real time makes it possible to access data when and where you need it. Technologies such as Apache Spark™ or Flink^® enable the shift toward real-time data streaming. Analyzing the evolution of the technology landscape is beyond the scope of this document; it evolves so rapidly. The requirements of well-architected streaming solutions are:

Enterprises create innovative services by coupling real-time data streaming with AI; for example, the Capital One Auto Navigator^® App [Groenfeldt 2018].

The new service idea started with a survey of 1,000 nationally representative US adults to learn how they approach the car shopping process.

Customers can use their smartphone to recognize a car, even in motion, that caught their attention. AI is used to classify the car and help find similar ones that are selling in nearby markets. The user can save the information to prepare for a visit to a dealership.

Once customers have identified cars they like, the app (see [AutoNavigator]) lets them personalize financing to simulate payments specific to the car they want. Pre-qualification has no impact on their credit score. When they are ready to purchase the car they have all the information needed, spend less time at the dealership, and get a better deal.

“I was very impressed with how easy my buying experience was. I had my financing ahead of time and was able to make my time at the dealership so much faster.” (Testimony from the Capital One Site)

Now that the power of combining great customer research with fast data and AI technology to develop innovative products has been described, the Data Architecture impacts can be analyzed.

Most fundamental data management principles and best practices remain valid even though some need to be updated to account for business and technology evolution. The emergence of new data engineering patterns requires a fundamental shift in the way data systems are architected.

The correspondence between things inside the information system and things in the real world still matters. Table 4 shows:

Data is only a model of reality, it is not reality. Depending on the context, the same territory (reality) can be represented using different maps (models).

“When files get integrated into a database serving multiple applications, that ambiguity-resolving mechanism is lost.” [Kent 2012]

To illustrate his point, the author uses the example of the concept of a “part”. In a warehouse context each part has a part number and occurs in various quantities at various warehouses. In a quality control context, part means one physical object; each part being subjected to certain tests, and the test data maintained in a database separately for each part.

Figure 45 represents the database that would serve the inventory management and quality control applications. In addition, a data model has been created to represent the part concept in each context.

The word “part” has a specific meaning in each context. In designing a shared database, the model would be unified at the expense of the vocabulary used in one of the contexts. As “Part” and “Item” mean the same thing, there are two options:

The shared database would start using a language of its own, which could lead to communication problems and errors.

“Total unification of the domain model for a large system will not be feasible or cost-effective.” [Evans 2003]

The example of Figure 46 illustrates the total data unification that is common in financial systems. This type of “monster” project often fails because it is impractical to fold all source system data models into one generic data model. This approach would create a number of problems; for example:

In a post on Martin Fowler’s blog, Zhamak Dehghani shows how it is possible to adapt and apply the learnings of the past decade in building distributed architectures at scale to the domain of data; and introduces a new enterprise Data Architecture that she calls “data mesh” [Dehghani 2019].

Zhamak argues that instead of flowing the data from domains into a centrally owned data lake or platform, domains need to host and serve their domain datasets in an easily consumable way. This requires shifting our thinking from a push and ingest, traditionally through ETLs and more recently through event streams, to a serve and pull model across all domains.

The monolithic Data Architecture is replaced by a modular one, and composed of:

Business facts are represented by domain events that capture what happened in reality.

A product-thinking mindset can help provide a better experience to data consumers. For example, source domains can provide easily consumable historical snapshots of the source domain datasets, aggregated over a time interval that closely reflects the interval of change for their domain.

There is a lot of data manipulation and transformation in Machine Learning (ML) systems. ML pipelines industrialize data manipulation and transformation. They are composed of data processing components that run in a certain order. Data components usually run asynchronously. Each component pulls in a large amount of data, processes it, and puts results in other data stores.

Each data component is fairly self-contained and communicates with other data components via data stores. If a component breaks down, downstream components can often continue to run normally by using the last output from the broken component. This makes the architecture quite robust unless a broken component goes unnoticed for some time. This is why proper monitoring is required to avoid data going stale.

TensorFlow Extended (TFX) is an end-to-end platform for deploying production ML pipelines. Figure 47, which is copied from [TFX User Guide], gives an example of an ML pipeline.

It includes a set of data components:

TFX provides several Python packages that are the libraries used to create pipeline components.

Mobile devices are now capable of leveraging the ever-increasing power of AI to learn user behavior and preferences, enhance photographs, carry out full-fledged conversations, and more [Kent 2012].

Leveraging AI on mobile devices can improve user experience [Singh 2020]:

The most popular apps now use AI to enhance user experience.

The shift toward real-time data streaming and modular Data Architecture changes the way enterprise data systems are designed and developed. When combined with AI, it is possible to create innovative products that deliver superior customer experience, such as the one illustrated by Section 18.2.

The goal of an event storming workshop is to explore a domain with several people collaboratively.

To do so, people will be placing domain events on sticky notes on a wall along a timeline with an unlimited modeling surface.

The workshop puts together three kinds of people: people with questions, people with answers, and a facilitator.

The orange color sticky note is the convention for the event.

The event is named with a past participle because of its simple semantic and notation. For example, order placed, order paid, order executed, product sent, etc.

Events are used because they are easy to grasp and relevant for domain experts.

Event storming workshops focus on domain events structured along a timeline.

The goal of the event storming workshop is to maximize the learning of all the participants. At the end of the workshop, the participants should have gained a shared knowledge of the domain subject of the workshop.

The physical setup is important: the surface of the wall represents an unlimited modeling surface and everyone should stand up to increase engagement in the workshop. There must be a continuous effort of every participant to maximize the engagement. The collaborative approach involves people with questions, people with answers, and a facilitator.

Markers and sticky notes should be available in different colors and in sufficient quantity.

There should be no limitation of the scope under investigation and the model should be continuously refined through low-fidelity incremental notation.

Event storming can have different goals, but mainly these two common ones are distinguished:

The concepts used during an event storming session, apart from the event concept, are voluntarily set by the participants whenever they feel they need one. However, the facilitator can suggest the following “usual” concepts:

The problem space holds what the enterprise does – its business capabilities – to keep it running and able to operate. A business capability is a specific function or ability that the enterprise possesses in order to achieve its goals.

The problem space describes several things:

The business capabilities are almost the same for different enterprises involved in the same business, but their implementations – the solution space – will differ. While sub-domains delimit the applicability of domains, bounded contexts delimit the applicability of domain models. As such, the bounded context is within the solution space.

Bounded context is the solution as it is designed. It describes the software architecture and is used to manage the complexity, and is, therefore, linked to the business.

Bounded context means different models of the same thing (e.g., books, customers, etc.) and is represented by models and software that implement those models. This is where patterns and heuristics are found.

Domain Model and Ubiquitous Language

The ubiquitous language is a deliberate language designed to be unambiguous and on which all stakeholders agree. This language is found in every artifact manipulated by the stakeholders (User Interface (UI), database, source code, documents, etc.). The concepts conveyed by the domain model are the primary means of communication; these words should be used in speech and in every written artifact. If an idea cannot be expressed using these concepts, the designers should iterate once again and extend the model, and they should look for and remove ambiguities and inconsistencies. The domain model is the backbone of the ubiquitous language.

Bounded Context

A bounded context delimits the applicability of a particular model so that team members have a clear and shared understanding of what has to be consistent and how it relates to other contexts. Bounded contexts are not modules.

Bounded contexts separate concerns and decrease complexity. A bounded context is the boundary for the meaning of a model. A bounded context creates autonomy, thus allowing a dedicated team for each. Bounded contexts simplify the architecture by separating concerns.

How to Identify a Bounded Context

Conflicts of naming suggest different contexts because they indicate that the model mixes different ubiquitous languages.

A context map describes the flow of models between contexts and provides an overview of the systems landscape. A context map helps to identify governance issues between applications and teams, and to reveal how teams communicate, as well as their “power” relationships. With a context map it is possible to get a clear view of where and how bad models propagate through Information System (IS) landscapes.

It is possible to use the metaphor of a flowing river to describe the relations between two bounded contexts: if you are upstream and pollute the river, those downstream will be impacted – not the opposite. And so, a relationship between two bounded contexts is one in which the actions of the upstream group affect the downstream group, but the actions of the downstream group do not affect the upstream group. It is not about the data flow’s direction, but about the model’s flow.

Context map patterns are categorized in three ways:

Modern software systems are composed of multiple components that communicate and interact to deliver whole “applications” that can now be composed of several user (e.g., a front-office and a back-office UI) and programmatic interfaces (API for the UI, API intended to be consumed by the customer, etc.).

Software architects deal with system decomposition and their responsibilities and dependencies. They use models, such as the C4 Model, to describe software system architecture effectively through simple diagrams with boxes and lines.

Software architecture deals with significant elements of the components of these systems and their structure, as well as how they interact together.

Software architecture has a strong focus not only on the structure of code but also on all the non-functional requirements:

and the cross-cutting inner concerns of software, detailed in Section 21.5, that include selecting technology, planned evolution, and defining roadmaps by following market and community evolution.

The process and practices of the team(s) that produce software systems are also closely related to the topic of software architecture; these practices include:

Besides the pure engineering role, software architects also need to have good soft and human skills in order to:

As its name implies, event-driven architecture is centered around the concept of an “event”; that is, whenever something changes an event is issued to notify the interested consumers of such a change. An event is a powerful concept to build architecture around because of the immutable and decoupling nature of events, as well as being a great way to design and build domain logic. The following sections detail the concepts and benefits of event-driven architecture, and then dive into the practical details of implementing such an architecture.

One of the greatest challenges of application architecture is to reach a good separation of concerns between the domain, application, and infrastructure logic in the code. Particularly it is necessary to keep the domain logic clean of any technologic concerns (persistence, distribution, UI, etc.). Big maintainability and evolvability troubles occur when domain logic is scattered all over the codebase. Hexagonal architecture decouples the application, domain, and infrastructure logic of the considered system.

Hexagonal architecture, created by Alistair Cockburn [Cockburn 2005], brings an interchangeability of adapter implementation and, therefore, a great suppleness in composing the domain with various ways of interacting with the software, as well as implementing infrastructure capabilities. Hexagonal architecture allows an effective decoupling of application, domain, and infrastructure concerns.

Hexagonal architecture is a departure from layered architecture that is considered as not solving correctly the coupling challenge between the different “layers”. The name “hexagonal architecture” was used because a hexagon is a shape that composes easily, has a great natural symmetry, and allows people drawing schema to have room to insert ports and adapters wherever they need to. There is a notable symmetry not only denoted by its shape, but also with the inbound ports and adapters on the left side and outbound on the right side.

Software architects must assess the priority of every non-functional requirement of the system they are building. Given that it is impossible to reach the maximum for every quality attribute, and that cost and delay add themselves to the equation, trade-offs must be made given the system context and the priorities given. By the way, a name with no negation would be better, so it may be better to talk about Operational Requirements instead of Non-Functional Requirements but people are used to NFRs.

The following sections give an overview of software quality attributes that must be dealt with by software architects.

Software architects need to address several cross-cutting concerns in their decisions while building software:

Infrastructure as Code is the process of managing and provisioning computer data centers through machine-readable definition files, rather than through physical hardware configuration or interactive configuration tools. The machine-readable definition files provide a descriptive model describing an abstract model of the hardware and infrastructure resources that are needed. This descriptive model is using the same versioning as DevOps teams use for source code. Infrastructure as Code tools transform this descriptive model using dynamic infrastructure platforms to create the actual configuration files required by the hardware and infrastructure components.

A dynamic infrastructure platform is the system that provides computing resources in a way that can be programmatically allocated and managed.

Most common dynamic infrastructure platforms are public clouds (e.g., AWS or Azure^®) or private clouds that provide server, storage, and networking resources.

Infrastructure can also be managed dynamically using virtualization systems or even bare metal with tools such as Cobbler or Foreman.

“SRE is what you get when you treat operations as if it is a software problem.” [sre.google]

SRE is a job role and a set of practices from production engineering and operations at Google and now largely adopted by the industry. An SRE team is responsible for the availability, latency, performance, efficiency, change management, monitoring, emergency response, and capacity planning of their services.

For details about SRE, refer to the series of books: [Beyer 2016], [Beyer 2018], and [Adkins 2020].