4. Architecture Development
This chapter defines the O-AA approach to Agile Architecture development. It is based on a set of modular building blocks that can be configured and assembled in a variety of manners. It supports several architecture development styles.
The scope of this document covers the enterprise as a whole, not just the alignment of business and IT; see Figure 5. It includes designing the enterprise business, organization, and operating models, which is the responsibility of senior executives who can be assisted by management consultants or Enterprise Architect profiles. Along with the authors of Designed for Digital: How to Architect your Business for Sustained Success, within this document the term Business Architecture is avoided because in many companies architecture is seen as the IT unit’s responsibility: “Right now, if you have a Business Architecture function, it’s probably buried in your IT organization (and having limited impact)” [Ross 2019].
Enterprise Architecture in this context will become more like an internal management consultancy, which implies that Enterprise Architects must develop their management consulting skills to include relationship building, problem solving, coaching, and negotiation as well as specialty skills, such as product management, design thinking, and Lean management.
The range of skills that should be considered part of the Enterprise Architect role includes the disciplines needed for management consultants who design business and operating models. This document incorporates these disciplines. It borrows concepts and methods from:
-
Strategic marketing and marketing research
-
User Experience (UX)
-
Design thinking
-
Lean Product and Process Development (LPPD)
-
Socio-technical systems
-
Organizational sociology
-
Operations strategy
-
Software architecture
Figure 5 illustrates the scope of this document, where:
-
“Ecosystem” refers to the interactions the enterprise has with its environment; see Section 2.19
-
“Product” refers to a bundle of services and/or goods produced by end-to-end processes or Lean value streams; see Section 2.49
-
“Experience” refers to how pleasant or unpleasant it is to interact with the enterprise
-
“Work System” refers to systems in which human participants and/or machines perform processes and activities using software, hardware, and other resources to deliver:
-
Products offered to internal and/or external customers
-
Experiences delivered to clients and employees
-
-
“Social System” refers to people, their behavior, cultural beliefs, skills, and expertise, and how work teams are forming and interacting as well as organizational leadership, strategy, structure, policy, and procedures
-
“Software” refers to something used or associated with, and usually contrasted with, hardware, such as a program for a computer [Merriam-Webster]
Software can be used to automate almost anything, ranging from IT infrastructure to decision-making; e.g., prescriptive analytics.
-
“Hardware” refers to tools, machines, wiring, and other physical components of a system
-
“Resources” refers to a source of supply or support by an available means – usually used in plural
4.1. Architecture
Martin Fowler wrote: “There is no such thing as a universally accepted definition of architecture” [Fowler]. His view of architecture is aligned with that of Ralph Johnson, who defines architecture as “the important stuff (whatever that is)” [Johnson 2008]. The lack of a universally accepted architecture definition justifies why this document borrows from three different sources to analyze what architecture means.
Most architecture definitions, such as these examples, adopt a systems thinking view that models the enterprise as a set of interrelated elements:
-
“The fundamental concepts or properties of a system in its environment embodied in its elements, relationships, and in the principles of its design and evolution” [ISO/IEC/IEEE 42010:2011]
-
“The structure of components, their interrelationships, and the principles and guidelines governing their design and evolution over time” [TOGAF Standard 2018]
Both of these definitions focus on “what the enterprise is”, its elements of form, rather than “what the enterprise does”, its functions; thus positioning architecture as a discipline which guides the design and evolution of the enterprise modeled as a system.
More detail on how the TOGAF Standard, Version 9.2 guides the delivery of architecture can be found in Sections 2.3, 2.4, and 2.5 of the TOGAF Standard; see [TOGAF Standard 2018].
To paraphrase Alfred Korzybski’s famous sentence [Korzybski 1958]: “a map is not the territory it represents, but if correct, it has a similar structure to the territory, which accounts for its usefulness”; the architecture model of the enterprise is not the enterprise. The enterprise makes architecture decisions that are reflected in the way it does business and operates. Architecture decisions may be implicit and are not necessarily documented or modeled.
Crawley [Crawley 2016] notes that “architecture is the embodiment of concept, and the allocation of physical/informational functions (processes) to elements of form (objects) and definition of structural interfaces among the objects”, and goes on to state that the systems engineering discipline adds two important ideas:
-
The allocation of functions or activities to the system’s elements or components
-
The definition of the structural interfaces that link the system’s elements or components
4.2. Development Building Blocks
As shown in Figure 6, the O-AA development is structured along two axes that represent:
-
The function-form axis: What the Enterprise “Does”, What the Enterprise “Is”
-
The perspective axis: Experience Perspective, Work System Perspective, Technical System Perspective
The “Experience Perspective” is in the problem space, while the “Work System Perspective” is in the solution space, as well as the “Technical System Perspective”.
The O-AA building blocks are positioned along the two axes, with the exception of: “Strategy”, “Value”, and “Data/Information & AI”.
The “Corporate Brand Identity” and “Corporate Culture” building blocks influence what the enterprise does.
4.2.1. Strategy
Michael Porter recommends distinguishing operational effectiveness from strategy. Strategy is about developing sustainable differentiation based upon strategic positioning.
“Strategic positioning means performing different activities from rivals' or performing similar activities in different ways.” [Porter 1996]
Porter’s strategy formulation helps to connect strategic intent with execution. Mapping activity systems helps to verify interactivity consistency and identify when activities are reinforcing.
Strategy formulation is also influenced by the Agile culture, which insists on the importance of experimentation. Strategy is seen as a set of strategic assumptions that need to be experimentally verified. Strategy professor, Adam Brandenburger, observes: “the assumptions underlying your business model are embedded in all your processes”. Therefore, he recommends challenging these assumptions when formulating strategy [Brandenburger 2019]:
-
Precisely identify the assumptions that underlie conventional thinking in your company or industry
-
Think about what might be gained by proving one or more of them false
-
Deliberately disturb an aspect of your normal work pattern to break up ingrained assumptions
The digital era has seen the rise in popularity of the concept of the business model. Sometimes the use of the words “strategy” and “business model” are interchangeable. Peter Seddon and Geoffrey Lewis examine in detail the meaning of two frequently used – and misused – terms, namely, “business model” and “strategy”. They argue that as used by leading thinkers these two terms might reasonably be interpreted as having roughly equivalent meanings [Seddon 2003].
A business model describes the rationale of how an organization creates, delivers, and captures value. It provides an analytical way of connecting strategy to execution. A business model has three components [Johnson 2008]:
-
A customer value proposition, which models how customers perform a specific “job” that alternative offerings do not address
-
A profit formula, which models how to generate value for the enterprise through factors such as revenue model, cost structure, margins, and inventory turnover
-
Key resources and processes, including the people, technology, products, facilities, equipment, and brand required to deliver the value proposition
Regardless of their quality, the odds that implementing the strategy and associated business model(s) will fail are high. Why?
“Culture eats strategy for breakfast.” (attributed to Peter Drucker)
In an HBR article, Donald Sull [Sull 2015] and his co-authors analyze why strategy execution unravels. They acknowledge that strategy alignment processes are fine; the issues are due to unreliable commitments from colleagues. Furthermore, strategic plans are just a set of assumptions that need to be verified.
“No plan survives first contact with the enemy.” (attributed to Helmuth van Moltke, Prussian military commander)
Other strategy execution issues are:
-
Strategy is often not well understood, even at the executive level
-
Past performance is over-valued at the expense of experimentation and agility
-
Execution should be driven by leaders who know the field, which is rarely the case when only driven by top executives
4.2.2. Corporate Brand Identity, Culture
A clear and unified corporate identity can be critical to competitive strategy. It serves as a “north star providing direction and purpose” [Greyser 2019]. Corporate brand identity starts from the enterprise mission and vision. It must be consistent with the culture and capabilities of the enterprise.
Jez Frampton believes that great brands are “business strategy brought to life” [Wind 2016]. The authors of Beyond Advertising: Creating Value Through All Customer Touchpoints point out that brands never had more ways to reach and engage people. The image of the brand is now the result of the experience of people across all channels.
In a digital world, brand experience depends on the effective bridging of the enterprise strategy with its operating model. Marketing can no longer manage brands in silos, relying on classical marketing and advertising tools. Marketing people must now become members of multi-disciplinary teams that manage brand experience holistically.
4.2.3. Value
Agile architecting borrows concepts from Value Engineering (VE), which was pioneered by L.D. Miles from General Electric®. VE is an approach directed at “analyzing the function of systems, equipment, facilities, services, and supplies for the purpose of achieving their essential functions at the lowest lifecycle cost consistent with required performance, reliability, quality, and safety” [Mandelbaum 2006].
In the digital age, value cannot be reduced to functional benefits. Value definition includes the emotional dimension; for example, reducing anxiety, providing fun, or a sense of affiliation.
The central idea is to provide benefits to clients and other stakeholders at the lowest reasonable cost. The value of a product can be increased either by maintaining its functions and features while reducing its cost, or by keeping the cost constant while increasing the functionality of the product.
4.2.4. Perspectives
The Experience perspective defines value from a client perspective. It analyzes a client’s job-to-be done; their pain points and gains. It also covers the emotional dimension of the experience, starting from an outside-in view that places customer needs and problems at the center. Design thinking and market research are incorporated into the broader Agile architecting discipline.
The Work System perspective defines the ability of the enterprise to deliver client benefits efficiently. It starts with analyzing commonalities across value streams; this helps to identify activities that could be shared, thus contributing to architecting the operating model.
The Technical System perspective covers the software and hardware parts of the enterprise. It starts from analyzing domains such as payments, mortgage lending, or federated identity management and also includes architecting the physical world. The software architecture discipline is part of the broader Agile architecting discipline.
4.2.5. What the Enterprise “Is”
Analyzing what the enterprise “is” starts by defining the boundaries that connect:
-
The enterprise to its environment
-
The enterprise parts to each other and to the environment
Defining boundaries covers:
-
Product Architecture, which is driven by the questions:
-
How should products be broken down into components?
-
Which interfaces should link product components?
-
Which modularity–integrality trade-off is appropriate?
-
Which product platform, if any, would bring value?
-
-
Operations Architecture, which is driven by the questions:
-
What are the key value streams and processes of your operations; and are they fit-for-purpose and efficient?
-
Are economies, skills, and scale being leveraged?
-
What is the platform strategy, if any?
-
Are the right resources, skills, and technologies deployed in the right facilities and locations?
-
-
Organization, which is driven by the questions:
-
What is the right organizational structure and culture?
-
How should organizational levels be defined, and how should the group, entity, team, and team of teams levels be articulated?
-
How should authority, responsibility, and accountability be distributed?
-
Which teams and teams of teams should be stream-aligned?
-
-
Domain-Driven Design segmentation, which is driven by the questions:
-
What are the enterprise domains and sub-domains?
-
How to decompose sub-domains into bounded contexts?
-
What are the upstream/downstream dependencies that link bounded contexts?
-
How to minimize inter-bounded context dependencies?
-
When modeling the enterprise as a Complex Adaptive System (CAS), architects should pay attention to leveling, boundaries, dependencies, and interactions: a CAS “is characterized by intricate hierarchical arrangements of boundaries and signals” [Holland 2012]. The definition of levels, boundaries, and interactions is a lever used in Agile Architecture to influence the evolution of the enterprise.
4.2.6. What the Enterprise “Does”
Experience Design combines customer research and product discovery during a set of design thinking iterations. This is not a linear process; it alternates between divergent and convergent thinking. Customer research borrows from marketing research, anthropology, and design thinking to capture and analyze customer needs and desires. Product discovery applies divergent and convergent thinking in the solution space to discover which products and features could satisfy customers. Customer research and product discovery apply outside-in thinking and aim at framing problems and solutions through the lens of the customer.
Journey Maps bridge outside-in thinking with inside-out thinking by defining which activities deliver the experience customers expect.
Value Streams complement the outside-in view by representing all the activities, both value and non-value creating, required to bring a product from concept to launch (also known as the development value stream) and from order to delivery or provisioning (also known as the operational value stream). More detail about value streams can be found in Section 2.49.
Event Storming helps Agile teams to explore the domain. The identification of domain events, commands, persona, or entities facilitates a structured conversation about the domain. The participants of an event storming workshop share knowledge of the domain to establish a common vocabulary or ubiquitous language. Domain events help to identify boundaries. Commands help to identify responsibilities to be allocated to services or components.
Readers who would like to zoom in on building blocks can read Chapter 10 and use Table 2.
4.3. Data, Information, and Artificial Intelligence
The digital enterprise collects and processes vast amounts of data. Data describes a portion of “reality” that is of interest to the enterprise. Data becomes information when it is interpreted, when it has meaning. Data can be interpreted by humans or algorithms. Artificial Intelligence (AI) transforms data into predictions, prescriptions, and automation capabilities.
The combination of data and AI is having a transformational impact on industries. For example, predictive maintenance can improve service quality while lowering costs. Vast numbers of images can teach machines how to recognize faces or interpret chest x-rays.
Architecting systems that handle data at scale is critical. Non-functional requirements, such as compliance (e.g., privacy) or security requirements, influence Data Architecture decisions. The move from batch processing to real-time data flows is having a profound impact on Data Architecture.
4.4. Software and Hardware Architecture
“Software is eating the world.”
Marc Andreessen explained in a famous Wall Street Journal article why software is eating the world [Andreessen 2011]. The rapid evolution of software technology has fueled the growth of digital business. Following the lead of Internet giants, some enterprises from the old economy are framing themselves as tech companies; for example, Banco Bilbao Vizcaya Argentaria (BBVA): “If you want to be a leading bank, you have to be a technology company”.
In 2002, Amazon™ was facing a complexity barrier. The size of its home page reached 800MB and it took 8 to 12 hours to compile. Jeff Bezos issued a mandate that profoundly changed the way software is created and how the enterprise is organized.
By shifting toward modularity and APIs, Amazon became well-positioned to open its distribution and logistics capabilities to third-party vendors. The self-service nature of the platform made it easy for vendors to sell and distribute their products in a frictionless manner. This helped Amazon to compete against eBay®, leveraging a business model which is different.
Software and hardware represents an increasing part of products and their supporting operations. Software architecture, product architecture, and operations architecture must be developed in a concurrent manner.
4.5. Architecture Development Styles
Except when they are in their creation phase (e.g., startups), enterprises have an “as-is” architecture, which may be explicit or implicit, documented or in the mind of employees.
This document distinguishes an architecture from its model; the map is not the territory. Architecture evolves due to emergence, which refers to what appears when the parts that compose a system come together and operate. An architecture model does not emerge; it is a representation of an “as-is” architecture or it is designed.
4.5.1. Emergence
Emergence refers to what appears, materializes, or surfaces when a complex system operates; desired or undesirable functions or outcomes emerge. An example of an undesirable outcome is the pollution that results from operating cars. Architecture aims to develop systems that deliver predictable and desirable outcomes.
However, as enterprise complexity grows, unpredictable and sometimes undesirable outcomes appear. Agile architecting aims at benefiting from emergence while minimizing unnecessary complexity and undesirable outcomes.
John Holland observes that “emergent properties often occur when coevolving signals and boundaries generate new levels of organization. Newer signals and boundaries can then emerge from combinations of building blocks at this new level of organization” [Holland 2012]. The more complex a system becomes, the more emergence is likely to appear.
As enterprises grow in complexity, they increase in resilience, become more capable of handling complex ecosystems, and are less likely to be imitated by competitors. However, complexity has a cost and can threaten the future of an enterprise if it gets out-of-hand.
To mitigate undesirable complexity growth, Agile architecting recommends using the levers below:
-
Modularity, to facilitate team autonomy and increase resilience
-
Standardization, to facilitate product or operating model reconfiguration
-
Architecting for a built-in responsiveness to change
Relaxing control favors emergent over designed or micromanaged solutions, which are often inferior. Favoring emergence does not mean it is impossible to steer the evolution of the enterprise.
John Holland suggests steering complex systems by modifying their signal/boundary hierarchies. The components of complex systems are bounded sub-systems or agents that adapt or learn when they interact. Defining the boundaries of sub-systems and their rules of interaction has a major influence on the evolution of the system [Holland 2014].
Agile architecting steers the evolution of the enterprise by modifying its organization and changing the allocation of authority, responsibility, and accountability.
4.5.2. Intentional Architecture
Does emergence mean that intentional architecture is no longer needed? This document makes the case that intentional architecture brings value. However, the architecture discipline needs to evolve. Enterprises need to shift from a Big Up-Front Design (BUFD) to continuous architecture. As James Coplien [Coplien 2010] argues:
-
Some up-front intentional architecture prevents waste and accelerates decision-making
-
Driving architecture from requirements is a mistake, allowing that sometimes this is the only realistic option
-
Intentional architecture should focus on the essence of the system without being unduly influenced by the functionality it provides
-
Partitioning (segmentation) allows each part of the system to evolve as autonomously as possible
This document views architecture development as a combination of intentional and emergent architecture. It promotes enterprise segmentation to facilitate concurrent development and architecture refactoring.
Lessons learned from the strategy field apply to intentional architecture too. The success of intentional architecture depends on the reliability of commitments, which is covered in Chapter 7.
Intentional architecture should be simple, focused, and compact because:
-
It is likely to evolve, so investing in a detailed model would be wasteful
-
It is guided by guardrails imposed by governance
-
Concentrating on the “important stuff” is likely to make the architecture model easier to understand
Chapter 5 will cover this topic in more detail and show how intentional architecture can be combined with continuous architecture.
4.5.3. Concurrent, Continuous, and Refactored
Agile architecting adopts Set-Based Concurrent Engineering (SBCE) practices from Lean Product Development [Ward 2014]:
-
Simultaneously explore multiple solutions for every sub-system
-
Aggressively explore them with rapid, low-cost analysis, and tests, progressively eliminating weak solutions
-
Converge on a solution only after it has been proven
SBCE requires architecture thinking to decompose the product or the product platform into sub-systems.
When weak solutions are eliminated, architecture has to evolve. Continuous architecture facilitates incremental changes to the product or platform architectures.
Architecture refactoring refers to the architecture restructuring that occurs when architecture evolves.
In an Agile world, evolvability is becoming a top-quality attribute. The objective should be to architect a system in such a way that will allow change without damaging any non-functional qualities.
4.5.4. Tailoring Architecture Development
The modularity of this document enables the formulation of specific architecture development approaches tailored to solve specific enterprise problems. Chapter 5 describes patterns that can guide the definition of tailored approaches that use and combine the O-AA building blocks.
A description of building blocks, their links, and how to segment the enterprise can be found in Chapter 10.
4.6. Security by Design
Security is a crucial element in any Agile Architecture, and enterprises must get it right. Businesses are in a world of digital enablement and iterative development, with Minimum Security Architecture (MSA) as a crucial element of Agile development. MSA is a continually evolving security architecture that fits into the cadence of a DevOps cycle, and is sufficient to guide developers toward the Minimum Viable Product (MVP) and to set architectural direction for future releases.
However, with this comes a danger: if an MVP is released with a security gap it is difficult to apply a solution once it has gained a user community. This means that products must correctly integrate security into the architecture from the beginning or risk the difficult – if not impossible – task of trying to remediate security gaps later on.
DevOps includes security as a fundamental aspect; it is not an add-on to the process. This prevents the silo thinking prevalent in literature, which attempts to add additional teams to DevOps leading to infinite variations of DevSecOps, ArchDevOps, etc.
The DevOps process (while technical in execution) is owned by the business. As such, the goal of DevOps is to help balance both business enablement and risk management (in the context of security). This can take the form of compliance and resiliency metrics that provide much needed assurance to the business in relation to system-related cybersecurity.
In addition to building security into the architecture from the beginning, security in an Agile Architecture must respond quickly through regular feedback loops that do not require extensive upfront planning. Because of these rapid feedback loops, there is a need for extra attention in order to balance security with other aspects of the architecture. The security team must consider the future state well beyond the current architecture to ensure that downstream teams also remain Agile. This underscores the purpose of security in an Agile Architecture – to enable the business to move rapidly in a world of defined and managed risk.
To this end, there are three core philosophies:
-
An Agile security architecture must be built in short, rapid cycles.
-
Any changes to the security architecture must apply to the environment of each team.
-
Security architects and champions will ensure any architecture complies with the organization’s predefined risk appetite.
For more information, see the O-AA Security Playbook 2021.