7. Architecting the Agile Transformation

In A Dual Transformation we recommended conducting concurrently the Digital and Agile Transformations of the enterprise. This chapter covers the change management dimension of this dual transformation, and zooms in on the Agile side of the overall enterprise transformation.

The Agile Transformation of the enterprise covers three areas:

  • Adopting new ways of working

  • Deploying new management systems

  • Changing the organizational structure

The new ways of working promote the practices below:

  • Rapid iteration and experimentation, which promotes continuous learning

  • Fact-based decision-making

  • Information sharing and transparency

  • Cohesive cross-functional teams coached by “servant” leaders

  • Performance orientation driven by peer pressure

Management systems evolve toward more autonomy for Agile teams, balanced by clear accountability. Freedom is required to empower teams to rapidly make decisions closer to the field. Accountability in an Agile organization is not about controlling people; it is about a two-way exchange where you agree to deliver something to another person.

7.1. Accountability

In an Agile organization employees are accountable to their peers, their manager, and their clients. Managers are accountable to their teams, the board of directors, and society. The management system cascades goals at all levels of the organization and promotes a constructive dialog to help set up accountability relationships between employees and managers. The reward system recognizes individual performance while promoting collaboration.

The organizational structure is flattened. Autonomous cross-functional teams, often named “feature teams” or “squads”, are formed. Cross-functional roles emerge to help construct robust communities of practice, often named “chapters” or “guilds”. Resource allocation is flexible and driven by the evolution of demand or activity level.

The left part of Agile Transformation represents the three transformation dimensions we have already introduced, with the addition of the “Enterprise Culture” dimension. Culture evolution results from changes in the three dimensions. For a culture change to take hold, people have to experience success operating in the new Agile organization.

Figure 10. Agile Transformation

The middle part of the figure lists a few important questions that the enterprise needs to address. For example, the waterfall scenario is likely to create intermediary stages that are suboptimal. New ways of working may conflict with the existing management system. The enterprise which deploys a new management system on top of an existing organizational structure will have to redeploy it at a later time.

Due to the interdependencies that link the transformation dimensions, it is tempting to conduct change on the three dimensions in parallel. This leads to either a big bang scenario or an incremental deployment. In the case of an incremental deployment, the challenge is to define the scope of each increment.

7.2. Incremental Agile Transformation

Transformation Increment shows that each transformation increment should cover the entire hierarchical line. Why? Because if you change the way one level of the organization operates while the hierarchical line continues to operate the old way, the enterprise runs the risk of submitting its employees and managers to a double bind; i.e., employees and managers becoming at risk of being confronted with two irreconcilable demands or a choice between two undesirable courses of action. For example, re-prioritize your backlog but stick to established budgets and product plans.

fig-transformation increment
Figure 11. Transformation Increment

Incremental transformation deployment is easier when two conditions are met:

  • Dependencies that link organizational units are minimal

  • The business and IT organizational structures are well-aligned

When those conditions are not met, it is better to change the organizational structure before deploying new ways of working and a new management system. The new organizational structure reflects an architectural intent, which is either implicit or explicit.

7.3. Architecting the Organization

We will illustrate the need to re-architect the organization with the example of an IT organization that evolved to become more Agile.

The legacy IT organization was structured by process/function (front-office, middle-office, finance, risk) rather than by product families (plain vanilla swaps, equity derivative, commodities, etc.). Two powerful central functions controlled operational teams:

  • Budgeting and financial control to rein in costs

  • Architecture to ensure the coherence of the information system

Pools of software development resources were organized by technologies; for example, server-side Java® development or mobile front-end development. Software development projects would on-board required resources for the duration of the project and release them when done. Last but not least, IT operations and support was separate from application development.

Though the specialization of the legacy organization was supposed to help the IT organization to capture economies of skill and scale, several problems surfaced.

The level of inter-team dependency is high because of the multiplication of organizational entities that have to intervene on a project. Time-to-market is increasing due to the high level of inter-silo coordination required; for example, between development and IT operations teams. Alignment between business and IT is complicated because of the lack of simple mapping between organizational structures.

7.4. DevOps Culture

The IT re-organization was inspired by the Spotify™ model. Small cross-functional teams named “squads” replaced projects. The size of a squad does not exceed 10 to 15 people. Unlike a project, a squad is a stable team composed of dedicated resources that develops an end-to-end perspective. Squads are product-centric, meaning they develop, deploy, and operate a service. Squads adopt a DevOps culture which translates into the mantra “you build it, you run it”.

Squads are grouped into tribes which are not bigger, on average, than 150 people. In order to maintain and develop functional or technical competencies, chapters and guilds are created; for example, chapters that regroup mobile developers or NoSQL Database Management System (DBMS) experts.

As the number of Agile teams grows to a few hundred squads running in parallel, it is important to define a taxonomy of Agile teams that clearly specifies the scope of each one and minimizes inter-team dependencies. True Enterprise Architecture thinking is required to discover an effective way to decompose the organization, and to draw boundaries that minimize inter-team dependencies.

The Taxonomy of Agile Teams represents a simplified taxonomy.

Figure 12. The Taxonomy of Agile Teams

The primary goal of an Agile team’s taxonomy is to minimize redundancy and duplication [Rigby 2018]. Because the taxonomy of Agile teams may be different from the formal Profit and Loss (P&L) structure of the enterprise, it is necessary to map it to the division, business unit, and P&L structure of the enterprise.

7.5. Leadership Drives Change

The Agile Transformation we describe in this chapter must be initiated by executives. Because Agile Transformation moves the organization from a siloed model to cross-functional cooperation, executives form coalitions to drive change across the enterprise.

Executives who drive change must be driven by an intrapreneurship mindset. Leadership Drives Change shows how they propel change throughout the organization.

Figure 13. Leadership Drives Change

Coalitions of executive intrapreneurs adopt an outside-in perspective to identify threats and opportunities that justify change. Middle management pioneers play several major roles:

  • Analyze threats and opportunities to determine possible responses to changes in the environment and market

  • Identify change agents, drilling down to define candidate responses in more detail

  • Report issues and problems to the executive intrapreneurs coalition

  • Identify innovation opportunities that require executive support

Middle management pioneers consolidate and analyze feedback coming from the field. They report problems to the executives. When change issues jeopardize successful implementation, middle managers initiate conversations that challenge executives when necessary. The change journey is incremental, inclusive, and experiment-driven. The change model moves away from classical command and control to engage all organizational levels.