The Turning Point: A Novel About Agile Architects Building a Digital Foundation

Six weeks later, the Data Warehouse and Business Intelligence work streams have made significant progress. The refactoring is completed and Release 1.0 of the new Digital Customer Intimacy strategy is ready for a go-live into production. Effectively, the new way of collaborating between architects and the Development team is well accepted and largely in use. The Enterprise Architecture team has been involved in the development of the data warehouse application and is being informed of the test results. This application is one of the key digital products for ArchiSurance because it will give executive leadership information about our customers concerning their likes and dislikes. It will also provide analytics on customer demographics to help determine the best way to deliver services to the customer base.

Craig Evans, the Product Owner for the data warehouse project, receives an email from the Release team confirming the test report has been approved and the Release Package is ready for deployment. He is pleased with the results and plans to contact the team immediately to launch the deployment to the cloud environment. Craig has been watching the emergence of a health crisis caused by a new virus, anxiously wondering what might happen if it continues to spread around the world. In addition to worrying about humanity in general, he’s been thinking about all the negative impacts that might be passed to customers, their businesses, and to all the people they employ and serve. What could he recommend now that would make the data warehouse application deployment more resilient for global use and to anticipate larger-scale use?

Only three weeks later, many digital technology employees are confined at home because of the global scale of the pandemic, except the teams in charge of the maintenance of the data centers. Kathleen is working from home with her husband and their two children, sharing her time between the children’s education and the management of the Enterprise Architecture team.

Kathleen is happy to note that the impact of the lockdown hasn’t disrupted work for the teams. The new way of working allows them to simply change to remote locations without losing a heartbeat. As she eats breakfast, realizing how relaxing it is to eat with the family every morning, her phone rings. It is not a number she recognizes, but in these trying times where work life is upside down, she now answers all calls.

“Hi Kathleen, this is Jasmine Williams from Customer Relations. I’m so sorry to disturb you this early, but we have a major incident going on with a lot of customers impacted. I’m setting up a war room and I need somebody from the Enterprise Architecture team to participate.”

“Of course,” says Kathleen, aware from the tone in Jasmine’s voice that the problem is serious. “But could you just tell me a little more about the problem, so I can work out who will be the best person to help you?”

“Sure. Since yesterday, we’ve been receiving lots of calls and emails in Customer Support from people who can’t access their ArchiSurance account with their mobile application. We already checked the changes that went through yesterday and the only one was the deployment of the new data warehouse application. However, customers do not have direct access to this application, so we need to investigate more deeply to find out whether and how that might have been the cause of this incident.”

Kathleen thinks about the Digital Customer Intimacy roadmap and finally answers Jasmine.

“I’ll ask Jamar Johnson. He is our IoT expert and the Mobile Application and IoT teams recently began working together.”

“That sounds great, Kathleen. Thank you.”

Kathleen’s hunch was right. Less than 15 minutes after Jamar Johnson joined the virtual war room, they discover that the mobile app had been updated to be compliant with the new Identity and Access Management (IAM) system that is now required for the security of IoT devices, which are now connected to the updated data warehouse system. Jasmine calls Kathleen to give her feedback on what they found.

“Good job, Jasmine!” says Kathleen, after hearing the cause of the incident. “But why did we authorize the deployment of the data warehouse application in that case?”

“The Enterprise Architecture team has been involved in the testing, but not in the final deployment,” replies Jasmine. “The architects are also invited to the Change Advisory Board (CAB), but the CAB is only concerned with Requests for Change (RFCs) that are registered in their system and which go into our own data centers. In this instance, the data warehouse application was in a public cloud, so it was missed by the CAB.”

Kathleen now remembers the new team is using a fully automated CI/CD pipeline. She remembers how proud everybody was when they did the first fully automated deployment to production. Since then, anybody in the team could submit code or other changes to production, as long as it passed all the testing. This basically bypasses the whole CAB; something she had not considered.

“In this case,” says Jasmine, “the decision to deploy the MVP of the data warehouse system, including the new IAM system and mobile app was made by Craig Evans, the Product Owner. Given that the pipeline itself was showing green, and all new and updated systems were working as designed, there was no indication of anything wrong, so the deployment itself was successful. Unfortunately, nobody involved in the deployment was aware that some of the oldest versions of the mobile app are still in use and as a consequence it stopped working. Perhaps, if the architects or the CAB had been informed of the deployment, they could have prevented the situation. However, as you know the architects were not informed up-front, and nor was anyone in the CAB aware of the deployment because the deployment was made to our public cloud environment.”

“OK,” says Kathleen, “so, that is something we need to look at again. It sounds like we still need to get our architects embedded with the Agile teams instead of working in our own silos. But what’s been decided to resolve the issue at hand? What’s the proposed resolution?”

“The problem is we do not know which customers are using the mobile app and which version they have installed. The app is available for free on different app stores and we have no feedback on who is downloading and installing the app. The only solution we have now is to send an email to all our customers to inform them that they need to update their mobile app if they are using it.”

Kathleen agrees and asks if Jasmine can get Corporate Communications involved to ensure the email is sent out in a professional manner.

“Yes,” says Jasmine. “That’s a good idea and should be pretty easy for them to do.”

Kathleen hangs up the phone as Sven steps into the room.

“I just finished helping the kids with their mathematics lesson, it’s your turn now. Oh, wait, are you ok? You seem upset. What just happened?”

Kathleen explains the cause of the major incident they had and how they are going to solve it.

“I thought you implemented that IT4IT Reference Architecture to get more transparency into the flow of products and make the gaps visible,” says Sven. “What happened?”

“We didn’t implement all of the IT4IT Standard, only the parts of it that would help solve some of our immediate issues. But you’re right, I’ll have a look at the IT4IT Reference Architecture again to see if this can be resolved. I have more time now that the next iteration is on hold during the pandemic confinement.”

The next day Kathleen takes a closer look at the IT4IT specification and especially the “Request to Fulfill” value stream (Request to Fulfill Value Stream Diagram); the one that addresses the delivery and consumption of services.

She discovers that the IT4IT Standard describes how to build catalog offerings that allow consumers to order a service that is executed through to the fulfillment of the order. More interesting is the subscription mechanism proposed by the standard. With this in place, ArchiSurance customers could order the mobile app service, and as soon as the app is installed they could keep track of their subscription to the service through the Order functional component. This would make it easy to know which customer is using the mobile service and which app-version they are using. The Order could be placed through the web portal already in use by customers. Information on updates could be delivered directly to customers via the portal whenever catalogs are updated with a newer version of their app. It is even possible, with the right monitoring in place, to follow the usage of the service for each customer.

Another interesting aspect she found by reading further into the IT4IT Standard documentation is that the same ordering mechanism can also be used by the staff. It dawns on Kathleen that the services consumed by people for things like a cloud infrastructure or an application deployment in the cloud could also be managed the same way as customer requests for business applications. If ArchiSurance had already put this in place, it would have been possible to have an approval workflow that included routing the approval to the Enterprise Architecture team.

Kathleen also notices the Fulfillment Orchestration component, which orchestrates the delivery of the various orders amongst fulfillment automation systems. The interesting part here is that it is triggered by a change. That is not something they had ever considered. What if the CI/CD pipeline could register a change automatically every time a change is going to production? That would also solve the issue of the CAB not being aware of changes. It might even be possible to tie releases of the CI/CD pipeline to a change that could be submitted for approval by the CAB even before it is being developed. In that way, the CAB would be able to perform its risk analysis, without stopping any deployment. Kathleen decides to look at that part at a later stage. First, they need to fix the ability to track requests.

Feeling motivated, Kathleen starts to build a course of action for what needs to be done to implement the subscription model.

Every service offering, she realizes as she types, would need to have the following data objects tracked: Service Offer, Order, and Subscription. And every Fulfillment Orchestration component, including the new CI/CD pipelines, would need to register a standard change, identifying the change they had made to production.

In less than half an hour, she has a “good enough” model, so she opens a web meeting and invites Philip Potter and Terri Nichols, the architects who have been working on the value stream stage “Serve Customers”, to join her. She explains her new Idea and tests its feasibility with them. They both really like the Idea and Philip confirms that the service management system they are currently using can be updated to implement the subscription mechanism and has an API to allow the CI/CD pipeline to open and process a change. Furthermore, Terri says that she is fine with interfacing the web portal with the service management system.

The following day, Kathleen logs into the portfolio management system from home and submits two new Ideas. First, to implement the subscription mechanism through the web portal and second, to let CI/CD pipelines create and process a change. She points to the recent service interruption incident and its negative impact on customer satisfaction as part of the rationale.

It is agreed to fund the Ideas, and the new features are added to the appropriate team backlogs.

Four weeks later, just two weeks after the change and CAB solution have been implemented, Kathleen starts to notice an increase in the workload of the CAB. This is of course necessary to help the organization get control, but Kathleen considers this to be an anti-pattern for an Agile organization. It creates a stage gate with the risk of delaying work and, suddenly, architecture decisions are validated as part of the CAB. To address the issue, she organizes a web meeting with her team to address what the next steps should be to reduce the risk she had identified.

Together they discuss the topic in detail and conclude that they have to find a way to allow the teams to be able to make decisions autonomously, but at the same time give the Enterprise Architecture team the right to step in. For this, they will need two things: the ability to see what is coming, and an understanding of why decisions were made.

The Enterprise Architecture team needs to find a more balanced approach to gain an understanding of why decisions were made, using Architecture Decision Records (ADRs) [22] rather than forcing everybody to submit their decisions for approval. The ADRs will be used by the Agile teams to record their decisions and then made available to the Enterprise Architecture team who can then validate amongst themselves whether these decisions are in line with the intended architecture. If they are not, a discussion can be started to see what can be done, removing the need for architecture decision validation as part of the CAB approval.

Gaining an ability to see what is coming, however, requires the team to think about how to present the planning of changes as agreed by the QBR [23]. The QBR should also be one of the approaches considered for providing governance in the transformed organization.

“I guess the next steps for us will be to begin migration planning,” says Kathleen. “I will have Tony set up some meetings for next week. We can get the enhancements of the Architecture Roadmap established that we have been talking about, so it can support the organization with migration planning. I would like Carl to lead this effort so that the way we document the migration becomes part of the Architecture Community of Practice.”

Everyone nods in agreement and Nick reminds Kathleen that the team has been looking at some standard formats for roadmaps and is just about ready to make a recommendation.

“That sounds wonderful, I can’t wait to see what the team comes up with!”

Kathleen notices that the TOGAF ADM itself is perfectly fit to be used to drive architecture development in an iterative way that is compatible with supporting Agile solution development teams. However, it certainly does not help to use the jargon from the ADM steps when simply wanting to implement a collaborative way of working with the Agile development teams. Moreover, creating the architecture iteratively ensures that architecture input is provided to the Agile teams when needed, without creating BDUF. To see the details of what is presented by Carl, how it is documented, and what transpires in this meeting, see Opportunities & Solutions and Migration Planning.