Ecosystems Architecture

To be blunt, change is not about evolution, but evolution is very much about change. In that regard, the former simply focuses on alteration and occurs without consideration for any grander plan. Evolution, on the other hand, points to change targeted toward the global betterment of those involved. Change can therefore be a negative thing, whereas evolution feeds on change to drive out negativity and promote advancement. Organizations can change, whereas, in the long term at least, ecosystems can only ever evolve if they are to remain viable. What is more, today we understand that evolution is a phenomenon not just limited to the natural world.

Scientists like Charles Darwin might well have taught us that evolution is tightly bound to skin and sinew, but more recent updates suggest that the idea is much more encompassing. For instance, in 1976 the eminent biologist and writer, Richard Dawkins, presented an information-based view in which he outlined the idea of selfishness in genes; thereby promoting the notion that they act only for themselves and similarly only replicate for their own good [40]. Dawkins also introduced the important distinction between replicators and their vehicles. It is the information held within the genes that is the real replicator, and the gene’s physical structure simply its carrier. A replicator can therefore be anything that can be copied, and that includes completely virtual capital like ideas, concepts, and Intellectual Property (IP) of significant commercial value. A vehicle is therefore only an entity that interacts with its environment to undertake or assist the copying process [40]. As a result, in modern business, any concept or idea embodied in the digital ether can be considered as a replicator, and human beings, alongside any hardware and software used in the form of IT, as its vehicle. This ties us, our musings, and our advancements together, as it always has, into the perpetually changing environment of commotion we awkwardly refer to as “progress”.

What results is the outcome of what Dawkins saw as a fundamental principle at work. In that, he suggested that wherever it arises, anywhere in the universe, be that real or virtual, “all life evolves by the differential survival of replicating entities”. This is the foundation for the idea of Universal Darwinism, which encompasses the application of Darwinian thinking beyond the confines of biological evolution, and which prompted Dawkins to ask an obvious, yet provocative, question: Are there any other replicators on our planet? The answer, he argued, was emphatically “yes”. Staring us in the face, although still drifting clumsily about in its primeval soup of evolutionary modernity, is another replicator — a unit of imitation [41] that is the very essence of sociotechnical progress itself. This is the fuel that truly powers the modern digital age, and it represents an important leap in understanding. By raising evolution above any level needing biological representation, it makes the digital ether an eminently suitable place for universal evolution. This is how the ongoing progress of the digital world maintains its momentum. Humans and the tools we build are inextricably bound together in an onward march. In realizing that, Dawkins found a name for his self-replicating ethereal unit of cultural exchange or imitation, taken from a suitable Greek root, and calling it a “meme”, inspirationally chosen for its likeness to the very word “gene” itself.

As examples of memes, Dawkins proposed “tunes, catch-phrases, clothes fashions, ways of making pots or building arches”. He mentioned specific ideas that catch on and propagate around the world by jumping from brain to brain or system to system. He talked about fashions in dress or diet, and about ceremonies, customs, and technologies — all of which are spread by one person copying another [41]. In truth, and although he could not have realized it at the time, Dawkins had captured the very essence of the post-Internet world, in a virtual memetic framework that would evolve just as all natural life had done before it. In a short space, Dawkins had laid down the foundations for understanding the evolution of memes. He discussed their propagation by jumping from mind to mind, and likened memes to parasites infecting a host in much the same way that the modern Web can be considered. He treated them as physically realized living structures and showed that mutually assisting memes will gang together in groups just as genes do. Most importantly, he treated the meme as a replicator in its own right and thereby for the first time presented to the world an idea that would profoundly influence global society [40] and business progress in one.

With specific regard to the emergence of sociotechnical ecosystems in the modern age, there are certain things we know, and which should not be underestimated. For instance, as with all ecosystems, be they naturally real-world, synthetically virtual, or an amalgam of both, their arrangement self-organizes and regulates in ways that generally benefit the whole and not necessarily individual parts. We also know that the self-regulation involved nearly always pushes to reduce waste and effort. To the untrained eye, this can appear unruly and chaotic, but closer inspection will show both order and disorder vying for position. Ecosystems, therefore, show well-defined regularities but can also fluctuate erratically [39]. They are complex systems in the sense that they exhibit neither complete order nor complete disorder. Rather, there is no fixed position at any scale to point to as an absolute definition. Instead, complexity relates to a broad spectrum of characteristics. At one end lies slight irregularity, while at the other complete randomness, without meaning or purpose. Both are extremes of the same thing. When slight variation is exhibited it is highly likely that the overall pattern of the whole can be accurately predicted just by examining one tiny part, but when complete randomness is encountered that would be pointless and impossible. In the middle lies some exciting ground; a sweet spot of complexity perched on the edge of chaos. Here, the pattern is neither random nor completely ordered. Regions of differing sizes can all be found exhibiting similar features, leading to the perception of some underlying theme at many different scales. This is where truly natural complexity lives and where the bullseye for Ecosystems Architecture can be found. It cannot be described in terms of simple extrapolations around a few basic regularities, as is the case with Enterprise Architecture. Instead, it displays nontrivial correlations and associations that are not necessarily reducible to smaller or more fundamental units [42]. In a nutshell, such complexity cannot be boiled out and served as if cooked from a list of simple ingredients. Only a deep understanding of all the participants, connections, and properties involved can ever lead to a true appreciation of the whole, regardless of any surface presentation. In that way, complexity behaves like a façade, an interface almost, standing guard over the wherewithal behind. Recognizing this guard and paying due respect is the first step on the road to Ecosystems Architecture. It also ushers in a need for Taoist-like practices. In simple terms, that implies a constant and equal appreciation of all architectural properties in play and at all times; the continual abandonment of preconceived ideas around both local and global patterns (unless blatantly obvious across multiple viewpoints), and a willingness to accept that architectural structure may just emerge without any traceable justification.

In the open then, what is fascinating about complexity’s sweet spot is that it is not only supported by, but promoted as a result of new technology. When looking at economic history, as opposed to economic theory, technology is not really like a commodity at all. It is much more like an evolving ecosystem itself. In particular, innovations rarely happen in a vacuum. They are usually made possible by other innovations already in place. For instance, a laser printer is basically just a photocopier with a laser and a little computer circuitry added to tell the laser where to etch on the copying drum for printing. So, a laser printer is possible with computer technology, laser technology, and photo-reproducing technology together. One technology simply builds upon the inherent capabilities of others [43].

In short, technologies quite literally form a richly connected matrix or web of ingenuity. Furthermore, most technological matrixes are highly dynamic. They can grow in a fashion that is essentially organic, as when laser printers give rise to desktop publishing software, and desktop publishing opens up a new niche for sophisticated graphics programs. There is literally a catalytic reaction taking place  [43]. What is more, technology networks can undergo bursts of spontaneous evolutionary creativity and massive extinction events, just like biological ecosystems. For example, when the automobile became affordable it replaced the horse as the primary mode of low-cost travel. Along with the popularity of the horse goes the primary need for blacksmiths, the pony express, the watering trough, the stables, and so on. Whole substrates of dependent technologies and support systems collapse in what the economist Joseph Schumpeter called “a gale of destruction”. But along with the automobile comes a new generation of change: paved roads, gas stations, drive-by fast food chains, motels, traffic police, and traffic lights. A whole new network of goods, services, and associated technologies begins to grow, spawning cascades of evolutionary sequences elsewhere, each one filling a niche opened by the redundant goods, services, and technologies that came before. The process of technological change is not just a mimic of natural eco-processes; it is like the origin of life itself   [43].

Neoclassical economic theory assumes that systems like the economy are entirely dominated by negative feedback — the tendency for small effects to die away and have no lasting impact on the wider environment. This propensity has traditionally been implicit in the economic doctrine of diminished returns [44]. But, as has been realized in more recent times, there is also positive feedback, or increasing returns present as well — influences that bubble up and build upon each other leading to higher plateaus of stability. These not only help promote significant change such as trends, but also help to explain the lively, rich, and spontaneous nature of many real-world systems.

All this presents a strong case in favor of the obvious; in that, modern companies are now intrinsically embedded into multiples ecosystems, in a number of ways and at a number of levels. Traditionally this has not been seen as the case, especially amongst large-scale enterprise organizations. Many still steadfastly believe that they are outside influencers, or even worse, creators of some end effect they perceive as relevant in their target markets. Real occurrences of this are becoming increasingly rare, however, and are at odds with the ecosystem model which sees growth stimulated from the inside out. In this model, the world is much less responsive to external brute force and deeply relies on new capabilities augmenting the value proposition of others.

The notion of brute force also becomes counterintuitive in an ecosystem world, by breaking the rule of minimized wasted effort wherever possible. For such reasons, ecosystems naturally organize in ways that scale easily. Around the complexity sweet spot, they often choose to be self-similar and self-referential; for instance, forming clusters of capability that naturally compliment and coordinate across their constituent parts. This pattern then repeats at higher levels so that clusters can come together into super-clusters and so on, all reflecting the challenge-focused structure that is echoed in many successful communities and organizations today. Look at how social media works, for instance, or the innards of the Internet. Examine how motorways are mapped or how the various veins and arteries of every living creature are organized. In all, you will see the same metapatterns in use. Instead of employing ridged hierarchies of command and control, many such organizations nimbly flex to dynamically allocate resources as need demands. This not only increases agility and local relevance but removes any preconceived obligation to align with particular definitions of need, market or customer. Such organizations are equally at home providing products or services directly to the lone consumer, the business unit, division, or enterprise. This is further backed by recent observations from major analysts like Gartner^® [45] [46], Forrester^® [47] [48] [49] [50], and McKinsey™ [51]. Overall, flexibility of organization and an openness to repeating structural pattern(s), at varying levels of scale and abstraction, are key, both to understanding the essence of ecosystems and to the design of the Technical Architectures needed in support of success at the hyper-enterprise level.

Likewise, such communities, groups, organizations, and arrangements are continually open to collaboration internally and externally. This not only covers relationships with contemporaries, but also with surrounding environment/market occupants at whatever level, be they suppliers or consumers. It is this ability to spontaneously empathize with, seize, and augment opportunity that adds to their stand-out credibility. In business, for example, by necessity, successful units never focus on one commercial play for too long, instead concentrating on how to develop models and platform plays that can quickly build on grounded success within their surrounding ecosystem networks. This is what gives these units and their parent organizations their defining operational advantage, and ultimately longevity in the face of accelerating change. Their center of gravity is therefore much lower and more dispersed. Within these organizations, high-level direction still filters down from the top, but the power that drives innovation and continual business growth comes not from within, but rather from the networks of support structures flowing out beyond the immediate control of their leadership teams.

Ecosystems form and thrive, therefore, out of an ability to seize latent advantage. In a word, the key is “serendipity” — the aptitude of an individual or group unit to recognize an adjacent or alternatively possible state and adapt to fulfill its potential in mutually beneficial ways. This adjacency not only ensures that appropriate skills, resources, and demand are close, but minimizes the cost and effort involved in innovation. In truth, however, innovation is too strong a word for it, as it suggests elements of independent expertise. Although this can be true, the trick to progress in an ecosystem is deep dependence, especially at the local network level. For that reason, ecosystems form from the adaptation or augmentation of whatever is already in place. They rely more on novelty, inspiration, and appreciation of latent need than any abstract ability to invent.

This is what experts like Stuart Kauffman [52] refer to as “Darwinian preadaptation”. In his work on evolution, Charles Darwin noted that an organ, say the heart, could have causal features that were not the function of the organ and had no selective significance in their normal immediate environment. But in a different environment, one of these features might indeed come to be significant [53].

This type of incidental latent ability to adapt is abundant in biological evolution, with the classic example being the swim bladder in fish. These organs are partially filled with air, partially with water, and allow fish to adjust their buoyancy. Paleontologists have traced their evolution back to early fish with lungs living in oxygen-poor water. Such lungs grew as outpouchings from the gut, allowing air bubbles to be absorbed and the fish to better survive. However, this led to both water and air mixing in the same organ and in such a way these early lungs were preadapted to evolve into a new function — the swim bladder [53].

It is important to understand that preadaptation is not exclusive to biology, however. To provide an example, the following story is said to be true: A group of engineers were trying to invent the tractor at the start of the 20^th century and, given the heavy work it would need to do, they knew they would need a massive engine. So, they obtained a huge engine block and mounted it on a chassis, which promptly collapsed under the weight. They then went on to try successively larger chassis, all of which also broke. All seemed lost until one of the engineers noted that the engine block was so rigid that it could be used to replace the chassis they were struggling with. And indeed, that is how tractors are made. The rigidity of the engine block is applied as a Darwinian preadaptation in the economic sphere [53].

As a further similar example, latent dependency in an economy can drive demand for one product because of change in another. In this way, if the amount of meat in circulation rises, the price of meat falls. Hamburgers might likely experience a rise in demand and therefore restaurants and outlets demand more bread buns. This leads to increased profit across local bakery outlets. In this way the equality of the ecosystem levels out, as one incumbent benefits as a result of another’s loss.

Various names, phrases, and classifications have been invented, reintroduced, or dusted down to help business accommodate the rise of ecosystems thinking. Most, however, are just signposts toward the overall swell of cultural and technological change needed to face off against the reality of the modern connected world. Gurus will talk of “digital initiatives” and “consumerization”, but no single term or phrase really does it justice. The overall change needed must be all-encompassing, coming both from the top and from grassroots levels. This must not only affect the way that organizations structure, but yield a fundamental change in the allocation of responsibility. The easiest way to understand this is to again think in biological terms. Living systems need to adapt constantly to their environment, which has led to the evolution of multiple sensory systems in living organisms. These provide continual feedback, and when external change is experienced, they react both consciously and subconsciously — not only sending signals back to the brain, but also inducing local reaction without the need for support. Without the ability for a reflex reaction to extreme heat, for instance, many animals would suffer far worse scalds or burns when waiting for a sensory signal to be sent to the brain and a muscle response to be triggered.

Such reactions provide a critical lesson for any business. If it is to remain responsive in a world where external stimuli are becoming more prevalent, rapid, and important. Not only are standard long-chain response systems needed, but responsibility must partially devolve to points of contact closest to surrounding influence networks. If done correctly, this will not only mirror the efficiencies of any self-referential problem-solving organizational structure but promote operational autonomy and catalyze change in harmony with the external environment. These lessons should be clear. Billions of years of biological and ecological evolution tell us so. Mother Nature is rarely wrong. What we need to do now is grasp the essences of her blueprints and learn how to infuse them at levels within the reach of direct architectural practice. Not just in the realm of IT architecture, but covering everything from organization structure, through business process design, out into cultural realignment, and so on.

Cumulatively, we are doing this already, but sometimes in nonobvious ways. When the early Internet pioneers set about designing their network, for instance, they were under military direction and keen to ensure the reliability of end-to-end connectivity while under sustained infrastructure attack. The rerouting of data packets across the Internet’s rats’ nest of wire-ways is therefore integral to its construction even today. Reframe that need, however, and the primary design objective becomes one of basic survival. As such, the Internet’s architecture is deeply primordial and completely in line with the most fundamental tenet of evolution; in that, the survival of the whole will always take priority over that of the individual, thereby assuring continuity of opportunity and advance regardless of local setbacks.

This same architecture was passed down from the world’s telecommunications networks and likewise passed on to the World Wide Web. As a result, the continuity of business-to-business and human-to-human communication will remain highly resilient in the face of global disaster.

Beyond that, the implications are more subtle and far-reaching. For instance, above the level of the World Wide Web, experts now openly talk of social machines [54]  [167] [168] [169] [170] [171] [172]. These see the planet’s networks bond whole communities (be they social or commercial) together into emergent amalgams able to act with unified, almost algorithmic, purpose. This is intelligence at the societal level [55] [56], and must lead us to question the very idea of intelligence itself.

Our natural inclination might well be, therefore, to consider intelligence, true intelligence that is, as a solely cerebral quality trapped within the confines of a single biological brain. But it is not. There are many different types of intelligence. Take, for example, the mind of the humble herring. Could such a creature outsmart the intelligence of any individual in their right mind? How about an ant, a bee, or a starling? Same question: how do you fancy their chances? Easy, it might be thought, but what if the rules are changed slightly? How about a shoal of herring, a swarm of bees, a murmuration of starlings, or a colony of ants? Could any individual construct their own skyscraper capable of housing several thousand of their brethren out of nothing more than dirt, for instance? That is exactly what a colony of ants can do, and to them it is one of the easiest things in the world. Furthermore, they are far from unique. It goes without saying that bees can produce great honeycombed hive complexes, and even the humble herring can organize itself with ease into a moving current of consensus, sometimes up to seventeen miles long. Regardless, would you ever consider a bee to be the brightest of creatures? It is not, and that is just the point. There are certain types of intelligence that emerge as if from nowhere. They are created by the summation of tiny snippets of capability — capability that is inconsequential on its own and nondifferential, but capability nonetheless. This works at lower levels in the main, but has just enough quirks, lumps, and bumps to fit precisely into a much greater, purposeful jigsaw of intellect — a jigsaw with no keystone, no ruler, or single manager. No mastermind, then, in such collective puzzles, but still puzzles that reveal a clear and concise picture once whole. A puzzle that is the norm in the realms of distributed intelligence.

“Where is the spirit of the hive, where does it reside?” asked an author as early as 1901 [55]. “What is it that governs her, that issues orders, foresees the future …​?” We are certain today that it is not the queen bee. When a swarm of bees decides to migrate to another location, the queen bee can only follow. The queen’s offspring manage the election of where and when the swarm should settle. A handful of anonymous drone workers scout ahead to check possible hive locations in hollow trees or wall cavities. They report back to the resting swarm by dancing on its contracting surface. During the report, the more vigorously a scout dances, the better the site it is championing. Other bees then check out the competing sites according to the intensity of the dances before them and will concur with the scout by joining in its jig. That encourages more followers to check out the lead prospects and join the commotion when they return by leaping into the dance troop of their choice [55]. It is uncommon to see a bee, except for the scouts that is, which has inspected more than one site. The bees see the message “Go there, it is a nice place.” They go and return to dance, saying “Yeah, it is really nice.” By compounding emphasis, the favorite sites get more visitors, thus increasing further visitors. As per the law of increasing returns, those with more votes get more votes, and those with less votes lose. Gradually, one overwhelming multitude of agreement is reached, and the hive moves as a whole   [55]. The biggest crowd wins, end of story.

History is littered with examples of technical movements, projects, and initiatives that claim to be ecosystems, but are not. For instance, many organizations work with collaboration tools like Box^® and Mural^®. Such workspaces allow for ideas to be shared across communities of interested parties, and can work well to both stimulate and advance progress. As such, they carry distinct value in certain situations, but do not always count as ecosystem catalysts. This is for several reasons. First, their placement often restricts their audience, which thereby limits their inputs, outputs, and subsequent opportunities for feedback, serendipity, and preadaptation — think of platforms like Facebook^®, WhatsApp™, or Twitter™ by comparison. Second, they are often offered within the broader cultural constraints of external command-control structures, like those found in large commercial organizations. These can impede the entrepreneurial spirit and unnecessarily influence evolutionary direction. They are also restricted by the mere fact that the skill needed to contribute precludes a significant number of those who might genuinely want to collaborate. What is more, individual participants can often be constrained by surrounding culture and various external rules or regulations, imposed because of physical location, surrounding politics, and so on. In total, therefore, the restricted application of collaborative tooling is more likely to encourage the development of closed communities, rather than genuinely evolving ecosystems per se.

All that said, such communities can sometimes qualify as ecosystems incubators, especially in climates of rapid and volatile disruption. This is due to reasons of diversity and reach.

There is strong evidence that diversity can improve community stability by influencing the response to disturbance and/or environmental fluctuations. For instance, more diverse assemblages are more likely to display a range of functional traits, increasing the probability that it is possible to compensate for negative responses to disturbance or environmental change [57]. A related phenomenon is the portfolio effect; that is, if the abundance of differing participants within a community fluctuates independently, or at least out of phase with one another, then these fluctuations can average each other out, leading to less variation over time in the diverse assemblage involved [58]. The net of this is that diversity helps protect against false bias and promotes a genuinely broad opportunity for evolutionary processes to take hold.

All of this points to the benefits and challenges associated with openness, decreased centralized control, increased self-regulation, and individual freedom within traditional organizations. To wrestle with these ideas, several models have already been suggested. For instance, the Platform as a Service (PaaS) model has been widely promoted [166] in recent years. This presents the idea of a stage or theatre of operation — an area where “a range of unique capabilities can be deployed and where organizations can seek to establish control over a range of value-creating activities”. At the surface, such models appear to support the type of problem-focused, self-referential patterns common to ecosystems, but when examined alongside established thinking on sociotechnical ecosystems [43] [39] [44] [41] [59]  [60], the evidence suggests that they will prove restrictive.

Here, the key challenge is that such platforms demand hierarchical structure and promote demarcation, thereby limiting evolving conversation flows at multiple levels. In essence, they seek to impose predefined views of the world based on historically successful business models. These just limit free thinking, stifle spontaneous serendipity, and restrict preadaptation in situations hopefully primed for disruption — even though value stream-orientated teams working on such platforms might be autonomous to build and run solutions to meet specific business value outcomes. This counterintuitively fights the primary purpose of any ecosystem, in the need to break free, expand, and find new ground. They further break the rules of minimal effort and devolved responsibility, distracting effort away from evolutionary problem-solving processes, toward the construction of unnecessary ancillary frameworks and management controls.

As an outcome of work on complex systems at scale [61] [62] [63], we have now come to realize that if the conditions in complex networks are right — in any given primordially connected soup, as it were — then there is no need to wait for random reactions to occur at all. The constituent parts, or participants, involved should simply gravitate to form a coherent, self-reinforcing web of interactions and reactions. Furthermore, each constituent in the web should catalyze the formation of other constituents, if appropriately left to its own devices — so that all the constituent parts steadily grow more abundant relative to parts that are not involved. Taken as a whole, in short, the network should catalyze its own formation and in doing so be categorized as an autocatalytic set [64] [65].

Experts, like Kauffman, recognized this as free order created by complex emergent behavior. This was natural order arising through basic rules rather than by any organizational imposition [66].

But was this the real essence of evolutionary success? No. An autocatalytic set has no internal blueprint from which to build itself, no DNA to speak of, no genetic code, no cell membrane — no architecture! In fact, it has no real independent existence except as a haze of constituents floating around in some particular space. Its fundamental nature is not to be found in any individual member of the set, but in the dynamic of the whole; in its collective behavior.

Even so, Kauffman believed that in some deeper sense, an autocatalytic set could be considered to be alive. Certainly, such systems can exhibit some remarkable lifelike properties. They can spontaneously grow, for example, and there is no reason in principle why an autocatalytic set should not be open-ended, producing more and more constituents as time goes on — and constituents that are more and more complex. Furthermore, autocatalytic sets can possess a kind of metabolism. Constituents can take in a steady supply of nourishment in the form of other constituents circulating around them, and catalytically glue themselves together to form more and more complex compounds [66].

Autocatalytic sets can even exhibit a kind of primitive reproduction. If a set from one space happens to spread out into a neighboring space — in a flood say, to use a suitable analogy from the natural world — then the displaced set can immediately start growing in its new environment, if the conditions are right. Of course, if another, different autocatalytic set were already in habitation, the two would intermingle in a competition for resources. And that, Kauffman realized, would immediately open the door for natural selection to purge and refine the sets. It is easy enough to imagine such a process selecting those sets that possessed the most appropriate fitness for the space — the landscape as it were. Eventually, in fact, it is easy to envisage the competitive process giving rise to a kind of DNA and all the other attributes we naturally associate with life. The real key is to find a set that can survive and reproduce. After that, evolutionary processes would kick in and could do their work in comparatively short order [44].

This may appear like speculation, but not to Kauffman and his colleagues. The autocatalytic set story was far and away the most plausible explanation for the origin of life that they had ever heard. If it is true, it means that the origin of life did not have to wait for a highly improbable event to produce a set of enormously complex molecules. It means that life could certainly have emerged from very simple ingredients indeed. Furthermore, it means that it had not been just a random accident but was part of nature’s incessant compulsion to resist entropy, its incessant compulsion to self-organize [66].

When Kauffman tackled the underlying mathematics of his ideas, the reality became obvious; that is, the number of reactions goes up faster than the number of polymers — connections and types of resources in the case of connected networks like the World Wide Web or the Internet. So, if there is a fixed probability that a polymer will catalyze a reaction, there is also some complexity at which that reaction becomes mutually autocatalytic. So, if any primordial soup passes a certain threshold of complexity, then it will undergo a peculiar phase transition. The autocatalytic set will indeed be almost inevitable [44]. And by such principles, if the conditions are right, the ever increasingly connected society in which we live, as well as the symbiosis of technologies that support it, is destined to live, too [66]. In that model, the reality of evermore complex and encompassing sociotechnical ecosystems becomes certain and certainly demands serious consideration; from a scientific perspective at least, if not from an engineering or architectural standpoint.

There is no trickery intended here. When looking at the ideas behind autocatalytic sets, it soon becomes apparent that they have the potential to be ubiquitous across all complex systems, not just those restricted to biology. Autocatalytic sets can be viewed as webs of transformation amongst components in precisely the same way that the economy is a web of transformation amongst goods and services, or the World Wide Web is a network of transformation across data and knowledge. In a very real sense, then, the World Wide Web and the global economy are both autocatalytic sets — as archetypal types of systems that consume resources (in goods, materials, and information) and convert them into something else, something much more powerful and useful too, as in profit and freely available insight [66].^[1]

After all the intensity of the text thus far, it is now time to pause before moving on to consider architectural practice.

A lot has been discussed in a very short space. We have gone from the history of IT, through ideas on ecosystems, added some hard-core theory along the way, and ended up making biological comparisons involving the idea of life itself. So, what on earth has this got to do with the future of IT architecture?

Well, the answer is actually remarkably simple. Rack up the evidence and the verdict will likely be that we are heading toward a much more complex world — a bottomless snake pit in which technology will play a far more critical role. Furthermore, the chances are that the complexity and scale of the problems we will face as IT architects will fall upwards of any human’s natural gifts. Add to that the increasingly dynamic and uncontrollable contexts in which these systems will live and an apparently impossible situation becomes clear. This is the world that the author, Kevin Kelly refers to as being “out of control” — the dawn of a “new era in which the machines and systems that drive our economy are so complex and autonomous as to be indistinguishable from living things” [55].

But wait. Let us not get too despondent just yet. Increased scale and complexity may well be a fait accompli in our IT systems, but within that can be found much to work with — elements that will indeed not only require, but demand, explicit architectural understanding and control. These live in and around the solidity found at complexity’s sweet spot, a solidity openly acknowledged by experts like Kauffman and others — the areas of essential stability, needed to uphold the chaotic spiral of ongoing progress around them. This is where the IT architects of the future will live. And in maintaining the vitality of such solidity, the overall importance of IT architecture will surely rise severalfold.

That means at least two things. First, that there is a clear responsibility on IT architecture to deliver systems that can tolerate (to some degree) both uncertainty and rapid change — no longer will concrete certainty in functional outcomes or operational performance suffice. Second, IT architects will need to deeply understand the mechanics at play in the world around them. In short, they will have to know as much, if not more, about the environment(s) of their solutions, as the solutions themselves. That will mean a broadening out of skills to include, at the very least, an understanding of complexity, basic physical and biological concepts, psychology, and sociology. So, IT architects will need to be increasingly polymathic going forward.

Beyond that is the real challenge. Bringing all these things together into some sort of framework that is readily consumable and eminently practical will be hard. Very hard. It is not something that should be underestimated under any circumstances. Where the base principles of IT architecture may have taken several years to settle, it may take decades for the same to happen with Ecosystems Architecture. Furthermore, whatever demarcations exist between IT architecture and other disciplines, they will surely be both added to and blurred beyond recognition. In that, IT architecture will likely become a blur itself, combining engineering rigor, advanced mathematics, applied science, and exquisite creativity. As such, what you will read here is only the first step on that journey. There is no intention to be definitive, complete, or even immediately relevant. Instead, what you have is more of an introduction to the introduction. A preface to what might come, as it were.

Over the coming pages, the intention is first to disassemble and distill the essence of several well-established architectural tools and approaches. These will then be built back up by extending the ideas at their very foundation. After that, we will look at the erosive challenges imposed by time and rapid change, before finally closing with a handful of conclusions.

So, consider this somewhat a voyage of discovery. Explorers and experimenters have long pushed the boundaries of what we know. They have, for instance, stretched the limits of what is humanly possible, journeyed to far-off places, and even reached out into the darkness of space. For those who have reported back, their stories have enthralled and educated us. We have also learnt from those who have not been so lucky. Take, for instance, the story of John Franklin’s ill-fated Arctic expedition in 1845 [67], as he went in search of the fabled North-West passage. Although not a happy story, his expedition was made possible because of the innovation we now know as canned food. Sadly though, for Franklin and his crew, and although the idea behind food canning was sound, the poor sealing of the cans he took ultimately led to his death and that of his crew through lead poisoning. If only they had waited until 1904 when the Max Ams Machine Company of New York patented the double-seam [68] canning process, which is still in use today.

After the event, we learned of Franklin’s demise and, in retrospect, it is not so much his attempt to expand our worldview that has held our attention. Rather, it is about lessons learnt; how to push out safely and come back with something new and useful.

Notwithstanding the perils of poorly canned food, that is what to expect as you read on here. You will hopefully find snippets of wisdom, some new insight, and perhaps even the faintest whiff of adventure. Nevertheless, nowhere will you find any deliberate direction or intention to land at a fixed destination. You may also uncover some wrong turns and dead ends along the way, but that is all part of the business of exploration. No, this text is more about direction and discovery, pointers and potentials. It is not about doctrines, promised lands, revolutions, or replacements. No doubt it will be a challenging journey for some, but not all. For those intrigued, we wish you well and ask you to persevere. For those who persevere and succeed, we ask you to join us as the quest continues.

Welcome to Ecosystems Architecture …​