In the Beginning
When I learned about TOGAF sometime in the early 21st Century, I recognised it immediately – even though it was new to me.
I guessed someone had observed my methods for the last 25 years and created a standardised way to apply them. That’s not exactly what happened, of course. But it’s partly true, in the sense that TOGAF aggregated an evolving set of best practices used to plan, design, understand and manage software, especially complex enterprise software at scale, and I was one of the practitioners in that mix in the 1990s.
In the decade before TOGAF was being born in 1995, I had led design and delivery of a half dozen first-of-a-kind enterprise systems. As the projects became increasingly large, the value of good holistic, well-documented design became increasingly obvious to me. That value scales up with complexity, and not in a linear manner. This was always the point of the thinking that created TOGAF and other architecture frameworks and methods – or at least the part I recognised as reflecting my own way of working to deliver business change based on IT and OT innovation. It is based partly on the recognition that in a nontrivial enterprise:
- A holistic, abstract model of the fundamentals of the business as well as its transitory needs for change – and how these fit together – should be the starting point for investing in change and measuring its value.
- This map of How Everything Fits Together in the enterprise, which is only partly about technology and which belongs to the business, is constantly changing in ways that can never completely align to changes in the technology environment.
- New system designs that ignore this will create massive, ongoing integration challenges and significant technical debt, and thus will be inherently and increasingly more unstable than holistic design.
- Almost every deployed system only exists in a matrix of other systems, all of which are changing constantly if only because the lower levels of their tech stacks keep going out of date.
An important job of the enterprise architect was and is to manage those business and IT maps as well as possible, to keep them reasonably aligned and to ensure a holistic, logical investment approach while everything is changing. TOGAF described a formal method for putting these insights into practice in a repeatable, understandable way, adding much value (along with others such as Zachman) as part of the community’s manifestation of the Holistic Architecture meme.
The Old and the New
By 1995 I was working out the IBM Labs in Hawthorne, where our team was trying to understand how a very large IBM data and process model called Insurance Application Architecture could be deployed effectively into the emerging world of OOA, OOD and very interestingly (and mostly disappointingly) OODB.
Controlling the Wild Coder
Why was our team focused on the transition to object orientation and model-based software design? Partly because OO solved some interesting technical problems that were very difficult and in some cases impossible in the world of procedural code and entity-relationship data modelling. We were fired up about Grady Booch and Hofstadter’s Godel, Escher, Bach: An Eternal Golden Braid
Partly for market reasons. OO and models and methodologies were important Shiny Objects for CIOs and developers, just like Agile and DevOps today. Vendors needed a story and, if possilble, products. These were popular topics energised by the idea that adoption would reduce the opportunity for developers to create untraceable bugs, integration nightmares, operational vibrations, and undecipherable code (OO code was going to be self-documenting, apparently). Part of the appeal of OO in particular was the idea that by encapsulating code and data, you could somehow encapsulate bad programming in a way that protected everyone else, sort of like imprisoning an immortal demon in a perpetual force field. And OO advocates persuaded many that we could transcend the expensive, slow, rigid, value-blind behaviour of the older generation of developers by freeing developers to just get on with the job and deliver great code overnight.
In the background of this debate was an explosion of development tools, methodology evangelists, and new ways of modelling that could perhaps be plugged into code generators, or at least that could provide traceability between imagined and delivered systems.
The Never-Ending Debate
Twenty years on, we have agile and DevOps and Continuous Integration and Continuous Deployment. We have cloud and more cloud. Low-code and no-code platforms such as SalesForce and ServiceNow and Pega have great traction.
And we have TOGAF 9.2, still part of the conversation. All still attack the same problem – striking the balance between urgently needed quick-and-dirty delivery on the one hand and on the other hand, effective management of security and technical debt and integration and operational stability. Some are fond of saying that Enterprise Architecture is a dead letter and TOGAF is finished.
I disagree. Most of those saying this loudest are making the old mistake of equating EA with software architecture, which is not the same thing. It’s very instructive that fewer EAs and Business Architects report the CIO. This is because while the application of TOGAF-style thinking should govern important IT investments, and can radically improve the ROI of software investments, TOGAF and EA are not ultimately about software development. They are about business value. Without the sort of formal methods and agreed semantics represented by things like TOGAF, there is no real hope of reducing chaos, cost and risk in large-scale IT. This must somehow be done while enabling innovation and artistry in areas where it’s actually valuable and enabling the quick delivery cycles promised by the Agilists.
Many enterprises try to dodge this bullet by outsourcing a lot (or all) of IT. They are outsourcing problems they couldn’t solve themselves (encapsulation) but doing nothing to reduce the world’s supply of friction between the need for agility and the need for governance. And you can’t outsource responsibility for your own bottom line. Increasing the variety in suppliers only makes understanding the holistic design of your own organisation more important. Now you must replace complex communications once based on tribal knowledge with formalised communications codified in contracts. TOGAF still has a lot to offer to all the parties in this game as a way of managing a disciplined holistic view of how things work, how the fit together, and how they might best be changed.
Still in the Game
The effects on business strategy and governance of digitization, outsourcing, cloud, mobile, IoT and IT/OT convergence are creating a new context in which it clearer than ever that Enterprise Architecture, using methods like TOGAF, is a fundamental business discipline for managing disruption and value. There are flaws in TOGAF and its extended family (MODAF, PEAF and all the rest). And not all EA efforts succeed. EA practitioners sometimes make themselves irrelevant by embracing absurdly rigid practices or boil-the-ocean-before-we-start approaches. Sometimes organisational culture is so anti-design and anti-systems thinking that the EA has no chance.
But on balance, TOGAF has done much more good than harm, can be used effectively in spite of flaws and is needed in the modern world. So I still like TOGAF, in spite of its weaknesses (yes, some are significant). We need this kind of thing to help us make sense not just of IT but of most businesses. I liked it before I knew about it, and still do.
0 Comments