Question

Enterprise Architecture is one of our clients responsibilities, and they would like to understand the best practices and approaches of managing EA in an Agile environment.

Terminology

We provide a brief outline of our understanding and experience of EA and Agile to set the context for our response:

EA
EA is a facilitator to organisational change by consolidating multiple areas of architecture and providing services to change initiatives while driving clear and measurable outcomes. This is done by integrating frameworks, techniques, methods, tools, standards and principles within a community of architects who recognise and understand the implications of business change and can use these to deliver quantifiable business benefit.

Agile
Agile is an umbrella term for software development that considers how people collaborate to deliver software, in an iterative manner, with an emphasis on early delivery of working software over long-term planning, analysis, documentation etc.  This includes the use of frameworks, methods and techniques and a mindset that supports creating and responding to change and dealing with uncertainty. These are achieved by empowering multidisciplinary teams.

Context

To understand EA in an Agile setting it is necessary to address two questions:

  1. How effective is your current EA?
  2. Are you using Agile as a set of techniques and methods vs has your organisation adopted an accelerated culture?

In this response, we’ll take a quick look at each of these and then consider the issues of EA in an Agile setting from the perspective of available frameworks, experience and research.

Current EA – experience shows that in many organisations EA is not a facilitator, that it is too academic and, in some cases, self-serving (with very fine and complicated diagrams to support this!) and with long cycle times for requirements gathering and solution modelling. Often there’s a real or perceived disconnect between the activities of EA and what is (or needs to be) delivered on the floor.

EA can be slow-moving; thinking deeply about the long-term implications of its actions before it acts. It also seeks to remove uncertainty, preferring to wait for clearer requirements before acting.

EA often holds a governance role, leading a board that approves technical decisions and occupying a seat on a board approving business change decisions.
The scope of EA may encompass aspects of organisational design, considering people, management structures, culture, business processes and technology. More frequently, however, its role is limited to the use of technology to support the business’ goals and change objectives.
Further detail is necessary to provide a fully qualified answer to the client and would include detail  regarding the current EA operating model, governance, models, maturity, tools, effectiveness etc.

A first consideration in exploring EA and Agile is, ‘how effective is your current EA?’ 

Agile – is an overloaded term that requires some consideration specifically concerning how it is used and implemented. The client has specified ‘in an Agile setting’; without further detail, we must address the assumptions of what this means before we consider EA in this context. 

Practically, Agile can be viewed as comprising several techniques and methods. It is often implemented as a first, or tentative step, with ‘digital’ projects – those that have clear CX and UX components and a balance in favour of coding activities. As these can be distinct, standalone projects, they will successfully adopt Agile techniques, methods, ethos and ways of working. A typical challenge arises where the apparent success of this implementation of Agile is required more widely – and this is where Agile may not prove, in the organisation’s eyes, to be the panacea it was expected to be. 

This is not a fault of the Agile approach or its tools or techniques but highlights the need for organisations to consider where it expects Agile to be most effective. Effective integration of Agile requires a cultural response at all levels in the organisation where benefits are expected.

‘Agile’ may give rise to concern about freedom and discipline. The most Agile ways of working we’ve seen are typically those with the most disciplined teams given the necessary freedom and authority and working hand-in-glove with people empowered to make decisions affecting business operations.

Many Agile practices are relevant across the spectrum of organisational change from coding and delivery through to senior executive reporting and strategic planning.

Using the techniques and approaches provided by Agile in this way has allowed Hivemind to introduce Agile benefits from the floor to the boardroom. 

The second consideration then is ‘Is my organisation and its EA capabilities ready for Agile?’

Experience

Hivemind’s experience of helping clients adopt and sustain the benefits of integrating Agile with architecture are included below:

Client 1 (national gas company)
Our experience shows that for Agile to work, culture must be addressed as a priority. In one of Hivemind’s clients, the CIO (who became a member of Hivemind), with the full backing of the CEO, recognised that significant change should begin with IT.  This could only be achieved by the proper adoption of Agile, not as a method or set of techniques, but as a cultural challenge first. ‘Hearts and minds’ – early and wide-ranging awareness and education activities were undertaken to introduce the concept of agility to a broad range of staff from the floor to the Board. 

This was followed by a change in the way projects were addressed – specifically the ‘perfect plans’ and traditional managed projects were replaced with workstreams, dynamic prioritisation and a ‘pull’ of activities by team members all of which more closely resembled the needs of the business. Amongst other benefits the CIO’s approach reduced development turnaround time (on like for like comparison)  from 108 days to 24 days.

Client 2 (global services organisation)
A current project with a global specialist services business has responded to tensions between architecture and innovation. The architecture governance had been put in place following a history of unsuccessful change initiatives and disjoint projects delivering point solutions. The resulting architecture governance became complex and decision making was too risk averse. Innovation was stifled because of too much bureaucracy, although architects saw the agile approach to innovation as potentially too disruptive.

The approach has been to better balance the imperative to establish a solid business and technology architecture whilst embracing an innovative ethos to permit rapid response to client-facing business needs.

Re-establishing architecture and innovation principles and a streamlined governance is helping build trust across teams. Adopting a digital products approach with clear ownership and an innovation incubator is helping the translation of rapid prototypes into production-ready. The refinement of an existing high-level business capability model is being used to prioritise innovation opportunities and thereby provides a common foundation across teams.

Observations

To work in an Agile setting, EA must be (a) effective and recognised/well established in the organisation (or clear to at least sponsor and consumer stakeholders) and (b) capable of adapting to and utilising changes in people, business, technology and other drivers and obligations that the business operates within.

EA in an Agile environment provides the capabilities necessary to determine, understand and guide the adoption of both strategic and tactical responses to ensure that business and technology change is outcome-driven, rather than model and diagram-driven.

While we’ve outlined our view of EA, its scope and its value to organisations is debated actively amongst architecture practitioners and IT at large. It is also true that EA can at times be its own worst enemy by requiring near end-to-end understanding, requirements, modelling and design before proceeding with delivering measurable business benefit. Many clients simply don’t have the resources for such an approach and therein lies the opportunity to integrate Agile and EA with a focus on business outcomes.

Equally we have seen a lack of EA (or ineffective employment of EA) alongside Agile result in significant levels of duplicated effort, misalignment of objectives across Agile teams, an inability to grapple with common or shared needs and, most concerningly, compromised security and low levels of compliance.
Our experience has shown that the battleship approach is not necessary, however consideration must be given to the following:

  • Culture and buy-in (across the board)
  • Clear linkage to the business
  • Adoption of dynamic prioritisation without the perceived collapse of all supporting and subsequent activities
  • Clarity on what decisions need to be made and when – without impacting the rate of delivery  
  • Effective and transparent governance
  • Recognition of what ‘good enough’ is
  • Recognition of what ‘done’ is and working to that without diminishing returns on perfection
  • Embracing innovation by utilising all  resources creatively
  • A focus on change as good and outcome-oriented rather than the downside
  • Trust in individuals 
  • Enable individuals to do what they’re good at
  • Transparency or action and risk

We believe that Agile Enterprise Architecture delivers just enough architecture, just in time to address business and technical needs without unnecessary resource utilisation while at the same time promoting measurable business outcomes and sustained business benefit through a common-sense approach allowing innovation in process, work practices, design and solution delivery.

Research

Research in the Agile EA space will often rely on existing frameworks and consider how to adopt Agile as a secondary capability across portions of the framework. To ensure that your EA will support, or itself adopt characteristics of Agile, it is necessary to understand the purpose of EA and then address the implications of Agile. We’re also aware of recognising if there are issues with regard to EA in the organisation, or whether it is being effective or perhaps even relevant, then considering Agile may just be a smokescreen being used to shore-up EA initiative(s).n We have a saying in the EA space – ‘always architect for a purpose‘.  

In terms of research papers we have included: 

  • Cooper and Sommer’s 2016 paper ‘The Agile–Stage-Gate Hybrid Model: A Promising New Approach and a New Research Opportunity’ 
  • Sommer’s 2019 paper ‘Agile Transformation at LEGO Group’
  • Fuchs and Hess’ 2018 paper ‘Becoming Agile in the Digital Transformation: The Process of a Large Scale Agile Transformation’
  • Kaddoumi and Watfa’s 2016 paper ‘A Proposed Agile Enterprise Architecture Framework’
  • Dorohyi, Tsuran, Telenyk and Doroha-Ivaniuk’s 2017 paper on a comparison of 10 Enterprise Architecture frameworks (also this is in the context of critical IT infrastructure design)

SAFe (Scaled Agile Framework) must be included here, however this is a large framework and has attracted some criticism such as defining some 12 different roles covering 4 layers (portfolio, large solution, program and team) and more recently allowing customer proxies ignores some of the core Agile principles. 

We also include mention of the The Open Group’s draft standard and as this is still draft caution is urged in reading. Preliminary analysis by one of the contributors to this Hivemind paper (also a contributor to TOGAF®) suggests that this draft framework may suffer from many of the same challenges as TOGAF® – loss of delivery focus to the business in innovative ways and compressed timescales. 

It is worth mentioning that while TOGAF® is perhaps one of the longest-standing and most recognised Enterprise Architecture Framework (now ‘standard’) and the Open Group’s move toward Agile is to be welcomed, it is disappointing to see comments made by the Open Group such as ‘Yet, in a lot of cases, deploying Agile at scale comes at the expense of architecture, causing a number of business-critical set-backs’ 

This is not Hivemind’s experience, in fact it is quite the opposite – we have found that adopting Agile provides the focus necessary to address the areas of most concern by using targeted skills, innovative activities, shorter turnaround times and delivery to dynamic business priorities. Done correctly this allows architecting between the boxes.

Contributors & Community

Alan Simmonds

Architecture

Richard Kernick

Architecture

Brad Arlington

Architecture

Bill Blackburn

Architecture

Matt Turner

Accelerated Delivery

David Clark

Accelerated Delivery

Focus In On: Responsible for Project and Programme Delivery

New Areas of Value:

Higher proportion of projects fit for purpose, on time and on budget

Greater acceptance of change – quicker to implement new changes

Increased credibility with and confidence from across the business

Higher delivery efficiency and effectiveness from clarity around process performance

Improvements around:

Disruption from business restructure or reprioritisations

Unclear or immature business strategy

Lack of clarity or understanding on operational readiness requirements