The challenge to integrate Architecture and Agile

The challenge to integrate Architecture and Agile

Recently on Simon Brown’s blog “Coding the Architecture”, Simon posed are interesting challenge of which this article provides my response. I enjoy Simon’s blog for he presents very well thought out articles on the challenging overlap between Agile software development and the value of architectural principles in providing software solutions. As those that frequent my blog have seen, I’ve written about the challenges of integrating pure Agile development methodologies into non-Agile organizations such as legal services, manufacturing as well as financial services. I’ve approached the topic of work estimation here which is a frequent challenge in non-agile organizations that make project and resource prioritization decisions based loosely on the estimated work effort. I’ve tried to successfully blend Agile story and blacklog project management with the preponderance for waterfall-esk program management here. Based on the comments to that article I didn’t succeed in my goal. I’ve even broached the Agile versus architecture question before here but the focus was on the Agile reducing over architect-ed solutions rather than taking Agile versus architecture head on. I’ve even discussed budget challenges before.

Simon’s challenge: Given a four month window to deliver a new software project in an Agile shop, how would one go about doing so given some interesting constraints? The below graphic is from his blog post that captures the unique constraints for this challenge:

Based on my multiple years in the financial services industry working for more than one bank, I feel the need to outline some immediate assumptions that pop into my mind that are tainted by my direct banking experience:

  1. Design Firms – Simon’s challenge involves the bank undergoing a new branding effort. In my experience, a bank’s internal marketing department partners heavily with an outside marketing firm for larger banding initiatives. Additionally, new branding efforts usually involve getting that new brand message out through a variety of channels.
  2. Middle-ware/SOA capability – Given the overall growing maturity in the service oriented architecture space and the relative IT budgets of most financial institutions, I am going to assume there is some middle-ware, messaging and/or enterprise service bus-like capability to leverage.
  3. Content Management System/CMS – Given the explanation of the current state online banking system, I am going to assume the bank does not currently have a mature content management platform which can be extended to provide online banking functionality.
  4. Backoffice – Also, I am assuming that the bank currently has a backoffice transaction system that currently performs the bank’s core bank product transaction needs. For the sake of this exercise, a “legacy” mainframe set of applications are in place that run the core bank transactional services that has some capability to expose those transactions to the middle-ware/SOA platform.
  5. Build versus Buy – A classic challenge for banks has been the decision to put the focus on building in-house capabilities versus buying mature vendor solutions and integrating those into the technology portfolio. Depending on the financial health of the bank itself as well as the predilection of the CIO, the build versus buy pendulum can swing to either extreme. Add the frequent turn over of senior IT leadership and this can be a real challenge for enterprise architectural patterns. For the sake of this exercise, the assumption is that the bank is willing to invest in the middle-ware transactional capabilities in-house as well as the supporting online banking application user experience but prefers to purchase a new content management solution to host the UI.
  6. Big Bang versus Functional Sequence based rollout – I am also assuming product management is risk adverse and open to sequencing in the new functionality rather than demanding a big bang approach. More on the sequencing later in this post.

Given the architecturally significant changes requested in 4 months, specifically the read-only to real-time transactions and the complete hosting re-platform need, I would convince management that it would be infeasible to deliver everything in 4 months. Yes, the project management side of my brain says don’t over commit given the technical complexity involved and the linkage to a significant customer impacting change.

Thus convinced, I would then break the effort into multiple parallel technical efforts:

Re-brand current site = I would spin up an Agile team including the online banking product management/manager to focus on re-branding the current site between now and the 4 month end date. The team’s priority would be:

  1. Integrate the new branding into the current site working heavily with the outside design firm
  2. Make changes, assuming no conflict with #1 above, to improve W3C compliance

Create re-useable transaction services = I would spin up another Agile team that has the mission of exposing the backoffice transactional services as “enterprise” consumable web services through the middle-ware platform. The stories would mix both the technical needs such as “Develop a way for the middle-ware components to authenticate to the mainframe components” as well as business capabilities “have the ability to transfer unlimited funds, subject to current available account balances, from one customer owned account to another account owned by the same customer”. The prioritization of the stories would be as follows:

  1. Architecturally needed stories such as component to component authentication, high availability, etc. Basically, everything needed to support #2 below.
  2. Transactions needed in the new online banking product based on the roll-out schedule.

One of the challenges for this team is the likelihood that the mainframe platform participants are not familiar with Agile practices. Thus, extra coaching is probably a must for this team.

Purchase Content Management Product = I would task the architecture group to work with procurement to select a vendor’s content management product that will become the framework for the new transactional customer user experience. The classic steps would be undertaken: RFI/RFP, product evaluation, proof of concept, vendor pricing negotiations and ultimately product selection. This needs to be completed in enough time for the next team to be able to deliver some functionality in the new CMS product.

New UI Development = Finally, I would spin up another Agile team to map out and build the new transactional UI. Their first challenge would be to reverse engineer the customer authentication system to determine how the system authenticates users and then links them to their accounts and products with the bank. They would need to work closely with the re-branding Agile team to determine the most seamless way to add navigation to the current site to link over to the new site for the real-time transactional capabilities. Additionally, this team would also have to work with the middle-ware team in order to understand the re-usable services being created and how to consume them. Lastly, this team would also have to keep tabs on the content management team to follow the procurement process and start to understand the framework capabilities of the new CMS product and how to link those to the navigational needs and transactional needs of the final online banking product.

This last team would have a heavy architectural staffing up front and transition to a more development heavy staffing towards the end for delivery. In a perfect world, the project management resources from this team would be the most senior in the organization and consistently work with this team as the skill-set transition occurs.

The below Gantt chart reflects the multiple parallel efforts and when they ultimately need to come together to deliver the final product.


The functional roll-outs would be linked to the customer marketing campaign in order to provide sequential customer communication on new product features as well as permit the new technology “kinks” to be addressed. Also, this allows call centers to ramp up temporary staff to address the increased call volume due to the changes impacting customers familiar with the prior system.

The actual sequence would need to be agreed upon by IT, product and call center management, but potentially the following sequence might fly:

  1. Launch the old site with the new branding.
  2. After a brief window of time to let the call center call volume return to normal levels, replace the link to current “account balance” information in the read-only system to the same “page” in the new system that requests that same balance through the new middle-ware services. The customer experience would be marginally impacted in that instead of seeing yesterday’s balance details they would be seeing their current account balances. Lastly, address any negative performance or functional kinks.
  3. Add account transfer links to the old site that take users to the new site to perform the transactions [first real new customer functionality]
  4. Same call center pause.
  5. Repeat the sequence for additional transactions such as external account-to-account, bill payment, bill presentment, etc.
  6. When all transactions are being handled both UI and middle-ware by the new site and the old site is essentially a set of static pages with links, cut the login process over to the new new site and disable the old site when web logs show no stray bookmarked pages are being accessed. Porting over the authentication system was part of the deliverables of the third Agile team.

Sure, this sounds like a full program of interdependent project activities, but from my experience, banks want to provide highly available and reliable services to their online customers which tend to be their highest revenue generating customers as well. Thus, all of the incremental functional roll-outs which come with the additional cost of testing and program management overhead deliver on the risk adverse, reliable and measured change.

I look forward to all the alternative approaches to this challenges Simon has presented. How far am I from the response norm? Any fundamental flaws?

, , , , , , , , , , , , ,