Midwest IT Survival

The challenge to integrate Architecture and Agile

Recently on Simon Brown’s blog “Coding the Architecture”, Simon posed are interesting challenge of which this article provides my response. I enjoy Simon’s blog for he presents very well thought out articles on the challenging overlap between Agile software development and the value of architectural principles in providing software solutions. As those that frequent my blog have seen, I’ve written about the challenges of integrating pure Agile development methodologies into non-Agile organizations such as legal services, manufacturing as well as financial services. I’ve approached the topic of work estimation here which is a frequent challenge in non-agile organizations that make project and resource prioritization decisions based loosely on the estimated work effort. I’ve tried to successfully blend Agile story and blacklog project management with the preponderance for waterfall-esk program management here. Based on the comments to that article I didn’t succeed in my goal. I’ve even broached the Agile versus architecture question before here but the focus was on the Agile reducing over architect-ed solutions rather than taking Agile versus architecture head on. I’ve even discussed budget challenges before.

Simon’s challenge: Given a four month window to deliver a new software project in an Agile shop, how would one go about doing so given some interesting constraints? The below graphic is from his blog post that captures the unique constraints for this challenge:

Based on my multiple years in the financial services industry working for more than one bank, I feel the need to outline some immediate assumptions that pop into my mind that are tainted by my direct banking experience:

1. Design Firms – Simon’s challenge involves the bank undergoing a new branding effort. In my experience, a bank’s internal marketing department partners heavily with an outside marketing firm for larger banding initiatives. Additionally, new branding efforts usually involve getting that new brand message out through a variety of channels.
2. Middle-ware/SOA capability – Given the overall growing maturity in the service oriented architecture space and the relative IT budgets of most financial institutions, I am going to assume there is some middle-ware, messaging and/or enterprise service bus-like capability to leverage.
3. Content Management System/CMS – Given the explanation of the current state online banking system, I am going to assume the bank does not currently have a mature content management platform which can be extended to provide online banking functionality.
4. Backoffice – Also, I am assuming that the bank currently has a backoffice transaction system that currently performs the bank’s core bank product transaction needs. For the sake of this exercise, a “legacy” mainframe set of applications are in place that run the core bank transactional services that has some capability to expose those transactions to the middle-ware/SOA platform.
5. Build versus Buy – A classic challenge for banks has been the decision to put the focus on building in-house capabilities versus buying mature vendor solutions and integrating those into the technology portfolio. Depending on the financial health of the bank itself as well as the predilection of the CIO, the build versus buy pendulum can swing to either extreme. Add the frequent turn over of senior IT leadership and this can be a real challenge for enterprise architectural patterns. For the sake of this exercise, the assumption is that the bank is willing to invest in the middle-ware transactional capabilities in-house as well as the supporting online banking application user experience but prefers to purchase a new content management solution to host the UI.
6. Big Bang versus Functional Sequence based rollout – I am also assuming product management is risk adverse and open to sequencing in the new functionality rather than demanding a big bang approach. More on the sequencing later in this post.

Given the architecturally significant changes requested in 4 months, specifically the read-only to real-time transactions and the complete hosting re-platform need, I would convince management that it would be infeasible to deliver everything in 4 months. Yes, the project management side of my brain says don’t over commit given the technical complexity involved and the linkage to a significant customer impacting change.

Thus convinced, I would then break the effort into multiple parallel technical efforts:

Re-brand current site = I would spin up an Agile team including the online banking product management/manager to focus on re-branding the current site between now and the 4 month end date. The team’s priority would be:

1. Integrate the new branding into the current site working heavily with the outside design firm
2. Make changes, assuming no conflict with #1 above, to improve W3C compliance

Create re-useable transaction services = I would spin up another Agile team that has the mission of exposing the backoffice transactional services as “enterprise” consumable web services through the middle-ware platform. The stories would mix both the technical needs such as “Develop a way for the middle-ware components to authenticate to the mainframe components” as well as business capabilities “have the ability to transfer unlimited funds, subject to current available account balances, from one customer owned account to another account owned by the same customer”. The prioritization of the stories would be as follows:

1. Architecturally needed stories such as component to component authentication, high availability, etc. Basically, everything needed to support #2 below.
2. Transactions needed in the new online banking product based on the roll-out schedule.

One of the challenges for this team is the likelihood that the mainframe platform participants are not familiar with Agile practices. Thus, extra coaching is probably a must for this team.

Purchase Content Management Product = I would task the architecture group to work with procurement to select a vendor’s content management product that will become the framework for the new transactional customer user experience. The classic steps would be undertaken: RFI/RFP, product evaluation, proof of concept, vendor pricing negotiations and ultimately product selection. This needs to be completed in enough time for the next team to be able to deliver some functionality in the new CMS product.

New UI Development = Finally, I would spin up another Agile team to map out and build the new transactional UI. Their first challenge would be to reverse engineer the customer authentication system to determine how the system authenticates users and then links them to their accounts and products with the bank. They would need to work closely with the re-branding Agile team to determine the most seamless way to add navigation to the current site to link over to the new site for the real-time transactional capabilities. Additionally, this team would also have to work with the middle-ware team in order to understand the re-usable services being created and how to consume them. Lastly, this team would also have to keep tabs on the content management team to follow the procurement process and start to understand the framework capabilities of the new CMS product and how to link those to the navigational needs and transactional needs of the final online banking product.

This last team would have a heavy architectural staffing up front and transition to a more development heavy staffing towards the end for delivery. In a perfect world, the project management resources from this team would be the most senior in the organization and consistently work with this team as the skill-set transition occurs.

The below Gantt chart reflects the multiple parallel efforts and when they ultimately need to come together to deliver the final product.

The functional roll-outs would be linked to the customer marketing campaign in order to provide sequential customer communication on new product features as well as permit the new technology “kinks” to be addressed. Also, this allows call centers to ramp up temporary staff to address the increased call volume due to the changes impacting customers familiar with the prior system.

The actual sequence would need to be agreed upon by IT, product and call center management, but potentially the following sequence might fly:

1. Launch the old site with the new branding.
2. After a brief window of time to let the call center call volume return to normal levels, replace the link to current “account balance” information in the read-only system to the same “page” in the new system that requests that same balance through the new middle-ware services. The customer experience would be marginally impacted in that instead of seeing yesterday’s balance details they would be seeing their current account balances. Lastly, address any negative performance or functional kinks.
3. Add account transfer links to the old site that take users to the new site to perform the transactions [first real new customer functionality]
4. Same call center pause.
5. Repeat the sequence for additional transactions such as external account-to-account, bill payment, bill presentment, etc.
6. When all transactions are being handled both UI and middle-ware by the new site and the old site is essentially a set of static pages with links, cut the login process over to the new new site and disable the old site when web logs show no stray bookmarked pages are being accessed. Porting over the authentication system was part of the deliverables of the third Agile team.

Sure, this sounds like a full program of interdependent project activities, but from my experience, banks want to provide highly available and reliable services to their online customers which tend to be their highest revenue generating customers as well. Thus, all of the incremental functional roll-outs which come with the additional cost of testing and program management overhead deliver on the risk adverse, reliable and measured change.

I look forward to all the alternative approaches to this challenges Simon has presented. How far am I from the response norm? Any fundamental flaws?

As much as you might want to avoid the pitfalls of estimating how long some chunk of work is going to take, it isn’t always possible.  How many times has this happened to you as an IT Manager?

Business Person <stopping Bob in the hallway>: “Hey Bob, we would like to add a simple drop down list of choices instead of that text box on the web site … how long will that take?”

Bob the Engineer: “Replace a text box with a drop down?  That shouldn’t take more than a few hours.”

Business Person: “Great, thanks for the info Bob!”

About to walk off the estimation cliff

Business Person <calling on the phone later that day>: “Sally, since it only takes a few hours to switch the web site’s text box to a drop down, can that be completed tomorrow?  We told the customers that have been asking for that enhancement that it would be ready tomorrow.”

Sally the IT Manager: “Um, err …” <how the heck did this happen?> “what about testing and who is going to populate the drop down with initial values and then add/subtract values as needed?  We don’t have a tool for that which you could use.  We don’t have a release scheduled for tomorrow and thus our Release Manager is on vacation and …” <… and now I have a big mess to sort out>

Once a non-technical person has some duration for a particular request, it is next to impossible to reset their expectations since the resetting usually involves increasing the duration with non-direct IT functions that don’t link directly to their request.  Testing?  Release schedules?  Administration?  Security?  All these don’t directly contribute to the immediate need to complete the request but are all critical to the overall quality delivery of the complete scope that the request dictates.

The first challenge to avoid these scenarios is to coach everyone on your team that they must never ever give out any work estimate of any kind unless you have empowered them to do so.  Make them comfortable with side stepping the artificial urgency of the request with uttering phrases such as:

• “I am not sure.  Can you send me an email with the request so I can track down an answer for you?”
• “I don’t know all the details.  Can I get back to you with an answer?”
• “I am not sure but I think I know who does. Can I get back to you with an answer?”

In my opinion, the best is some permutation of the first option which asks the requestor to write down, i.e., do some work on their part, what they want and send it along.  I am continually amazed at how many verbal requests with urgency never become written requests when asked to be submitted as such.  They couldn’t really be urgent if it takes all of five minutes to craft a quick email with what you just asked for in it and send it along.

In a follow-up second part to this pitfall avoidance I’ll suggest a format to reply to work estimation requests with “CYA” coverage.

Anyone have any other tried and true methods to show responsiveness and concern for a request yet channel the request into some mechanism to respond rationally and with appropriate “CYA”?