The challenge to integrate Architecture and Agile

Recently on Simon Brown’s blog “Coding the Architecture”, Simon posed are interesting challenge of which this article provides my response. I enjoy Simon’s blog for he presents very well thought out articles on the challenging overlap between Agile software development and the value of architectural principles in providing software solutions. As those that frequent my blog have seen, I’ve written about the challenges of integrating pure Agile development methodologies into non-Agile organizations such as legal services, manufacturing as well as financial services. I’ve approached the topic of work estimation here which is a frequent challenge in non-agile organizations that make project and resource prioritization decisions based loosely on the estimated work effort. I’ve tried to successfully blend Agile story and blacklog project management with the preponderance for waterfall-esk program management here. Based on the comments to that article I didn’t succeed in my goal. I’ve even broached the Agile versus architecture question before here but the focus was on the Agile reducing over architect-ed solutions rather than taking Agile versus architecture head on. I’ve even discussed budget challenges before.

Simon’s challenge: Given a four month window to deliver a new software project in an Agile shop, how would one go about doing so given some interesting constraints? The below graphic is from his blog post that captures the unique constraints for this challenge:

Based on my multiple years in the financial services industry working for more than one bank, I feel the need to outline some immediate assumptions that pop into my mind that are tainted by my direct banking experience:

1. Design Firms – Simon’s challenge involves the bank undergoing a new branding effort. In my experience, a bank’s internal marketing department partners heavily with an outside marketing firm for larger banding initiatives. Additionally, new branding efforts usually involve getting that new brand message out through a variety of channels.
2. Middle-ware/SOA capability – Given the overall growing maturity in the service oriented architecture space and the relative IT budgets of most financial institutions, I am going to assume there is some middle-ware, messaging and/or enterprise service bus-like capability to leverage.
3. Content Management System/CMS – Given the explanation of the current state online banking system, I am going to assume the bank does not currently have a mature content management platform which can be extended to provide online banking functionality.
4. Backoffice – Also, I am assuming that the bank currently has a backoffice transaction system that currently performs the bank’s core bank product transaction needs. For the sake of this exercise, a “legacy” mainframe set of applications are in place that run the core bank transactional services that has some capability to expose those transactions to the middle-ware/SOA platform.
5. Build versus Buy – A classic challenge for banks has been the decision to put the focus on building in-house capabilities versus buying mature vendor solutions and integrating those into the technology portfolio. Depending on the financial health of the bank itself as well as the predilection of the CIO, the build versus buy pendulum can swing to either extreme. Add the frequent turn over of senior IT leadership and this can be a real challenge for enterprise architectural patterns. For the sake of this exercise, the assumption is that the bank is willing to invest in the middle-ware transactional capabilities in-house as well as the supporting online banking application user experience but prefers to purchase a new content management solution to host the UI.
6. Big Bang versus Functional Sequence based rollout – I am also assuming product management is risk adverse and open to sequencing in the new functionality rather than demanding a big bang approach. More on the sequencing later in this post.

Given the architecturally significant changes requested in 4 months, specifically the read-only to real-time transactions and the complete hosting re-platform need, I would convince management that it would be infeasible to deliver everything in 4 months. Yes, the project management side of my brain says don’t over commit given the technical complexity involved and the linkage to a significant customer impacting change.

Thus convinced, I would then break the effort into multiple parallel technical efforts:

Re-brand current site = I would spin up an Agile team including the online banking product management/manager to focus on re-branding the current site between now and the 4 month end date. The team’s priority would be:

1. Integrate the new branding into the current site working heavily with the outside design firm
2. Make changes, assuming no conflict with #1 above, to improve W3C compliance

Create re-useable transaction services = I would spin up another Agile team that has the mission of exposing the backoffice transactional services as “enterprise” consumable web services through the middle-ware platform. The stories would mix both the technical needs such as “Develop a way for the middle-ware components to authenticate to the mainframe components” as well as business capabilities “have the ability to transfer unlimited funds, subject to current available account balances, from one customer owned account to another account owned by the same customer”. The prioritization of the stories would be as follows:

1. Architecturally needed stories such as component to component authentication, high availability, etc. Basically, everything needed to support #2 below.
2. Transactions needed in the new online banking product based on the roll-out schedule.

One of the challenges for this team is the likelihood that the mainframe platform participants are not familiar with Agile practices. Thus, extra coaching is probably a must for this team.

Purchase Content Management Product = I would task the architecture group to work with procurement to select a vendor’s content management product that will become the framework for the new transactional customer user experience. The classic steps would be undertaken: RFI/RFP, product evaluation, proof of concept, vendor pricing negotiations and ultimately product selection. This needs to be completed in enough time for the next team to be able to deliver some functionality in the new CMS product.

New UI Development = Finally, I would spin up another Agile team to map out and build the new transactional UI. Their first challenge would be to reverse engineer the customer authentication system to determine how the system authenticates users and then links them to their accounts and products with the bank. They would need to work closely with the re-branding Agile team to determine the most seamless way to add navigation to the current site to link over to the new site for the real-time transactional capabilities. Additionally, this team would also have to work with the middle-ware team in order to understand the re-usable services being created and how to consume them. Lastly, this team would also have to keep tabs on the content management team to follow the procurement process and start to understand the framework capabilities of the new CMS product and how to link those to the navigational needs and transactional needs of the final online banking product.

This last team would have a heavy architectural staffing up front and transition to a more development heavy staffing towards the end for delivery. In a perfect world, the project management resources from this team would be the most senior in the organization and consistently work with this team as the skill-set transition occurs.

The below Gantt chart reflects the multiple parallel efforts and when they ultimately need to come together to deliver the final product.

The functional roll-outs would be linked to the customer marketing campaign in order to provide sequential customer communication on new product features as well as permit the new technology “kinks” to be addressed. Also, this allows call centers to ramp up temporary staff to address the increased call volume due to the changes impacting customers familiar with the prior system.

The actual sequence would need to be agreed upon by IT, product and call center management, but potentially the following sequence might fly:

1. Launch the old site with the new branding.
2. After a brief window of time to let the call center call volume return to normal levels, replace the link to current “account balance” information in the read-only system to the same “page” in the new system that requests that same balance through the new middle-ware services. The customer experience would be marginally impacted in that instead of seeing yesterday’s balance details they would be seeing their current account balances. Lastly, address any negative performance or functional kinks.
3. Add account transfer links to the old site that take users to the new site to perform the transactions [first real new customer functionality]
4. Same call center pause.
5. Repeat the sequence for additional transactions such as external account-to-account, bill payment, bill presentment, etc.
6. When all transactions are being handled both UI and middle-ware by the new site and the old site is essentially a set of static pages with links, cut the login process over to the new new site and disable the old site when web logs show no stray bookmarked pages are being accessed. Porting over the authentication system was part of the deliverables of the third Agile team.

Sure, this sounds like a full program of interdependent project activities, but from my experience, banks want to provide highly available and reliable services to their online customers which tend to be their highest revenue generating customers as well. Thus, all of the incremental functional roll-outs which come with the additional cost of testing and program management overhead deliver on the risk adverse, reliable and measured change.

I look forward to all the alternative approaches to this challenges Simon has presented. How far am I from the response norm? Any fundamental flaws?

Earlier this week I was given the opportunity to share my “aha moment” when it comes to project management.  Below is a brief quote from my article submission:

“Like it or not, technical people need project managers. Yet, ask the average software developer, systems integration engineer or infrastructure engineer about what their company needs more of role-wise and they will almost without a doubt say: “We need more X’s. I’m buried in work!” Feel free to replace “X” with whatever job title applies to whom you are questioning. Software developers indicate they need more developers because the product owners keep asking for more features faster than can be developed.”

Feel free to read the article in its entirety here.

Building structure without command and control ... isn't that an oxymoron?

I can’t say enough positive things about the stimulating agile management views shared in Jurgen Appelo’s new Management 3.0:Leading Agile Developers, Developing Agile Leaders (Addison-Wesley Signature Series (Cohn)) book currently available here on Amazon.  Jurgen has assembled an extremely thought provoking text that covers the breath of the challenges to effective team management today.  From the basics of self-organization to leading people to effective communication to competing management models, Jurgen has provided chapter after chapter of excellent insight into what is needed to manage in today’s dynamic business climate.  Even though I could go on and on about my interpretation of each chapter, this article will focus on one aspect of Jurgen’s “Martie the management model” depicted in the below graphic and introduced in Chapter 16 labeled “grow structure”.

Even though my background is focused on working in corporate IT departments rather than tech startups or consulting, I found great validity in Jurgen’s “views” outlined in his management model including his own admission that it is “wrong” (p.371) to some degree as all models are inherently wrong in being comprehensive and time proven.  Some may say that corporate IT and in particular, large corporate IT departments are unable to be “agile” and thus such an agile management model is less relevant.  I disagree in that there is a constant barrage of blog articles and news stories pressuring corporate IT to be more nimble, deliver more value quicker and with less people.  Thus, managing less people who are expected to wear even more role and functional “hats” requires a more progressive management style compared to the more traditional command and control approach.

With all that being said, wouldn’t one conclude that to “grow structure” one would have to invest in even more command and control management techniques?  Quite the opposite is proposed in Jurgen’s model.  Covered in depth in chapter 13, Jurgen suggests that in order to be in a position to efficiently adjust to the changing demands of the overall organization, technology, products and people, a manager must institute concepts he identifies as “generalizing specialists, wide job titles and informal leadership” as summarized on page 309.  I interpret, from a corporate IT perspective, the structure he is proposing to be one of recognizing the multiple hats and positioning the team to best adapt to change.  Examples in my mind include working with that technical expert in one narrowly defined domain to assist them in appreciating the need for expanding outside of their discrete technical silo through coaching, mentoring and stretch assignments.  Additionally, challenging team members that have been previously placed in narrow job definitions to think of delivering more expansively; while at the same time, providing some structure to that expansiveness in order to provide a level of focus so team members can actual accomplish a task or function.

So, implement wide job titles, push team members to take the lead on assignments (informal leadership) and stretch specialists to become generalists … this doesn’t sound like creating structure?  I encourage you to pick up a copy of Jurgen’s book and explore how he eloquently presents a compelling argument for why such destruction of traditional command and control management is most effective in being an effective manager and leader in today’s business climate.

Can I get the impossible delivered next week?

How many times have you participated in this typical IT conversation?

Business: “How long is it going to take to enable that new FlimFlam engage-the-client-better module with those extra customizations we talked about?”

IT: “Last time we looked at that we said it would take a little over three months including changing the custom code in order to be able to use that new module.”

Business: “Well, we need it turned on by the end of next month when the marketing campaign starts.”

IT: “Um, err, change all the custom code, install the new module and enable it with those additional features in two not three months?”

Business: “Yes.”

The theme of this classic IT delivery challenge is the same:

In order to meet a communicated business need within a business defined time-frame, a perceived number of technical tasks need to be accomplished that don’t initially appear feasible in that given time-frame.

The initial reaction by most technically minded people is this is completely impossible.  And yes, Agile or other project delivery methodologies have built in capabilities to handle fixed dates and variable scopes.  But if you are faced with this common theme of questioning, I am going out on a limb and guessing you are not working in a truly Agile shop or else this question isn’t likely to be asked in this manner in the first place.

So, you rush out and grab your FlimFlam experts to communicate the need and the desired end date.  You tell them this is the most important thing to work on and then you go back to your desk to get ready for the next crisis of systems delivery.

Nope.

Consider taking the time to put together a work delivery estimate as your first priority.  Why do this seemingly futile exercise when the business has already stated what they need and when they need it by?

You need to have some estimate data in hand to have a conversation with the business on what is truly feasible within a pre-communicated time-frame.

This conversation serves a few purposes that are critical to you and your team’s ability to deliver a quality solution:

• Establishes what in-flight work will be put on hold/delayed until this higher priority request is completed.
• Durations of time on sub-tasks enable the business to prioritize what features they truly need versus those that are just “nice to have”.
• Establishes a baseline so when other high priority requests come in or new feature requests are added, you can dust off the previous estimate, revise with the new needs, and re-engage in the conversation.

I can’t count the number of times I have chatted with a business sponsor that swore up and down they just had to have every item in their request delivered by an irrational date.  Yet when presented with some work estimate that indicated everything wasn’t realistically possible within the time-frame [based on the cumulative hours/weeks associated with their request broken down into granular tasks], that same business sponsor started cutting “low priority” features left and right to meet their date.

Business: “You mean it is going to take a week just to get that data on the screen within the application?  We can just use that old report that shows the same data on paper.  Scratch that feature.”

Technical/Engineering Challenge

The typical technical/engineer mindset applied to the theme of delivering unfamiliar technology within an aggressive (arbitrary?) time frame is to think it impossible given the number of unknowns.  Get ready for panic and shortness of breath from your less seasoned technical staff.  Giving them coaching and a framework to break down the seemingly huge and complicated requests into a logical sequence of executable tasks is the other side of the estimation challenge.

To help enable the technical resources to break down “the work” into meaningful, estimate able and negotiable chunks, I’ve attached a simple work estimation spreadsheet:

Sample IT Work Estimation Template 10-05-10 [xls]

Sample IT Work Estimation Template 10-05-10 [xlsx]

Below is a brief explanation of how the template works:

There are three tabs:

1.      Template = where one fills in the data to create an IT work estimate.

2.      Calculations = where certain values are used in calculations on the Template sheet.  Changing numbers here causes the whole estimate to change.

3.      Assumptions = certain assumptions, copyright and reference back to this article for explaining how the template works.

Template

The top portion of the template (red arrow below) is designed to capture the basics about the estimate to tell it apart from other estimates: Name, project, dates, etc.  Think of all the fields that would help you and your organization know what you are estimating.

The main section of the template is where the work break down and granular task estimating occurs.  Gray fields throughout are auto-calculated but can be typed over if needed.  The first task heading “1. Architecture Tasks” shown below is a section to capture the non-negotiable tasks that need to be executed in order for any business functionality to be delivered.  This could be getting servers installed or setting up a new project in MS Visual Studio 2039 or creating a new code branch for the required new features which involves an act of Congress in your organization.

The “description” column is for the estimator to describe, in low tech language, what granular task is needed.  Since one may need to sit down and discuss this with a non-technical business person, some effort to use language that is more descriptive and less heads down techie would be beneficial.

The “Low” and “High” are for the estimator to enter the approximate minimum and maximum hours needed to complete the task.  The “Average” column in gray is calculated as simply the average to be used for the roll-up calculations at the end of the template.  If there are a number of unknowns or the tasks could be quick but depending on X, Y or Z, might run long, use a wide range for “low” and “high”.  This also presents the business conversational element of “Well, if this step goes well, it could be as short as 3 hours.  But, if turns out we do need to engage the storage team and get more storage allocated, then this could take as long as 20 hours.”  This is useful in setting the business’s expectations around variability in the estimate so they don’t get too fixed on a single, implicit number when things don’t follow the “happy path”.

The “Actual” column is useful for the engineer to record the actual time it took them to complete the tasks or the number of hours consumed up to the date the template is revised for some status reporting reason.  This also paves the way for estimators to get better at more evidence based estimating/scheduling.

The “Complete” column is for the task executor to record if they indeed completed the task or if it  still needs work to finish.  Entering a “Y” means the task is complete.  Blank or anything else means the task still needs work to complete.

The “Estimate Remaining” is gray and thus calculated as the remaining hours of the “Estimate Average” minus the “Actual” if the “Complete” column doesn’t have a “Y”.  If the “Actual” turned out to be more than the “Estimate Average” then the column is left blank.

The “Notes” column (not shown here) is for any task notes that help the estimator or anyone reading the estimate to know some additional details about task that is driving the estimated hours.

The next section called “2. Development of Functional Unit 1” represents a block of work that roles up to some business identifiable chunk of work.  Feel free to change the section name to reflect something all project stakeholders would understand.  These sections are designed to be the negotiable features that can serve as that business conversation to determine what are the exact features needed and the corresponding work durations.  Feel free to cut/paste as many new sections as is representative of the work break down requested.

In the above example from the template, Sample Task 2.1 represents a task that was estimated to be 7.5 hours but actually took 4.5 and is compete.

Sample Task 2.2 represents a task with 18.00 hours completed so far, thus calculated is the remaining 5 hours from the 23 hour estimate.

The “Sub Total” represents the min/max of the total work effort (142.25 and 181.25 in this example, which is a full ~40 hour delta) for an average of 161.75 hours of which 65 have been completed and 101 hours remain against the average.  Thus, the business expectations can be set relative to the ~40 hour swing to help the planning for best and worst case delivery.

Below the “Sub Total” section are tasks that are relative to the overall IT work:

In this section, any standard deliverables or work associated with the doing the actual development work can be captured.  In this example section, I added “unit testing” which I calculated as a percentage of the sub total hours of development work above.  The percentage is pulled from the “Calculations” tab.  In this case, I am calculating that unit testing takes 80% of the hours estimated towards development.  You can add/remove entries or adjust calculations on the “Calculations” tab to capture the hours needed to deliver a quality solution that so many developers forget to include in their hard core development work estimating.

The two documentation entries represent either a fixed amount of time, in this case 10 hours, or hours that are a percentage of the total development work, in this case, 20% of the total hours.  Feel free to add and subtract items that come up regularly to make the overall estimate more complete.  Need production turn over documentation?  Add an entry here.  Need some change control document to push a solution into the next environment?  Add an entry here.  Over time, this section will settle into capturing the work that is regularly needed in every project but is easy to forget to estimate for each time.

Lastly, the final section includes the total hours for all the work which is especially useful in answering that initial question: “How long is it going to take to enable that new FlimFlam engage-the-client-better module?”  One additional element that helps beyond the “how many hours” is the “Total Work Days” calculation which is based on a more realistic number of productive hours per day plus any reduction in time to cover other assignments that aren’t specific to project work such as researching a new technology or working on a special assignment of some kind.  The calculations are in the “Calculations” tab.  In this example, the productive work day is 6 hours (not 8 as some might consider) and 80% of those 6 hours should go to projects such as this one.  Hence the “Total Work Days” is greater than simply 348.50 divided by 40.  Again, feel free to adjust these calculations to aide in matching what your resources truly can dedicate to a particular project.  Want to show the “cost” of assigning two concurrent projects to a single resource?  Drop the hours to 3 and add another 20% to cover the cost of “context switching” and see how your estimates come to reflect reality a bit more as an example.

Additional Value

Once established as the “baseline” estimate, as new requests/changes come in, add them to the previous estimation sheet, change the date and quickly be able to predict when the overall solution will now be delivered.  Get ready for another “what is pushing out the date?” discussion armed with your estimating data.

Once established on multiple projects in the “portfolio”, now you can hold these up against competing resources to show “if project X goes first, when would I get project Y next?”  This is extremely handy if your resources effectively cost “zero dollars” in a non-charge back type corporate IT model.

Please give this template a try and let me know feedback on how effective it is with technical resources as well as a conversation tool with project sponsors.

For additional practical estimation articles, consider these I’ve written in the past:

• Is the Gantt Chart Useless in Agile Projects?
• More Pitfalls of Work Estimation
• The Art of IT Work Estimation

Also, for an excellent article on using a similar technique on prioritizing multiple projects, consider this great post by Peter Kretzman, “The Practical CIO: Difficulties in project prioritization & selection, part 2“.

For additional caution in getting too carried away with the accuracy of your estimate, consider Todd Williams’s article “Good Estimates Only Have a 50% Chance of Being Made”.

Everyone participates in iteration planning meetings

Today is the third and final day of the Agile Boot Camp being hosted by ASPE SDLC Training. I’m excited to spend three days focusing on how to leverage the Agile methodology for software project delivery. I’ve read a number of books and articles and blogs over the years, but this is the first formal Agile training I’ve been able to enjoy. In order to appreciate this article, I suggest you read my thoughts and experiences from the first and second days.

First, the excellent Davisbase, LLC instructor finally revealed his contact information. I confirmed with him it was ok to share so here goes:

Rod Ashton

rod@davisbase.org

801-918-4092

I can’t stress enough that Rod’s real world experience in corporate IT project rescue and his on premise company Agile consulting experience were extremely valuable to making this course very pragmatic. Rod would physically step from the center of the room (Agile practice, process and theory) to his immediate left (real world) frequently to segue from theory to practical real world experience based guidance. This made the course triple in value for me. Lastly, at the end of the day today on the course feedback sheet, the best I could come up with on “needs improvement” was having white-boards for team exercises.

The third day is where everything really came together. Thus, without much lead in, here are my bullet point notes from the day:

• When asking why, consider 5 level of “why’s” to get to the bottom of the real need
• For developers that are stuck or struggling to break down a blob of work into tasks, lead them through their internal thought process of developing the blob with “what are you doing to do first? What are your going to do next? Next? Next? …”
• For product folks having trouble articulating what they really are asking for, try “if you could wave a magic wand, what would you want?” to break the potential log jam of real world constraints and competing priorities to get them to spill.
• Make all changes in small increments in order to “tweak” the output in order to achieve cadence.
• Always, rule of thumb = “simplicity”
• Considering the above two bullets, start with 2 week duration iterations in order to get feedback sooner and see results of incremental changes sooner.
• Agile plus continuous integration maybe the next hot thing in efficient and effective software development.
• To start to build team commitment, PM or Agile Coach or Scrum Master lock eyes with team members to get personal commitment on initial story execution until team auto-commits by default.
• No brainer, but how many organizations are realistic on capacity? 6 hours a day max on average of real employee productivity.
• Everyone, everyone, everyone involved in an iteration participates in iteration planning.

I captured some helpful tips on how to establish a new team’s initial velocity:

• Plan to do the hardest stuff first
  • Tolerance for mistakes from management, nay saying peers and others are higher initially and less so as time marches on.
• Time box one hour for estimating to encourage quick decisions and less conversational off topic drift.

All in all, it was a lot of material crammed into three days. Rod did a great job keeping the class moving and keeping attendees awake and engaged. Plus, as I said before, his real world experiences merged with the Agile principles and process really made the course outstanding.

Now, my challenge is bring all this knowledge into practical execution with multiple agile and non-agile development teams plus non-agile infrastructure and all the classic corporate IT structures.

Can you really multitask?

Today is the second of a three day Agile Boot Camp being hosted by ASPE SDLC Training.  I’m excited to spend three days focusing on how to leverage the Agile methodology for software project delivery.  I’ve read a number of books and articles and blogs over the years, but this is the first formal Agile training I’ve been able to enjoy.  In order to appreciate this article, I suggest you read my thoughts and experiences from the first day.

With the basics of the history and the initial ground rules out of the way, now the class can dig into the meat of Agile.  The instructor started the second day with an interesting exercise. We were asked to do three things, each time boxed to two minutes:

On a piece of 8.5 x 11 regular white paper, do the following (2 minutes each):

• Write as many numbers as you can, 1, 2, 3, 4+
• Write as many letters of the alphabet as you can.  When you reach Z, start again with A
• Write as many letters and numbers as you can: A1, B2, C3, D4+

At the end of each 2 minute writing exercise, he captured the top number of entries from the class.  What was interesting and the point of the exercise: no one did particularly well on the letters and numbers exercise.  Even the clever people that tried various techniques to write faster failed to write as many entries as either numbers or letters only.

The point = people can’t multi-task as effectively as they think they can.

Or as I’ve come to appreciate this topic:

Context switching has a price.  That price can be around a 20% loss of productivity as a whole when someone is assigned an additional concurrent task to complete.

Many, many articles support my statement above. One classic is Joel Splosky’s “Human Task Switches Considered Harmful”

Early in the day, someone asked the instructor to share his opinion on if a team commits 100% to start being Agile, how long should it take to go from starting with high frustration to running smooth with little to no frustration.  Mind you, this is just his experience seeing companies and teams take on Agile, but in his opinion:

New Agile Team, 100% dedicated resources = 12 weeks or approximately 6 iterations

New Agile Team working with a vendor = 24 weeks or approximately 12 iterations

New Agile Team + vendor + contractor(s) = 36 weeks or 18 iterations

In more Agile terms, the project team requires the durations above in order to have more of a constant/predictive velocity and established cadence.

It was striking to see how the external elements of vendors and contractors double and triple the time it takes for a team to achieve a predictive Agile rhythm.

As we learned about good story creation and planning poker and relative estimating, I jotted down some Agile success themes that struck me throughout the day:

• Product Roadmap creation forces collaboration amongst all stakeholders.
• Serious Agile success is achieved when the business starts asking how to use Agile for their business work to be as productive as IT.
• Stories are from the user perspective, who, what and why … if you can’t define the “who”, then stop all work/discussion on that story.
• Agile requires a culture that encourages everyone to ask questions of everything
• Stories can come from anyone on the team … developer has an interesting idea about a product feature?  Write up a story for the backlog.

One of the more challenging topics to get ones waterfall head around was the planning poker or story estimating.  I wasn’t alone, a number of technical folks were asking questions around how it is possible to say this work takes “8 units” without completing that phase with “ of hours/days/weeks”.  Most classic corporate IT budgeting and projects need estimates of actual X features completed by Y.  Agile works under more of a “line of credit” concept stating: given Y amount of time and Z resources, X features will most likely get done assuming some velocity with cadence achieved.  I’ve tried to capture this challenge in more detail in this previous post on “Agile versus Classic IT Budgeting”.  For many classic corporate IT shops, this is a huge cultural challenge to overcome.

I am looking forward to getting back together tomorrow for the final day of talking about how to slot in stories based on priority and work units against the roadmap as well as release planning.

Demo Early, Demo Often?

Today is the first of a three day Agile Boot Camp being hosted by ASPE SDLC Training.  I’m excited to spend three days focusing on how to leverage the Agile methodology for software project delivery.  I’ve read a number of books and articles and blogs over the years, but this is the first formal Agile training I’ve been able to enjoy.

The first day covers the history and an introduction to Agile.  It also covers the forming of Agile teams, creating the product vision, focusing on the customer and the product road map.  The instructor was a little slow to get going, but once he got past the high level introductions and the brief history of Agile, things started to move quicker and get more interesting.  The class is full of people from all industries and job roles.  There are people from manufacturing, health care, insurance, government, consulting and financial services.  There are developers, managers, business analysts and project managers.  The class is mostly people without much if any Agile experience, so the pace of the course is perfect for the majority of the attendees.  Given my personal research and light weight implementation of a hybrid of Agile principles in past professional lives, I could see the course moving through material and concepts much quicker.

The instructor had us split into three and four person teams to do some exercises that support the content of the course.  Each team is given some very loosely defined requirements such as make paper airplanes that can achieve a certain goal or organize a pile of colored paperclips.  Once the exercises are completed, the instructor reveals the Agile specific principles involved in the particular activity.  This teaching technique is a creative way to get introduced to the theory before actually being told that theory from a strictly academic sense.  In the examples of the paper airplanes, team members self-organized into airplane designers, developers and testers based on the ad hoc team members basic personal/professional strengths.  I was amazed my high school mentally distracting interest in creative paper airplane construction coupled with the CWRU Strosacker quest to make paper airplanes glide gently over the crowd to softly bump into the main screen to the gleeful cheers of the students in the audience that came in handy today.

After today’s session, in chatting with the instructor, I learned he is a 25+ year veteran of globetrotting in order to rescue projects that are in trouble for time and/or budget reasons for American Express and other multi-national firms.  His hands on real world corporate and consulting experience gave additional credibility to the course material.  Some of his helpful, but maybe not strictly textbook knowledge shared included:

• Demo early, demo often = don’t wait till the end of the Scrum sprint to demo the product increment.  Get feedback ASAP in order to handle possible course correction.
• Team best practices =  Open and Honest Feedback = Direct, Specific and Non-punishing
• Plans are useless, planning is indispensable.
• Documentation is required, but only to the lowest level necessary.  “Do what it takes to barely get by”

I am looking forward to getting back together tomorrow to talk about creating and maintaining a product backlog, prioritizing that backlog and estimating work plus release planning.

Is everyone at the same level of competency?

As I was trolling through my RSS feeds of blogs I try and keep current on and I ran across another thought provoking post by Todd (@backfromred) entitled “The Failure in Gating Process”. I wrote a brief comment to share with Todd and his readers. In the comment I stated that I both agreed and disagreed with Todd’s premise. This blog article expands on that conflict.

Todd’s premise is that having PMO enforced project gates:

• Interrupt project progress momentum
• Allow management and sponsorship stakeholders an excuse to zone till “quality gate #4 next month” before paying attention to the project
• Efficiencies lost to stop, starting and context switching

It is difficult to argue that gating process or projects does have a “cost” associated with the stoppage.

Todd’s solution is better stakeholder engagement.

Again, I can’t disagree with his solution in theory. Where is disagree is in the practical implementation of “better stakeholder engagement” in the real world. First, project teams need strong, competent and knowledgeable participation from these disciplines:

• Project and Executive Sponsorship and Prioritization
• Project and Program Management
• Enterprise and Application Architectural Alignment
• Requirements Gathering, Documenting and Tracking
• Business/Systems Analysis
• Vendor Management (if 3^rd party providers involved in solution delivery)
• Technical Development/Integration Leadership/Delivery
• Testing (from unit through final business acceptance)

Simply having resources “engaged” from these disciplines does equal success if the engaged resources lack competency in their discipline and/or knowledge in their discipline to bring into the project fold.

How practical is it to have strong, competent and knowledgeable resources on every IT project in your portfolio?

I argue it is rare and even if all those resources exist in the same organization, they are assigned to the highest priority and visible project(s) alone. This leaves mid to low priority projects with a less than optimal resource mix.

How do you get timely visibility into when a less than optimal resource mix staffed project team is trending off the optimal trajectory?

You need a project gating system applied to specific projects to strategically pause, evaluate and course correct those specific projects.

I chose the previous statement’s phrasing carefully and specifically to avoid claiming that having a default project gating system as a “silver bullet”1 is the answer. Again, I’m not arguing against Todd’s claim that better stakeholder engagement is needed for successful IT projects. I’m saying the practical application in the real world still requires a project quality gating system due to the inability to have strong, competent and knowledgeable resource.

I have some thoughts on attributes of a more efficient project quality gating system, but I think I’ll need another article to collect and share these thoughts bouncing around in my head.

1. Top notch blog publisher Peter Kretzman on his blog “CTO/CIO Perspectives” has an excellent article on IT “silver bullets” and how to get early indications of projects in jeopardy.

How to marry the classic IT budget and Agile?

I’ve been reading more and more about managing large projects with Agile.  I am wondering how best to optimize the merger of the classic IT yearly budget cycle for large scale projects with the Agile principles and agile project delivery.  There has to be a way to balance the constraints of an upfront budget process with the benefits of agile-ly deployed business prioritized deliverables.  Or in other words, how does one keep the dollars and cents focused people happy and the business project sponsors enjoying the benefits of Agile delivery?

The classic IT yearly budget cycle I’m referring to is one of wild guesses at capital such as servers or software licenses, maintenance, expense and internal plus external resource costs to complete projects that involve IT in the next yearly financial calendar.  Projects maybe completely IT driven with IT as the end customer or projects maybe requested by the business to deliver business value that need IT support in delivering that value.  In either case, the classic budgeting versus agility challenges is the same.  Take the typical project mixing IT and business changes:

New Project Request Form:

Project: A456 Enable Call Center Order Confirmation

Scope: Upgrade the FlimFlam call center system from 5.0 to 6.0 to enable the customer communication widget and change the call center workflow to use the widget and not carrier pigeon to confirm orders with customers.

Budget Questions:

• How much does the upgrade cost?  (easy, ask software vendor)
• How many consulting hours and dollars are needed to support upgrade? (medium, ask software vendor’s professional services for a typical customer upgrade number, then add fudge factors to map over to your company)
• How many hours from your internal shared IT resources to support the entire effort? (hard, don’t have the time to go through the project requirements gathering and design phase or collect all stories, get story points, etc., etc., so take a wild guess based on past projects or other similar work, add fudge factors, add more fudge factors, then double it … you get the idea)

Project Budget Submission Criteria:

? = Capital

? = Maintenance

? = Expense

? = Internal Resource Hours

? = External Resources Hours

? = External Resources Hourly Rate

? = Total External Labor Costs

The budget process demands a number to be provided for each question mark.  You know that certain magic number thresholds, if cross, will doom your project or almost as bad, push it into a category that requires a whole new level of business case and ROI sizing effort. This classic IT project budgeting process doesn’t seem to support the majority of the Agile project delivery process.

Does anyone have any experience or references to best practices in this area of marring legacy budgeting and agility?

Don't give up on the Gantt!

I’ve written before that I am not an “Agile” nor “agile” development nor project management expert. I’ve previously proposed that one by product of “agile” development and project management in general is a reduction in over architect-ed software solutions. With project requirements being represented as stories and tools such as Kanban boards for lean software development to show the flow of work through a process, one might think the classic waterfall project management work and schedule reporting tool, the Gantt chart, is obsolete. Before you abandon this tried and true project schedule reporting solution for the more transparent status inherent in agile project management, you may want to keep reading.

So, you have succeeded in transitioning your IT project management and delivery methodology to one that is more aligned with “agile” than “waterfall”. Your non-IT stakeholders are more engaged than ever in the project requirements definition, prioritization and sprint/release scheduling process. You are tempted to stop trying to use MS Project or other tools to represent schedules in Gantt form. In a word: “don’t”.

One of the main criticisms of complete agile project management is: “The dangers are the loss of recognition that systems/solutions change continually over time as well as team members” Put in more direct, bullet point form from my experience of this criticism expressed by product or management stakeholders:

• What is the big picture?

• I have the big picture in mind, when am I going to get X?

• At the current burn rate, how much time will be invested before I get Y?

• If I add/subtract resources, what will be the impact on the big picture?

These are all criticism subsets that can be directly addressed with the data collection associated with producing a classic Gantt chart. So don’t throw away those Gantt chart creation disciplines just yet.

What is the big picture? I have the big picture in mind, when am I going to get X?

Let the sprint/release iterations continue, but don’t let too much time go by past the releases to meet with the product stakeholders and update your rendition of the product road-map. (You have your road-map, right? You aren’t letting the business surprise you with all requests, right?) This is a great opportunity to get an early indicator if product stakeholders have become caught up in their immediate needs and lost sight of the “cost” of those features. By “cost” I mean the investment in features now means pushing out the product road map. You can gently remind them how the “cost” appears graphically in a Gantt. The Gantt view of their product can clearly show “milestone 45” getting pushed into the next quarter due to their recent feature bonanza. It is much easier to have the “hey, I thought I was getting milestone 45 this quarter” discussion as soon as the schedule shows initial signs of slippage rather then at the start of the next quarter.

At the current burn rate, how much time will be invested before I get Y? If I add/subtract resources, what will be the impact on the big picture?

Here again is where your mastery of tools such as MS Project and the Gantt chart view of the product road map are exceedingly important. If you are working for a MidWestern company, rather than say, a start-up, you can’t ignore traditional budget and resource management constraints. As a start-up in growth mode, your focus is getting your product out the door with the resources you have or your resources plus the on boarding of additional resources. In a MidWestern company, you are most likely trying to maximize the resources you have or being asked to reduce your head count while still meeting project expectations. Thus, prioritizing features to be delivered against a shrinking resource pool is a given. You need something beyond the agile sprints/releases under way to manage project stakeholder expectations on what can be accomplished when.

The Gantt view of the work allows for a graphical view of “if this is more important, what else is impacted” realities. Additionally, add in the need to manage a fixed pool of resources across multiple agile projects and you absolutely have to have some way to represent a view of the prioritized work across all resources. In addition, you may need additional tools to help represent the “what if our team member Sally gets assigned to the VP’s ‘special project’, what does that do to our resource model and what can get done when?”

Conclusion

As an IT manager or IT project lead in a MidWestern company that is moving towards more agile project management and technology delivery, you may be tempted to relax the project management disciplines that come with the more traditional waterfall approach, specifically tracking project schedules in a Gantt chart. In order to avoid the inevitable product stakeholder expectation mismatches as well as pending budget cutbacks and/or uncontrollable resource re-allocations, keep collecting that “program management” data. Continue to have re-occurring meetings to review the “big picture” Gantt chart view of the work and use other tools to reflect “what if” scenarios and their impact on the big picture.