Midwest IT Survival

AWS Makes Coding in the Cloud Easy

With all of the IT punditry talking about how everyone who is anyone is “moving to the cloud”, I thought I would take a serious look at what Amazon’s Amazon Web Services (AWS) has to offer for hosting applications in the cloud. Since I’ve already written about my perspective that “the cloud” is evolutionary rather than revolutionary, I thought I would roll up my sleeves and challenge myself to interact directly with some “cloud” services. What also helped propel me forward was discovering that AWS has a free “get starting” package that includes the ability to provision a server with Internet access, storage and all the AWS development packages and libraries pre-setup.

[Feel free to skip down to the source code if you aren't interested in the next section on business context]

Business Context

Now if you have read any of my articles on this blog you know I mostly cover the challenges of working in a large, corporate IT environment both from a staff and management perspective. So, this is a bit off the beaten path for me. But the rate of business groups pushing corporate IT to implement cloud solutions, especially in the on-line product space, is on a significant up tick. Now, especially in financial services, integrating on-line products with “cloud/SaaS/ASP” hosted applications as product extensions is nothing new. It seems almost as soon as financial firms had an on-line application, they were looking to integrate with existing partners that also were standing up on-line versions of their service offerings: think on-line banking and viewing statements electronically, etc.

The trend difference I’ve observed from the late 90s and early 00s of “ASP” integration to the present is the non-traditional “cloud” companies looking to work with banks. Prior, companies that were already working with banks to provide outsourced off-line services progressed to offer on-lines services. Thus, the maturity of the pre- and post-sales process was familiar to both parties. The ASP providers knew how to address data protection, regulatory compliance and complex/unique technology integrations. The new “cloud” application service providers are using all of the cloud infrastructure as a service (here is the tie-in with AWS) offerings to produce new robust products, but they are completely unfamiliar with how to architect a complete product and service solution for financial services. Thus, many are having to address retrofitting their solutions to be akin to the needs of regulated, conservative banking institutions including all of the security assurance overhead needed (think SAS 70s, penetration tests, security standards and procedures, site visits, lengthy contracts, etc.).

What does all this mean?

In summary, current cloud service providers such as AWS, offer a great suite of building blocks to stand up a robust application. But choose your technologies strategically, especially if you are planning to integrate your product in any way with financial services customers. Be prepared to have to transition to company owned and managed application infrastructure including data storage for the foreseeable future until cloud providers, such as AWS, are universally accepted by the financial services security community as “secure”.

Technical Stuff

Ok, now for a bit more fun technical stuff, I went ahead and signed up for the free AWS package which was incredibly easy. Just a few mouse clicks and I am sitting in the AWS web based management console. Without any serious investigation, I was off creating my own “bucket” of storage in their Simple Storage Service (S3). Next step was to provision a server to host my application experiment. The Elastic Compute Cloud (EC2) tab was equally easy to click through a wizard of picking basic server configuration options. I opted for the Amazon Linux Micro Instance (specifically the Amazon AIM platform as I assumed it would be optimized for using AWS services) in order to stay within the “free” parameters. At the conclusion I was provided all the pertinent remote connection details including a client/user certificate and literally the ssh command syntax to cut/paste and connect.

Since I am clearly taking AWS for a spin years after it first came on the market, I am assuming I am benefiting from significant end user functional improvements made within that time duration. It has been over a decade since any server I built or any code I wrote actually was deployed in a corporate production environment, so to say I have been relegated to a tinkerer in my technical career would be an understatement. But the simple wizard based configuration of the server and storage provisioning clearly allows even a novice technician to be exceedingly productive within AWS.

The Goal – Functional Application Running in AWS

Now that I have cloud storage and a cloud server I needed an application development challenge to solve. So after some thought, here is what I came up with:

Java based application service that will replicate my Dropbox files into my new AWS S3 storage “bucket”.

Note: Yes, Dropbox uses AWS as it’s back-end storage platform so I’m really duplicating my data within the same storage cloud so what am I gaining? Ok, real world, not much gained but this is a throw away experiment to begin with so just permit me this architectural short-sighting.

This experiment involves:

• Installing the Dropbox GUI-less client on the Linux Micro Instance
• Connecting all the Java AWS libraries together to access my S3 storage “bucket”
• Scheduling the application to periodically replicate the Dropbox files to my S3 “bucket”

By using AWS’s example “S3Sample.java” code from their Java SDK, in a matter of a few hours (those hours mostly spent getting all the correct jars linked together in the classpath), I was able to start copying files. Of course, after I reverse engineered how their sample program worked I ran across this article on AWS’s blog that hand holds you through everything.

I was able to follow the directions provided on Dropbox’s site I was able to download and install the Dropbox client on my Linux Micro Instance without a single hick-up.

As I mentioned above, it has been quite a long time since I cracked open an editor and started coding, so any comments on the lack-o-elegance of my Java is most likely very accurate. Plus, I didn’t go so far as add any mechanism to traverse directory trees to copy nested files. Additionally, all I achieved was a one way copy of all files rather than a true sync or any date/time check to see if a file even needs to be re-copied if it already exists.

Goal Achieved!

Here is a link to my (lame, err, not production ready) Java source here.

I welcome any comments around reader’s thoughts on cloud application development and AWS specifically.

How do you survive without SMART goals in today's Corporate IT?

There are plenty of great resources on the Internet that offer excellent perspectives on management and leadership that can be readily applied to those working in corporate IT. And one would think with the vast amount of excellent free advice, all managers would excel at their jobs. Alas, today the demands on IT management make readily putting that advice into practice exceedingly challenging. Recently I’ve been contemplating on how to best articulate what I feel is the dichotomous role a corporate IT professional has in today’s workplace.

Dichotomous Role:

1. Deliver on what your manager of the moment expects
2. Deliver on what your role is expected to deliver to the organization

Why “dichotomous”? More often than not, what your manager expects can be incongruent with what the organization expects.

One might think all you have to do is understand your job description, your department/team/personal goals and objectives and go off every day and do your job. And for some they maybe enjoying this straight forward, obvious job function clarity. But for most, I would feel confident in saying that seeking this expectation clarity can consume a significant number of brain cycles everyday with varying degrees of success. Frequently, your manager’s expectations differ with what the organization expects. What forces are at play creating this dichotomy and what can you do to stay sane over time?

Biggest Contributors to Role Dichotomy? Lagging Goals + Manager Shuffling

First, Lagging Goals

I know of no study or statistical evidence to support my claim, but I feel rather confident in saying that the rate of change in IT has increased dramatically in recent compared to prior years. Step back and take a sample of recent IT management articles. How many are asking the CIO role to change? How many are saying you have to have a mobile work force, outsource development or leverage “the cloud” or risk falling behind? With all that rapid change, in my opinion, pragmatically, gone are the days of SMART goals. Recently, Pawel Bordzinski posted an article similarly calling SMART goals into question here. Sure, MBA academics and management blog pundits will tout the benefits of clearly articulated goals leading to reports having increased delivery success and improved job satisfaction.

Let me be clear up front; I am not contradicting the sound fundamentals of solid goal setting. But unfortunately, with corporate fiscal cycles starting/ending and thus “trickle down” goals trailing six months or more from the cycle start, the average corporate IT employee is lucky to get written goals if they get any goals at all. In looking back over my last five years I probably can point to only two situations where I actually was given documented goals for my job role. In both cases, the fiscal year had already been underway for a good five or six months before I got those written goals.

Why the lag in goal delivery when all sound management principles suggest timeliness equals improved organizational success? In a phrase:

The current corporate business climate expects IT change at such a rapid rate that lagging goals can’t easily, if at all, keep up with the organizational change and subsequent overlapping vision changes.

These typical corporate IT scenarios may seem extremely similar to many and they help illustrate my point. Consider how established goals would need to be handled in each case:

1. The company hires a new “chief marketing officer” who has a new chunk of budget to spend on a “mobile strategy”. Suddenly, new IT projects are kicked off to deliver mobile solutions.
2. An IT Director of the “something” department retires and a new Director is hired from outside the organization. Managers reporting to the previous Director either start reporting to new areas of the organization or start leaving the company. The new Director starts hiring replacement mangers from his prior company.

In the first scenario, assuming managers, teams and individuals had goals that reflected pre-CMO priorities, all now have to wind down a bit on what was previously being worked on and wind up on what the new CMO sponsored projects entail. Sticking to pre-CMO priorities are just not an option. The company clearly has a strategic gap hence the CMO was hired in the first place. Thus, ignoring the CMO’s “high priority” projects because they don’t fit nicely into prior communicated priorities and goals is effectively ignoring the business needs of the company.

In typical corporate IT fashion, the priority of these new CMO projects has been communicated from the top of the house down thus the entire IT delivery management structure is trying to figure out how to reshuffle in-flight work in order to accommodate them. The crisis of the moment has shifted from whatever was the previous crisis to the new CMO project delivery crisis. The company wasn’t strategic enough to see the need for a CMO earlier as new media outlets were creating new demand, what is to say the organization is strategic in addressing new IT project priorities? Lastly, with IT departments cut staff and budget-wise due to the recent recession, what management structure is going to stop and revise all previously documented goals? The demand for flexibility, agility and rapid change makes it next to impossible to be able to cleanly re-write goals as priorities shift.

If the goal setting challenge faced a stagnant organizational chart, then there might be some HR efficiencies all could leverage, but on top of priorities changing, org structures rarely stay static for more than a few months. The second part of this article will dive into what compounds the goal problem for corporate IT employees: rapid organization and management reporting structure changes.

Does IT have a strong or weak partnership with the Business?

Whether you are working in a complete custom software development shop with little vendor interaction or a technology integration shop with vendor solutions integrated with other vendor solutions on top of yet other vendor solutions, you will have to manage vendor relationships to some degree as an IT manager in a MidWestern company.  This series looks at the complex arena of IT vendor management and offers some tips to make the arduous process a bit less arduous and possibly discover some additional benefits along the way.

Vendor Management Categories

• Who Owns the Relationship

In the previous article, I used an example of a company CIO bringing in a software vendor with the CIO’s brother as a majority owner of the software company and how to navigate the need for technical assistance from the vendor to get their product installed successfully by your team.  The notion of needing to investigate and understand the non-technical dynamics of who owns the vendor relationship is critical to avoid a potentially career limiting exchange involving vendor bashing or venting frustrations too openly.

Continuing in the theme of understanding who owns the vendor relationship, especially when you or your immediate management don’t, what other dynamics come into play?  Another challenging dynamic is when the business owns the relationship yet IT needs significant interaction with the vendor to establish the vendor’s service for the business.  Starting from the beginning during the vendor selection process, depending on the business plus IT partnership, the vendor selected will have a significant impact on how smoothly the service will be implemented.

Strong Partnership between IT and Business

If the intra-department partnership is strong and thus IT has an equal seat at the vendor selection table, then there is a greater potential for the vendor selected will include critical due diligence around the vendor’s IT strength and competency.  To determine the vendor’s capability to smoothly integrate the service, IT needs an efficient yet thorough due diligence process to quickly understand if the vendor successfully integrated their service with another customer recently that matches key attributes of your technology framework.  The vendor that can site examples without prompting of integration challenges and how they over came those challenges that closely match your anticipated challenges given your technology frameworks is in a stronger position.  A vendor that can’t cite examples with confidence and clarity will most likely be learning “on the job” with your company.  Note, not all “learning on the job” situations are negative.  In a future article I’ll describe a “strategic customer” relationship that turns learning on the job into a significant win for both company and vendor.

Weak Partnership between IT and Business

If the intra-department partnership is weak, the business will most likely engage vendors and make a product or service selection based more strongly on their goals and objectives and not consider the IT integration costs heavily in the due diligence process.  The business could be conducting the due diligence process and not have a way to engage IT partners easily due to how IT is structured within the organization.  Or, the business could be specifically down playing the role IT needs to play in the service integration effort due to lack of service implementation experience and just how important IT’s role is in that success.  Additionally, the business may be lulled into a sense of artificial comfort by the vendor’s slick sales pitches that create the impression the IT side of things is a “no brainer” and they’ve done this integration in the recent past with countless customers with nothing but success.

Sounds like the strong partnership between the business and IT fosters more open communication in an effort to more equally way the IT costs in with the business integration costs.  How can one leverage techniques to generate conversation when the partnership isn’t strong?  Continuing this MidWestern IT [] perspective on the topic of “Who Owns the Relationship?” in the next article.

Are you meshing with the vendor relationship owner?

Whether you are working in a complete custom software development shop with little vendor interaction or a technology integration shop with vendor solutions integrated with other vendor solutions on top of yet other vendor solutions, you will have to manage vendor relationships to some degree as an IT manager in a MidWestern company.  This series looks at the complex arena of IT vendor management and offers some tips to make the arduous process a bit less arduous and possibly discover some additional benefits along the way.

Vendor Management Categories

• Who Owns the Relationship

When looking into the broad topic of vendor management, one could easily be hit with confusion as to where to start the investigation.  For lack of a better place, I picked “Who Owns the Relationship” as a starting point.  Ultimately, someone or some group within the organization is the interface between the service the vendor provides and the benefit the business receives from that service.  Someone or some group within the organization knows who to contact within the vendor’s organization for answers to all facets of questions pertaining to the service the vendor provides, including:

• Service level agreements for deliverables
• Over all service cost structure
• Service benefits to the organization
• Out of scope elements
• Who to contact to reach subject matter experts within the vendor’s resource pool

The service could be as simple as a software widget that converts data from one format to another with the widget being used to complete that data conversion within a larger software application.   Or, the service could be as complex as a complete business process outsourcing arrangement where a major portion of a core business function is effectively provided beginning to end by a vendor loosely at the direction or more specifically, within the operating parameters established by the business.

As an IT Manager in a MidWestern company, knowing who owns the vendor relationship by service is absolutely critical to functioning successfully in your role and to avoid career challenging missteps such as:

You <out loud in an IT all hands meeting as you wait for the meeting to start>: “Who ever makes this FlimFlam software application must be the worst vendor on the planet!  The documentation is completely wrong.  It crashes every five minutes with completely useless error messaging and no logs!  Why can’t we just scrap it and run the market leading software by …”

Peer <in the meeting as well>: “Um, the FlimFlam software was developed by the CIO’s brother-in-law who also happens to be a member of our company’s board of directors.”

You: “Ummmmm, errrrrr …”

This is somewhat an extreme example, but the point I am trying to convey is vendors are sometimes selected not because they are best of breed or because every technical person within a 200 miles radius of your desk would all agree on vendor X for a particular solution, but rather, due to disconnected value propositions.  Propositions such as due to a personal relationship or due in part to a larger arrangement that accepts a lower quality of service solution for an associated arrangement that provides higher overall value to the organization.

Extending the above example evern further: given a software application that you have been tasked to get functioning but is not performing as advertised, how does knowing the vendor relationship owner assist in your need for success?  The vendor probably knows at some level they aren’t quite providing the best solution to the business need.  Yet, they have a similar stake in maintaining credibility in providing this software service given their company owner is directly involved in this customer relationship.  The last thing employees of the software company want to be dealing with is their owner coming back to them with:

Software Company Owner: “Hey, I’m hearing our product isn’t working for ABC Company.  You know I am directly involved in ABC Company and don’t want to hear our product isn’t working for them.  What are we doing to fix this situation so I only hear good things about our products from ABC Company?”

So, how can you leverage this example’s relationship ownership knowledge to your advantage?  Consider approaching the CIO or appropriate members of the organizational chain that can interface with the CIO to present the challenge this way:

“We are having some trouble getting the FlimFlam software up and running.  It is probably something we didn’t quiet understand from the limited interaction and documentation provided to us from the FlimFlam company.  I understand you may have some knowledge on how to put us in contact with the right people at the company to help us get past our current challenges.  Can you help us get that contact established?”

Since, in this example, the CIO is connected to someone high up in the management/ownership of the vendor, slamming the vendor is also a slam against the CIO for choosing and thus  endorsing this vendor.  Flipping this around, the CIO has chosen and is thus endorsing this vendor; hence, the CIO wants the vendor to succeed so he or she is viewed positively for bringing in this vendor to the organization.  Thus, presenting the problem of bad software instead as a need to gain access to strong vendor resources to partner with your team to get things installed correctly shows your respect for the vendor relationship and the credibility of the CIO yet still affords you the ability to complete your task.  Or stated yet another way, given this query, how can the CIO not pick up the phone, call someone important at the vendor and push them to get their top people engaged in order to preserve his credibility?  The CIO involvement can go between two hypothetical extremes in this example:

Extreme = Ultra Positive

Top engineering resources at the vendor are engaged.  They are informed of the stakes and provide high touch service to you and your team.  Maybe they have a good product but the documentation is weak and they can fill in the gaps.  Maybe they, in working with your team, discover a gap in their product and with you and your team’s help; they are able to provide technical fixes that work specifically in your technical framework while strengthening their product for other customers.  CIO appears to be the hero.  He or she was able to bring the right people together to make this happen.  CIO views you as a strong management team partner as you enabled him to reinforce his leadership perception with little risk and little effort on his part.

Extreme = Somewhat Negative

The vendor doesn’t have top engineering resources or the product is indeed a wrong fit or the vendor fails to manage the relationship on their side of the vendor management equation.  Whatever the failure, you have kept up your side of vendor management arrangement.  If you continue to keep good notes and use email to document major communication exchanges, you will have evidence to show you really tried to partner with the vendor but things fell down on the vendor’s side.  Be prepared for unpredictable external events to trickle down to you and require you to quickly assemble your “here is what we did and said when” list to provide your perspective that even after the high level vendor resource request, the vendor is not coming to the table with solutions.  Again, bashing the vendor or using this forum as an opportunity to vent your frustrations will trend towards career limiting rather than career bolstering.

Continuing this MidWestern IT perspective on the topic of “Who Owns the Relationship?” in the next article.

Is build versus buy like pushing your team off a cliff?

Whether you are working in a complete custom software development shop with little vendor interaction or a technology integration shop with vendor solutions integrated with other vendor solutions on top of yet other vendor solutions, you will have to manage vendor relationships to some degree as an IT manager in a MidWestern company.  This series looks at the complex arena of IT vendor management and offers some tips to make the arduous process a bit less arduous and possibly discover some additional benefits along the way.

As I reflect on the topic of vendor management, I tried to break down vendor management into subcategories.  For each subcategory below, I plan to dedicate an article or two to drill down into the nuances I’ve come across, what techniques worked for me and what didn’t turn out so well:

Vendor Management Categories

• Who Owns the Relationship
• Vendor Service Integration Challenges
• Role of the Sales Rep.
• Sales Cycle and Pricing
• How to Leverage Tech Support
• Product Versioning and the Upgrade Cycle
• Cost of Swapping versus Maintaining
• Product Selection/Feature Evaluation
• Being a Strategic Customer

Why spend so much time covering this topic?  Don’t MidWestern companies pay decent money to vendors for technology they can install, get the features they need cheaper and more robustly than building themselves and pay a monthly/yearly fee for this great benefit without further worry?  Well, technically, on paper, yes, but in my experience, it has never worked out so simply.

As a typical IT manager in a MidWestern company that primarily leverages IT for operational efficiency gains rather than direct core technical product offerings (see this article for more on this IT conceptual nuance), there are plenty of business cases that tip the build versus buy decision towards the buy.  If the business case says buy, then buy means vendor.  And believe me, the minute you start asking the “hey, does someone make a technology product that does X” question, there will be thirty if not forty different vendors all clamoring to sell you on their product that does X and for the cheapest price you can find anywhere.  Now nailing down that price is quite a challenge, but more on that aspect in a future article.

A point of clarification when I say “vendor” and “buy” what I am referring to is more directly the aspect of purchasing a software or infrastructure technology that you and your team have some hand in installing, configuring and/or supporting it’s operations going forward.  An example would be database products from companies such as IBM, Oracle or Microsoft.  You are clearly looking to leverage the expertise these companies have invested in their product over the years rather than build your own database solution from scratch.  An example of what would not fit in this category would be a vendor/product relationship where a majority if not all of the IT services are outsourced to the vendor.  “Software as a Service” models would fit in this “outsourced to the vendor” category or effectively any web site that the company leverages a vendor to provide the complete business service with little to no integration or direct support from in-house IT.

Cloud computing [definition: Wikipedia] is a recently maturing technology offering that is blurring the lines a bit between, say, buying “software” and install it on your “box”.  The “software” may be a portion of the overall software application that supports the business need that happens to be hosted by a third party both in terms of the platform as well as the business logic/rules that are being executed as part of the overall business solution.  The “box” may be from a logical interaction perspective the same as a server powered, networked and operating within your company’s data center but in reality, a virtual server powered in the vendor’s datacenter and connected to your company’s network via the Internet or similar arrangement.  Cloud computing is rapidly gaining attention as a viable and cost effective solution to “outsourcing” more IT functionality to a vendor while continuing to maintain control over core business differentiated IT services and pushing off control of commodity/non-core business services to third parties.  The concept involves moving more non-core business services “to the cloud” allowing cloud vendors to focus on up time, reliability, disaster recovery, capacity and performance at a price point lower than in-house since it is spread across all the customers of the cloud.  The basic concepts of vendor management still apply to cloud environments.  I encourage readers considering cloud computing to continue to evaluate the growing experiences of peers as cloud computing grows and matures from its current state.

So, let’s dive into the realm of vendor management from a MidWestern IT perspective starting with the topic of “Who Owns the Relationship?” in the next article.