Midwest IT Survival

AWS Makes Coding in the Cloud Easy

With all of the IT punditry talking about how everyone who is anyone is “moving to the cloud”, I thought I would take a serious look at what Amazon’s Amazon Web Services (AWS) has to offer for hosting applications in the cloud. Since I’ve already written about my perspective that “the cloud” is evolutionary rather than revolutionary, I thought I would roll up my sleeves and challenge myself to interact directly with some “cloud” services. What also helped propel me forward was discovering that AWS has a free “get starting” package that includes the ability to provision a server with Internet access, storage and all the AWS development packages and libraries pre-setup.

[Feel free to skip down to the source code if you aren't interested in the next section on business context]

Business Context

Now if you have read any of my articles on this blog you know I mostly cover the challenges of working in a large, corporate IT environment both from a staff and management perspective. So, this is a bit off the beaten path for me. But the rate of business groups pushing corporate IT to implement cloud solutions, especially in the on-line product space, is on a significant up tick. Now, especially in financial services, integrating on-line products with “cloud/SaaS/ASP” hosted applications as product extensions is nothing new. It seems almost as soon as financial firms had an on-line application, they were looking to integrate with existing partners that also were standing up on-line versions of their service offerings: think on-line banking and viewing statements electronically, etc.

The trend difference I’ve observed from the late 90s and early 00s of “ASP” integration to the present is the non-traditional “cloud” companies looking to work with banks. Prior, companies that were already working with banks to provide outsourced off-line services progressed to offer on-lines services. Thus, the maturity of the pre- and post-sales process was familiar to both parties. The ASP providers knew how to address data protection, regulatory compliance and complex/unique technology integrations. The new “cloud” application service providers are using all of the cloud infrastructure as a service (here is the tie-in with AWS) offerings to produce new robust products, but they are completely unfamiliar with how to architect a complete product and service solution for financial services. Thus, many are having to address retrofitting their solutions to be akin to the needs of regulated, conservative banking institutions including all of the security assurance overhead needed (think SAS 70s, penetration tests, security standards and procedures, site visits, lengthy contracts, etc.).

What does all this mean?

In summary, current cloud service providers such as AWS, offer a great suite of building blocks to stand up a robust application. But choose your technologies strategically, especially if you are planning to integrate your product in any way with financial services customers. Be prepared to have to transition to company owned and managed application infrastructure including data storage for the foreseeable future until cloud providers, such as AWS, are universally accepted by the financial services security community as “secure”.

Technical Stuff

Ok, now for a bit more fun technical stuff, I went ahead and signed up for the free AWS package which was incredibly easy. Just a few mouse clicks and I am sitting in the AWS web based management console. Without any serious investigation, I was off creating my own “bucket” of storage in their Simple Storage Service (S3). Next step was to provision a server to host my application experiment. The Elastic Compute Cloud (EC2) tab was equally easy to click through a wizard of picking basic server configuration options. I opted for the Amazon Linux Micro Instance (specifically the Amazon AIM platform as I assumed it would be optimized for using AWS services) in order to stay within the “free” parameters. At the conclusion I was provided all the pertinent remote connection details including a client/user certificate and literally the ssh command syntax to cut/paste and connect.

Since I am clearly taking AWS for a spin years after it first came on the market, I am assuming I am benefiting from significant end user functional improvements made within that time duration. It has been over a decade since any server I built or any code I wrote actually was deployed in a corporate production environment, so to say I have been relegated to a tinkerer in my technical career would be an understatement. But the simple wizard based configuration of the server and storage provisioning clearly allows even a novice technician to be exceedingly productive within AWS.

The Goal – Functional Application Running in AWS

Now that I have cloud storage and a cloud server I needed an application development challenge to solve. So after some thought, here is what I came up with:

Java based application service that will replicate my Dropbox files into my new AWS S3 storage “bucket”.

Note: Yes, Dropbox uses AWS as it’s back-end storage platform so I’m really duplicating my data within the same storage cloud so what am I gaining? Ok, real world, not much gained but this is a throw away experiment to begin with so just permit me this architectural short-sighting.

This experiment involves:

• Installing the Dropbox GUI-less client on the Linux Micro Instance
• Connecting all the Java AWS libraries together to access my S3 storage “bucket”
• Scheduling the application to periodically replicate the Dropbox files to my S3 “bucket”

By using AWS’s example “S3Sample.java” code from their Java SDK, in a matter of a few hours (those hours mostly spent getting all the correct jars linked together in the classpath), I was able to start copying files. Of course, after I reverse engineered how their sample program worked I ran across this article on AWS’s blog that hand holds you through everything.

I was able to follow the directions provided on Dropbox’s site I was able to download and install the Dropbox client on my Linux Micro Instance without a single hick-up.

As I mentioned above, it has been quite a long time since I cracked open an editor and started coding, so any comments on the lack-o-elegance of my Java is most likely very accurate. Plus, I didn’t go so far as add any mechanism to traverse directory trees to copy nested files. Additionally, all I achieved was a one way copy of all files rather than a true sync or any date/time check to see if a file even needs to be re-copied if it already exists.

Goal Achieved!

Here is a link to my (lame, err, not production ready) Java source here.

I welcome any comments around reader’s thoughts on cloud application development and AWS specifically.

Initial Foray into Tablet Computing

In keeping up with my personal trend of being an early technology investigator yet a late technology adopter, I finally made the leap into tablet computing. After following the blogosphere cover the literal explosion of tablets onto the consumer and corporate market, I finally decided I needed to try and commit to integrating a tablet into my daily computer using habits. After getting my spouse a Kindle Fire as a Christmas gift in a response to her request for an ebook reader and helping her get it setup, I knew I would have to make a product selection for myself. I immediately got the sense in handling the Kindle Fire that today’s tablet fits a computing need I really didn’t know I had. That need, for me, is filling the gap between when one is logistically barred from one’s primary desktop computer and yet equally inconvenient to fire up the laptop/netbook to access Internet content. And so, this week arrived a RIM Blackberry PlayBook to my door.

What? A PlayBook? Isn’t RIM the mobile incumbent vendor that has let the market pass them by.

Yes, thus permit me a few minutes to explain … here goes: I use a Blackberry Curve mobile phone with quite some time still left on my service contract and find the tethering option to access the Internet via my personal netbook invaluable in my effort to keep my personal computing pursuits completely separate from my full-time professional demands. In my mind, having a security related position ratchets up the need to fully and completely embrace all policies and standards surrounding using company resources for only company business. Thus, when I researched the tight integration between the PlayBook and Blackberry phones, that became a plus for me.

My primary intention for the tablet is for consumption of Internet hosted, primarily written, content. In other words, I’m not really interested in playing games or watching movies or even outright purchasing ebooks. These interests don’t seem to constrain anyone to any particular tablet manufacturer but RIM has been focused on the corporate user, rather than consumer to their suggested demise, and with the addition of having a strong security framework around mobile computing, I drifted towards the PlayBook with that in mind.

Then came the “fire sale” and the prospective of having a corporate designed tablet device with 64gb of storage originally priced at $699 for $299. It seems unclear as to RIM’s specific motivations for such deep discounting of a product that has taken a beating in the media since being announced last year right before a major upgrade (OS 2.0) next month (Feb. ’12). Add RIM’s significant investment in this year’s Consumer Electronics Show (CES ’12) showing continued investment in the PlayBook product and picking up a discounted PlayBook, still a gamble, owning one could be intriguing.

Learning that OS 2.0, from folks brave enough to install the development beta version, will include the ability to run native PlayBook apps as well as Andoid apps, thus even if RIM is unable to gain significant leaps in market-share and corresponding development investment to enrich the product’s capabilities, the product should benefit from enabling the Android Marketplace as a source of additional usefulness.

All things considered, the price point is what really pushed me over the edge to make the purchase. If RIM continued to leave the PlayBook 16gb entry level point at $499 I am pretty certain I would still be researching the vast tablet landscape.

So, after a few more days of use, I’ll post my initial experience of how tablet computing and the PlayBook specifically is working out for me.

Today's IT Projects Need Transparency to Change

For large organizations it seems that as technology grows more and more integrated, IT related projects become more complex and thus longer in overall duration. There is no doubt the rise in cloud/SaaS solutions has exacerbated this increase in overall IT project complexity. I’ve written on the impact of cloud in this manner prior here. Gone are the days of a large corporate IT shop having a project manager engage the same three or four familiar delivery stakeholders and with little outside involvement, execute the project beginning to end. This increase in technical integration means a project manager can no longer count on those three or four stakeholders having the cross systems knowledge and technical systems access to implement changes as crisply with few artifacts as to what the project has done/is doing/when/etc.

To help illustrate this evolving shift, consider the following hypothetical large corporate IT conversation:

PM: “Welcome everyone to the FlimFlam upgrade project’s twentieth weekly status meeting and a special welcome to Jim who is joining to help sort out all the changes that impact others outside of our core team.”

<General welcoming gestures and verbal niceties ensue>

Jim: “Ok, is there any diagram that captures all the flows of data in and out of the current FlimFlam system?”

Core Team: “Um, no, we just know them from working on FlimFlam for the last five years.”

Jim: “Um, ok, have you mapped out what new features of the upgrade are turned on compared to off and who would be affected? Or say, documented the link between the features and business requirements?”

Core Team: “Well, not documented, but we know HR wants the real-time instead of batch interaction and Operations wants better reports. But HR outsourced last year to a cloud provider and we have no idea what Operations is doing …”

Jim: <thinking to himself> “… oh boy, good people, but this project is looking like a train wreck already …”

Clearly a “business as usual” approach to this upgrade isn’t going to work any more.

In the past, with so few stakeholders having comprehensive access to the silo-ed systems impacted by these types of changes, the need for easy to digest transparency into what changes were going to happen when and how was not critical. Sometimes the only visibility to what such a small project team was doing was in the production change management review and approval process:

Change Control Board: “Ok, next up is change record number 72,578 which reads ‘Enable the employee web portal to support the time off calendar’. Anyone here have any concerns with this change? Hearing none, approved. Next on the list …

Today’s Problem: IT systems are too interconnected for lack of project transparency to change

Sounds like 72,578 is a simple change that an HR delivery team of the past could have easily implemented without much cross team impact. But today, that example time off calendar may need to interact with the HR system to record those time off days against how many the employee actually has as part of their compensation package. There probably is a need to support some management approval work-flow. Plus, there are probably other work scheduling systems and PMO resource planning tools that need a feed of that data in order to accurately support their user base. There is probably some single sign on/web access management technology involved to support all employees accessing the web portal, some central provisioning system to handle access plus some remote access needs to support today’s mobile workforce. It is probably safe to assume that some of those integrated systems are in-house and some are cloud/SaaS or a mix of all of the above.

Additionally, with matrix-ed internal and external project resources with contracted and off-shore delivery coupled with the “cloud” vendor resource engagement model, a simple change could have a variety of stakeholders in need of agreement on what is changing when, etc.

Thus, hopefully I’ve convinced you that something as simple as a web portal for employee time off entry can involve a number of different internal and external teams and systems that all need to coordinate changes to support the business objectives of this example project.

So how does this all drive the need for “transparency”? Isn’t this just a basic PMO 101 issue of dependency management and cross project impacts?

Yes and No

The project team needs to produce deliverables that don’t just get the core team in agreement to pass the next quality gate in the project life-cycle (never to be revised again); the project team needs to produce deliverables that outline, at a high-level, the following basic project elements:

• Scope of the project in a sentence or two
• What is changing from present to future state
• Who is impacted by the change (and have they been engaged)
• Lastly, what isn’t in scope (that a non-core stakeholder might assume is in scope)

… for non-core stakeholders to easily digest and understand … and update the material frequently to have at the ready anytime it might be needed.

Besides an effective communications vehicle, another subtle yet important aspect to this deliverable is its ability to build confidence in the effective management of your project in outside stakeholders. This confidence can lead to senior management getting the impression the project is “under control” and move on to another project for increased scrutiny rather than assigning all kinds of ancillary people to dig into your project to figure out why they don’t have that “under control” feeling.

Stated another way, large corporate IT projects today need to adopt a bit of “program management”, specifically, some of the enterprise reporting themes. A Gantt chart (which I’ve extolled the benefits of before here) isn’t the end-all-be-all here. A slide deck that contains a few slides covering these topics with lots of pictures and drawings where ever possible would be more effective in serving this communication need.

So if you are a project sponsor or a project manager, consider having a communication deliverable that is actively maintained, even if your PMO PLC doesn’t explicitly call for one, to provide simple and easy to digest transparency into key aspects of your project at the ready at all

For any of those in the Cleveland/Akron, Ohio, USA area the week of 1/22, I’ll be speaking at the University of Akron on the topic of Identity Management in “the cloud” and general career opportunities in the Information Security industry.  More specifically, the title of the presentation is “Identity and Access Management Reference Architecture for Cloud Computing” and I’ve already published the slides on SlideShare here.

I’m looking forward to good interaction with the students and faculty.  If you are in attendance, please stop by and say hello!

Statistics for 2011

Hard for me to imagine but 2011 represented another year for MidwestITSurvival.com. I thought it would be interesting to share some statistics from 2011 in the same format as I did last year:

(Per Google Analytics)

3,708 Total unique visitors

6,783 Pages viewed

Viewer traffic sources:

I enjoyed that 23% of visitors that come directly here to read articles and the additional 31% that come here from other sites that link people here which was statistically similar to last year.

The top 3 articles in terms of number of unique visitors reading them are:

1. Organizational Structure and Enterprise Architecture [502]
2. Is the Gantt Chart Useless in Agile Projects? [389]
3. Conflict Between Agile and Architecture [287]

A total of 33 articles were published in 2011 with a total of 33 comments submitted to those articles.

All in all, having no goal of X number of comments per article or Y number of unique visitors per month, etc., I continue to be pleased with the blogging experience overall.

In 2011 I changed employers and roles in late February and switched from a general development/delivery role to an enterprise architecture role focusing on information security. Thus, the content of my articles switched about a third into the year from the trials and tribulations of IT delivery in a large IT shop to general security issues facing all IT organizations today. IT delivery challenges tend to be similarly themed across industries and rather easy to generalize to avoid sharing any direct information about one’s employer. On the topic of enterprise security, I found it a bit more difficult to generalize on matters that allowed depth of material presentment while not exposing any specific employer information. Hence, blog posting frequency started strong in the beginning of the year and tailed off towards the end of the year. I am hoping to find more general IT topics to share this year.

On the positive, on certain posts, especially the ones surrounding the 2011 FFIEC Guidance on Internet Banking for Financial Institutions, the comment dialog significantly enhanced the material I was presenting by adding additional industry depth and opinions. I strongly encourage you to check out those articles if you haven’t already and read through the comments.

Here is to 2012 and all the new challenges ahead!