Midwest IT Survival

AWS Makes Coding in the Cloud Easy

With all of the IT punditry talking about how everyone who is anyone is “moving to the cloud”, I thought I would take a serious look at what Amazon’s Amazon Web Services (AWS) has to offer for hosting applications in the cloud. Since I’ve already written about my perspective that “the cloud” is evolutionary rather than revolutionary, I thought I would roll up my sleeves and challenge myself to interact directly with some “cloud” services. What also helped propel me forward was discovering that AWS has a free “get starting” package that includes the ability to provision a server with Internet access, storage and all the AWS development packages and libraries pre-setup.

[Feel free to skip down to the source code if you aren't interested in the next section on business context]

Business Context

Now if you have read any of my articles on this blog you know I mostly cover the challenges of working in a large, corporate IT environment both from a staff and management perspective. So, this is a bit off the beaten path for me. But the rate of business groups pushing corporate IT to implement cloud solutions, especially in the on-line product space, is on a significant up tick. Now, especially in financial services, integrating on-line products with “cloud/SaaS/ASP” hosted applications as product extensions is nothing new. It seems almost as soon as financial firms had an on-line application, they were looking to integrate with existing partners that also were standing up on-line versions of their service offerings: think on-line banking and viewing statements electronically, etc.

The trend difference I’ve observed from the late 90s and early 00s of “ASP” integration to the present is the non-traditional “cloud” companies looking to work with banks. Prior, companies that were already working with banks to provide outsourced off-line services progressed to offer on-lines services. Thus, the maturity of the pre- and post-sales process was familiar to both parties. The ASP providers knew how to address data protection, regulatory compliance and complex/unique technology integrations. The new “cloud” application service providers are using all of the cloud infrastructure as a service (here is the tie-in with AWS) offerings to produce new robust products, but they are completely unfamiliar with how to architect a complete product and service solution for financial services. Thus, many are having to address retrofitting their solutions to be akin to the needs of regulated, conservative banking institutions including all of the security assurance overhead needed (think SAS 70s, penetration tests, security standards and procedures, site visits, lengthy contracts, etc.).

What does all this mean?

In summary, current cloud service providers such as AWS, offer a great suite of building blocks to stand up a robust application. But choose your technologies strategically, especially if you are planning to integrate your product in any way with financial services customers. Be prepared to have to transition to company owned and managed application infrastructure including data storage for the foreseeable future until cloud providers, such as AWS, are universally accepted by the financial services security community as “secure”.

Technical Stuff

Ok, now for a bit more fun technical stuff, I went ahead and signed up for the free AWS package which was incredibly easy. Just a few mouse clicks and I am sitting in the AWS web based management console. Without any serious investigation, I was off creating my own “bucket” of storage in their Simple Storage Service (S3). Next step was to provision a server to host my application experiment. The Elastic Compute Cloud (EC2) tab was equally easy to click through a wizard of picking basic server configuration options. I opted for the Amazon Linux Micro Instance (specifically the Amazon AIM platform as I assumed it would be optimized for using AWS services) in order to stay within the “free” parameters. At the conclusion I was provided all the pertinent remote connection details including a client/user certificate and literally the ssh command syntax to cut/paste and connect.

Since I am clearly taking AWS for a spin years after it first came on the market, I am assuming I am benefiting from significant end user functional improvements made within that time duration. It has been over a decade since any server I built or any code I wrote actually was deployed in a corporate production environment, so to say I have been relegated to a tinkerer in my technical career would be an understatement. But the simple wizard based configuration of the server and storage provisioning clearly allows even a novice technician to be exceedingly productive within AWS.

The Goal – Functional Application Running in AWS

Now that I have cloud storage and a cloud server I needed an application development challenge to solve. So after some thought, here is what I came up with:

Java based application service that will replicate my Dropbox files into my new AWS S3 storage “bucket”.

Note: Yes, Dropbox uses AWS as it’s back-end storage platform so I’m really duplicating my data within the same storage cloud so what am I gaining? Ok, real world, not much gained but this is a throw away experiment to begin with so just permit me this architectural short-sighting.

This experiment involves:

• Installing the Dropbox GUI-less client on the Linux Micro Instance
• Connecting all the Java AWS libraries together to access my S3 storage “bucket”
• Scheduling the application to periodically replicate the Dropbox files to my S3 “bucket”

By using AWS’s example “S3Sample.java” code from their Java SDK, in a matter of a few hours (those hours mostly spent getting all the correct jars linked together in the classpath), I was able to start copying files. Of course, after I reverse engineered how their sample program worked I ran across this article on AWS’s blog that hand holds you through everything.

I was able to follow the directions provided on Dropbox’s site I was able to download and install the Dropbox client on my Linux Micro Instance without a single hick-up.

As I mentioned above, it has been quite a long time since I cracked open an editor and started coding, so any comments on the lack-o-elegance of my Java is most likely very accurate. Plus, I didn’t go so far as add any mechanism to traverse directory trees to copy nested files. Additionally, all I achieved was a one way copy of all files rather than a true sync or any date/time check to see if a file even needs to be re-copied if it already exists.

Goal Achieved!

Here is a link to my (lame, err, not production ready) Java source here.

I welcome any comments around reader’s thoughts on cloud application development and AWS specifically.