Archive

AWS

If you live in or around the south sound region near Olympia, would like to hear about the AWS Toolkit and SDK for Visual Studio, come and check out the South Sound .NET Users Group on at Olympia Center, 222 Columbia NW, Olympia, Washington.  The meeting will be on January 12th at 7:00pm.

Slides & Links to Code are already available!

Overview:  During this presentation I will provide an overview of what is needed to get started using Visual Studio 2010 with the AWS Toolkit & SDK. We’ll also cover the basic design ideas behind the do’s and don’ts of cloud architecture and development. There will be some hands on coding (if you’d like to bring a laptop to follow along) and we will deploy code (pending a wireless/cat5 connection) into AWS Cloud Services & get EC2 instances up and running live!

I’ve been diligent and started a search of Platform as a Service Providers, so far my list includes:

  • EngineYard
  • Heroku
  • AWS Beanstalk
  • Windows Azure
  • AppFog
  • Tier3
  • CloudFoundry
  • OpenShift
  • IBM PaaS
  • Google App Engine
  • CloudBees

Who else is there? Help me out in creating a list of every possible offering we can find!  Cheers! Please leave a comment or three below with any I’ve missed.  Thanks!

I’m sitting on the bus this morning. As happens almost every day of the week. I’m flipping pages, sort of, it’s an eBook on my Kindle App. I’m reading about Steve Jobs taking over the Macintosh Program at Apple. How things started to fall into place for Apple, for the Macintosh, and how Jobs saw what could be a pushed for it. Everybody else; Microsoft, Xerox, Canon, and practically every single other company was missing it. Xerox Parc had it right in front of them, the GUI, Mouse, Object Oriented Language, and about every single thing we assume for computer use and development today but wasn’t doing anything with it. They were all missing it, except Jobs. The eccentric, crazed, reality distortion field generating Jobs pushed forward and found those that agreed, this was absolutely the future. Today’s computers owe so much to Jobs efforts to pull these people together, to what he saw as the future, and our modern computing world will forever be indebted to Steve Jobs.

Howard Hues had done this 50 years earlier. He simply stated, “nobody wants to fly on a plane at 10k feet and get shaken to pieces, planes need to fly at 30,000 feet or more where the air is smooth!” He then went about working to get a plane built that could do this! The Government was in his way, the industry was fighting him, everybody said this wasn’t the way to go. Nobody could build a plane that would do that right now! It’s absurd. He did it, and bought every single one of them he could putting the airline (TWA) in hock at the same time! But it paid off, and his airline had the nicest planes, best flight in the world, easily. Today’s airlines are all modeled after this ideal, our modern travel owes a huge debt to what Howard Hughes pushed forward.

The competition, the fighting pushed the envelope, but in both cases a visionary could see the future. To them it was plain as an image on a clear sunny day. To them, the future didn’t need to be tomorrow, it was ready right now. The future just needed dragged kicking and screaming directly into today! They did this, they pulled people together who could make these changes, and they with their teams yanked the future right into humanity’s grasp.

Utility Computing / Cloud Computing

With those thoughts flying around at Warp 10 in my mind, everywhere, at every moment it seemed to occur to me. We’re merely putting the motherboards and cassette tape drives together right now in cloud computing. We have no Macintosh of cloud computing, we have no clear direction, there has to be something bigger, much bigger. At this point we’re merely making small steps, slight little strides toward the future. What we need to do is create the future and pull it directly into now!

There could be more though. Some of these things are being put together by individuals at various companies, oriented toward the platform level. There is, somewhere, a growing movement toward that next big shift in the way things are done. The gap between big architectures, big ideas, and launching these things is decreasing by the day – literally!

With these big ideas and big architectures and all the small steps and small pieces the industry is moving in the right direction. We’ve experienced shifts over the years and some more are definitely coming up very soon!

The Playing Field : Sitrep

With these thoughts racing around I felt compelled to look at where the industry stands right now. These are in no particular order, they all provide some type of building blocks for the next big thing, all in some aspect of the industry.

Amazon Web Services : This one should not need explaining. They’re probably the most utilized, nearly the most advanced, robust, price conscious utility storage, compute, and services provider in existence today. They continue to defeat the innovator’s dilemma over and over again, this company, and the departments in the company are hungry, very hungry and they fight the fight to stay in the lead.

Cloudability : This company is about keeping utility/cloud computing costs in check, and knowing where and when you’re pushing the pricing limits among all the various building blocks. There has been more than a few issues with billing, and people blowing through budgets by inadvertently leaving on their 1000 node EC2 instances and Cloudability helps devops keep these types of things under control stopping overages cold!

New Relic : The key to this offering is monitoring of everything, everywhere, all the time. New Relic offers absolutely beautiful charting and information displays around services, compute, storage, and a zillion other metrics among Ruby, PHP, Python, .NET, and about everything else available.

Puppet Labs : Imagine operations, IT, and systems administration all rolled into a single bad ass company’s product efforts. Imagine ways to automate and monitor ritualized machines, get them deployed, all with elegant and extremely powerful tools. Imagine that power now, you’ll know what Puppet Labs provides.

Opscode : The cloud needs management, hard core powerful management. Opscode and their respective chef product does just that. The influence of chef has gone so far as to influence Amazon Web Services (and others) to design their systems automation in a way as to enable chef usage. The devops community around Opscode is growing, the inroads to systems agility they’re making is getting to a point as to even be considered a disruptive market force!

Joyent : The birthplace of node.js, do I need to add more? Well, ok, I will. Joyent has a host of amazing devs, and amazing ops goals. The advances coming out of  Joyent aren’t always associated back to the company (maybe they should be) but rest assured there is some heavy duty research and dev going on over there. Things to check out would be their SmartDataCenter and of course the JoyentCloud.

MongoHQ : Mongo HQ is one of the distributed cloud hosting provider for Mongo DB. Mongo HQ  is also a supported provider in several of the other PaaS Providers such as Heroku and AppHarbor.

MongoLabs : Mongo Labs, another distributed cloud hosting provider for Mongo DB. Mongo Labs is also a supported provider in several of the other PaaS Providers such as Heroku and AppHarbor.

Nodester : Nodester is a hosting solution for node.js applications beautifully distributed in a horizontal way.

Nodejitsu : One of the leading node.js hosting providers and a very active participant in the community in and around New York.

AppFog : AppFog is a Platform as a Service (PaaS Provider) that is working on providing a cloud based horizontally distributed platform for creating applications with a wide variety of frameworks and languages. Some of those include .NET, Ruby on Rails, Java, and many others.

PhpFog : This is the PHP root of the PaaS Provider AppFog. They have a good history and an absolutely spectacular architecture for PHP Applications with a screaming simple and fast deployment model to cloud/utility based systems. They have a really great product.

Heroku : Deploy Ruby, Node.js, Clojure, Java, Python, and Scala. Probably the leader in PaaS based deployment right now. Got git, get Heroku, get push heroku master is about all the gettin’ for your application to be running there.

EngineYard : Think Ruby, Ruby on Rails, Rubinius, or any other aspect of Ruby and you’ll probably arrive at EngineYard in short order. The teams at EngineYard are heavily active in the cloud & Ruby scene. They are easily one of the leaders in PaaS based git workflow deployment in the Ruby & Ruby on Rails Community. They also, however, support tons of other technologies so don’t think they’re limited to just Ruby & Rails.

AppHarbor : The .NET Framework, often thought to be left completely out in the cold when it comes to serious cloud computing git based agile work flows, finally got included with AppHarbor! With the release of AppHarbor the trifecta of IaaS and a solid PaaS offering were finally available for the .NET stack.

Windows Azure : Windows Azure is Microsoft’s official cloud service, which supports a host of capabilities centered around a mostly PaaS based service. Windows Azure has however spread into SaaS and IaaS also. Some of the frameworks and tools they support include Ruby on Rails, Java, PHP, .NET (of course), node.js, Hadoop and others.

CloudFoundry : Cloud Foundry is an open source PaaS Solution that serves to link up various back end and front end architectures. Currently it is supported by a host of companies including VMWare, AppFog, and others.

Putting the Pieces Together

That’s where we stand in the industry today. We have all the pieces and they need fit together to create something great, something awesome, something truly remarkable. I fully intend to create part of the future, will I see you there? I’d hope so!

The Environment

Recently a conversation came up about high availability in a traditional Enterprise Environment. Let me paint the picture for this environment;

“This environment has several hundred servers, and several hundred applications. These application range in simple client server applications to n-tier applications strung across multiple services and machines. Some are resilient, some are not so resilient. These applications have administration that ranges from needing rebooted on a daily basis to not being touched for months at a time. Needless to say the range of applications is vast.

In addition to all these applications the data center had a mix of hardware concerns that directly effected how applications were built.”

With that basic idea, one can imagine that planning for high availability is by no means a simple thing. However there are opportunities now available, that Enterprises have never had previously. In the past an Enterprise would usually have some big heavy hitter come in, such as EMC. The Enterprise would then pay them hundreds of thousands or even millions of dollars to do an analysis. Then the Enterprise would probably fork over another couple hundred grand here and there. This would happen time and time again, until some level of high availability would be achieved.

Well, to put it simply, a lot of that effort is unneeded today. The effort that is needed, with the right team, is in the hands of the Enterprise itself. Some people who know me, would immediately think I’m about to say “just setup an account at AWS and build to the cloud…” which is obviously the easiest, secure, and most progressive route to go. But no, I’m going to step in with some other solutions, that can be provided on-premise. I’ll elaborate at a later time the reasons behind this.

I’m going to now step through some key technologies available today. These can be used to provide high availability from the software architecture points of view. In your enterprise, if you have off-shored, outsourced, or otherwise attained your Enterprise Software, these functionalities and capabilities will be up to the creating provider. You’ll have to go to them for further information on how to change or adapt the architecture.

Software Architecture

For in house software here are some APIs, SDKs, and tools to help attain the much sought after high availability (Always aiming for that mythical five 9′s).

Dell’s Cloud Solutions

So without significant research time, the Dell Solutions can be thoroughly confusing at first glance. They don’t offer anything related to actual “cloud services” such as AWS, Windows Azure, or Rackspace. What they’re simply offering is hardware to build out resilient data centers and contributing actively to open source software solutions.

The Dell Cloud Edge Software is available on Github at dellcloudedge. The best places to start researching what is available are on two key blogs; JBGeorge Tech Blog and Rob Hirschfeld’s Blog.

Another key part of the Dell Solution is Crowbar. Dell open sourced Crowbar at the 2011 OSCON Conference. Even though most of the sample configurations revolve around Dell Poweredge Servers and Rackspace Cloud Builder Solutions, the software is available for use on system that are completely unrelated from Rackspace or Dell Solutions. Crowbar, simply put, is the software used to get servers up and running. As quoted on the Dell Announcement released during OSCON,

“Bringing up a cloud can be no mean feat, as a result a couple of our guys began working on a software framework that could be used to quickly (typically before coffee break!) bring up a multi-node OpenStack cloud on bare metal. That framework became Crowbar. What Crowbar does is manage the OpenStack deployment from the initial server boot to the configuration of the primary OpenStack components, allowing users to complete bare metal deployment of multi-node OpenStack clouds in a matter of hours (or even minutes) instead of days.”

That quote now brings up the next piece of software, OpenStack. When building out a data-center it is a solid idea to begin building a platform on which things will operate. OpenStack enables just that. There are two major elements of OpenStack that are key; OpenStack Compute and OpenStack Storage. This is where the architectural paradigm begins to change dramatically for traditional software. This is also where there will  a major sticking point for traditional Enterprise Software that relies primarily on a database on a server, with a web server on a server, and maybe some middleware or a service bus on another server. The massive problem is applications need to focus around horizontal scalability with compute and storage being the two key elements.

In many enterprises this is unfortunate, because a safe estimate would be 95% or more of enterprise applications don’t scale horizontally, or scale at all. If you’re an SOA shop, you’re much farther along than most. Most enterprises simply rely on the traditional vertical stack. This is a major problem. So how do we bridge this gap between the compute plus storage architectural design goal versus traditional architecture? That’s where the follow software comes to the rescue.

Windows Server AppFabric

Windows Azure AppFabric (Click to visit the MS Azure AppFabric Site)

Windows Azure AppFabric (Click to visit the MS Azure AppFabric Site)

(Not to be confused with the Windows Azure AppFabric, for differences review this article)

Windows Server AppFabric (Click for the MS Site)

Windows Server AppFabric (Click for the MS Site)

The Windows Server AppFabric has several capabilities that help an enterprise application leap forward into the modern era of horizontal scalability with a more clear way to focus on compute and storage. The feature set of the AppFabric includes these key functions that enable this leap forward (more information available in this article):

  • Workflow Instance Management
  • Scaling Out Distributed Applications

These by no means are the only features of AppFabric. For a thorough description of scenarios and applications around AppFabric check out Introducing Windows Server AppFabric.

In Summary

Where Does This Leave Enterprise Environments? The simple answer is, “A really long way away from achieving the scalability, cost savings, integrity, agaility, and capabilities of public cloud computing“. You can quote me on that. The effort to acheive data integrity and up time to perform standard business, it’s already here for Enterprises, but to go beyond that and extend hours of operation, acheive 5 9′s of up time, and decrease costs in a dramatic way is generally cost prohibitive in private cloud infrastructure and especially in traditional data center operations. The fact is, things will still go down. Applications have a long way from being resilient, idempotent, or designed with an architecture that allows them with the concept of the public cloud “design for failure” concept.

So what to do about this? The best thing for an Enterprise Application Environment is simply to start building applications with horizontal scalability in mind. Build with the concept of systems being nodes, with idempotent messaging, clear and redundant messaging queues, and thinking – even while limited by traditional data centers and limited visualization technologies – thinking in a resilient architectural style instead of the traditional vertical mindset.

These tools I’ve outlined can help your Enterprise move forward in a traditional data center environment, a private cloud infrastructure environment, and be prepared for public cloud scale and capabilities.

I’ll be presenting on the AWS Toolkit for Visual Studio 2010 in the very near future (Check out the SAWSUG Meetup on October 12th, that’s this Wednesday). I’ll be covering a number of things about the new AWS Toolkit for Visual Studio. My slides are available below (with links to the Google Docs and Slideshare Versions).

Direct link to Google Docs Presentation or the SlideShare Presentation.

The code for the presentation is available on Github under AWS-Toolkit-Samples. Beware, this code will be changing over time, the core will stay the same though.

Follow

Get every new post delivered to your Inbox.

Join 3,273 other followers