Archive

Tag Archives: open source

What is Riak? Who builds it? Who maintains it? Can I download it? How does it work? What are the features?

Here’s the start of answers to these questions and more.

First, the basic high level description:

Riak is an open source, highly scalable, fault-tolerant distributed database.”

That’s the first line you’ll read when checking out the product via the Basho product link. It provides good information, but here I’m going to add more to the definition without the need to dig around yourself. Maybe I can save you some time & provide some links directly to solid information in the docs. Kind of a “Cliff Notes” of Riak. Let’s take this feature by feature which will in turn get us to a definitive definition of what exactly Riak is.

Riak is Open Source.

Riak is built and contributed to by the community, with Basho being the steward and an active member that extends, builds and provides support for additional products. The avenues to reach the Riak Open Source Community members is pretty straight forward, following known avenues of communication. Hit us up on the email list, especially feel free to contribute & ask questions via the Github Basho organization, there is the Basho Riak Blog, the weekly recap and jump into the IRC chat room #riak on freenode. Oh, and there’s a twitter feed @basho.

So what exactly does this get you, when you become a user or contributor of Riak? The entire community is behind you, will help you get started using Riak and provide help whenever you run into problems. If you want SLAs or 24 hour support Basho can provide this for you. But for bugs, issues, queries, searching and all sorts of other related development questions there is the community. An open source community like this is passionate, which means you’ll have support like no closed source company will ever provide you, and absolutely no closed source product’s community will provide you. We’re talking about a different level of interest, passion and levels of personal involvement.

Riak is a key value based database store.

Riak is a key value store. What exactly is a key value store? It’s pretty simple and you’re probably already familiar with what a key value store is. A key value is made up of two pieces of data, the first is the identifier for the second element within the data structure. This gives a system or developer using key value storage a schema-less way of working with data.

Riak is designed for highly distributed environments.

This type of distributed isn’t the “we put one database over here and one database over here and you gotta figure out how they work together” type of distribution. So this isn’t some of that oddball pretend stuff Oracle keeps hoisting on people. This is the honest to goodness distribution of the sort, when one node goes down you don’t blink, you don’t stop eating dinner, you don’t sweat it. You just continue onward with life knowing full well that you’ll just spool up another node when you need to.

Riak is master-less, with no single point of failure.

This is one of self explanatory features. But what does a master-less system provide us? One thing is no single point of failure. Being that all nodes can act autonomously to work around the loss of one or more nodes it also helps add to the high availability of the system.

Riak is fault tolerant, like a disk drive you wish was real.

Ever have a backup disk drive? What? You don’t have one of those? Ugh. Ok, so imagine you had a backup disk drive that had an unfortunately high failure rate. Well, why, because you know, they have an oddly high failure rate. If you do backups like good practices dictate, eventually you’ll end up with some dead drives.

RAID, both software and hardware, are built specifically to deal with this type of failure. With a distributed system like Riak, it bumps the level of abstraction above software or hardware RAID, enabling another level of even greater fault tolerance. Not to remove the relevance of RAID capabilities, but with a multi-node system like Riak, you can easily remove nodes and swap them out as needed, keeping costs down by using simple drives in simple machines. If you want to, you could indeed get higher I/O machines and faster drives, but it isn’t necessary to insure fault tolerance in a Riak Database System.

Riak scales, with hot swappable nodes enabling zero downtime.

The ability to commit hot swappable changes while in the midst of operating starts at a very low level for Riak. The language used to build Riak, Erlang has the ability to change pieces of an application system in realtime built into the precepts of the language. This provides, at the core, the inherent capability to change out systems, and by proxy of architectural design, the ability for nodes in Riak to be changed out simply by removing them from a cluster ring. Once that is done it is just as simple to add another node or nodes back into the cluster ring, enabling a number of additional practices around upgrades, hot swaps for failures, or even version changes.

Riak can be used as a building block for distributed (aka cloud) infrastructure.

The concepts and contractual components that Riak Database is built on are available for use via the Riak Core Project. If you’re looking into starting a project around distributed systems this is a great place to get start. Also be sure to do a general web engine (re: google) search for “riak core” and you’ll find lots of material around the project, and projects people have started with the project as a base. I’m currently in the process of putting together one of these projects myself.

Riak is eventually consistent.

The term eventually consistent is becoming more and more common place. Riak is one of the many systems, that inherently often apply to distributed systems, that use the concepts of eventual consistency. The idea, is that even though all nodes may not immediately receive a new piece of data, or updated piece of data, they eventually will receive that update and by synchronized with the cluster ring of nodes. This goes back to the equality of nodes and removal of the master-less concepts, providing the availability and other capabilities, with some trade off in the synchronization of data through eventual consistency.

In Summary

That’s round one for the many features of Riak. I’ll be adding more in the future, but for now this is a good starting point in knowing about and knowing what Riak is, what it can be used for, and how it might help you extend, maintain or invent the next great piece of technology.

Today Microsoft released, with partnerships with a number of companies including Basho, Hupstream and Bitnami, the VM Depot. I’ve always followed Bitnami, so it’s really cool to see their VM releases for Jenkins (CI Build Server), WordPress, Ruby 1.9.3 stackNode.js and about everything you can imagine out their along side our Basho Riak CentOS image. If you want a great way to get kick started with Riak and you’re setup with Windows Azure, now there is an even easier way to get rolling.

Over on the Basho blog we’ve announced the MS Open Tech and Basho Collabortation. I won’t repeat what was stated there, but want to point out two important things:

  1. Once you get a Riak image going, remember there’s the whole community and the Basho team itself that is there to help you get things rolling via the mail list. If you’re looking for answers, you’ll be able to get them there. Even if you get everything running smoothly, join in anyway and at least just lurk. :)
  2. The RTFM value factor is absolutely huge for Riak. Basho has a superb documentation site here. So definitely, when jumping into or researching Riak as software you may want to build on, use for your distributed systems or the Riak Key Value Databases, check out the documentation. Super easy to find things, super easy to read, and really easy to get going with.

So give Riak a try on Windows Azure via the VM Depot. It gets easier by the day, and gives you even more data storage options, distribution capabilities and high availability that is hard to imagine.

New Relic & The Rise of the New Kingmakers

In other news, my good friends at New Relic have released a new book in partnership with Redmonk Analyst Stephen O’Grady @, have released a book he’s written titled The New Kingmakers, How Developers Conquered the World. You may know New Relic as the huge developer advocates that they are with the great analytics tools they provide. Either way, give a look see and read the book. It’s not a giant thousand page tomb, so it just takes a nice lunch break and you’ll get the pleasure of flipping the pages of the book Stephen has put together. You might have read the blog entry that started the whole “Kingmakers” statement, if you haven’t, give that a read first.

I personally love the statement, and have used it a few times myself. In relation to the saying and the book, I’ll have a short review and more to say in the very near future. Until then…

Cheers, enjoy the read, the virtual images and happy hacking.

The Iron Foundry Team are big advocates of open source software. We write code across all sorts of languages, just like many of the development shops out there do. Sometimes we’re heavy on the .NET, other times we’re all up in some Java, Ruby on Rails, spooling up a Node.js Application or something else. So keeping with our love of open source and our polyglot nature we’ve created the Thor Project with three distinct apps.

Before jumping into the applications though, a little context for what and where Thor is in the grand scheme of things. We need to roll back to the Cloud Foundry Project to get into that. The Cloud Foundry Project is an open source project built around software for PaaS (Platform as a Service) which can be used to build your own PaaS internally or externally, in a cloud provider or directly on hardware. It’s your choice how, when and where you want to use it. For more context on PaaS check out my previous entry “The Confusions of IaaS, PaaS and SaaS“.

Thor Project

Cocoa for OS-X

Thor Odinson

Thor Odinson, God of Thunder

You know who Thor is right? He’s this mythic Norse God, also known as the God of Thunder. Since we’re all about bringing the hamma we welcomed Thor into our team’s stable of applications. So starting immediately we’ve released Thor into the realms for contributions and fighting the good open source software battle! If you’d like to join the effort, check out the github project and feel free to join us!

Technically, what is the Thor Application? This is a Cocoa Application built for OS-X that is used for managing, deploying and publishing applications to Cloud Foundry enabled and or Iron Foundry extended PaaS Environments.

.NET for Windows 7

The .NET Metro version of the Thor Application is also released via github with a provided installer. We’ve almost taken the same path, except of course for the very different UX and UI queues with Windows 7 and the Metro UX design guidelines.

WinRT for Windows 8

I wasn’t really sure what to call this version. Is it Metro or WinRT or Windows 8 or something else? Anyway, there is a project, it is albeit empty at this point, but it is the project where the Windows 8 version of Thor will go! For now get the Windows 7 version and install it on Windows 8, it won’t have touch interface support and things, but should work just like a regular application on Windows 8.

The Code

To get started with these, generally you’d just clone the repo and do a build, then get started checking out the code. There is one catch, for the OS-X version you’ll want to pull down the sub-modules with the following command.

git clone git@github.com:YourForkHere/Thor.git
git submodule update --init --recursive

Once you do that in XCode just make sure to then select the right project as the starting build project.

…then when the application is launched…

Thor Running in OS-X

Thor Running in OS-X

I’ll have more in the coming days and weeks about Thor & Iron Foundry. For now, check out the blog entry on the Iron Foundry Blog and subscribe there for more information.

Tuesday night, as usual ended with great technical conversation at Bailey’s Taproom. Bailey’s is basically the epicenter of the Portland tech scene. Almost every programmer, devops, or technical person either goes about once a month or has this establishment as a regular watering hole! It’s great, the atmosphere is chill, the beer is SUPERB, the beer menu kicks ass (see: Beer Dashboard Kick’s Ass) and the list of fun cool things just continues on and on.

This week of course OSCON adds a little spice to the regular roll call at Bailey’s. There were a number of conversations that broke out, which I’ve broken out the key topics below:

vert.x => To summarize as is written on the site itself, “Write your application components in JavaScript, Ruby, Groovy or Java. Or mix and match several programming languages in a single application. Create real, scalable applications in just a few lines of code. No sprawling xml config. Scale using messaging passing and immutable shared data to efficiently utilise your server cores. Super-simple concurrency model frees you from the hassles of traditional multi-threaded programming.

Here’s an example from the site in a few of the languages:

Java

import org.vertx.java.core.Handler;
import org.vertx.java.core.http.HttpServerRequest;
import org.vertx.java.deploy.Verticle;

public class Server extends Verticle {
    public void start() {
        vertx.createHttpServer().requestHandler(new Handler() {
            public void handle(HttpServerRequest req) {
                String file = req.path.equals("/") ? "index.html" : req.path;
                req.response.sendFile("webroot/" + file);
            }
        }).listen(8080);
    }
}

JavaScript

load('vertx.js')

vertx.createHttpServer().requestHandler(function(req) {
    var file = req.path === '/' ? 'index.html' : req.path;
    req.response.sendFile('webroot/' + file);
}).listen(8080)

Ruby

require "vertx"

Vertx::HttpServer.new.request_handler do |req|
    file = req.uri == "/" ? "index.html" : req.uri
    req.response.send_file "webroot/#{file}"
end.listen(8080)

Wednesday Roughness

I felt beat up a bit start Wednesday, but rolled into it after a short while. Needless to say, the intensity of conversations (and maybe a few of those rounds of beer) and number of ideas, new things to check out and fitting it all in can wear one out.

The morning sessions were solid, I attended most of “Comparing Open Source Private Cloud Platforms“. Lance did a solid job of laying out the tooling, virtualization software and where these things come together to form a number of OSS options for cloud computing. Check out more from Lance on his @ramereth, his blog Lance Albertson, or check out his band he’s in “The Infallible Collective“.

Wednesday, Thursday and Friday Meets

I met a ton of people. All of whom I must say, I hope to get to talk to again, work with on projects, or just sling some code sometime. Absolutely great people, friendly, intelligent and highly motivated. Some of these people I met included:

Andy Piper (@andypiper) – Part of Great Britain’s contingent of VMware Cloud Foundry advocates and such. We got to hang out and talk about a zillion different topics at a number of events. Andy was kind enough to show me a few tips and tricks he’s been using with Cloud Foundry, the VMC, and in general working with the platform.

Josh Long (@starbuxman) – I met Josh once before on the Cloud Foundry open tour, where he brought COBOL programming… oh no wait, he brought some great Sprint Java samples and such to demo on the Cloud Foundry Platform. I fulfilled Josh’s dreams by telling him that COBOL, could indeed run on Cloud Foundry thanks to the .NET capabilities of Iron Foundry! (ya know, if anybody is into that type of thing)

Erica Brescia (@ericabrescia) – I finally got to meet Erica in person, after chit chatting on Twitter about all the great applications her company Bitnami helps to deploy in the cloud. There are some really great deployment hosting solutions from them, check them out if you’re looking for some streamlined deployment practices. She also mentioned I need to meet…

Jono Bacon (@jonobacon) – I managed to meet Jono by randomness. He’s, well, let’s say he does some absolutely great work in the tech industry for Canonical and in the open source universe. In addition Jono has some superb tastes in music.  \m/  \m/  Check out some of his work:  Blog, personal site, and you can probably google him too. Do it, he’s got a lot of great material out there.

As I was saying, these aren’t the only people that I met. To all those people I didn’t mention, it was awesome hanging out, catching up and hearing about what everyone is working on and creating.

PaaS, IaaS and The Driving Open Source Coders

On the topic of PaaS, it continues to expand into new realms of publicly (or privately) run services. PaaS is quickly expanding past mere framework services around .NET, PHP, Rails, Sinatra and such and moving into the realm of databases, services buses, and other capabilities as a service. As laid out with the SOA mindset. Even though enterprises failed to bring SOAP to an effective worldwide use, RESTful services are expanding rapidly. *aaS is pushing those even further, to do what the enterprise had wanted but failed to do. Creating a universal acceptance of scalable, powerful, expandable and extensible services through APIs.

As more services are extended we’ll start seeing a lot of offerings around truly scalable databases with various feature sets around those databases offered as a key service. Examples would include “atomic database as a service”, “transactional data store as a service”, or “document store as a service”. In the end it will include the amount of usefulness for the services while eliminating a need to know each in intimate detail. Knowing the core capabilities of an option and just using the service will grossly outpace the attempt to implement these services internally.

So keep watching PaaS to grow in many various ways. Consuming the service being the driver over attempting to build the service. Of course, if the service doesn’t exist, get on that it’s business opportunity!

Random OSCON Diversions

I had a great time visiting with family while at OSCON also. To whom they all send a hello and horns up, thrash on salute to the coders of the world!

Voodoo Donut Break with Florida Family Contingent

Voodoo Donut Break with Florida Family Contingent.

My brother Adam, the IT Department

My brother Adam, the IT Department

My Brother Runs an IT Shop of One…

…thanks to cloud computing capabilities.

This kind of blew my mind. I sort of of knew what he did, but it didn’t hit me how close our professional lives are until this trip. He’s just recently moved several hundred miles away from the main office, but still manages the entire company.

One of the unique happenstances is, my brother (the guy next to the bald guy that is me, he’s wearing a Tesla t-shirt) is the top IT guy for a little billion dollar a year company. Which, in this case, he’s proven the power of cloud computing. Why do I say this? Because traditionally this organization would have needed an army of PC techs, network knob fiddlers, and such. But with the advantages of cloud computing, both on premise and off premise, and have a DevOps Guy that knows what he’s doing they are able to efficiently run their entire company with one single guy.

Needless to say, with the synergy of OSCON we had more than a few conversations around tech. Some of those included the replacement of PCs with mobile devices, such as iPads or smart phones. Another was the mix of on-premise data that couldn’t easily be transferred or utilized form cloud services. These are just a few fo the things that have helped him to run the show, the entire show.

Summary

OSCON was awesome. Next time I will be taking off a day or two before and a day or two afterwards so that I can do an even more elaborate write up of the event. My aim is to have interviews, video and otherwise, and really step it up in relation to providing an eye into the event from a developer’s point of view.

OSCON 2012 Opening Doors

OSCON 2012 Opening Doors

Today kicked off with a monster Reggie Biscuit from Pine State Biscuits. If you live in Portland or are visiting just for the conference and like soul food of the tastiest nature, check it out.

My first day ended up not as planned. Instead of attending sessions I ended up meeting a number of people and discussing the future of Cloud Foundry, where it is headed and in general, the direction of PaaS Technologies. I met Andy Piper (@andypiper) and Raja Rao (@rajaraodv) and discussed Node.js and Cloud Foundry specifically. We then dove into trying out some of the CLI features in the latest VMC builds.

After that I met Mark Atwood for a brief few moments. As always, Mark’s a friendly guy, and might I add pretty smart too. I’ve enjoyed our conversations in the past during the AWS Meetups in Seattle too. He’s always got interesting thoughts and perspectives on open source, linux and now on PaaS Technology too. Ya see, Mark has become the Red Hat OpenShift Advocate. It’s a perfect fit, as Mark loves this stuff!

Ignite!  ….or Bailey’s for more tech talk and #nodejs discussions.

After all of this I almost, and had planned, to attend the Ignite Presentations after OSCON, but instead ended up heading over to talk with some Node.js & JavaScript Coders about some of our latest efforts around getting concrete performance benchmarks for Node.js and some of the various libraries in use.

That brings us to Tuesday…

Tuesday brought forth a super busy, exciting and educational day. I headed straight to OSCON for the OpenShift Workshop with Mark Atwood & Krishna Raman (Mark’s Twitter is @fallenpegasus). The session was great and they hit on a lot of hugely important topics. Let’s go through each of these real quick, as this is where more than just the tech bits were involved.

OpenShift is Truly Open Source Software

Mark & Krishna made a strong point to outline and show how and why OpenShift is open source. For instance, they are following the original precepts of a particular guy named Stallman (http://stallman.org/ if you’re unfamiliar with Richard, he’s the guy who got GNU happening and a major originating advocate of open source software). Mark pointed out that Red Hat is open to keeping the governance of the project completely open, would even cede it to another governance entity when it grows beyond just Red Hat, and they intend to keep all the communication very open and public, as intended with open source projects.

Another thing that Mark and Krishna pointed out, was that the software is on github, and not just in a psuedo “read-only” state, but in an actively useful way, with interactions and tracking on github. The point being that there is no hidden processing of the code or private repositories of code. What you see is what you get in this regard. In addition all of the code that is available, is the exact code that Red Hat is using to actually host the OpenShift PaaS that they provide for testing and demoes. Simply, it is all there available in a completely open, contribution based, interactive, and publicly accessible way.

So far this is even more evident if you do a google search or even trace the twitter activity. They definitely have the search engines working in their favor with all of that searchable content publicly available.

Cloud Foundry & OpenShift

I’m still a huge Cloud Foundry fan, the team and effort and product is getting to be in pretty solid shape. However OpenShift is definitely here to provide some competitive interest. In the end, I’m a fan of PaaS Technology and what it can do for software developers and what we’re trying to achieve on a daily basis. The potential of PaaS to improve, dramatically, the software development lifecycle while reducing the overhead cost is pretty huge. The key is, people have to be aware of and start utilizing the technology well. Just implementing it and saying “I have PaaS” is one thing, but improving your software development process to use PaaS technologies well is where the seriously powerful advantage is.

I’m looking forward to seeing the market unfold and start making progress with these technologies. On that note, day #1 and #2 are finished for me. Cheers!

midnight mystery ride

at midnight, we ride.

Riding on Roadways

Writing on Riding on Roadways

Not Rich Yet

It's going to happen. Gotta find something to do until then.

Portland Transit Lane

The Untold stories of Portland Transit

craftedincarhartt

Carhartt Women's Blog

heydev

For the love of code

Nathan Evans' Nemesis of the Moment

My nemesis of the moment

Open Source Bridge: Presentation Proposals

Snippets, software architecture, lean, agile, management, and leadership bits.

Captured Refractions

A collection of my latest adventures, past reflections and other photos.

for the love of Nike

for the love of Nike

The Cloud Dev

Developing {for/ on/ the} Cloud...

Project Manager in a Cloudy IT World

Thoughts, comments and ideas from experiences as a Project Manager in IT

iBikeuBike

If I can bike... So can you!

MAX FAQs

Portland Light Rail

UX Success

User Experience Design, Agile Development, Lean UX, Start Up

The lost outpost

a weblog by Andy Piper about technology, photography, and life

SaintGimp

Agile development, software craftsmanship, continuous improvement - Eric Lee's blog

Clang and Clamour

pardon the construction noises while we build the internet

Kristen Mozian

social {good} design + experience

RightScale Blog

Cloud Management News & Conversations

Follow

Get every new post delivered to your Inbox.

Join 5,492 other followers

%d bloggers like this: