JavaScript

Getting in Some Code Stylings, Looking Good for the Code Dance

In every language there are opinions about how to format code. With JavaScript, the community abounds with opinions about how the code should look, how variables are declared, whether there should be semi-colons to end each statement, spaces before or after parenthesis, and more than I care to list in a simply worded paragraph like this. Recently the team at Deconstructed sat down to determine what our ongoing code style format would be and how we can enforce it.

The first thing we did was figure out what we could use for enforcement of the coding style. Milan (@milanloveless) quickly discovered node-jscs per suggestion from Adam Ulvi (@s5fs). He implemented that code as follows.

Continue reading

JavaScript

More on Using the Nools DSL and Engine…

Nools Objects

In the helloworld.nools object there is a single object defined called Message. This object has two elements defined; the text property and the constructor. The specific object definition is shown below.

define Message {
    text : '',
    constructor : function(message){
        this.text = message;
    }
}

This object can now be referred to by name throughout the nools file. The other way to reference this object is to call the getDefined function from the flow object that is being used in code processing the business rules. In the nools language any javascript has can be put inside the define blog. By defining the constructor as shown, it overrides the default constructor behavior. Continue reading

JavaScript

Learning “nools” Rules Engine

Recently I sat down to work up a solution around a rules engine. There were a few things I noticed right off.

  1. When there is a request to implement or build a rules engine it is very often (I’m guessing a solid 40-60% of the time) reasoned that there is a need solely based on a lack of understanding around what the problem space is that actually needs a solution. The simple assumption, is 40-60% of the time somebody says “let’s implement a rules engine to solve these unknown problems” really translates to “we really don’t know much about this domain so let’s implement something arbitrary as a stop gap”.
  2. Implementing a business rules engine can quickly become a “support the user” scenario for the developers that implement the rules engine. This is a situation in which the developers actually have to help the people writing the rules to be processed. This is not an ideal situation at all, generally developers supporting users writing rules is a quick way to ensure burn out, misappropriation of skills and turnover.
  3. Many developers will, without hesitation, spout out “are you sure you want to implement a rules engine?” and then follow that up with “let’s discuss your actual problem” with that leading to “are you sure you want to implement a rules engine?”. Other developers upon hearing that one will implement a rules engine immediately respond with, “shit, I’m out.”

At this point I realized I had X, Y and Z reason to use it and would just have to persevere with all of the threats that are inclusive of implementing a rules engine. Sometimes one just has to step into the realm of scary and get it done.

So here’s what I dug up. I’m really not sure about the name of this project, as it appears to be some sort of odd usage, so whatever, but it is indeed called nools (github repo). Nools is a business rules engine based on the Rete Algorithm, something that is helpful to read up on when implementing. The main deployment for nools is to a Node.js server, but I’ve read that it is prospectively deployable in most browsers too.
Continue reading

Strongloop

Framework: Strongloop’s Loopback

Recently I did a series for New Relic on three frameworks, both for APIs and web apps. I titled it “Evaluating Node.js Frameworks: hapi.js, Restify, and Geddy” and it is available via the New Relic Blog. To check out those frameworks give that blog entry a read, then below I’ve added one more framework to the list, Strongloop’s Loopback.

Strength: Very feature-rich generation of models, data structures and related enterprise-type needs. Solid enterprise-style API framework library.
Weakness: Complexity could be cumbersome unless it is needed. Not an immediate first choice for a startup going after lean and clean.
Great for: Enterprise API Services.

When I dove into StrongLoop, I immediately got the feel that I was using a fairly polished package of software. When installing with ‘sudo npm install -g strongloop’ I could easily see the other packages that are installed. But instead of the normal Node.js display of additional dependencies that are installed, the StrongLoop install displayed a number of additional options with a shiny ASCII logo.
Continue reading

Coding on Orchestrate.io & Orchestrate.js & Orchestrate.NET

First context, then I’ll dive in.

Orchestrate

http://orchestrate.io/

Orchestrate is a service that provides a simple API to access a multitude of database types all in one location. Key value, graph or events, some of the database types I’ve been using, are but a few they’ve already made available. There are many more on the way. Having these databases available via an API instead of needing to go through the arduous process of setting up and maintaining each database for each type of data structure is a massive time saver! On top of having a clean API and solid database platform and infrastructure Orchestrate has a number of client drivers that provide easy to use wrappers. These client drivers are available for a number of languages. Below I’ve written about two of these that I’ve been involved with in some way over the last couple of months.

Orchestrate.NET

https://github.com/RobertSmith/Orchestrate.NET

This library I’m currently using for a demonstration application built against the Deconstructed.io services (follow us on twitter ya! @BeDeconstructed), a startup I’m co-founding. I’m not sure exactly what the app will be, but being .NET it’ll be something enterprisey. Because: .NET is Enterprise! For more on this project check out the Deconstructed.io Blog.

Some of the latest updates with this library.

But there’s still a bit of work to do for the library, so consider this a call out for anybody that has a cycle they’d like to throw in on the project, let us know. We’d happily take a few more pull requests!  The main two things we’d like to have done real soon are…

Orchestrate.js

https://github.com/orchestrate-io/orchestrate.js

With the latest fixes, additions and updates the orchestrate.js client driver is getting more feature rich by the day. In addition @housejester has created an orchestrate-brain project for Hubot that uses Orchestrate.js. If you’re not familiar with Hubot, but sure to check out the company robot that can dramatically improve and reduce employee efficiency! Keep an eye on that project for more great things, or create a Hubot to keep a robotic eye on the project.

Here are a few key things to note that have been added to help in day-to-day coding on the project.

  • The travis.yml file has been added for the Travis Continuous Integration build. This build runs against node.js v0.10 and v0.8.
  • Testing is done with mocha, expect.js and nock. To get the tests up and running, clone the repo and then build with the make file. The tests will run in tdd format.
  • Promises are provided via the kew library.

If you’re opening up the project in WebStorm, it’s great to setup the mocha tests with the integrated mocha testing as shown below. After you’ve cloned the project and run ‘npm install’ then follow these steps to add the Mocha testing to the project. We’ve already setup exclusions in the .gitignore for the .idea directory and files that WebStorm uses.

First add a configuration by clicking on Edit Configurations.

Edit Configurations

Edit Configurations

Next click on the + to add a new configuration to run. Select the Mocha option from the list of configurations.

Mocha & Other Configurations in WebStorm

Mocha & Other Configurations in WebStorm

On the next screen set a name for the configuration. Set the test directory to the path for the test directory in the project. Then finally set the User interface option for Mocha to TDD instead of the default BDD.

Edit Configuration Dialog

Edit Configuration Dialog

Last but not least run the tests and you’ll see the list of green lights light up the display with positive results.

Test Build

Test Build

Fixing Up Passport.js ‘passport-http’ for Express v4

Even though it isn’t in the primary trunk of code for the ‘passport-http’ Passport.js Strategy, I’ve upgraded the packages.json and app.js file for the basic username and passport authentication to Express.js v4. If you’re using Express.js and are looking to migrate to v4 from v3 or earlier a great starting place is to check out the “Migrating from 3.x to 4.x” wiki page. As for the passport-http strategy, here’s the updated example I put together in a commit here with my own fork here, with the code changes below.

First step was to bump to the latest Express.js v4 Module. I did this with a simple edit to the packages.json file. The original looked like this

{
  "name": "passport-http-examples-basic",
  "version": "0.0.0",
  "dependencies": {
    "express": ">= 0.0.0",
    "passport": ">= 0.0.0",
    "passport-http": ">= 0.0.0"
  }
}

which I changed the depedency from >= 0.0.0 to >= 4.0.0 so that it would require something above v4.

{
  "name": "passport-http-examples-basic",
  "version": "0.0.0",
  "dependencies": {
    "express": ">= 4.0.0",
    "passport": ">= 0.0.0",
    "passport-http": ">= 0.0.0"
  }
}

Technically the old file would have pulled the latest (which as of today I believe is 4.1.1) but it would also not do anything if you’d already pulled the example down. It just make sit more specific that the version is v4+ now.

After changing that dependency I added Morgan. Morgan is a replacement middleware for the logger. The final packages.json file looked like this when I was done.

{
  "name": "passport-http-examples-basic",
  "version": "0.0.0",
  "dependencies": {
    "express": ">= 4.0.0",
    "passport": ">= 0.0.0",
    "passport-http": ">= 0.0.0",
    "morgan": "~1.0.0"
  }
}

Once that was done I nuked my node_modules directory and ran npm install to pull down the latest bits. Once I did that, starting with the app.js I made a few changes. Below is what the app.js file looked like when I started with.

var express = require('express')
  , passport = require('passport')
  , util = require('util')
  , BasicStrategy = require('passport-http').BasicStrategy;

var users = [
    { id: 1, username: 'bob', password: 'secret', email: 'bob@example.com' }
  , { id: 2, username: 'joe', password: 'birthday', email: 'joe@example.com' }
];

function findByUsername(username, fn) {
  for (var i = 0, len = users.length; i < len; i++) {
    var user = users[i];
    if (user.username === username) {
      return fn(null, user);
    }
  }
  return fn(null, null);
}

// Use the BasicStrategy within Passport.
//   Strategies in Passport require a `verify` function, which accept
//   credentials (in this case, a username and password), and invoke a callback
//   with a user object.
passport.use(new BasicStrategy({
  },
  function(username, password, done) {
    // asynchronous verification, for effect...
    process.nextTick(function () {
      
      // Find the user by username.  If there is no user with the given
      // username, or the password is not correct, set the user to `false` to
      // indicate failure.  Otherwise, return the authenticated `user`.
      findByUsername(username, function(err, user) {
        if (err) { return done(err); }
        if (!user) { return done(null, false); }
        if (user.password != password) { return done(null, false); }
        return done(null, user);
      })
    });
  }
));

var app = express.createServer();

// configure Express
app.configure(function() {
  app.use(express.logger());
  // Initialize Passport!  Note: no need to use session middleware when each
  // request carries authentication credentials, as is the case with HTTP Basic.
  app.use(passport.initialize());
  app.use(app.router);
  app.use(express.static(__dirname + '/public'));
});

// curl -v -I http://127.0.0.1:3000/
// curl -v -I --user bob:secret http://127.0.0.1:3000/
app.get('/',
  // Authenticate using HTTP Basic credentials, with session support disabled.
  passport.authenticate('basic', { session: false }),
  function(req, res){
   res.json({ username: req.user.username, email: req.user.email });
  });

app.listen(3000);

First changes, add some requires, remove some requires.

var express = require('express')
  , passport = require('passport')
  , util = require('util')
  , BasicStrategy = require('passport-http').BasicStrategy;

and changed it to

var express = require('express')
  , passport = require('passport')
  , util = require('util')
  , BasicStrategy = require('passport-http').BasicStrategy
  , morgan  = require('morgan')
  , app     = express();

Then I deleted the entire section shown below.

var app = express.createServer();

// configure Express
app.configure(function() {
  app.use(express.logger());
  // Initialize Passport!  Note: no need to use session middleware when each
  // request carries authentication credentials, as is the case with HTTP Basic.
  app.use(passport.initialize());
  app.use(app.router);
  app.use(express.static(__dirname + '/public'));
});

I replace that with a nicely cleaned up Express.js v4 section of code and the replacement for logger, the morgan() library. Initializing passport however is still done in the same ole’ trusty way with initialize().

app.use(morgan());
app.use(passport.initialize());

Ordering of code has changed a bit for express.js, which meant I needed to have the app.use commands before the following section, which I moved right up underneath the two app.use statements.

// curl -v -I http://127.0.0.1:3000/
// curl -v -I --user bob:secret http://127.0.0.1:3000/
app.get('/',
    // Authenticate using HTTP Basic credentials, with session support disabled.
    passport.authenticate('basic', { session: false }),
    function(req, res){
        res.json({ username: req.user.username, email: req.user.email });
    });

Finished app.js File

After those changes the app.js file should look like this.

var express = require('express')
  , passport = require('passport')
  , util = require('util')
  , BasicStrategy = require('passport-http').BasicStrategy
  , morgan  = require('morgan')
  , app     = express();


app.use(morgan());
app.use(passport.initialize());

// curl -v -I http://127.0.0.1:3000/
// curl -v -I --user bob:secret http://127.0.0.1:3000/
app.get('/',
    // Authenticate using HTTP Basic credentials, with session support disabled.
    passport.authenticate('basic', { session: false }),
    function(req, res){
        res.json({ username: req.user.username, email: req.user.email });
    });


var users = [
    { id: 1, username: 'bob', password: 'secret', email: 'bob@example.com' }
  , { id: 2, username: 'joe', password: 'birthday', email: 'joe@example.com' }
];

function findByUsername(username, fn) {
  for (var i = 0, len = users.length; i < len; i++) {
    var user = users[i];
    if (user.username === username) {
      return fn(null, user);
    }
  }
  return fn(null, null);
}

// Use the BasicStrategy within Passport.
//   Strategies in Passport require a `verify` function, which accept
//   credentials (in this case, a username and password), and invoke a callback
//   with a user object.
passport.use(new BasicStrategy({
  },
  function(username, password, done) {
    // asynchronous verification, for effect...
    process.nextTick(function () {
      
      // Find the user by username.  If there is no user with the given
      // username, or the password is not correct, set the user to `false` to
      // indicate failure.  Otherwise, return the authenticated `user`.
      findByUsername(username, function(err, user) {
        if (err) { return done(err); }
        if (!user) { return done(null, false); }
        if (user.password != password) { return done(null, false); }
        return done(null, user);
      })
    });
  }
));

app.listen(3000);

If you execute either of the curl commands shown in the comments in the app.js code you should see the respective response when running the application.

curl -v -I http://127.0.0.1:3000/
curl -v -I --user bob:secret http://127.0.0.1:3000/

…and that should get you running with Passport.js and Express.js v4.

Configuring Node.js Web Applications… Manually || Convict.js

There’s more than a few ways to configure node.js applications. I’ll discuss a few of them in this blog entry, so without mincing work, to configuring apps!

Solution #1: Build Your Own Configuration

Often this is a super easy solution when an application just needs a single simple configuration. Here’s an example I found that’s pretty clean that Noli posted on Stackoverflow to the question “How to store Node.js deployment settings/configuration files?“.

var config = {}

config.twitter = {};
config.redis = {};
config.web = {};

config.default_stuff =  ['red','green','blue','apple','yellow','orange','politics'];
config.twitter.user_name = process.env.TWITTER_USER || 'username';
config.twitter.password=  process.env.TWITTER_PASSWORD || 'password';
config.redis.uri = process.env.DUOSTACK_DB_REDIS;
config.redis.host = 'hostname';
config.redis.port = 6379;
config.web.port = process.env.WEB_PORT || 9980;

module.exports = config;

…then load that in with a require…

var config = require('./config')

The disadvantage is when the application gets a little bigger the configuration can become unwieldy without very specific, strictly enforced guidelines.

Solution #2: Use a Library/Framework Like Convict.js

The use of a library provides some baseline in which to structure configuration. In the case of convict.js it uses a baseline schema that then can be used to extend or override based on configurations needed for alternate environments. A first steps in setting up convict.js for the fueler project looks like this.

Setup a convict.js file:

var convict = require('convict');

// Schema
var conf = convict({
    env: {
        doc: "The App Environment.",
        format: ["production", "development", "test"],
        default: "development",
        env: "NODE_ENV"
    },
    port: {
        doc: "The port to bind.",
        format: "port",
        default: 3000,
        env: "PORT"
    },
    database: {
        host: {
            default: "someplace:cool",
            env: "DB_HOST"
        }
    }
});

// perform validation
conf.validate();

module.exports = conf;

The main two configuration values are the environment and port values. Others will be added as more of the application is put together, but immediately I just wanted something to put in the project to insure it works.

Next get the convict.js library in the project.

npm install convict --save

The save gets it put into the package.json file as a dependency. Once this is installed I opened up the app.js file of the project and added a require at the top of the file after the path require and before the express() call.

var path = require('path');
var config = require('./config');

var app = express();

In the app.set line for the port I changed the setting of the port to be the configuration parameter.

app.set('port', process.env.PORT || config.get('port'));

Now when I run the application, the port will be derived from the config.js file setting.

Now What Did I Do?

I’ll write more about this in the near future, but for now I’ve run into something not being setup right. I’m still working through various parts of customizing my setup. In the instructions for convict.js, which aren’t very thorough beyond the most basic use, is how to insure that the other environments are setup with *.json files. What I mean by this is…

I’ve setup a directory with three json files. It looks like this.

My Config Directory

My Config Directory

Each of these files (or at least one of the files) I would think, based on the instructions, get loaded and merged into configuration based on the code in my app.js as shown below.

var env = conf.get('env');
conf.loadFile('./config/' + env + '.json');

The order of override for the configuration values starts with the base config.js, then any *.json files override those config.js settings and any environment variables override the *.json set configuration variables. Based on that, unless of course I’ve missed something for this snippet of code, I should be getting the configuration settings from the *.json files.

My config file data looks like this. Since it is using cjson I went ahead and stuck comments in there too.

/**
 * Created by adron on 3/14/14.
 * Description: Adding test configuration for the project.
 */

{
    "port": {
        "doc": "The port to bind.",
        "format": "port",
        "default": 1337,
        "env": "PORT"
    }
}

Until later, happy coding, I’m going to dive into this and figure out what my issue is. In my next blog entry I’ll be sure to post an update to what the problem is.

Oh, and that fueler project. Feel free to ping me and jump into it.