Using Javascript Generators

If you've checked other posts about ES6, you've probably seen Generators explained with the help of code that looks something like:

function task() {
  yield 1;
  yield 2;
  yield 3;
}

var iterator = task();
console.log(iterator.next().value);
console.log(iterator.next().value);
console.log(iterator.next().value);

It's hard to see how the above can be used to eliminate nesting.

Generators don't magically make asynchronous code synchronous. All they do is return an iterator which can be used to step through a function (which is what the above code shows). For a real code, we need something that understands and automatically executes iterators. Popular libraries are co, Q and suspend. If you're using Koa (or some other generator-aware framework) this should all be setup for you; your code will already be running within a generator-aware control flow pipeline.

The other thing to understand is that Generators work in conjunction with, not as a replacement for, promises. While this isn't a strict rule, it's how non-trivial examples work. Understanding how the two work together was my aha moment. Specifically, yielding to a promise stops execution until said promise is resolved.

Back to our goal: connecting to a database and querying it. Ideally, our target code looks something like:

query = (sql, params) ->
  conn = connect()
  result = conn.query(sql, params)
  conn.close()
  return result

Of course, we know that connect and conn.query are asynchronous. The first step is to make this code leverage promises (I'm going to do this the long way first). Let's do connect:

# cs
connect = ->
  new Promise (resolve, reject) ->
    pg.connect connectionString, (err, conn) -> resolve(conn)

// js
function connect() {
  return new Promise(function(resolve, reject){
    pg.connect(connectionString, function(err, conn){
      resolve(conn);
    });
  });
}

Using the above, we end up with:

query = (sql, params) ->
  connect().then (conn) ->
    result = conn.query(sql, params)
    conn.close()
    return result

The next step is turning the above function into a generator. In CoffeeScript, a function that yields is a generator. In JavaScript the function would need to be declared with *: function* query(sql, params) {...}.

# cs
query = (sql, params) ->
  conn = yield connect()
  result = conn.query(sql, params)
  conn.close()
  return result

// js
function* query(sql, params) {
  var conn = yield connect();
  var result = conn.query(sql, params);
  var conn.close();
  return result;
}

Again, the aha momenet is understanding that yield waits for our promise to be resolved. Furthermore, while resolve gets assigned, reject gets raised:

connect = ->
  new Promise (resolve, reject) ->
    pg.connect connectionString, (err, conn) ->
      return reject(err) if err?
      resolve(conn)

query = (sql, params) ->
  try
    conn = yield connect()
    ...
  catch err

Before we do the same change to conn.query, let's clean up our promise code by switching from the native ES6 Promise to Bluebird. With bluebird, we can easily create promises for an objects methods via promisifyAll:

pg = Promise.promisifyAll(require('pg'))
Promise.promisifyAll(pg.Client.prototype)

connect: ->
  pg.connectAsync(connectionString).spread (conn, close) ->
    conn.close = close
    conn

query: (sql, params) ->
  conn = yield connect()
  result = yield conn.queryAsync(sql, params)
  conn.close()
  return result

The first lines generates an *Async version of every method exposed directly by pg (Async methods return promises). The second line does the same thing for all instance methods of the Client object.

That's kind of all there is to it. You do end up using yield in a lot of places, but it does wipe out all nesting (something we couldn't quite get done with Promises alone). It also brings back more conventional exception handling, which again, never felt 100% right with promises.

To get a better feel for things, here's some code you can play with. It's a bit more standalone.

// npm install co
// you'll need to be running an engine that supports ES6 (io.js or node with harmony enabled)

var fs = require('fs');
var co = require('co');

function readFile(path) {
  return new Promise(function(resolve, reject){
    fs.readFile(path, 'utf-8', function(err, text) {
      err ? reject(err) : resolve(text);
    });
  });
}

// can use Q or suspend. Using co is what makes it so we don't need to
// manually iterate our generator and makes it all promise-aware.
// In a framework like Koa, the fact that your code is running inside
// a similar pipeline is completely transparent.
co(function*() {
  try {
    contents = yield readFile('test');
    console.log(contents);
  } catch(err) {
    console.log(err);
  }
});