Flow-JS
===============

Overview
---------------

Flow-JS provides a continuation-esque construct that makes it much easier to express
multi-step asynchronous logic in non-blocking callback-heavy environments like
Node.js or javascript in the web browser.

The concept is best explained with an example. The following code uses a simple
asynchronous key-store to look-up a user's ID from his username and then sets his
email address, first name, and last name.

In this example, the dbGet and dbSet functions are assumed to rely on asynchronous
I/O and both take a callback that is called upon completion.

	dbGet('userIdOf:bobvance', function(userId) {
		dbSet('user:' + userId + ':email', 'bobvance@potato.egg', function() {
			dbSet('user:' + userId + ':firstName', 'Bob', function() {
				dbSet('user:' + userId + ':lastName', 'Vance', function() {
					okWeAreDone();
				});
			});
		});
	});

Notice how every single step requires another nested function definition. A
four-step process like the one shown here is fairly awkward. Imagine how painful a
10-step process would be!

One might point out that there is no reason to wait for one dbSet to complete before
calling the next, but, assuming we don't want okWeAreDone to be called until all
three calls to dbSet are finished, we'd need some logic to manage that:

	dbGet('userIdOf:bobvance', function(userId) {
		var completeCount = 0;
		var complete = function() {
			completeCount += 1;
			if (completeCount == 3) {
				okWeAreDone();
			}
		}
		
		dbSet('user:' + userId + ':email', 'bobvance@potato.egg', complete);
		dbSet('user:' + userId + ':firstName', 'Bob', complete);
		dbSet('user:' + userId + ':lastName', 'Vance', complete);
	});

Now look at the same example using Flow-JS:

	flow.exec(
		function() {
			dbGet('userIdOf:bobvance', this);
			
		},function(userId) {
			dbSet('user:' + userId + ':email', 'bobvance@potato.egg', this.MULTI());
			dbSet('user:' + userId + ':firstName', 'Bob', this.MULTI());
			dbSet('user:' + userId + ':lastName', 'Vance', this.MULTI());
		
		},function() {
			okWeAreDone()
		}
	);

A flow consists of a series of functions, each of which is applied with a special
`this` object which serves as a callback to the next function in the series. In
cases like our second step, `this.MULTI()` can be used to generate a callback that
won't call the next function until all such callbacks have been called.


Installing
---------------

Flow-JS is a CommonJS compatible module. Place the "flow.js" file in any directory
listed in your `require.paths` array and require it like this:

	var flow = require('flow')

Or you can just put "flow.js" next to your script and do this:

	var flow = require('./flow')


Defining a Flow
---------------

`flow.define` defines a flow given any number of functions as parameters. It returns
a function that can be used to execute that flow more than once. Whatever parameters
are passed each time that flow is called are passed as the parameters to the first
function in the flow.

Each function in the flow is called with a special `this` object which maintains the
state of the flow's execution, acts as a container for saving values for use between
functions in the flow, and acts as a callback to the next function in the flow.

Here is an example to make this clear:

	// define a flow for renaming a file and then printing its stats
	var renameAndStat = flow.define(
	
		function(fromName, toName) {
			// arguments passed to renameAndStat() will pass through to this first function
			
			this.toName = toName; // save to be used in the next function
			fs.rename(fromName, toName, this);
		
		},function(err) {
			// when fs.rename calls the special "this" callback above, this function will be called
			// whatever arguments fs.rename chooses to pass to the callback will pass through to this function
		
			if (err) throw err;
			
			// the "this" here is the same as in the function above, so this.toName is available
			fs.stat(this.toName, this);
		
		},function(err, stats) {
			// when fs.stat calls the "this" callback above, this function will be called
			// whatever arguments fs.stat chooses to pass to the callback will pass through to this function
			
			if (err) throw err;
			
			sys.puts("stats: " + JSON.stringify(stats));
		}
	);
	
	// now renameAndStat can be used more than once
	renameAndStat("/tmp/hello1", "/tmp/world1");
	renameAndStat("/tmp/hello2", "/tmp/world2");


Executing a Flow Just Once
---------------

`flow.exec` is a convenience function that defines a flow and executes it immediately,
passing no arguments to the first function.

Here's a simple example very similar to the one above:

	flow.exec(
		function() {
			fs.rename("/tmp/hello", "/tmp/world", this);
		},function(err) {
			if (err) throw err;
			fs.stat("/tmp/world", this)
		},function(err, stats) {
			if (err) throw err;
			sys.puts("stats: " + JSON.stringify(stats));
		}
	);


Multiplexing
---------------

Sometimes, it makes sense for a step in a flow to initiate several asynchronous tasks and
then wait for all of those tasks to finish before continuing to the next step in the flow.
This can be accomplished by passing `this.MULTI()` as the callback rather than just `this`.

Here is an example of `this.MULTI()` in action (repeated from the overview):

	flow.exec(
		function() {
			dbGet('userIdOf:bobvance', this);
			
		},function(userId) {
			dbSet('user:' + userId + ':email', 'bobvance@potato.egg', this.MULTI());
			dbSet('user:' + userId + ':firstName', 'Bob', this.MULTI());
			dbSet('user:' + userId + ':lastName', 'Vance', this.MULTI());
		
		},function() {
			okWeAreDone()
		}
	);

  You can identify the results of a function by passing a result identifier to MULTI. The results of a function can retrieved using this key in the final step. The result will be a single value if callback receives 0 or 1 argument, otherwise it will be an array of arguments passed to the callback.

  Example:

	flow.exec(
		function() {
			dbGet('userIdOf:bobvance', this.MULTI('bob'));
			dbGet('userIdOf:joohndoe', this.MULTI('john'));
		},function(results) {
		  dbSet('user:' + results['bob'] + ':email', 'bobvance@potato.egg');
		  dbSet('user:' + results['john'] + ':email', 'joohndoe@potato.egg');
		  okWeAreDone();
		}
	);

In many cases, you may simply discard the arguments passed to each of the callbacks generated
by `this.MULTI()`, but if you need them, they are accessible as an array of `arguments`
objects passed as the first argument of the next function. Each `arguments` object will be
appended to the array as it is received, so the order will be unpredictable for most
asynchronous APIs.

Here's a quick example that checks for errors:

	flow.exec(
		function() {
			fs.rename("/tmp/a", "/tmp/1", this.MULTI());
			fs.rename("/tmp/b", "/tmp/2", this.MULTI());
			fs.rename("/tmp/c", "/tmp/3", this.MULTI());
		
		},function(argsArray) {
			argsArray.forEach(function(args){
				if (args[0]) then throw args[0];
			});
		}
	);


serialForEach
---------------

Flow-JS comes with a convience function called `flow.serialForEach` which can be used to
apply an asynchronous function to each element in an array of values serially:

	flow.serialForEach([1, 2, 3, 4], function(val) {
		keystore.increment("counter", val, this);
	},function(error, newVal) {
		if (error) throw error;
		sys.puts('newVal: ' + newVal);
	},function() {
		sys.puts('This is the end!');
	});

`flow.serialForEach` takes an array-like object, a function to be called for each item
in the array, a function that receives the callback values after each iteration, and a
function that is called after the entire process is finished. Both of the second two
functions are optional.

`flow.serialForEach` is actually implemented with `flow.define`.

Thanks to John Wright for suggesting the idea! (http://github.com/mrjjwright)
