Node fibers: Patterns and anti-patterns using synchronize.js

SHARE ON

Get rid of callbacks without the overhead of promises and generators

For all the discussion of “callback hell” in the JavaScript community—it
even has its own website—there’s no standard solution to the problem,
and isn’t likely to be until ES6 promises and generators become more widely
supported.

That’s a particular problem in Node, given that even v0.12 implements a paltry
17% of ES6. But Node also supports the best workaround, one which
has received fairly little attention compared to promises: fibers.

This is unsurprising given that there is very little information on what fibers
are and what they can do. The core implementation is intentionally low-level
and understanding how fibers work requires an intimate understanding
of Node’s event loop.

But, you don’t need to understand how fibers work in order to reap their benefits.
All you need to know is that fibers let you write asynchronous code as if it
were synchronous
. They eliminate callbacks, offer a minimum of API overhead
compared to promises, and don’t require syntactic support like generators.

Then, you need some examples to see how fibers can work: a little library on top
like the core project recommends.

At Mixmax, our fibers library of choice is synchronize.js. Here’s
how to use it to write beautifully-straightforward async code.

Basics

synchronize.js lets you execute asynchronous functions synchronously—
within a fiber. That way, the fiber can yield while the event loop—other
fibers, timers, callbacks, etc.—keeps running. Fibers “suspend”, but the
event loop never blocks.

You create and run a fiber using sync.fiber:

Within the fiber, you can block on the result of an asynchronous function
using sync.defer and sync.await. Calling sync.defer creates a Node-style
(err, result) callback. Pass that to an asynchronous function and then call
sync.await to suspend execution until the callback is called.

Using sync.defer and sync.await won’t block the code executing
outside the fiber:

But the code inside the fiber will run synchronously:

sync.await lets you handle asynchronous results and errors synchronously:

What sync.await does is convert the arguments to sync.defer(), err and
result, into exceptions and return values:

  • If sync.defer()’s first argument is truthy, sync.await will throw that
    value as an error.
  • Otherwise, sync.await will return sync.defer()’s second argument, whatever
    that may be.

Compare how you’d handle the errors and results using callbacks:

As you can see, the synchronize.js form is much more straightforward.

For more examples, including running functions in parallel, see synchronize.js
documentation.

How to structure an application using fibers

Using synchronize.js in production requires that you consider how your fibers
will interoperate with regular old synchronous and asynchronous functions. This
isn’t much more difficult than the examples I’ve shown above. But your first
impulse might trip you up.

There are two simple rules to happy development with fibers. The first and most
important is:

Functions should never assume that they’re running in a fiber.

Here’s what I mean. In contrast to promises or generators, fibers make it
dangerously easy to refactor code. If I start with:

and then decide that I want to move the reading-and-processing operation into
its own function, it’s super easy to do so:

processFile doesn’t have to be declared using special syntax, like a generator
function. It doesn’t have to return a result or an error using special APIs like
promises.

…but…

processFile cannot be called outside of a fiber!

To preserve modularity and avoid unpleasant surprises when using unfamiliar APIs,
asynchronous functions should accept callbacks.


DON’T PANIC. The difference between callback hell and fibers is that (second rule):

You don’t call the callbacks yourself.

To fix processFile you just have to wrap its contents in a fiber and pass
the callback to the fiber:

And synchronize.js will call done for you!

  • If the fiber throws an error, done will be called with that error as its
    first argument.
  • If the fiber returns a value, done will be called with that value as its
    second argument.

In this way, fibers interoperate seamlessly with Node-style callbacks.

To drive home this lesson, consider calling done yourself in a more complicated
processFile:

Look at all the places that done could be called! And the superfluous return
necessary to guard against calling it multiple times! It’s barely better than
if you were only using callbacks. But by passing done to sync.fiber,
you can strip away almost all the cruft:

Marvelous.

Use magic sparingly

Now that you’ve learned the rules above, I can share with you the most powerful
ability of synchronize.js: to do away with the API calls altogether.

If you pass an object to sync, you make its asynchronous methods synchronize.js-aware.
Afterward, calling such a method without a callback implicitly calls sync.await
and sync.defer:

But: this can make it very difficult to tell what code needs a fiber! As before,
if you’re worried you might be running outside of a fiber, make one. There’s
negligible overhead to doing so.

At Mixmax, we sync only the most commonly-used asynchronous functions, like
database accessors.

Smooth sailing

Fibers bring the simplicity of synchronous programming to Node without compromising
on I/O. Using fibers, you get to code as if Node had threads without
ever blocking the event loop.

As long as you follow the rules above, fibers are a powerful yet simple alternative
to callbacks, promises, and generators. To wit:

  • Give every function its own fiber and callback
  • Let synchronize.js call that callback
  • And use magic sparingly.

Questions about fibers? Want to tell us that promises are the one true way to
write asynchronous JavaScript? Email careers@mixmax.com
or send us a tweet @Mixmax!

Want to explore cutting-edge JavaScript technologies at your day job?
Email careers@mixmax.com and let’s grab coffee!

SHARE ON

Written By

Jeff Wear

Jeff Wear

From Your Friends At