Asynchronous programming with JavaScript can lead to what is disparagingly termed "callback hell."

I feel that Node.js beginners are told far too often that the only way to accomplish asynchronous JavaScript is to use an async library. In my experience, solving relatively simple async problems is the best way to learn Node, and will give you a deeper more intuitive understanding of the environment. (It will also give you the tools you will need to implement more complicated async solutions.)

Callback hell

"Callback hell" refers to the excessive code nesting created as you attempt to code a series of logical steps that each require a call to an asynchronous (non-blocking) function. For example, the following function processes three asynchronous steps one at a time:

function nestedExample() {
    setTimeout(function() {
        console.log('Do step one');
        setTimeout(function() {
            console.log('Do step two');
                setTimeout(function() {
                    console.log('Do step three');
                    setTimeout(function() {
                        console.log('Finalize process');
                        return;
                    }, 200);
                }, 200);
        }, 200)
    }, 200);
}

Note: Using setTimeout() it is a great way to simulate an asynchronous function when you are learning. Much like a database or http call, it takes a callback function to execute when it completes. Here the delay is set to 200ms.

Notice the way to code marches to the right, with each successive callback nested inside the previous step. This code can be very hard to read and manage.

A recursive approach

Before jumping to a full fledge asynchronous solution such as async, or q, consider whether a recursive pattern might work. The above example would look something like this:

function recursiveExample() {

    var step = 'one';
    processData();

    function processData() {

        switch (step) {
            case 'one':
                setTimeout(stepOne(), 200);
                break;

            case 'two':
                setTimeout(stepTwo(), 200);
                break;

            case 'three':
                setTimeout(stepThree(), 200);
                break;

            case 'finalize':
                stepFinalize();
                break;
        }
    }

    function stepOne() {
        console.log('Do step one');
        step = 'two';
        processData();
    }

    function stepTwo() {
        console.log('Do step two');
        step = 'three';
        processData();
    }

    function stepThree() {
        console.log('Do step three');
        step = 'finalize';
        processData();
    }

    function stepFinalize() {
        console.log('Finalize process');
        return;
    }

}

This approach creates more code, but it is very simple, easy to read, and permits a large degree of flexibility.

Note that the step variable is global to all the step functions because of the closure.

I have used steps named "one", "two", and "three" above, but in practice you can name these to fit your process - such as "validateuser", "processorder", and "send_email" for example.

You can easily expand this approach to handle some parallel tasks, and branching conditionals in the async process steps. (You can also start with an array of records to process and .pop() each next value to process until the array is empty.)

This is certainly not the best approach for many cases - but it is a great way to become more intimate with asynchronous programing before jumping into an async library. And you might be surprised how manageable your async challenge becomes once you start to become comfortable with some new patterns.

I was able to implement all of the MySQL transaction functionality for my eCommerce sites in a clean readable manner without using an async library.

I think the jump to a formal async library comes when the complexity level increases to the next level, or when you are implementing a larger scope project that requires integrating asynchronous functionality across multiple modules.

Read more on how to implement promises here.

Next post I will talk about Moving user state to the browser.

For a full introduction and index to this blog: Node.js: One New Approach

Cheers!