Node's asynchronous (non-blocking) model is the most challenging hurdle to learning Node, but is also the fundamental reason Node is so compelling. This new "parallel programming" technique is a consequence of Node's inherently "event-driven" approach which naturally fosters efficient and scalable solutions for I/O intensive server applications.

Procedural programming

If you have done any front-end browser programming with the browser DOM and AJAX, you are already familiar with an event-driven environment and some asynchronous programming. Since the front end tends to respond to user events, the event-driven approach is easier to understand in this context.

But on the back end, the instinct to build services in a procedural manner is hard to change. The procedural approach closely mimics our logical approach to solving problems. We see the solution as in a flow diagram, with conditional branches, process loops, and an eventual output. Everything happens one step after another. This is great for the programmer, but creates a challenge for the server. Each request to the server essentially fires up a dedicated program to process the request. The process may do many things - query a database, request data from another server, write a file, etc. The server must keep this ball in the air while still serving other requests. To keep things moving, the server manages many threads at the same time - even when they are just sitting there waiting for a response from the database, file system, or external server.

Asynchronous programming

Node.js does not directly support the procedural approach. Instead, it leverages the asynchronous capability of JavaScript and puts the burden (and the control) in the hands of the programmer.

In Node, you still implement a procedural process, but you must design an asynchronous coding solution for it. As a byproduct of this design process, you will craft a fundamentally more efficient program.

In Node, whenever you reach a step that might take some time (like querying a database, writing a file, or making an http request to another server), you call an asynchronous function. The asynchronous function is written in such a way that it returns control back to your process flow right away, but continues to execute on the server. This asynchronous function most always takes a "callback" as an argument, which is a function you pass to it to be executed when it completes. Because of the "closure", the callback function has access to the caller's variables and keeps their state on the server until the asynchronous function completes.

If you don't already understand closure in JavaScript - here is a nice quick explanation.

On a busy Node server, there may be many simultaneous requests active on the server at any one time, which seems to go against the notion you get when you hear that Node is single-threaded. The answer to this apparent contradiction comes from the way JavaScript manages function scope using closure. Even though there are multiple independent client requests active at one time, each function called to handle a request maintains its own set of variables until it completes.

Here is a helpful overview of JavaScript functions and scope: Functions and function scope

Great for I/O intensive work

I like to think of Node as a "conductor", orchestrating requests and the various I/O tasks required to fulfill the request. An I/O task might be querying a database, reading a file, or making an http request to an API. Also, Node.js supports delegating a CPU-intensive task to a "child process" which gets its own thread.

Callback hell

This is a disparaging term that refers to the syntactical challenge that can crop up when you try to implement a long branching procedural process flow that requires many nested asynchronous function calls. There are a couple ways to mitigate this scenario:

• There are special libraries and approaches to support complicated logical flows in an asynchronous environment. Popular solutions include promises, async, and other approaches
• Define functions outside your logic flow code block instead of putting them in-line. With some clever design and function naming, this can yield surprisingly impressive results
• Using recursion can solve many asynchronous list-looping patterns with a small amount of nice readable code
• Just accept some level of function nesting - it is not as bad as it seems once you accept it as part of the language. I find I can nest up to three async calls without confusion, and that I can code a majority of my solutions without needing to go further.

Next post I give an example of using recursion to tame callback hell.

For a full introduction and index to this blog: Node.js: One New Approach

Cheers!