/ javascript

Async Programming Patterns with NodeJS

Async What?!..

Recently I was contacted by a friend with some questions about async programming in NodeJS. This forced me to explore several different patterns of writing async JS in Node and I wanted to share my experiences. But first why do we write code asynchronously as opposed to synchronously? Well this is largely due to the fact JS is single threaded. As such it can only do one thing at a time - if a function call takes time to complete then all other code is blocked until it finishes. Not ideal when you may want to load a page! So async allows us to "kick" off operations and continue on our merry way. When there is something we need to deal with we'll come back. I'm sure you can see the benefits to this - We avoid blocking code with the added bonus of being able to do more with our resources. How do you implement this? Read on...

In this quick blog post we will take a look at 2 different methods to run things asynchronously. They are:

  • Callbacks - The most common way of writing non blocking code. A callback is a function which is passed as a parameter to an operation. When the operation is executed and a result or error is available the callback is run accepting the data as parameters. The callback can now pass this back to our program flow so we can carry on. Clear as mud? I thought so lets see an example...

I have create a file with some text in it called data.txt for this example

data.txt

Hello from data.txt

The Code

const fs = require('fs');
const file = 'data.txt';

const data = fs.readFile(process.cwd() + '/data.txt'); // This wont work - returns undefined
fs.readFile(process.cwd() + '/data.txt', (err, data) => {
  console.log(data.toString()); // That's better!
});

console.log(data);

The Output

undefined
Hello from data.txt

We can nest as many async operations and keep passing data as much as we like. However even with the new ES6 syntax this is getting hard to keep track of isn't it? There is a better way and that's with Promises!

(data cb) => {
  cb(data1, cb1) => {
    cb1(data2, cb2) => {
      cb2(data3, cb3) => {
        cb3(data4, cb4) => {
          cb4(data5, cb5) => {
            // This is getting a bit insane!
          }
        }
      }
    }
  }
}
  • Promises - Promises are like callbacks but better! They are a control structure which allows us to flatten the depth of our code and avoid callback hell which we saw above. Basically a promise allows us to "wrap" our code in a function which keeps track of the async operation in contains creating a chain. We can then create many operations which can be executed as part of that chain sequentially. This makes our code a lot more linear and easier to maintain. Taking the example above lets rewrite it using a Promise:

const promise = new Promise((resolve, reject) => { // A promise accepts a resolve and reject parameters
  fs.readFile(process.cwd() + '/data.txt', (err, data) => {
    if(err) {
      reject(err); // If it fails we reject
    }

    if(data) {
      resolve(data); // If it passes we resolve and move on to the next chain
    }
  });
});


promise.then((data) => {
  console.log(data.toString()); // Resolved!
})
.catch((err) => {
  console.log(err); // Error
});

We get the exact same result except now its readable due to the Promise structure - then do this catch that. It may not be immediately apparent from a single promise this is better but what if we wanted to read 3 files async one after the other? Well compare the two versions:

With Callbacks

fs.readFile(process.cwd() + '/' + file, (err, data) => {
  if(data) {
    fs.readFile(process.cwd() + '/' + file2, (err, data2) => {
      if(data2) {
        fs.readFile(process.cwd() + '/' + file3, (err, data3) => {
          console.log(data.toString(), data2.toString(), data3.toString());
        });
      }
    });
  }
});

With Promises

const promise = new Promise(function(resolve, reject) {
  fs.readFile(process.cwd() + '/' + file, (err, data) => {
    if(data) {
      resolve(data);
    }

    if(err) {
      reject(err);
    }
  });
});

const promise2 = new Promise(function(resolve, reject) {
  fs.readFile(process.cwd() + '/' + file2, (err, data2) => {
    if(data2) {
      resolve(data2);
    }

    if(err) {
      reject(err);
    }
  });
});

const promise3 = new Promise(function(resolve, reject) {
  fs.readFile(process.cwd() + '/' + file3, (err, data3) => {
    if(data3) {
      resolve(data3);
    }

    if(err) {
      reject(err);
    }
  });
});

promise.then((data) => {
  console.log(data.toString());
  return promise2;
})
.then((data2) => {
  console.log(data2.toString());
  return promise3;
})
.then((data3) => {
  console.log(data3.toString());
})
.catch((err) => {
  console.log(err);
});

Both versions return the same results and although promises means more code we can read it much easier and in a linear way. Neat huh?

Wrapping things up...

That's it! Two different async control flows - Both can be used in isolation or together and both are valid approaches. Its down to you as the developer to decide which best serves your purpose for what your trying to achieve.