Internals of async / await in JavaScript

If you have ever used JavaScript in the past, there is a high chance you have encountered the async / await syntax. async / await makes it easy to define asynchronous logic in a synchronous way that our brains can comprehend better. Some of you JavaScript veterans might know that async / await is merely a syntactic sugar over the existing Promises API. This means that we should be able to achieve the functionality of async / await in JavaScript without using async and await keywords albeit with some verbosity. This is exactly what I wanted to explore in this post.

What are we trying to achieve?

Let’s look at some boilerplate code to understand what we are trying to achieve.

1
function wait() {
2
  return new Promise((resolve, reject) => {
3
    setTimeout(() => {
4
      resolve('Timeout resolved');
5
    }, 2000);
6
  });
7
}
8
9
async function main() {
10
  console.log('Entry');
11
12
  const result = await wait();
13
  console.log(result);
14
15
  console.log('Exit');
16
  return 'Return';
17
}
18
19
main().then((result) => {
20
  console.log(result);
21
});

The output of the above code:

Entry
// Pauses for 2 seconds
Timeout resolved
Exit
Return

Given the above code snippet and its respective output, can we re-write the main() function to not use async and await keywords, but still achieve the same output? Conditions being the following:

• Cannot use Promise chaining within main(). This makes the problem trivial and deviates from the original goal. Promise chaining might work in the contrived example above but it does not capture the complete essence of async / await and the problems it solves.
• Not changing any function signatures. Changing function signatures would require updating function calls as well which is complex in a large project with many interdependent functions. So try not to change any function signatures.

Even though I have defined the conditions above, if you feel stuck and don’t see a way forward without breaking the above conditions, do take the liberty to try that approach. Maybe breaking the condition will lead you to an approach that satisfies the above condition. No one is expected to come up with the right solution on their first try(even I didn’t while preparing this 🙂).

Playground

This blog post is hands-on and I strongly suggest you use the below playground to test out your ideas to implement something like async / await without using async and await keywords. The rest of the post has hints and solutions on how you can achieve this, but feel free to take a moment and try out some code whenever you get an idea to proceed.

function wait() {
  return new Promise((resolve, reject) => {
    setTimeout(() => {
      resolve('Timeout resolved');
    }, 2000);
  });
}

async function main() {
  console.log('Entry');

  const result = await wait();
  console.log(result);

  console.log('Exit');
  return 'Return';
}

main().then(result => {
  console.log(result);
});

Hint #1: Looking at pausing and resuming a function execution

At a high level, what does async / await do? Well, it pauses the execution of the async function whenever it encounters an await statement. And resumes the execution when the awaited promise is resolved(or throws an error). A natural question to ask here would be how to go about pausing and resuming a function’s execution. Functions tend to execute to completion right?

Is there a native feature in JavaScript that mimics this behavior? Yes! It’s Generators.

Generators are a special type of function that can return multiple pieces of data during its execution. Traditional functions can return multiple data by using structures like Arrays and Objects, but Generators return data whenever the caller asks for it, and they pause execution until they are asked to continue to generate and return more data.

I won’t be diving deep into what Generators can do, please use the linked reference. Here’s a code snippet explaining the concept of Generators, try to follow the comments to understand what is happening.

1
function* main() {
2
  console.log('Entry');
3
4
  const message = yield 'Result 1';
5
  console.log(message);
6
7
  console.log('Exit');
8
  yield 'Result 2';
9
10
  return 'Return';
11
}
12
13
const it = main();
14
/**
15
 * No output in console right now even though main() was called.
16
 * Calling .next() on the object returned by the generator starts the execution.
17
 */
18
19
console.log(it.next());
20
/**
21
 * Output:
22
 *   Entry
23
 *   { value: "Result 1", done: false }
24
 *
25
 * The generator has paused the execution at line 4.
26
 * It resumes when .next() is called again.
27
 */
28
29
console.log(it.next('Message Passing'));
30
/**
31
 * .next() also takes an argument that is made available to the yield
32
 * statement where the generator is paused.
33
 *
34
 * Output:
35
 *   Message Passing
36
 *   Exit
37
 *   { value: "Result 2", done: false }
38
 */
39
40
console.log(it.next());
41
/**
42
 * Output:
43
 *   { value: "Return", done: true }
44
 */

Now that you know about Generator functions, I suggest you take a pause and go to the playground to see if you can proceed from here.

Hint #2: When to resume function execution?

Generators provide a way to pause and resume function execution. Recalling how async / await works, an async function is paused when it encounters await statement. So we can treat the async function as a generator function and place a yield statement near a promise so that it pauses at this step. Looks something like this:

1
function* main() {
2
  console.log('Entry');
3
4
  const result = wait();
5
  yield;
6
  console.log(result);
7
8
  console.log('Exit');
9
  return 'Return';
10
}
11
12
const it = main();
13
it.next(); // Start the execution, when should it.next() be called again?

But when should the generator function resume its execution? It should resume when the promise near the yield is resolved. How can the caller know about the promise when it is within the generator function? Is there a way to expose the promise to the caller so that they can attach a .then() callback to it which will call the .next() on the generator object to resume execution?

The answer to all the questions above is to simply yield the promise we want to wait for so that the caller can use this yielded promise and call .next() when it is resolved.

1
function* main() {
2
  console.log('Entry');
3
4
  const result = yield wait();
5
  console.log(result);
6
7
  console.log('Exit');
8
  return 'Return';
9
}
10
11
const it = main();
12
it.next().value.then(() => {
13
  it.next();
14
});

Hint #3: Making promise’s resolved data available to the generator

In the previous code snippet, we were able to successfully pause and resume function execution when the promise was resolved. But the generator function does not get the resolved data from the promise. result variable in the main() function is supposed to have "Timeout resolved" as we see when using async / await. But in our implementation, it does not get the data that the promise gives when it is resolved. Is there a way to pass the resolved data of the promise to the generator? After all, the caller has access to resolved data since the generator yields the promise. So can it pass this data back to the generator function when the caller calls .next() on the generator object? We have already come across this message-passing behavior above in this post 😉

.next() function takes an argument that is made available to the last yield statement where the generator was paused. So to pass the promise’s resolved data, we simply call .next() with the resolved data from the promise.

1
function* main() {
2
  console.log('Entry');
3
4
  const result = yield wait();
5
  console.log(result);
6
7
  console.log('Exit');
8
  return 'Return';
9
}
10
11
const it = main();
12
it.next().value.then((resolvedData) => {
13
  it.next(resolvedData);
14
});

With this change, we have a basic implementation of async / await without using async and await keywords. Notice the main() function and compare it with its async counterpart. They are shockingly similar, right? Instead of using async function, it is function *, and instead of await, it uses yield keyword. That’s the beauty of this implementation!

We have come quite far! If you were able to figure out some of these steps on your own then give yourself a pat on the back 💚

Hint #4: Extending it to work with multiple yield statements

The next step is to make our implementation work with an arbitrary number of yield statements. The above snippet only works with one yield as it calls .next() only after the first promise is resolved. But the generator can have an arbitrary number of promises being yielded. Can we write an abstraction that dynamically waits for any yielded promise to be resolved and then calls .next()?

This abstraction can be a function(say run) which takes in the generator function. What should run() return? Again, compared with the async function counterpart, every async function implicitly returns a Promise, which gets resolved when the async function has completed execution. We can mimic this behavior by returning a Promise from the run() function, and resolving it only when the generator has finished execution.

This is how the code looks like, your implementation might vary.

1
run(main).then((result) => {
2
  console.log(result);
3
});
4
5
function run(fn, ...args) {
6
  const it = fn(...args);
7
8
  return new Promise((resolve, reject) => {
9
    // TODO: Call it.next() as long as something is there to yield
10
  });
11
}

Hint #5: Calling .next() arbitrary number of times

Now, let’s focus on implementing the run() function. It should call .next() on the generator object as long as promises are being yielded. Can we use loops to do this? Would it work as expected when we are using promises? Of course not, we can’t use loops since it will keep calling .next() on the generator object without waiting for the promises being yielded to be resolved. Is there a better way to loop which does not have this problem?

It’s Recursion! By using Recursion, we can keep calling .next() on the generator object when yielded promises get resolved. What’s the exit condition or the base case for the recursion to end? We want to stop when the generator ends. What does .next() return when the generator has reached the end? The done property on the returned object is set to true!

1
function run(fn, ...args) {
2
  const it = fn(...args);
3
4
  return new Promise((resolve, reject) => {
5
    function step() {
6
      const result = it.next();
7
8
      // Exit condition
9
      if (result.done) {
10
        return;
11
      }
12
13
      result.value.then((resolvedValue) => {
14
        step();
15
      });
16
    }
17
18
    // Call step() to start recursion
19
    step();
20
  });
21
}

We are not passing resolvedValue from the promise back to the generator. To do this, let’s make the step() function accept an argument. Also, notice how the promise returned by run is never resolved. Because we don’t call the resolve() function anywhere! When should the promise be resolved? When the generator ends and there’s nothing else to execute. What should the promise resolve with? With whatever the generator function returns, as that matches the behavior of async functions.

1
function run(fn, ...args) {
2
  const it = fn(...args);
3
4
  return new Promise((resolve, reject) => {
5
    function step(resolvedValue) {
6
      const result = it.next(resolvedValue);
7
8
      // Exit condition
9
      if (result.done) {
10
        resolve(result.value);
11
        return;
12
      }
13
14
      result.value.then((resolvedValue) => {
15
        step(resolvedValue);
16
      });
17
    }
18
19
    // No need to pass anything to start the execution of the generator
20
    step();
21
  });
22
}

There you have it – async / await without async and await!

That completes the implementation of async / await without using async and await keywords. Async functions are represented as generator functions and instead of using await we use yield statements to wait for promises to get resolved. For a developer using our implementation, it still feels similar to async / await and it doesn’t break the two conditions set at the beginning of this post.

function* main() {
  console.log('Entry');

  const result = yield wait();
  console.log(result);

  const result2 = yield wait();
  console.log(result2);

  // ...

  console.log('Exit');
  return 'Return';
}

run(main).then(result => {
  console.log(result);
});

function run(fn, ...args) {
  const it = fn(...args);

  return new Promise((resolve, reject) => {
    function step(resolvedValue) {
      const result = it.next(resolvedValue);

      // Exit condition
      if (result.done) {
        resolve(result.value);
        return;
      }

      result.value.then(resolvedValue => {
        step(resolvedValue);
      });
    }

    // No need to pass anything to start the execution of the generator
    step();
  });
}

function wait() {
  return new Promise((resolve, reject) => {
    setTimeout(() => {
      resolve('Timeout resolved');
    }, 2000);
  });
}

This is exactly what transpilers like Babel do when converting async / await to older versions of JavaScript that did not have this feature natively. If you see the transpiled code, you can draw a lot of parallels to our implementation above!

Next steps

The implementation above only covers the successful path of async / await. Our implementation does not handle error scenarios when a promise gets rejected. I would like to leave this as an exercise to the readers, as this is very similar to the success path, just the functions used are different. Do take a look at the Generators API to see if there is a way to propagate errors back to a generator similar to the .next() function to start with! If you do solve this, share it with me on Twitter, and don’t forget to tag me – @blenderskool.

Conclusion

When I first came across this implementation, I was awestruck by its beauty and simplicity. I was aware that async / await was a syntactic sugar, but did not know what it looked like under the hood. This post covered exactly this aspect of async / await, and how it ties in with Promises and Generators. I like to unravel abstractions from time to time to get a sense of how things work under the hood. And that is where I find interesting concepts to learn. I hope this post also motivated you to stay curious and learn things practically instead of just following a tutorial 🙌.