Asynchronous Iterator

The problem of the synchronization traverser

As mentioned in the chapter "Iterator", the Iterator interface is a protocol for data traversal. As long as you call the next method of the iterator object, you will get an object that represents the information of the current traversal pointer. The structure of the object returned by the next method is {value, done}, where value represents the value of the current data, and done is a boolean value that indicates whether the traversal is over.

function idMaker() {
  let index = 0;

  return {
    next: function () {
      return { value: index++, done: false };
    },
  };
}

const it = idMaker();

it.next().value; // 0
it.next().value; // 1
it.next().value; // 2
// ...

In the above code, the variable it is an iterator. Every time the it.next() method is called, an object is returned, representing the information of the current traverse position.

There is an implicit requirement here that the it.next() method must be synchronous, and it must return a value immediately as long as it is called. In other words, once the it.next() method is executed, the two attributes of value and done must be obtained synchronously. If the traversal pointer just points to a synchronous operation, of course there is no problem, but for asynchronous operations, it is not suitable.

function idMaker() {
  let index = 0;

  return {
    next: function () {
      return new Promise(function (resolve, reject) {
        setTimeout(() => {
          resolve({ value: index++, done: false });
        }, 1000);
      });
    },
  };
}

In the above code, the next() method returns a Promise object, so this won't work, it doesn't conform to the Iterator protocol, as long as the code contains asynchronous operations. In other words, the next() method in the Iterator protocol can only contain synchronous operations.

The current solution is to wrap the asynchronous operation into a Thunk function or Promise object, that is, the value property of the return value of the next() method is a Thunk function or Promise object, waiting for the real value to be returned later, and done The attributes are also generated synchronously.

function idMaker() {
  let index = 0;

  return {
    next: function () {
      return {
        value: new Promise((resolve) =>
          setTimeout(() => resolve(index++), 1000)
        ),
        done: false,
      };
    },
  };
}

const it = idMaker();

it.next().value.then((o) => console.log(o)); // 0
it.next().value.then((o) => console.log(o)); // 1
it.next().value.then((o) => console.log(o)); // 2
// ...

In the above code, the return value of the value property is a Promise object, which is used to place asynchronous operations. But writing in this way is very troublesome, it is not intuitive, and the semantics are more convoluted.

ES2018 Introduced "Async Iterator" (Async Iterator) provides native iterator interfaces for asynchronous operations, namely value and done Both of these attributes are generated asynchronously.

Asynchronous traversal interface

The biggest grammatical feature of the asynchronous iterator is that the next method of the iterator is called, which returns a Promise object.

asyncIterator
  .next()
  .then(
    ({ value, done }) => /* ... */
  );

In the above code, asyncIterator is an asynchronous iterator. After calling the next method, it returns a Promise object. Therefore, you can use the then method to specify that the state of this Promise object becomes the callback function after resolve. The parameter of the callback function is an object with two attributes of value and done, which is the same as a synchronous traverser.

We know that the interface of an object's synchronous iterator is deployed on the Symbol.iterator property. Similarly, the object's asynchronous iterator interface is deployed on the Symbol.asyncIterator property. No matter what kind of object it is, as long as its Symbol.asyncIterator property has a value, it means that it should be traversed asynchronously.

The following is an example of an asynchronous iterator.

const asyncIterable = createAsyncIterable(["a", "b"]);
const asyncIterator = asyncIterable[Symbol.asyncIterator]();

asyncIterator
  .next()
  .then((iterResult1) => {
    console.log(iterResult1); // { value: 'a', done: false }
    return asyncIterator.next();
  })
  .then((iterResult2) => {
    console.log(iterResult2); // { value: 'b', done: false }
    return asyncIterator.next();
  })
  .then((iterResult3) => {
    console.log(iterResult3); // { value: undefined, done: true }
  });

In the above code, the asynchronous iterator actually returned the value twice. The first time it is called, it returns a Promise object; when the Promise object resolve, it returns an object representing the current data member information. That is to say, the final behavior of the asynchronous traverser and the synchronous traverser is the same, but the Promise object is returned first as an intermediary.

Due to the next method of the asynchronous iterator, a Promise object is returned. Therefore, it can be placed after the await command.

async function f() {
  const asyncIterable = createAsyncIterable(["a", "b"]);
  const asyncIterator = asyncIterable[Symbol.asyncIterator]();
  console.log(await asyncIterator.next());
  // { value: 'a', done: false }
  console.log(await asyncIterator.next());
  // { value: 'b', done: false }
  console.log(await asyncIterator.next());
  // { value: undefined, done: true }
}

In the above code, after the next method is processed with await, there is no need to use the then method. The whole process is very close to synchronization.

Note that the next method of the asynchronous iterator can be called continuously, and it is not necessary to wait until the Promise object resolve generated in the previous step to be called later. In this case, the next method will accumulate and run automatically in the order of each step. The following is an example, put all the next methods in the Promise.all method.

const asyncIterable = createAsyncIterable(["a", "b"]);
const asyncIterator = asyncIterable[Symbol.asyncIterator]();
const [{ value: v1 }, { value: v2 }] = await Promise.all([
  asyncIterator.next(),
  asyncIterator.next(),
]);

console.log(v1, v2); // a b

Another usage is to call all the next methods at once, and then await the last step.

async function runner() {
  const writer = openFile("someFile.txt");
  writer.next("hello");
  writer.next("world");
  await writer.return();
}

runner();

for await...of

As mentioned earlier, the for...of loop is used to traverse the synchronized Iterator interface. The newly introduced for await...of loop is used to traverse the asynchronous Iterator interface.

async function f() {
  for await (const x of createAsyncIterable(["a", "b"])) {
    console.log(x);
  }
}
// a
// b

In the above code, createAsyncIterable() returns an object with an asynchronous iterable interface, and the for...of loop automatically calls the next method of this object's asynchronous iterable to get a Promise object. await is used to process this Promise object. Once resolve, the obtained value (x) is passed into the loop body of for...of.

One use of the for await...of loop is to deploy an asynchronous interface for the asyncIterable operation, which can be directly put into this loop.

let body = "";

async function f() {
  for await (const data of req) body += data;
  const parsed = JSON.parse(body);
  console.log("got", parsed);
}

In the above code, req is an asyncIterable object used to read data asynchronously. As you can see, after using the for await...of loop, the code will be very concise.

If the Promise object returned by the next method is reject, for await...of will report an error and use try...catch to catch it.

async function () {
  try {
    for await (const x of createRejectingIterable()) {
      console.log(x);
    }
  } catch (e) {
    console.error (e);
  }
}

Note that the for await...of loop can also be used for synchronous iterators.

(async function () {
  for await (const x of ["a", "b"]) {
    console.log(x);
  }
})();
// a
// b

Node v10 supports asynchronous iterators, and Stream deploys this interface. The following is the difference between the traditional way of reading files and the way of writing asynchronous traversers.

// Traditional writing
function main(inputFilePath) {
  const readStream = fs.createReadStream(inputFilePath, {
    encoding: "utf8",
    highWaterMark: 1024,
  });
  readStream.on("data", (chunk) => {
    console.log(">>> " + chunk);
  });
  readStream.on("end", () => {
    console.log("### DONE ###");
  });
}

// Asynchronous traverser writing
async function main(inputFilePath) {
  const readStream = fs.createReadStream(inputFilePath, {
    encoding: "utf8",
    highWaterMark: 1024,
  });

  for await (const chunk of readStream) {
    console.log(">>> " + chunk);
  }
  console.log("### DONE ###");
}

Asynchronous Generator function

Just as the Generator function returns a synchronous iterator object, the purpose of the asynchronous Generator function is to return an asynchronous iterator object.

Syntactically, the asynchronous generator function is a combination of the async function and the generator function.

async function* gen() {
  yield "hello";
}
const genObj = gen();
genObj.next().then((x) => console.log(x));
// { value: 'hello', done: false }

In the above code, gen is an asynchronous Generator function, which returns an asynchronous Iterator object after execution. Call the next method on this object and return a Promise object.

One of the design goals of the asynchronous traverser is to be able to use the same set of interfaces when the Generator function processes synchronous and asynchronous operations.

// Synchronous Generator function
function* map(iterable, func) {
  const iter = iterable[Symbol.iterator]();
  while (true) {
    const { value, done } = iter.next();
    if (done) break;
    yield func(value);
  }
}

// Asynchronous Generator function
async function* map(iterable, func) {
  const iter = iterable[Symbol.asyncIterator]();
  while (true) {
    const { value, done } = await iter.next();
    if (done) break;
    yield func(value);
  }
}

In the above code, map is a Generator function, the first parameter is the traversable object iterable, and the second parameter is a callback function func. The function of map is to use func to process the value returned at each step of iterable. There are two versions of map above. The first one handles the synchronous traverser, and the latter handles the asynchronous traverser. You can see that the two versions are basically written in the same way.

The following is another example of an asynchronous Generator function.

async function* readLines(path) {
  let file = await fileOpen(path);

  try {
    while (!file.EOF) {
      yield await file.readLine();
    }
  } finally {
    await file.close();
  }
}

In the above code, the asynchronous operation is marked with the keyword await, that is, the operation after await should return a Promise object. Wherever the yield keyword is used, it is the place where the next method stops, and the value of the expression after it (ie the value of await file.readLine()) will be returned as the object of next() This is consistent with the synchronous Generator function.

Inside the asynchronous Generator function, the await and yield commands can be used at the same time. It can be understood that the await command is used to input the value generated by an external operation into the function, and the yield command is used to output the value inside the function.

The usage of the asynchronous Generator function defined in the above code is as follows.

(async function () {
  for await (const line of readLines(filePath)) {
    console.log(line);
  }
})();

Asynchronous Generator function can be used in combination with for await...of loop.

async function* prefixLines(asyncIterable) {
  for await (const line of asyncIterable) {
    yield "> " + line;
  }
}

The return value of the asynchronous Generator function is an asynchronous Iterator, that is, every time its next method is called, a Promise object is returned, that is to say, after the yield command, it should be a Promise object. If like the example above, the yield command is followed by a string, it will be automatically wrapped into a Promise object.

function fetchRandom() {
  const url =
    "https://www.random.org/decimal-fractions/" +
    "?num=1&dec=10&col=1&format=plain&rnd=new";
  return fetch(url);
}

async function* asyncGenerator() {
  console.log("Start");
  const result = await fetchRandom(); // (A)
  yield "Result: " + (await result.text()); // (B)
  console.log("Done");
}

const ag = asyncGenerator();
ag.next().then(({ value, done }) => {
  console.log(value);
});

In the above code, ag is the asynchronous iterator object returned by the asyncGenerator function. After calling ag.next(), the execution sequence of the above code is as follows.

  1. ag.next() immediately returns a Promise object.
  2. The asyncGenerator function starts to execute, and Start is printed out.
  3. The await command returns a Promise object, and the asyncGenerator function stops here.
  4. When A becomes fulfilled, the generated value is put into the result variable, and the asyncGenerator function continues to execute.
  5. The function is suspended at yield at B. Once the yield command gets the value, the Promise object returned by ag.next() becomes the fulfilled state.
  6. The callback function specified by the then method after ag.next() starts to execute. The parameter of this callback function is an object {value, done}, where the value of value is the value of the expression after the yield command, and the value of done is false.

The functions of the two lines A and B are similar to the following code.

return new Promise((resolve, reject) => {
  fetchRandom()
    .then((result) => result.text())
    .then((result) => {
      resolve({
        value: "Result: " + result,
        done: false,
      });
    });
});

If the asynchronous Generator function throws an error, it will cause the state of the Promise object to become reject, and then the thrown error will be caught by the catch method.

async function* asyncGenerator() {
  throw new Error("Problem!");
}

asyncGenerator()
  .next()
  .catch((err) => console.log(err)); // Error: Problem!

Note that the normal async function returns a Promise object, while the asynchronous Generator function returns an asynchronous Iterator object. It can be understood that the async function and the asynchronous generator function are two methods of encapsulating asynchronous operations, and both are used to achieve the same purpose. The difference is that the former has its own executor, the latter is executed by for await...of, or you can write your own executor. The following is an executor of an asynchronous Generator function.

async function takeAsync(asyncIterable, count = Infinity) {
  const result = [];
  const iterator = asyncIterable[Symbol.asyncIterator]();
  while (result.length < count) {
    const { value, done } = await iterator.next();
    if (done) break;
    result.push(value);
  }
  return result;
}

In the above code, the asynchronous iterator generated by the asynchronous Generator function will be automatically executed through the while loop. When the await iterator.next() is completed, it will enter the next cycle. Once the done property becomes true, it will break out of the loop, and the asynchronous iterator will end.

The following is an example of the use of this automatic actuator.

async function f() {
  async function* gen() {
    yield "a";
    yield "b";
    yield "c";
  }

  return await takeAsync(gen());
}

f().then(function (result) {
  console.log(result); // ['a', 'b', 'c']
});

After the emergence of the asynchronous Generator function, JavaScript has four functional forms: ordinary function, async function, generator function and asynchronous generator function. Please pay attention to distinguish the difference between each function. Basically, if it is a series of asynchronous operations performed in sequence (such as reading a file, then writing new content, and then saving it to the hard disk), you can use the async function; if it is a series of asynchronous operations that produce the same data structure (such as a line Read the file one line), you can use the asynchronous Generator function.

Asynchronous Generator functions can also receive external data through the parameters of the next method.

const writer = openFile("someFile.txt");
writer.next("hello"); // execute immediately
writer.next("world"); // execute immediately
await writer.return(); // Wait for the end of writing

In the above code, openFile is an asynchronous Generator function. The parameters of the next method are used to pass data to the operations inside the function. The next method is executed synchronously each time, and the last await command is used to wait for the end of the entire write operation.

Finally, for synchronous data structures, asynchronous Generator functions can also be used.

async function* createAsyncIterable(syncIterable) {
  for (const elem of syncIterable) {
    yield element;
  }
}

In the above code, since there is no asynchronous operation, the await keyword is not used.

yield* statement

The yield* statement can also be followed by an asynchronous iterator.

async function* gen1() {
  yield "a";
  yield "b";
  return 2;
}

async function* gen2() {
  // result will eventually be equal to 2
  const result = yield* gen1();
}

In the above code, the final value of the result variable in the gen2 function is 2.

Like the synchronous Generator function, the for await...of loop will expand the yield*.

(async function () {
  for await (const x of gen2()) {
    console.log(x);
  }
})();
// a
// b