Transform Stream Data With a Node Transform Stream

Instructor: [00:01] Here I have some code that sets up a read stream, a read from a data.JSON file that I prepared ahead of time. If that stream encounters an error, we'll log that error.

[00:10] Once that stream becomes readable, we'll read each incoming chunk off the stream and log the chunk value. If you look to the right, you'll see that we're receiving a buffer. That buffer is the JSON data being read from the file.

[00:24] Now, I want to convert that buffer into a JavaScript object, so I'm going to use a transform stream. I'm going to create a class to object that extends stream.transform. Any transform stream must implement transform method.

[00:42] The arguments to that method are an incoming chunk, the encoding of the chunk or stream, and a callback. We can then call the callback with any possible error and apply some transformation to the chunk if we're sending it out.

[00:58] Then, I can just pipe that incoming stream to a new instance of to object. That new instance is going to have object mode set to true. That just means that any outcoming value is expected to be an object, rather than a string or a buffer. As you can see, on the right, we're getting the JSON out from data.JSON, our email and our password.

[01:21] Now, if I want to create additional transform streams, I'll need to implement a new class that extends from transform, or I can pass my transformation function to an option in the constructor. Even better than that, I can use a helper library, like through, provided by Mississippi.

[01:40] Now, I can create a map function that takes some function as an argument and returns a through stream, with object mode set to true. The first argument to this function is going to be a transform function, just like above.

[01:54] The arguments will be chunk, encoding, and callback. Just as above, we will call that callback with a potential error, and the result of applying our function to the chunk. Now, rather than pipe to some new object, I can say map. I take my chunk argument and apply my transformation.

[02:18] I can clean this up a little bit. Since map is easy to implement with any transformation, I can easily extend this. I can take my chunk and, just as an example, I can return just the email field.