This Lesson is for Members

Subscribe today and get access to all lessons! Plus direct HD download for offline use, enhances transcripts, member comment forums, and iTunes "podcast" RSS feed. Level up your skills now!

Unlock This Lesson

Already subscribed? Sign In

Autoplay

    Transform Stream Data With a Node Transform Stream

    Matt RossMatt Ross

    Node's Stream API provides a stream.Transform class for easily and intuitively transforming incoming data. Piping streams through this class preserves natively provided back-pressure and high watermarking, while also keeping you working in the same paradigm (streams!) from start to finish.

    javascriptJavaScript
    nodeNode.js
    Code

    Code

    Become a Member to view code

    You must be a Member to view code

    Access all courses and lessons, track your progress, gain confidence and expertise.

    Become a Member
    and unlock code for this lesson
    Transcript

    Transcript

    Instructor: 00:01 Here I have some code that sets up a read stream, a read from a data.JSON file that I prepared ahead of time. If that stream encounters an error, we'll log that error.

    00:10 Once that stream becomes readable, we'll read each incoming chunk off the stream and log the chunk value. If you look to the right, you'll see that we're receiving a buffer. That buffer is the JSON data being read from the file.

    00:24 Now, I want to convert that buffer into a JavaScript object, so I'm going to use a transform stream. I'm going to create a class to object that extends stream.transform. Any transform stream must implement transform method.

    00:42 The arguments to that method are an incoming chunk, the encoding of the chunk or stream, and a callback. We can then call the callback with any possible error and apply some transformation to the chunk if we're sending it out.

    00:58 Then, I can just pipe that incoming stream to a new instance of to object. That new instance is going to have object mode set to true. That just means that any outcoming value is expected to be an object, rather than a string or a buffer. As you can see, on the right, we're getting the JSON out from data.JSON, our email and our password.

    01:21 Now, if I want to create additional transform streams, I'll need to implement a new class that extends from transform, or I can pass my transformation function to an option in the constructor. Even better than that, I can use a helper library, like through, provided by Mississippi.

    01:40 Now, I can create a map function that takes some function as an argument and returns a through stream, with object mode set to true. The first argument to this function is going to be a transform function, just like above.

    01:54 The arguments will be chunk, encoding, and callback. Just as above, we will call that callback with a potential error, and the result of applying our function to the chunk. Now, rather than pipe to some new object, I can say map. I take my chunk argument and apply my transformation.

    02:18 I can clean this up a little bit. Since map is easy to implement with any transformation, I can easily extend this. I can take my chunk and, just as an example, I can return just the email field.

    Discuss

    Discuss