Spawn and Manage Child Processes and Node.js Streams with execa

Instructor: [0:00] Before you use any libraries to manage child processes, you should be comfortable with the Node child process API and, by extension, Node streams API. I have here a stream file, and in it, I'm destructuring the spawn function from the Node child process module.

[0:20] I'm using it to spawn a simple command. Here, I'm using the built-in Unix LS command, and I'm passing in some flags. Let's run it. Nothing seems to be output to the screen, and that's expected, because by default, the behavior of the spawn function is to swallow all the output that is generated by the underlying command.

[0:44] Now, this child variable is actually a Node stream. Streams have one input, and there are three input-output streams associated with a Node stream. There's one standard in stream and two output streams, one for standard out, where everything is just standard logging, and one for standard error, where everything that is pertaining to an error is logged.

[1:08] You can pipe streams from one to the other. Here, I'm piping the parent stream, process.stdin, into the child standard in stream, in case I need to pass in anything to the child. This is a very use case, and most likely, you won't use this.

[1:23] The more common one is to care about the child standard out stream being piped to the parent standard out, and the child standard error stream being piped to the parent standard error. If I run this code again, I can now see the contents of what's being run by the underlying LS command.

[1:42] Because this piping behavior is such common use case, it's actually built into the platform as the third parameter to the spawn function. Just supply an object that has the standard IO key. It's got these shortcuts.

[1:57] For example, if I wanted to ignore the standard in, I just pass a string called ignore. I can just pass the process.stdout or process.stderror for piping for each of these standard IO streams for that child stream.

[2:08] For now, with just these few lines of code, if I run this stream again, I can run that child process internally, output everything from that child process, and then terminate when it's done. Piping is not the only thing you can do with streams.

[2:21] You can actually take the streams and do creative things with them. For example, you can use that standard out stream and listen to specific events. For example, if data's coming through that stream, you can listen to the data event, take those specific chunks of data, and transform them however you like.

[2:42] This is actually how write filestream and other stream APIs work under the hood. You're doing something on a chunk-by-chunk basis, so you're not using a lot of memory to do so. This is especially important for large files.

[2:56] You can also listen to child processes and close when they close. For example, if the child process closes, you can also do some other action with that information. Now, when I run this code again, I'm also logging out, transforming that stream chunk-by-chunk.

[3:12] For example, there's only one chunk here, but I'm adding that standard out colon to that first chunk and logging it out. Then, when the child process exits, closes, it gets an exit code, and it exits with that code when I log it out.

[3:27] Exiting with a code of zero means that there are no errors that happened. Typically, if there's any fatal error, you can exit a code of one. There are other standards for other exit codes, if you care to look them up.

[3:39] Now, you might be asking yourself, if the API is so easy to use, why do we need any libraries at all? I think is a very fair question, until you consider Windows and Linux users, or if you're a Windows user, you might want to consider Mac users.

[3:56] There are a bunch of different libraries that help you do cross-platform shell command execution. There's Shell JS and Cross-Spawn, but probably by far, the biggest and most widely used is Execa. They all have very similar APIs, because they're thin wrappers over the underlying child process API.

[4:15] Execa, in particular, patches the child process methods with some other niceties, like a promise-based interface, garbage collection of processes, and interleaving outputs very nicely, so there's no surprise that this is the standard in Node.js.

[4:30] For an exercise, going to try and use Execa to run an install of our templated project, right after we copy it out. Definitely, feel free to the read the errors for whatever use case you have in mind. I'm going to run yarn add execa in my CLI.

[4:48] Add execa, and this adds it into the mycli workspace. Inside of my init command, I'm going to import Execa. I guess I can just const execa equals require execa. After I'm done copying it out, instead of logging out everything I copied out, I can just paste in my async code, just turn this into an async function, and run install.

[5:14] You probably also want to change directories before you do so, so change to dir. I'm going to chdir to the out dir. I think that would be it. I'm going to run yarn mycli init, supply a project name, My New App. It's going to hopefully cd into that project, and it's going to install the dependencies accordingly.

[5:41] That's a single example of going the extra mile for your user by, after doing some sort of business logic, actually spinning up any particular installation code for them and logging it out.

[5:54] As long as you're familiar with the command line, you can run any other command line as a child process and manage it, the manage the input and output through the stream API. That's a very, very powerful way to manipulate multiple streams at once.

[6:08] For example, you could very easily write a wrapper CLI over an existing CLI that you use, adding or removing capabilities to your liking. One final nuance to take care of is that, in JavaScript land, there may be people with very strong preferences over Yarn or NPM, or they may not even have Yarn installed.

[6:28] You should be sensitive to their preferences. Fortunately, it's already been written for you in a package conveniently called Yarn or NPM. There are a number of things to think about checking, and these are very sensible defaults, so you might as well use them.

[6:44] We're going to try to use them in a Yarn or NPM library. I'm going to install Yarn workspace, mycli add yarnornpm. This is itself a very thin wrapper over the child process API. We don't need Execa anymore if we just use this specialized library for Yarn or NPM.

[7:04] We can take this spawn, which is a proxy for either Yarn or NPM, and we can just sub that in there, and no longer need to specify Yarn or NPM. That seems cleaner all-around. Basically, there are a bunch of similar libraries, all addressing different use cases around cross-platform stream management and child process management.

[7:25] If you're firm on the fundamentals of child process management, then you can always figure out your issues.