Build Monitoring into your API Server
There are a lot of great monitoring tools available for Node.js. It is also incredibly easy to build monitoring into your application. In this lesson, we will add precision monitoring to the function calls in the Todo API server, log it to the console, and persist those results to Elasticsearch where they can later be reviewed, graphed, and analyzed.
Instructor: [0:01] I'm going to use process.hrtime as the timing function to measure or monitor our function calls within our API.
[0:12] Process.hrtime returns a relative time based on whenever the service was actually started and it's an array consisting of two numbers. The first number is the number of seconds, and the second number is microseconds.
[0:27] Now, the cool part about this is that when you recall process.hrtime without passing anything into it, it returns the relative timestamp. The second time you call it, though, if you provide a process.hrtime object to it, it returns the delta.
[0:47] You can see in this instance that it was just a little over 19 seconds from the time that I defined variable A until I called process.hrtime with the variable A. With that information, here's what we're going to do.
[1:00] In my helper's folder, I'm going to create a new file and I'm going to call it monitor.js. Inside, I'm going to define a function called Monitor, and it's going to have a couple of parameters. There's going to be a start time and a tag. Both of those are optional.
[1:16] Before I forget, I'm going to export this. Here's what we're going to do. If we pass in a start time, then that means we're giving it the start times we want to calculate the duration. We'll say our end time is equal to process.hrtime.
[1:36] Then provide the start time to get that differential. Then we're going to return an integer where we take the seconds from the start time, multiply that by 1,000 to get the number of milliseconds. We'll add that to the micro seconds from the end time that's divided by one million.
[1:59] Those two summed together will give us the number of milliseconds that were elapsed between the start and end time. Otherwise, if we don't provide any parameters, we're just going to return the process.hrtime, which will later be used as the start time.
[2:18] Actually, we don't want to return this. I'm going to call it duration. Then we want to...we'll just write out to the log the duration and the tag that we specified. That looks good. If I go in to get all ToDos, I can bring in the monitor with a require statement.
[2:39] When this function loads, I can save our start is equal to monitor. Then down here at the end after we've sent our response back, I can call monitor again providing the start parameters, and that should log out. It's actually going to take two parameters. We'll say, start and then get all ToDos.
[3:03] That should log out the duration, and the fact that we ran our get all ToDos.
[3:09] Now by looking at the server logs or the console log, if I hit that endpoint a bunch of times, you can see that each time I hit it, it writes out to the console, the duration, which endpoint I hit, and the amount of time that it took.
[3:25] With that, I can now go back through all of the other functions -- the Add ToDo, Delete, Find, and Update. I can add the same thing, a start at the beginning of the function call. Then at the end, I can call the monitor statement again, passing that start time in.
[3:40] Then provide a tag saying which function was called. Then from watching our console logs, we'll be able to see how everything is performing. I'm just going to go through and do this again through the other functions here. There's no need for you to really watch me do that.
[3:56] I've added the monitor function to all of the different endpoints. If I hit multiple endpoints at the same time, you can see in the console logs here that it logs all of them out along with the time.
[4:09] Let's say we wanted to take that one step further and do something a little more persistent with our monitoring. Well, we're already storing our ToDo data in Elasticsearch. Let's go ahead and bring that Elasticsearch client back in here.
[4:23] Then after we write out to the console, we'll do a client.create, and we'll specify an index. This time, we're going to specify an index called Monitoring. Then the type we're going to name after our application, which is ToDo-API, our ID. I'm just going to do a timestamp.
[4:44] That should be enough precision for what we're working on. Then in the body of the document we store in Elasticsearch, I'm going to write a duration field that's going to be equal to the duration we calculated, and a tag, which will be our tag for the application or the function call.
[5:07] We can save that. I can run a curl command against my ToDo application. Response gets returned. In the right hand pane there, you can see that it wrote out the duration to the console log. Now I can also take a look at Elasticsearch. Let's clear that. We'll take a look at Elasticsearch. It's running on Port 9200.
[5:28] I'll use the cat interface and show the indices stored. There's a new index called Monitoring there. If we query that index, if you query it for everything, you can see there's several operations there that I've already logged.
[5:47] Each one contains a document with the duration and the tag showing the function name, the ID number we timestamped so they can be listed sequentially. If you're using something like Kibana, you can use this to build a really cool chart that shows the performance of your API server.
[6:05] Again, I did this with Elasticsearch because that's what we were already using. You could do the same technique to store the data into any data store that you were using or had access to. There's one last thing I want to show you here that may be of interest to you.
[6:20] One of the things you can do is, you could say if process.env.node env is development, that's when you want to log out to your console. You could even move your client, the Elasticsearch store, up into here as well.
[6:39] What this does for you is it allows you to have all of your monitoring code built in already. If you launch in development mode, you start getting all of your development statements.
[6:49] Then when you run into production, you don't get the console log output because it's a production environment.
[6:55] That code's already there, which gives you the ability to just use the same code base so you're not trying to insert monitoring after there's already a problem that you need to be researching.