Parse Image Files with the Google Vision API and NodeJS

Instructor: [00:00] We start with a basic Node.js Express-based application. We've got an additional package here called serve static. We're using this to serve our indexed HTML page.

[00:11] The next thing we're going to include is a package called Multer. Multer makes our life a lot easier for dealing with file uploads. We're going to then use the Google Cloud library. Specifically, we only need the Vision part. We will include that here.

[00:24] When you set up the Google Vision API on the Google Console, you would have been given option to set up a service account which in turn would have created a JSON file. That JSON file I've referenced here and is in the root of my project. It looks like this.

[00:43] The next thing we're going to do is add a way for our Express server to receive the file. We'll create a new root. This root will be a post. As it's dealing with color, we will call it color.

[00:56] Next, we're going to use the Multer middleware. This is going to take the file from the file upload. It's expecting a single file. We're going to use the file field name. It will put the file on the request object, so we can now deal with it.

[01:13] Next, we're going to use the Google Vision API client. We're going to get the image properties. We're going to pass it the file that has been saved by Multer to the file system. We will then take the result and pipe it straight through to the response. If we get an error, we will catch it and send that through with a status of 500.

[01:42] Let's now take a look at our HTML page which we're going to use to allow people to upload files they send to the Google Vision API for our server. We're using a standard Bootstrap layout. I've referenced the Bootstrap stylesheet here. We're also going to be using the Axios library which is useful for dealing with Ajax requests in the browser.

[02:06] Looking at the body, we have a standard container and row. Then we have a file upload control within our form here.

[02:15] We're only going to allow images to be selected. We restrict to JPEGs, GIFs, & PNGs on the file upload control. We have a button. Then we have a div which we'll populate with a preview of the image before it's uploaded to the Google Vision API. Then the result that comes back will be pushed into this output div here.

[02:34] Let's have a look what that looks like in the browser. Here we fired our Node server up on the local host 8088. It's opened the index file which currently looks like this with our file picker and a button.

[02:46] Let's now increment the file preview first. The first thing we need to do is add a script block to our page. I got to wrap my code inside a medially invoked function. The first thing we're going to do is add an event handler to the file upload control. We're going to look for an on change event. Then we're going to call a separate function which will then do the preview.

[03:09] This preview function, the first thing it's going to do is get hold of the file upload control. It's then going to check to make sure that there is actually a file being selected. If there has, it's going to create a new file reader object, add an event handler for that file read log object for the on load event.

[03:28] It's going to create a new image in the DOM. It's going to set the source to the result of the file reader. It's going to set a task which is used by Bootstrap. It's then going to get hold of our image holder, preview div. Finally, it's going to blank the image holder in case it's got on previously and then append the image element to it.

[03:49] Now we just need to trigger the on load event. Using the reader's data URL method on the reader object, we'll feed it the files from the input. In this case, it's an array of files. But we only need the first one. That is it.

[04:03] Let's take a look. We have our Node server running. We're going to try and select a file and see if it triggers the on event handler. Nothing happens. Going back to the script, I've noticed that I've missed off the brackets so the self-executing function actually executes. We'll go back to the browser. Try it again. Choose a file. Refresh the page. Choose a file. It's triggered the preview.

[04:36] Back into our index HTML file, let's do something with that image. First thing we're going to do is wire up an event handler to our get color button. We need to make sure that if we've already got our result, we need to just clear it out of our output div. We'll just use an HTML here just to make things a bit easier.

[04:54] Next, we need to get the form data. The easiest way to do that is use the form data object. Then we're going to append our file to the form data object. We're then going to use the Axios library to do our post to our Node server. The root we want is color. That's what we defined previously. We pass it the form data object. Axios returns a promise. We'll deal with that now.

[05:23] We're just going to log out the result for a minute. We're also going to deal with the errors if there are any. Again, just for a second, we're going to log them out.

[05:34] Let's see it in action. We've got our server running. I've refreshed the index page. Let's select a file. We'll go for the colorful balls again. I'll click the get colors button. We should see a result appear in the console from the Google API. Here it is.

[05:48] We actually selected the image properties annotation which is the one we want. Let's have a look at that. We've got an array of dominant colors which has individual color objects. Let's see if I can scroll this up.

[06:03] Then we have a individual color object itself which is consisting of red, green, and blue. Its alpha channel, which in this case because it's not transparent, is null. We also get a pixel fraction and a score.

[06:15] Let's try and make something more useful with the result. First of all, let's get rid of our console logging here. Next, let's get the higher level objects from our result. Then we're going to grab the colors array from that property's object.

[06:30] What we're going to do now is sort the colors so that the most prominent color is at the top of the list. We're going to use an in-line arrow function. Next, we're going to loop over our sorted array.

[06:49] For each color object, we will output a container div which will have the background color of that color object. We're going to use another function to actually build out those divs. This new function that's going to create these divs for us we're going to call build color box. It's been passed the color object.

[07:18] First, we'll create a new container element. We're going to set the background color to be the RGB values from our color object. For this, we're going to use ES6 backticks. This will allow us to use templates and insert our JavaScript variable directly in the string.

[07:38] We're also going to add a CSS class. This will give our containers some height and width. While we're at it, we might as well display inside the container the actual pixel fractions and the RBG colors.

[07:55] Finally, we return the new div. Let's have a look and see how that works. We have our server running on 8088. We're going to select our first file. Let's go for the color balls again. Hit get colors. Comes back from the Google Vision API. We created our colored squares based on the RGB colors. We've sorted them based on how prominent the colors are.

[08:22] Let's try a different file. Let's choose the shark picture. We'll expect blue to come back as a predominant color here. It has. That's working fine.