Transform and Return JSON Data From an API in Cloudflare Workers
In this lesson, we'll parse the incoming data from Unsplash's API and transform it for our frontend client. Because Cloudflare Workers is serving as "middleware" - in-between the Unsplash API and our frontend client we'll build shortly - we can use it to format or simplify data coming through it that our client doesn't care about. We'll also use the Response class to return a JSON response from our API. JSON responses need to be formatted in a specific way to be able to be parsed by clients, so we'll look at the various pieces of a Response (body and headers) and how they work together to represent a JSON response.
Instructor: [0:00] With our basic API request to Unsplash being returned from our worker serverless API, we can now customize it to be more interesting.
[0:08] Earlier we looked at the query that was coming from a request, which was a little piece of data that we sent along with our request to say, let's search for this value.
[0:16] We can use that to search for photos specifically in Unsplash's API, like how you would go to the search bar on Unsplash and search. Similar to if I came here into Unsplash's search and just search for something like guitar.
[0:28] To do this we'll use Unsplash's search and point, which is just that /search/photos. This takes a parameter called query which is just a search term or terms, and then returns a list of results.
[0:40] You can see here down in the response I get this object back with a key called results, which is just an array of different photos that it found that matches that search query that I made. To use this I'm going to take this line of code that we wrote earlier and I'm going to paste it up here at the top.
[0:55] Just getting the query value out of the request.JSON. I'm also going to update this URL here. Instead of going to /photos, I'll update this to search/photos. Then I'll turn this into an ES6 template literal and then parse in the query parameter, query, set to the value, query.
[1:12] Now that the API requests we're making using fetch has been updated, we'll go back through this code and make sure that yes, const data equals await.resp.JSON.
[1:20] We're getting that JSON data back from that request, and then we're stringifying it into a JSON string that's being returned with the correct content type set to application/JSON. Back in our terminal, I still have Wrangler dev running.
[1:35] In another window I can come here and I can run this exact same query request that we meet earlier. Just a refresher, it sets the header to content type application JSON. Then sends this data query set to guitar to my local host that is serving the workers function.
[1:49] You can see that there is a ton of data that comes back, but if I take a look here I can see there's things like, type search, title, electric guitar, or if I scroll up randomly more, you can see that there is this bio here that says this person is into travel, music, and photography.
[2:05] It appears that this stuff is guitar related, but there's a lot of extra data in here that we don't need. To solve that we're going to take the API response that we're getting back from Unsplash and we're going to transform it into something that is easier to parse.
[2:19] Down underneath data, we're going to set up a new variable called images. Images is going to map through data.results. You'll remember that back here in the API response for /search/photos, Unsplash's API returns an array of results, which is a list of these images.
[2:39] In our code we're going to say, map through data.results, where map is a function that allows you to call some function for every single thing inside of the array. I'm going to parse in a function here. I'm going to say, image.
[2:52] Then I'm going to return an object back from this map function. I'm saying, for every image inside of data.results, call this function which returns this object. This object is going to be a small sampling of the data that's inside every image.
[3:08] Things like ID, width, height, likes. All of that stuff is set, but in this case we want ID. We want the image itself which is available here under URLs. Any of these raw, full, regular, these are different sized images of the original image.
[3:25] In our case we'll probably use small, because the file size is manageable. Then, we'll also use links.html which is the actual URL for viewing the Unsplash image in browser. For instance, if I came here and looked at say this image which is an Unsplash guitar image, you can see there's this unsplash.com/photos here and that will be the final URL for this image.
[3:48] Back in our code, we want to create a smaller object based on each of these images. I'll say, ID, which is image.ID. Image which is image.url.small and link which is image.links.html.
[4:06] We've just taken these three values out of that larger set of data and turned images into basically a transformed subset of that larger API response. Now that we've set up these images, we can come down here to return new response.
[4:20] Instead of parsing the entire data object here, I'm going to return images. It's going to take that smaller array of data and it's going to JSON-stringify it into a big string of data, returning it with this header set to content type application/JSON.
[4:36] Now that I'm using this query that I defined earlier, I can delete this other response that I've been returning. If I open up my terminal and run this again you can see it's still a big amount of data. In order to make this data easier to look at, I highly recommend a tool called jq, which is a command line tool for processing and viewing JSON data.
[4:57] You can install it by going to the download page here and it has precompiled stuff for Windows, for Mac, for Linux. Once you have it installed, you can come in here, and you can see I still have the curl request I've been making, but what I'm going to do is use the pipe operator which will take all of the output from that command. I'll parse it into jq. You can see the format here is an array of images. This is the beginning of the array.
[5:25] For each image here, I have, ID, image and link. I can click on one of these URLs, and you can see it returns the direct image here, which appears to be of a ukulele, as well as the actual in-browser ukulele image link here.
[5:44] Instead of returning big, big amounts of JSON data from an external API, one thing that Workers is great at is being able take incoming data, parse it and transform it and filter it as a middleman between larger APIs and whatever use case you're trying to build.
[6:00] In our case, we took this huge amount of data, and we filtered it and transformed it into a smaller amount of data that returns an ID, an image and a link for the pieces of data that we care about.