Make HTTP Requests in Bash with `curl`

Cameron Nokes
InstructorCameron Nokes
Share this video with your friends

Social Share Links

Send Tweet
Published 6 years ago
Updated 5 years ago

curl is how you make HTTP requests from the command line. It's awesome for testing out APIs because you have full control over all the headers that are sent, and you don't have to interact with a UI to send a request.

We’ll learn how to make requests and change the HTTP method, set headers, and send JSON and url encoded form data. We'll also see how to increase readability by splitting long requests into multiple lines and formatting JSON responses with jsome, a node module.

Instructor: [00:00] To make a get request to a website, we'll just do curl and then we'll pass the URL here as a parameter. This is sort of curl's most basic form. Just by default, it does a get request and it takes the response body, which in this case is HTML, and it returns it to your standard out. Let's see what curling a JSON API might look like.

[00:22] This is the Star Wars API. Let's see what that returns. That just returns the JSON body right to my standard output. Sometimes it can be useful to inspect the headers of the response, as well. To do that, we're going to pass the I flag. I stands for include, meaning include headers. Let's call the same Star Wars API. Let's try that and see what happens.

[00:46] We were getting a 301. We can see that right there. If we look down here at the location header, it's telling us that it's trying to redirect us. Curl by default does not follow redirects like a browser might. If we make the same request again but we add this trailing slash, it works. We could also tell curl to follow redirects.

[01:07] If we go back here, let's take off this slash so that it will redirect. If we add the capital L flag, that will tell it to follow redirects. If we look at our output, we can see we get the 301 and then there's the 200 and the JSON response that we're expecting.

[01:24] Let's just into how to build more complicated HTTP requests. I have a server running locally in a separate tab here. The code for this server will be available in the description of this video. It's running locally at port 3000. Oftentimes, we need to pass an authorization header or a cookie to access a protected resource. We do that with the H flag, H is for header. Let's try that out.

[01:49] Do curl H. When doing a header flag, be sure to wrap it in quotes. Then this isn't a real API, so the token it's looking for is 123. Now, let's pass our URL, which is localhost:3000. This end point is expecting this authorization header. This will do a get request and return some fake blog post. We can see that worked.

[02:16] Let's see how we change the request method of our HTTP request and see how we post JSON. We use the capital X flag to change the method, so we do a post. Then let's pass some headers. We'll do content type application JSON. Then to pass a request body, we're going to use a lower case d flag. This is where we build our JSON string. I'm going to use single quotes because we need double quotes in JSON.

[02:44] I'm just going to post a new blog post to my collection there. I'll give it a title, author, and then let's pass our URL. Great, that worked. To verify it, we can just quickly do a get request. If they're both there, there's the new blog post.

[03:01] Posting a URL in coded form is similar. Data slash URL encode. Then we're going to pass the key. We'll do title and then the value here. Then to pass another field in the form, we do the same thing again, but we'll do author and then we'll pass our URL. Great, we can see that worked because it returned a 201 HTTP status.

[03:27] When you're building large curls, it can be useful visually to slit it into multiple lines. Let's do a put request on one of the blog posts that we just posted. Do curl and say I to include the response headers. Set the method to put. To start going to another line, we just pass the backslash character and hit enter and it gives us another line to do stuff on.

[03:50] Let's do JSON. Do D and then let's put our JSON string here. We'll just change the title. Then we want another line, we'll do a backslash again and hit enter. Let's pass the content type header. Another line. Let's pass our URL. We'll do this on the second blog post that we created. It looks like they work. We got a 200 back from our API.

[04:14] In dealing with large output, it can be useful to dump the response in a file. We can do that with the O flag. If we do curl and I'll say include the response headers, follow redirects, and let's just do Google.com. Then we'll say dash O, output that to a file that we'll call Google.txt. It'll create that file on the fly.

[04:36] We don't get the normal output because it all went into that text file. Let's inspect that text file. You can see there's HTTP header information, we've got a 301 and then a 200 here. Here's the HTML response from Google.com.

[04:58] When dealing with large JSON responses, it can sometimes be hard to read in just the default formatting. Like we saw earlier, if we do a curl against the Star Wars API, that's really hard to read. One thing we can do is we can pipe the output of that curl into another command. I've installed a global node module called JSOM. If we run that, it pipes the output to this Node.JS module and then formats it, it's a lot easier to read large JSON responses.

Jwan Khalaf
Jwan Khalaf
~ 5 years ago

How do you install jsome and is there any setup required after you've installed it to make it work in bash like you showed?

Cameron Nokes
Cameron Nokesinstructor
~ 5 years ago

How do you install jsome and is there any setup required after you've installed it to make it work in bash like you showed?

Nothing special, just npm install -g jsome. NPM handles putting it on your PATH automatically. Another option to check out is jq which not only does json formatting but allows you to query and manipulate the json. (It's not a node module, you'll have to install it via brew or something like that). See https://egghead.io/lessons/jq-read-and-use-json-in-bash-with-jq for how to use that. Hope that helps 😀

Markdown supported.
Become a member to join the discussionEnroll Today