curl: methods, follow redirects, response headers, response time, multipart upload, authentication… and more

Basic usage: simple GET method

This will make a GET request to the given URL and print response to stdout.

$ curl https://example.com

Ignore invalid and self-signed SSL connection errors with curl

Use -k or --insecure option

$ curl -k https://domain-with-insecure-ssl.com

Print response headers

Use -i option to print response headers and the response itself, -I option to omit the response and print response headers only.

$ curl -I https://example.com

Sample response:

HTTP/2 200
accept-ranges: bytes
age: 135602
cache-control: max-age=604800
content-type: text/html; charset=UTF-8
date: Wed, 06 Jan 2021 07:16:58 GMT
etag: "3147526947"
expires: Wed, 13 Jan 2021 07:16:58 GMT
last-modified: Thu, 17 Oct 2019 07:18:26 GMT
server: ECS (oxr/8310)
x-cache: HIT
content-length: 1256

Print payload size and response time

For this, we need a custom output template. First, create a template file, curl-format.txt for example, with following content:

        size_request:  %{size_request} bytes\n
size_upload: %{size_upload} bytes\n
size_download: %{size_download} bytes\n
size_header: %{size_header} bytes\n
time_namelookup: %{time_namelookup}s\n
time_connect: %{time_connect}s\n
time_appconnect: %{time_appconnect}s\n
time_pretransfer: %{time_pretransfer}s\n
time_redirect: %{time_redirect}s\n
time_starttransfer: %{time_starttransfer}s\n
----------\n
time_total: %{time_total}s\n

Then using -w option to specify output format, as following:

$ curl -w "@curl-format.txt" example.com

Sample output:

        size_request:  76 bytes
size_upload: 0 bytes
size_download: 0 bytes
size_header: 351 bytes
time_namelookup: 0.001756s
time_connect: 0.236285s
time_appconnect: 0.000000s
time_pretransfer: 0.236358s
time_redirect: 0.000000s
time_starttransfer: 0.478430s
----------
time_total: 0.478695s

Follow redirects

It’s extremely common for HTTP servers to return a 301 or 302 redirect for a given URL. One common example of this is to redirect your browser from an HTTP URL to HTTPS, like http://ucode.vn to https://ucode.vn.

$ curl -i ucode.vn

And we can see HTTP/1.1 301 Moved Permanently code:

HTTP/1.1 301 Moved Permanently
Server: nginx/1.18.0 (Ubuntu)
Date: Wed, 06 Jan 2021 07:31:13 GMT
Content-Type: text/html
Content-Length: 178
Connection: keep-alive
Location: https://ucode.vn/
<html>
<head><title>301 Moved Permanently</title></head>
<body>
<center><h1>301 Moved Permanently</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>

Running curl command with -L or --location options will automatically handle any 3XX redirects and will retrieve whatever data is returned by the resulting URL.

$ curl -I -L ucode.vn

and we can see:

HTTP/1.1 301 Moved Permanently
Server: nginx/1.18.0 (Ubuntu)
Date: Wed, 06 Jan 2021 07:33:53 GMT
Content-Type: text/html
Content-Length: 178
Connection: keep-alive
Location: https://ucode.vn/
HTTP/1.1 200 OK
Server: nginx/1.18.0 (Ubuntu)
Date: Wed, 06 Jan 2021 07:33:53 GMT
Content-Type: text/html; charset=utf-8
Content-Length: 85913
Connection: keep-alive
X-Powered-By: Express
Set-Cookie: UCODE_LOCALE=vi; Path=/; Expires=Thu, 06 Jan 2022 07:33:53 GMT; SameSite=Lax
Set-Cookie: UCODE_PRODUCTION_IDE_CONFIGS=%7B%22theme%22%3A%22dark%22%2C%22layout%22%3A%22vertical%22%2C%22language%22%3A%7B%22id%22%3A%22python%22%2C%22name%22%3A%22Python%20(Interactive)%22%7D%7D; Path=/
Set-Cookie: UCODE_LOCALE=vi; Max-Age=2592000; Path=/
ETag: "14f99-LowxBo/SA1VGY9E29rvOz4t6Zck"
Accept-Ranges: none
Vary: Accept-Encoding

Making a POST request

Useful options:

  • Request method: -X POST
  • Content-type header: -H "Content-Type: application/x-www-form-urlencoded" (default) or -H "Content-Type: application/json"
  • Form url-encoded data: -d "param1=value1&param2=value2" or -d @data.txt
  • Json data: -d '{"key1":"value1", "key2":"value2"}’ or -d @data.json

Examples:

POST application/x-www-form-urlencoded:

$ curl -d "param1=value1&param2=value2" -X POST {url}

or with a data file:

$ curl -d "@data.txt" -X POST {url}

sample content of data.txt:

param1=value1&param2=value2

POST application/json:

$ curl -d '{"key1":"value1", "key2":"value2"}' -H "Content-Type: application/json" -X POST {url}

or with a data file:

$ curl -d "@data.json" -X POST {url}

sample content of data.json file:

{
"key1":"value1",
"key2":"value2"
}

HTTP Basic authentication

$ curl http://username:passwd@domain.com

or using -u option:

$ curl -u username:passwd {url}

Multipart file upload

Using -F or --form option:

$ curl -F upload=@localfilename -F press=OK [URL]

The corresponding HTML form could be written like this:

<form method="POST" enctype='multipart/form-data' action="upload.php">
<input type=file name=upload>
<input type=submit name=press value="OK">
</form>

--

--

--

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

DotVVM Feb Meetup on 25th

Use Python to Hang Your Picture Frames on Your Wall

wall with two picture frames hanging centered

GraphQL vs REST APIs | What’s the Best Kind of API?

StereoLOGIC and GoAutomate Partnership

One restaurant that has always done it right is Bareburger. If you don’t live in

The Partial Options Monoid

MongoDB integer primary key (nodejs example) using server side functions

How to incrementally migrate the data from RDBMS to Hadoop using Sqoop Incremental Last Modified…

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Chi Thuc Nguyen

Chi Thuc Nguyen

More from Medium

Headfirst Java (Chapter 3)- Know Your Variables

Know After 10 years How you will see Luxury Limousines: The Future of Limo Service in Toronto

Telos Oracle & Data Logging Service (data.scribe)

Implement the Concept of Template and Do-while Loop in python