Some commands with cURL that I wanted to save somewhere for reference.
Say that you’d like a simple check on time it takes for the entire operation to complete, to request a webpage. Then you can try this:
$ curl --output /dev/null --silent -w "Type: %{content_type}\nCode: %{response_code}\nTime: %{time_total}\n" https://nahusznaj.github.io/
Type: text/html; charset=utf-8
Code: 200
Time: 0.169270
Let’s break down the command. We have --output /dev/null, --silent, and -w "Type: %{content_type}\nCode: %{response_code}\nTime: %{time_total}\n".
--output /dev/null
From man curl, --output is:
-o, –output
: Write output to instead of stdout.
And what does --output /dev/null do? From https://en.wikipedia.org/wiki/Null_device, “the null device is a device file that discards all data written to it but reports that the write operation succeeded”. If we don’t use it, we’d see the response in the command line. For my site: https://nahusznaj.github.io/, this will be a HTML file. We don’t need to see this now. So, with this option, we’re asking cURL to write the output of the GET request into the folder /dev/null, which is like a virtual dump (a blackhole).
For simplicity, I will show just a bit of the repsonse in the case we don’t use this option:
$ curl --silent -w "Type: %{content_type}\nCode: %{response_code}\nTime: %{time_total}\n" https://nahusznaj.github.io/
<!doctype html>
<!--
Minimal Mistakes Jekyll Theme 4.11.2 by Michael Rose
Copyright 2013-2018 Michael Rose - mademistakes.com | @mmistakes
Free for personal and commercial use under the MIT license
https://github.com/mmistakes/minimal-mistakes/blob/master/LICENSE.txt
-->
<html lang="en" class="no-js">
<head>
<meta charset="utf-8">
<!-- begin _includes/seo.html --><title>Bio - My personal website</title>
<meta name="description" content="Home">
[...]
</body>
</html>Type: text/html; charset=utf-8
Code: 200
Time: 0.369955
You can see the page in plain HTML. But for this purpose, we don’t want this. So the option --output /dev/null will put this HTML in a file in that folder, which effecitvely makes it to go away. If we wanted to save the output in a file, we could pass another folder.
--silent
Without passing the silent option, curl will show progress meter or error messages.
This option will make curl mute. Let’s see what the output is without the silent mode:
$ curl --output /dev/null -w "Type: %{content_type}\nCode: %{response_code}\nTime: %{time_total}\n" https://nahusznaj.github.io/
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 8295 100 8295 0 0 26250 0 --:--:-- --:--:-- --:--:-- 26167
Type: text/html; charset=utf-8
Code: 200
Time: 0.316982
The first part is the progress.
-w
Finally, -w . From man curl,
-w, –write-out
: Make curl display information on stdout after a completed transfer.
And there are many options. We used: content_type, response_code and time_total.
content_type The Content-Type of the requested document, if there was any.
response_code (http_code) The numerical response code that was found in the last retrieved HTTP(S) or FTP(s) transfer.
time_total The total time, in seconds, that the full operation lasted.
Another useful one could be this:
time_namelookup The time, in seconds, it took from the start until the name resolving was completed.
Let’s try it out:
$ curl --output /dev/null --silent -w "Type: %{content_type}\nCode: %{response_code}\nTimeTotal: %{time_total}\nTimeNameLookup: %{time_namelookup}\n" https://nahusznaj.github.io/
Type: text/html; charset=utf-8
Code: 200
TimeTotal: 0.379680
TimeNameLookup: 0.001620
Two nice pages that I read to learn this are https://ec.haxx.se/usingcurl/usingcurl-verbose/usingcurl-writeout, and https://ops.tips/gists/measuring-http-response-times-curl/, and curl’s manual.